* feat: Set which browser to launch via PUPPETEER_PRODUCT
This change introduces a PUPPETEER_PRODUCT environment
variable as a first step toward using Puppeteer with
many different browsers. Setting PUPPETEER_PRODUCT=firefox, for
example, enables Firefox-specific Launcher settings.
The state is also exposed as `puppeteer.product` in the API
to support adding other product-specific behaviour as needed.
The bulk of the change is a refactoring in Launcher
to decouple generic browser start-up from product-specific
configuration.
Respecting the puppeteer-core restriction for PUPPETEER_
environment variables, lazily instantiate the Launcher
based on a `product` Puppeteer.launch option, if available.
* test: Distinguish Juggler unit tests from Firefox
The funit script is renamed to fjunit (j for Juggler, which is
used only by the experimental puppeteer-firefox package.
In contrast, the funit script now refers to running Puppeteer
unit tests against the main puppeteer package with Firefox.
To do so with Firefox Nightly, run:
`BINARY=path/to/firefox npm run funit`
A number of changes in this patch make it easier to run
Puppeteer unit tests in Mozilla's CI.
Node.js v6 was end-of-life'd in April, 2019, with AWS Lambda prohibiting updaets to the Node.js v6 runtime since June 30, 2019.
This makes it quite safe for us to remove the Node 6 support from the repository.
We'd like to pass an abortion signal inside Helper.waitForEvent in order to interrupt it when browser/page closes. Several approaches have been considered:
1. Pass CDPSession instance as a another parameter to the helper method and listen to Disconnected event on it. It would introduce undesired dependency on the session object.
2. Listen to the CDPSession closure at the call sites (e.g. waitForRequest) and pass an abortion promise which would be fulfilled when such event is fired. The listeners would have to be removed from the session on successful completion of waitForEvent so we'd have to pass some kind of DisposablePromise which would be disposed during cleanup. Such parameter looked somewhat hairy.
3. Create DisconnectPromise on CDPSession. One potential risk with that is all chained promises would hang around until the event is fired which might inadvertently cause memory leaks. On the other hand, adding such promise to Promise.race will remove dependency as soon as the race is finished. So this is the approach we're taking with one tweak: the promise is created locally inside Page.
Ideally the disconnectPromise would throw when the session is closed but it may lead to uncaught promise errors if all chained promises are resolved, to avoid that the promise is resolved with an Error and Helper.waitForEvent throws it later.
Fix#4733
This patch introduces a page.waitForFileChooser() method
that adds a watchdog to wait for file chooser dialogs.
This lets Puppeteer users to capture file chooser requests
and fulfill/cancel them if necessary.
Fixes#2946
Introduce `//lib/api.js` that declares a list of publicly exposed
classes.
The `//lib/api.js` list superceedes dynamic `helper.tracePublicAPI()` calls
and is used in the following places:
- [ASYNC STACKS]: generate "async stacks" for publicy exposed API in `//index.js`
- [COVERAGE]: move coverage support from `//lib/helper` to `//test/utils`
- [DOCLINT]: get rid of 'exluded classes' hardcoded list
This will help us to re-use our coverage and doclint infrastructure
for Puppeteer-Firefox.
Drive-By: it turns out we didn't run coverage for `SecurityDetails`
class, so we lack coverage for a few methods there. These are excluded
for now, sanity tests will be added in a follow-up.
This adds `browser.waitForTarget` and `browserContext.waitForTarget`. It also fixes a flaky test that was incorrectly expecting targets to appear instantly.
This patch traces all public async methods and wraps them
in a helper method that tags the sync stack trace.
Later on, if the method call throws an exception, we add
a captured stack trace to the original stack trace with the "--ASYNC--"
heading.
An example of a stack trace:
```
Error: net::ERR_ABORTED at http://localhost:8907/empty.html
at navigate (/Users/lushnikov/prog/puppeteer/lib/Page.js:622:37)
at process._tickCallback (internal/process/next_tick.js:68:7)
-- ASYNC --
at Page.<anonymous> (/Users/lushnikov/prog/puppeteer/lib/helper.js:147:27)
at fit (/Users/lushnikov/prog/puppeteer/test/page.spec.js:546:18)
at process._tickCallback (internal/process/next_tick.js:68:7)
```
This patch removes all dynamic requires in Puppeteer. This should
make it much simpler to bundle puppeteer/puppeteer-core packages.
We used dynamic requires in a few places in lib/:
- BrowserFetcher was choosing between `http` and `https` based on some
runtime value. This was easy to fix with explicit `require`.
- BrowserFetcher and Launcher needed to know project root to store
chromium revisions and to read package name and chromium revision from
package.json. (projectRoot value would be different in node6).
Instead of doing a backwards logic to infer these
variables, we now pass them directly from `//index.js`.
With this patch, I was able to bundle Puppeteer using browserify and
the following config in `package.json`:
```json
"browser": {
"./lib/BrowserFetcher.js": false,
"ws": "./lib/BrowserWebSocket",
"fs": false,
"child_process": false,
"rimraf": false,
"readline": false
}
```
(where `lib/BrowserWebSocket.js` is a courtesy of @Janpot from
https://github.com/GoogleChrome/puppeteer/pull/2374/)
And command:
```sh
$ browserify -r puppeteer:./index.js > ppweb.js
```
References #2119
We had (and still have) a ton of pull requests to support
PUPPETEER_EXECUTABLE_PATH and PUPPETEER_CHROMIUM_REVISION in puppeteer launcher.
We were hesitant before since env variables are not scoped
and thus don't make a good interface for a library. Now, since we
determined `puppeteer-core` as a library and `puppeteer` as our end-user
product, it's safe to satisfy our user needs.
This patch:
- teaches PUPPETEER_EXECUTABLE_PATH and PUPPETEER_CHROMIUM_REVISION
env variables to control how Puppeteer launches browser
- makes sure these variables play no role in `puppeteer-core` package.
This patch adds a new require, `puppeteer/Errors`, that
holds all the Puppeteer-specific error classes.
Currently, the only custom error class we use is `TimeoutError`. We'll
expand in future with `CrashError` and some others.
Fixes#1694.
Since Node 10, `console.assert` no longer throws an AssertionError.
(This is generally good since it aligns Node.js with Browsers.)
This patch migrates all usages of `console.assert` in our codebase.
- All the `lib/` and testing code is migrated onto a handmade `assert`
function. This is to make Puppeteer transpilation / bundling easier.
- All the tooling is switched to use Node's `assert` module.
Fixes#2547.
This uses the `/json/protocol` endpoint to generate type definitions for the protocol.
Currently it is lacking protocol events and commands, but I will add those later.
This patch introduces `BrowserFetcher` class that manages
downloaded versions of products.
This patch:
- shapes Downloader API to be minimal yet usable for our needs. This
includes removing such methods as `Downloader.supportedPlatforms` and
`Downloader.defaultRevision`.
- makes most of the fs-related methods in Downloader async. The only
exception is the `Downloader.revisionInfo`: it has stay sync due to the
`pptr.executablePath()` method being sync.
- updates `install.js` and `utils/check_availability.js` to use new API
- finally, renames `Downloader` into `BrowserFetcher`
Fixes#1748.
feat: expose raw devtools protocol connection
This patch introduces `target.createCDPSession` method that
allows directly communicating with the target over the
Chrome DevTools Protocol.
Fixes#31.
This patch:
- introduces `helper.promisify` - a simple polyfill for the `util.promisify`. The
`util.promisify` could not be used due to Node6 compatibility issues.
- migrates all sync filesystem operations to the async replicas
Fixes#884.
This patch starts using typescript to lint JSDoc annotations.
Note: this uses typescript's bleeding edge. We should migrate to stable once
it has all the necessary bugfixes.
References #65.
This patch:
- introduces ExecutionContext class that incapsulates javascript
execution context. An examples of execution contexts are workers and
frames
- introduces JSHandle that holds a references to the javascript
object in ExecutionContext
- inherits ElementHandle from JSHandle
Fixes#382.
It turns out that [undefined, 1].join(',') results in ",1" instead
of "undefined,1". This causes a syntax error when trying to pass undefined
as a first argument to `page.evaluate` method.
Fixes#572.
It's very bad to have 'unhandled promise rejection' that can't be
handled in user code. These errors will exit node process in a near
future.
This patch avoids 'unhandled promise rejection' while sending protocol
messages.
This patch:
- introduces `puppeteer:error` debug scope and starts using it for all
swalloed errors.
- makes sure that every `client.send` method is either awaited or its
errors are handled.
- starts return promises from Request.continue() and Request.abort().
- starts swallow errors from Request.contine() and Request.abort().
The last is the most important part of the patch. Since
`Request.continue()` might try to continue canceled request, we should
disregard the error.
Fixes#627.
This patch:
- switches to objects instead of maps for headers (in Request, Response and
page.setExtraHTTPHeaders)
- converts all header names to lower case
Fixes#547, fixes#509
This patch:
- rolls chromium to r494365
- starts using Runtime.evaluate(awaitPromise: true), with new semantic
we can avoid additional Runtime.awaitPromise call
- stops resolving promises for Console event
This patch
- rolls chromium to 492629
- migrates connection establishing to use browser target. This migration means
that now we have a single websocket connection to browser (implemented
in Connection class). A connection to a particular target is
incapsulated in a new Session class.
The `DEBUG=*page npm run unit` is too verbose due to events spamming
the console.
This patch starts tracing emitted events only if there are any
listeners.
The page.waitForFunction method allows to wait for a general predicate.
The predicate will be continiously polled for in page, until
it either returns true or the timeout happens.
The polling parameter could be one of the following:
- 'raf' - to poll on every animation frame
- 'mutation' - to poll on every dom mutation
- <number> - to poll every X milliseconds
References #91
This patch:
- implements a basic public API coverage based on 'helper.tracePublicAPI' methods
- adds `npm run coverage` command which reports coverage after running all of the unit tests
References #50.
This patch improves on page.evaluate to accept a string.
The string can have a trailing '//# sourceURL=' comment which would
name the evaluation to make stacks beautiful.
In order to make sourceURL comments possible, this patch:
- removes wrapping of the client function into `Promise.resolve()`
- stops passing `awaitPromise` parameter to `Runtime.evaluate`
- starts to await promise via the `Runtime.awaitPromise` if the return type of the evaluation
is promise
closes#118
This patch stops serializing console API arguments unless there are
listeners of the 'console' event in puppeteer.
This saves quite a lot CPU cycles.
Fixes#117.
This patch:
- introduces helper.addEventListener/helper.removeEventListeners
to simplify event management
- moves NavigatorWatchdog over to the helper.addEventListener to
stop leaking event listeners
