* chore: Use devtools-protocol package
Rather than maintain our own protocol we can instead use the devtools-protocol package and pin it to the version of Chromium that Puppeteer is shipping with.
The only changes are naming changes between the bespoke protocol that Puppeteer created and the devtools-protocol one.
This change started as a small change to pull types from DefinitelyTyped over to
Puppeteer for the `evaluateHandle` function but instead ended up also fixing
what looks to be a long standing issue with our existing documentation.
`evaluateHandle` can in fact return an `ElementHandle` rather than a `JSHandle`.
Note that `ElementHandle` extends `JSHandle` so whilst the docs are technically
correct (all ElementHandles are JSHandles) it's confusing because JSHandles
don't have methods like `click` on them, but ElementHandles do.
if you return something that is an HTML element:
```
const button = page.evaluateHandle(() => document.querySelector('button'));
// this is an ElementHandle, not a JSHandle
```
Therefore I've updated the original docs and added a large explanation to the
TSDoc for `page.evaluateHandle`.
In TypeScript land we'll assume the function will return a `JSHandle` but you
can tell TS otherwise via the generic argument, which can only be `JSHandle`
(the default) or `ElementHandle`:
```
const button = page.evaluateHandle<ElementHandle>(() => document.querySelector('button'));
```
As far as I can tell these became irrelevant as of v1.15 which added
`puppeteer.errors` and `puppeteer.devices [1]. This is a breaking change
but one that's easily mitigated. We've said that we don't consider
changes to our folder/file structure a breaking change, but we can't
really do that if we have these two top level files that we've
documented.
[1]: e3abb0aa32 (diff-522b24108d7446af4c59873472a90444)
Just one was used externally and I wrapped that up in a method. I think
it's a useful method to provide (I can imagine wanting to know if JS is
enabled on a page) so I think there's no harm here (I'd rather that then
have JSHandle reach into a private variable).
Replacing the Node EventEmitter with Mitt caused more problems than
anticipated for end users due to the API differences and the amount of
people who relied on the EventEmitter API. In hindsight this clearly
should have been explored more and then released as a breaking v4.
This commit rolls us back to the built in Node EventEmitter library
which we can release to get everyone back on stable builds. We can then
consider our approach to migrating to Mitt and when we do do that we can
release it as a breaking change and properly document the migration
strategy and approach.
* chore: migrate to Mitt as the EventEmitter
This commit moves us to using Mitt [1] for the event emitter in
Puppeteer. This removes our dependency to Node's EventEmitter which is
part of a larger stream of work to enable a Puppeteer-web version that
doesn't depend on Node.
There are no large breaking changes as we support the main methods that
EventEmitter had, but it also provides some methods that Puppeteer
didn't use. Technically end users could depend on this but it's
unlikely.
[1]: https://github.com/developit/mitt
It conflicts with an inbuilt TypeScript `Request` type so can cause
confusion when in TS land. Note: `Response.ts` and `Worker.ts` also
suffer from this; PRs to rename them are incoming.
Fixes an edge case where Puppeteer looked for a Chromium revision when launching Firefox.
Allow appropriate Launcher to be instantiated when calling `Puppeteer.connect`.
Add an example of running Firefox.
The recommended Dockerfile uses node:10-slim image as a base,
but the base image does not contain wget command anymore.
(About the reason, see https://github.com/nodejs/docker-node/issues/1185)
So fixed the problem.
* (feat) Add option to fetch Firefox Nightly
Add Firefox support to BrowserFetcher and the install script.
By default, the latest Firefox Nightly is downloaded
directly from archive.mozilla.org (dmg, tar.bz2 and zip)
This also required changes that impact `puppeteer.launch()`
and `puppeteer.executablePath()`
Fixes#5151
* Update docs/api.md
Co-Authored-By: Mathias Bynens <mathias@qiwi.be>
* Clean up revision promise
* Improve error handling in revision check
* Remove matchAll
* Use explicit octal mode
* Update .gitignore
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
* chore: update relevant Node.js versions from 8 to 10
* chore: remove node6 and node8 folders from puppeteer-firefox ci
* fix: loosen definition for proc.stdio
* fix: update typescript version used in npm run test-types
This makes it more clear that the callback receives an actual array of nodes instead of just a NodeList.
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
This changes the Chromium revision to r722234 (Chrome 80.0.3987.0),
since that's the most recent version in the Chromium 80 range for
which a download exists for all supported platforms.
* feat: Set which browser to launch via PUPPETEER_PRODUCT
This change introduces a PUPPETEER_PRODUCT environment
variable as a first step toward using Puppeteer with
many different browsers. Setting PUPPETEER_PRODUCT=firefox, for
example, enables Firefox-specific Launcher settings.
The state is also exposed as `puppeteer.product` in the API
to support adding other product-specific behaviour as needed.
The bulk of the change is a refactoring in Launcher
to decouple generic browser start-up from product-specific
configuration.
Respecting the puppeteer-core restriction for PUPPETEER_
environment variables, lazily instantiate the Launcher
based on a `product` Puppeteer.launch option, if available.
* test: Distinguish Juggler unit tests from Firefox
The funit script is renamed to fjunit (j for Juggler, which is
used only by the experimental puppeteer-firefox package.
In contrast, the funit script now refers to running Puppeteer
unit tests against the main puppeteer package with Firefox.
To do so with Firefox Nightly, run:
`BINARY=path/to/firefox npm run funit`
A number of changes in this patch make it easier to run
Puppeteer unit tests in Mozilla's CI.
Adds note about Jest maxWorkers as well as the base image to start with. This is an improvement over the previous section I wrote, from me banging my head against a YAML file all week 🙃.
This came from personal difficulties in running Puppeteer tests on CircleCI. I tried to keep the note as brief as possible, while being helpful for an entire CI platform.
This patch introduces a page.waitForFileChooser() method
that adds a watchdog to wait for file chooser dialogs.
This lets Puppeteer users to capture file chooser requests
and fulfill/cancel them if necessary.
Fixes#2946
A freetype update broke bitmap fonts. Adding freetype-dev to the Alpine dependencies resolves related issues.
This resolves these errors when launching chromium:
```/usr/bin/chromium-browser
Error relocating /usr/lib/chromium/chrome: FT_Get_Color_Glyph_Layer: symbol not found
Error relocating /usr/lib/chromium/chrome: FT_Palette_Select: symbol not found```
Background:
https://bugs.alpinelinux.org/issues/10309https://github.com/stark/siji/issues/28https://github.com/lucy/tewi-font/issues/35
The documentation for frame.goto() and page.goto() were updated to make
it clear that the method will not throw an error if the HTTP requests
results in any valid HTTP status code being returned by the remote
server.
Going from `AXNode` -> `ElementHandle` is turning out to be controversial.
This patch instead adds a way to go from `ElementHandle` -> `AXNode`. If the API looks good, I'll add it into Firefox as well.
References #3641
- Pins Alpine Chromium version to prevent updates from causing issues
When using these instructions today, I found that the Chromium version in the latest Alpine edge was 73, which caused errors with Puppeteer.
Pinning to the latest 72 version in Alpine registry resolved the issue, and should prevent others from running into it in the future.
This roll includes:
- https://crrev.com/653809 - FrameLoader: ignore failing provisional loads entirely
- https://crrev.com/654750 - DevTools: make sure Network.requestWillBeSent is emitted on time for sync xhrs
The FrameLoader patch is the reason behind the test change. It's
actually desirable to fail frame navigation if the frame detaches - and
that's consistent with Firefox.
Fixes#4337
These getters are introduced as a more convenient substitute for
a `require('puppeteer/Errors')` and
`require('puppeteer/DeviceDescriptors')`.
This way we can make cross-browser story nicer - a single require
of `puppeteer` or `puppeteer-firefox` fully defines Puppeteer
environment.
* removing libgconf-2-4 install since no longer needed according to https://bugs.chromium.org/p/chromium/issues/detail?id=795759#c7
* wget is already included in `node:8-slim` image, so removed lines related to install/cleanup
* node 8 has EOL this year, so incremented to node:10-slim
* use "docker run --init" if available (available in docker-engine >= 1.13.0)
* make dumb-init optional
* combine permission changes and 'npm install' of puppeteer into same line to reduce image size by few hundred MB
* overall image size reduction: 1.21GB -> 865MB
A link on line 525 is pointing to a undefined branch (`lkcr`) in `chromium.googlesource.com/chromium/src/`. Change it to point to `lkgr` instead, since it's the closest defined branch in name.
Method `page.setDefaultTimeout` overrides default 30 seconds timeout
for all `page.waitFor*` methods, including navigation and waiting
for selectors.
Fix#3319.
`page.waitForSelector` should return `null` if waiting for `hidden:
true` and there's no matching node in DOM.
Before this patch, `page.waitForSelector` would return some JSHandle
pointing to boolean value.
Makes Running on Alpine up to date:
- Chrome is now available in LTS Node 10
- Chrome version is updated to the latest alpine `@edge`, 71
- Corresponding Puppeteer is updated to v1.9.0
- `harfbuzz` is now required by dynamic linking
The proposal adds a drop-down list in a similar fashion as Dependencies list since it feels a little weird to have list for a continuing detail as I assume that all the three options belong to the same level of information.
Very small change in light of operational experience while getting it running on Centos in Jenkins pipeline.
Without the `-p`, the permissions set in the `chmod` before this command are not carried over chrome cannot start.
This patch teaches `page.setContent` to await resources in
the new document.
**NOTE**: This patch changes behavior: currently, `page.setContent`
awaits the `"domcontentloaded"` event; with this patch, we can now await
other lifecycle events, and switched default to the `"load"` event.
The change is justified since current behavior made `page.setContent`
unusable for its main designated usecases, pushing our client
to use [dataURL workaround](https://github.com/GoogleChrome/puppeteer/issues/728#issuecomment-334301491).
Fixes#728
This adds `page.accessibility.snapshot()`. It serializes and returns the accessibility tree for the page. By default, uninteresting nodes are filtered out of the snapshot.
fixes#2033
This adds `browser.waitForTarget` and `browserContext.waitForTarget`. It also fixes a flaky test that was incorrectly expecting targets to appear instantly.
This patch:
- adds experimental "transport" option to pptr.connect
- uses "transport" option to make sure Puppeteer-Web works with
Target.exposeDevToolsProtocol
Drive-by: add `browser.target()` to access browser target.
This patch introduces API to manage frame navigations.
As a drive-by, the `response.frame()` method is added as a shortcut
for `response.request().frame()`.
Fixes#2918.
We had (and still have) a ton of pull requests to support
PUPPETEER_EXECUTABLE_PATH and PUPPETEER_CHROMIUM_REVISION in puppeteer launcher.
We were hesitant before since env variables are not scoped
and thus don't make a good interface for a library. Now, since we
determined `puppeteer-core` as a library and `puppeteer` as our end-user
product, it's safe to satisfy our user needs.
This patch:
- teaches PUPPETEER_EXECUTABLE_PATH and PUPPETEER_CHROMIUM_REVISION
env variables to control how Puppeteer launches browser
- makes sure these variables play no role in `puppeteer-core` package.
If referer is passed to the options object its value will be used as the referer instead of the value set by `Page.setExtraHTTPHeaders()`.
This is the correct way to set referer header: otherwise, the `referer` header will override all the document subrequests.
Fixes#3090.
Introduce an API to manage permissions per browser context:
- BrowserContext.overridePermissions(origin, permissions)
- BrowserContext.clearPermissionOverrides()
Fixes#846.
It turned out that almost any usecase requires helper methods to access
DOM inside the ExecutionContext.
Instead of exposing execution contexts as-is, we should introduce
IsolatedWorld as a first-class citizen that will hold execution contexts
inside.
This patch adds a new require, `puppeteer/Errors`, that
holds all the Puppeteer-specific error classes.
Currently, the only custom error class we use is `TimeoutError`. We'll
expand in future with `CrashError` and some others.
Fixes#1694.
