* (feat) Add option to fetch Firefox Nightly
Add Firefox support to BrowserFetcher and the install script.
By default, the latest Firefox Nightly is downloaded
directly from archive.mozilla.org (dmg, tar.bz2 and zip)
This also required changes that impact `puppeteer.launch()`
and `puppeteer.executablePath()`
Fixes#5151
* Update docs/api.md
Co-Authored-By: Mathias Bynens <mathias@qiwi.be>
* Clean up revision promise
* Improve error handling in revision check
* Remove matchAll
* Use explicit octal mode
* Update .gitignore
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
* chore: update relevant Node.js versions from 8 to 10
* chore: remove node6 and node8 folders from puppeteer-firefox ci
* fix: loosen definition for proc.stdio
* fix: update typescript version used in npm run test-types
This changes the Chromium revision to r722234 (Chrome 80.0.3987.0),
since that's the most recent version in the Chromium 80 range for
which a download exists for all supported platforms.
https-proxy-agent requires agent-base, which currently monkey-patches the core `https` Node module, causing problems in unrelated code. The latest version of https-proxy-agent uses the latest version of agent-base which no longer does this monkey patching.
* fix: prepare jsHandle.uploadFile for CDP Page.handleFileChooser removal
https://chromium-review.googlesource.com/c/chromium/src/+/1935410
removes Page.handleFileChooser from the CDP.
* fix: improve binary file support
UTF-8-decoding the input file could fail for binary files, and so we
now read the raw file buffer and base64-encode it. To base64-decode it
within the page context, we use the Fetch API in combination with a
data URL. This requires knowing the proper MIME type for the input
file, which we now figure out using the new mime-types dependency.
* feat: Set which browser to launch via PUPPETEER_PRODUCT
This change introduces a PUPPETEER_PRODUCT environment
variable as a first step toward using Puppeteer with
many different browsers. Setting PUPPETEER_PRODUCT=firefox, for
example, enables Firefox-specific Launcher settings.
The state is also exposed as `puppeteer.product` in the API
to support adding other product-specific behaviour as needed.
The bulk of the change is a refactoring in Launcher
to decouple generic browser start-up from product-specific
configuration.
Respecting the puppeteer-core restriction for PUPPETEER_
environment variables, lazily instantiate the Launcher
based on a `product` Puppeteer.launch option, if available.
* test: Distinguish Juggler unit tests from Firefox
The funit script is renamed to fjunit (j for Juggler, which is
used only by the experimental puppeteer-firefox package.
In contrast, the funit script now refers to running Puppeteer
unit tests against the main puppeteer package with Firefox.
To do so with Firefox Nightly, run:
`BINARY=path/to/firefox npm run funit`
A number of changes in this patch make it easier to run
Puppeteer unit tests in Mozilla's CI.
Node.js v6 was end-of-life'd in April, 2019, with AWS Lambda prohibiting updaets to the Node.js v6 runtime since June 30, 2019.
This makes it quite safe for us to remove the Node 6 support from the repository.
This roll includes:
- https://crrev.com/681997 - Turn on default SiteInstance by default.
The SiteInstance by default was breaking "devtools: true" option, so
there's a new feature we disable now by default.
This keeps pressuring us towards OOPIF support since that's an
inevitable future.
* feat(chromium): roll Chromium to r665405
This roll includes:
- https://crrev.com/665226 - DevTools: make interception respect cross-process frame boundaries
This fixes page loading with dynamic OOPIFs - test is added.
Fix#4442
* fix lint
This roll includes:
- [inspector_protocol:8ec18cf](8ec18cf088) Support STRING16 in the template when converting CBOR map keys
to protocol::Value.
- [inspector_protocol:37518ac](37518ac421) fix parsing of the last ASCII character
This fixes protocol handling of UTF8 in both V8 and Chromium.
Fixes#4443.
This roll includes:
- https://crrev.com/653809 - FrameLoader: ignore failing provisional loads entirely
- https://crrev.com/654750 - DevTools: make sure Network.requestWillBeSent is emitted on time for sync xhrs
The FrameLoader patch is the reason behind the test change. It's
actually desirable to fail frame navigation if the frame detaches - and
that's consistent with Firefox.
Fixes#4337
Introduce a `npm run funit` script that runs puppeteer tests
against Puppeteer-Firefox.
Next steps:
- bring Puppeteer-Firefox unique tests to Puppeteer
- skip failing tests and run Puppeteer-Firefox on CI
- work through tests to pass them all with Puppeteer-Firefox
This roll includes:
- https://crrev.com/624247 - DevTools: Allow DOM.resolveNode to resolve
into isolated worlds
- https://crrev.com/624486 - DevTools: addScriptToEvaluateOnNewDocument
should work with disabled javascript
This roll includes:
- https://crrev.com/619087 - DevTools: support interception for file: schema
- https://crrev.com/616936 - Complete the screen capture color space plumbing
This should allow us to switch to network service by default.
Note: We now have to force a specific color space since https://crrev.com/616936
tries to pick the system one.
This roll includes:
- https://crrev.com/609886 - DevTools: force-detach worker sessions
on navigation
This should eliminate flakiness with our worker test.
Also, new Chrome now exposes a new type in its protocol - binary.
It becomes a raw C++ array once used through C++ bindings, but for
us it's still a base64 string.
This roll includes:
- https://crrev.com/599782 DevTools: force a style recalc before reporting final CSS coverage
- https://crrev.com/599769 DevTools: Expose more properties through the accessibility protocol
References: #2033
This upgrades us to TypeScript 3.1.1, which fixes some build failures. Annoyingly TypeScript does a better job of checking `process.stdio`, which exposes that the DefinitelyTyped definition for it is wrong. See https://github.com/DefinitelyTyped/DefinitelyTyped/issues/11587. I'll look into submitting a patch for that later.
This patch:
- adds "browser" field to the package.json with default
bundling options.
- introduces "bundle" and "unit-bundle" commands to
create bundle and test bundle
- starts running bundle tests on Travis Node 8 bots
Fixes#2374.
This roll includes:
- https://crrev.com/584293 - DevTools: execute scripts in addScriptToEvaluateOnLoad in order
- https://crrev.com/585630 - DevTools: introduce Browser.grantPermissions
- https://crrev.com/587156 - Revert "[Base] Use background mode for ThreadPriority::BACKGROUND threads (behind feature) (reland)."
The "revert" patch fixes headless functionality on windows.
References #846.
Fixes#3106.
This patch rolls Chromium to r579032. The patch includes:
- https://crrev.com/577366 - DevTools: report redirect responses only if response interception is enabled
- https://crrev.com/577212 - DevTools: intercept requests resulting from redirects
- https://crrev.com/578934 - DevTools: Add a protocol method to insertText
Interception Logic in DevTools protocol has changed regarding redirects;
this patch migrates interceptions to dispatch "request" events based on
requestWillBeSent event.
This roll includes:
- https://crrev.com/574785 - DevTools: allow tracing over the remote debugging pipe.
This fixes tracing over the remote debugging pipe.
This roll includes:
- https://crrev.com/567104 - DevTools: introduce Target.exposeDevToolsProtocol() method
The patch includes a drive-by fix to DevToolsAgentHostImpl that
eliminats chromium crashes in certain cases.
This patch drops the markdown-toc module and instead rolls out
our own simple markdown table-of-contents generator.
As a side effect, it fixes links to `page.$` and `page.$$`.
Previously protocol.d.ts was generated on `npm run tsc`. This was inconvenient because it meant that vscode checking was wrong until type checking was run manually, and was inefficient because it necessarily regenerated the types even if no new Chromium was downloaded. This patch generates the types when npm install is run from the github checkout, assuming a new Chromium revision was downloaded.
This roll includes:
- https://crrev.com/551261 - DevTools: page.navigate should fail when server returned HTTP 204
- https://crrev.com/550319 - DevTools: fix resource mimetype for request interception of file:// urls
References #1879
References #1506
This roll includes:
- https://crrev.com/549003 - DevTools: make pptr tests pass with DCHECKs.
The patch fixes a browser crash that happens during browser close.
As a result, cookies were not saved properly (and thus the flaky test we
had).
Fixes#1537.
This uses the `/json/protocol` endpoint to generate type definitions for the protocol.
Currently it is lacking protocol events and commands, but I will add those later.
This roll includes:
- https://crrev.com/547982 - v8 roll that includes [fixed
Runtime.callFunctionOn](1637818671) method
The upstream fix makes it possible to run frame.waitFor* functions on
pages with strict CSP.
References #1229.
This roll includes the following important CLs:
- https://crrev.com/545955 - DevTools: Update page scale immediately in InputHandler
- https://crrev.com/544446 - Make HeadlessWebContentsImpl::Close wait for the renderer to go
- https://crrev.com/546865 - DevTools(Headless): close WebContents when closed via
'window.close'
- https://crrev.com/546685 - Headless: Add support for the --remote-debugging-pipe flag
This should drastically reduce flakiness for pptr input events.
This patch:
- removes the "tags: true" field, we want to publish every commit
- makes sure we publish from node7 only
- prebuilds node6 before publishing
This patch rolls chromium to r524617. This roll includes:
- http://crrev.com/523674 - Headless printing: refactor print template
to allow header/footer customization.
References #373
This patch unifies node6 transpilation:
- instead of generating multiple top-level directories, prefixed with
`node6-`, all transpiled code gets placed under single `node6/` folder
- transpilation doesn't change require paths of transpiled modules any
more
This patch introduces a tiny test runner to run puppeteer tests.
The test runner is self-container and allows parallel (wrt IO) test execution.
It will also allow us to split tests into multiple files if necessary.
Comparing to the jasmine, the testrunner supports parallel execution, properly
handles "unhandled promise rejection" event and signals.
Comparing to ava/jest, the testrunner doesn't run multiple node processes,
which makes it simpler but sufficient for our goals.
This roll includes the following revisions:
- crrev.com/515281 DevTools: fix crash on intercepting request that
posts a blob
- crrev.com/515368 DevTools: wait for navigation to be committed
upon Page.navigate on the browser side.
Fixes#894, References #1218
This roll brings in a bunch of important patches:
- crrev.com/512647 Changed headless browser profile dir to use Default profile path
- crrev.com/512760 DevTools: stop idleness detector when pending navigation commits
- crrev.com/512905 DevTools: introduce Page.getFrameTree
- crrev.com/513373 DevTools: report loaderId in the lifecycle events
- crrev.com/513419 DevTools: introduce Page.setLifecycleEventsEnabled
- crrev.com/513422 DevTools: return loaderId from Page.navigate
Fixes#921
BREAKING CHANGE:
Headless user profile structure is changing. Custom profiles set with --user-data-dir flag will no longer be read in Chrome 63 and will have to be recreated.
Alternatively, you can migrate old headless profile to a new structure. if you stored your profile in `<profile>` folder, you would run the following bash commands:
```bash
cd <profile>
mkdir Default
mv * Default
```
Full headless-dev PSA announcement: https://groups.google.com/a/chromium.org/forum/#!msg/headless-dev/asX8WgktXIE/zTUfmHDcAQAJ
This roll includes:
- crrev.com/510651 that changes request interception methods in protocol
- s/Page.setRequestInterceptionEnabled/Page.setRequestInterception
BREAKING CHANGE
Page.setRequestInterceptionEnabled is renamed into
Page.setRequestInterception.
This roll includes the following important revisions:
- https://crrev.com/509990 - DevTools: Allow location to be specified in
Input.dispatchKeyEvent
- https://crrev.com/509486 - DevTools: fix extra HTTP headers not sent
on browser navigation request
References #877, references #777
This patch improves life of puppeteer contributor on Windows:
- Setting environment variables using cross-env since Windows requires the SET command.
- Calling Jasmine in the script debug-unit using jasmine's JavaScript binary instead of shell.
- Add /test/test-user-data-dir* to .gitignore since temporary user data directories, in case of test
fails, remains in the test directory.
This patch starts using typescript to lint JSDoc annotations.
Note: this uses typescript's bleeding edge. We should migrate to stable once
it has all the necessary bugfixes.
References #65.
This patch rolls chromium to r503964.
Note: since the plznavigate is not supported by puppeteer right now, the
patch also starts passing the `--disable-browser-side-navigation` flag.
This is a temporary work around for us.
References #877.
The new chromium:
- supports executionContextId in Runtime.callFunctionOn protocol method
- supports passing objects as arguments in Runtime.callFuntionOn
protocol method
This roll also fixes#487.
The license used is Apache License 2.0, so I think it makes sense to update the package.json file with the correct string as a some software use this field to approve or not packages.
This patch kills PhantomShim, a lightweight layer that helped to
bootstrap Puppeteer API.
At this point, PhantomShim:
- passes 139 phantomJS tests
- fails 13 tests
- lacks capabilities to run 26
The failings and unsupported tests don't have any value for Puppeteer
API, so it doesn't make sense to maintain PhantomShim any longer.
This patch:
- introduces a transpiler which substitutes async/await logic with
generators.
- starts using the transpiler to generate a node6-compatible version of puppeteer
- introduces a runtime-check to decide which version of code to use
Fixes#316.
This patch bumps version to 0.9.1-alpha.
This should emphasize that the documentation is related to the tip-of-tree
version of puppeteer, not to the latest release.
This patch rolls chromium to r496140. This includes the r496130 that
introduces multiple sessions for single target.
With this patch, it is possible to run puppeteer in headful mode
and open devtools over the automated pages without puppeteer losing
connection to the page.
