Commit Graph

646 Commits

Author SHA1 Message Date
Maja Frydrychowicz
33f1967072
(feat) Add option to fetch Firefox Nightly (#5467)
* (feat) Add option to fetch Firefox Nightly

Add Firefox support to BrowserFetcher and the install script.
By default, the latest Firefox Nightly is downloaded
directly from archive.mozilla.org (dmg, tar.bz2 and zip)

This also required changes that impact `puppeteer.launch()`
and `puppeteer.executablePath()`

Fixes #5151

* Update docs/api.md

Co-Authored-By: Mathias Bynens <mathias@qiwi.be>

* Clean up revision promise

* Improve error handling in revision check

* Remove matchAll

* Use explicit octal mode

* Update .gitignore

Co-authored-by: Mathias Bynens <mathias@qiwi.be>
2020-03-10 21:59:03 +01:00
Changhao Han
807fbbdc20
chore: drop Node.js v8 support (#5365)
* chore: update relevant Node.js versions from 8 to 10

* chore: remove node6 and node8 folders from puppeteer-firefox ci

* fix: loosen definition for proc.stdio

* fix: update typescript version used in npm run test-types
2020-03-10 21:55:32 +01:00
Maja Frydrychowicz
0b1a9ceee2
chore: Replace Juggler with Firefox Nightly in CI (#5395)
Fixes #5150.
2020-02-20 13:57:15 +01:00
Pavel Feldman
9923e56b3e
fix(filechooser): quick fix for the headful file chooser (#5369)
Fixes #5363
2020-01-31 08:43:34 +01:00
Mathias Bynens
013a86cf28
feat(chromium): roll Chromium to r722269 (#5289)
This corresponds to Chromium 80.0.3987.0.

This roll includes:

- Implement support for the new ARIA `generic` role
  https://chromium-review.googlesource.com/c/chromium/src/+/1872305
- Expose button's children to accessibility tree
  https://chromium-review.googlesource.com/c/chromium/src/+/1845810
- Remove `Page.handleFileChooser` from CDP
  https://chromium-review.googlesource.com/c/chromium/src/+/1935410
2020-01-27 14:44:53 +01:00
Mathias Bynens
6091a34a36
fix: prepare jsHandle.uploadFile for CDP Page.handleFileChooser removal (#5196)
* fix: prepare jsHandle.uploadFile for CDP Page.handleFileChooser removal

https://chromium-review.googlesource.com/c/chromium/src/+/1935410
removes Page.handleFileChooser from the CDP.

* fix: improve binary file support

UTF-8-decoding the input file could fail for binary files, and so we
now read the raw file buffer and base64-encode it. To base64-decode it
within the page context, we use the Fetch API in combination with a
data URL. This requires knowing the proper MIME type for the input
file, which we now figure out using the new mime-types dependency.
2019-12-03 09:18:18 +01:00
Mathias Bynens
8b49dc62a6
fix: don’t disable BlinkGenPropertyTrees anymore (#5159)
Ref. #5080.
2019-11-27 12:58:09 +01:00
Maja Frydrychowicz
35d5ba5830 feat(launcher): Set default Firefox prefs (#5149) (#5195)
Add recommended automation preferences to profile
setup when launching Firefox. This profile can be overridden
by using the `userDataDir` launch option, or individual prefs
can be overwritten with the `extraPrefsFirefox` option.

The preferences have been reviewed by peers at Mozilla
over at https://bugzilla.mozilla.org/show_bug.cgi?id=1596888

Co-Authored-By: Mathias Bynens <mathias@qiwi.be>
2019-11-27 07:57:43 +01:00
Mathias Bynens
eddb23b521
chore: update URLs (#5185) 2019-11-26 13:12:25 +01:00
Maja Frydrychowicz
c5a72e9887 feat(launcher): add option to run Puppeteer with different browsers (#5137)
* feat: Set which browser to launch via PUPPETEER_PRODUCT

This change introduces a PUPPETEER_PRODUCT environment
variable as a first step toward using Puppeteer with
many different browsers. Setting PUPPETEER_PRODUCT=firefox, for
example, enables Firefox-specific Launcher settings.

The state is also exposed as `puppeteer.product` in the API
to support adding other product-specific behaviour as needed.

The bulk of the change is a refactoring in Launcher
to decouple generic browser start-up from product-specific
configuration.

Respecting the puppeteer-core restriction for PUPPETEER_
environment variables, lazily instantiate the Launcher
based on a `product` Puppeteer.launch option, if available.

* test: Distinguish Juggler unit tests from Firefox

The funit script is renamed to fjunit (j for Juggler, which is
used only by the experimental puppeteer-firefox package.

In contrast, the funit script now refers to running Puppeteer
unit tests against the main puppeteer package with Firefox.
To do so with Firefox Nightly, run:

`BINARY=path/to/firefox npm run funit`

A number of changes in this patch make it easier to run
Puppeteer unit tests in Mozilla's CI.
2019-11-26 10:23:19 +01:00
theeko
7f3e372b14 docs(api.md): improve page.waitFor docs for the options object (#5021) 2019-10-24 11:22:21 +02:00
Mathias Bynens
1248a19135
feat(api): add page.emulateTimezone(timezoneId) (#4949)
This enables dynamically changing the timezone of the page.
2019-10-23 15:49:39 +02:00
Mathias Bynens
a86363fda6
feat(api): add page.emulateMedia{Type,Features} (#5012) 2019-10-23 13:55:00 +02:00
Mathias Bynens
06dbb72c73
chore: use map instead of plain object (#5068) 2019-10-23 13:12:33 +02:00
Mathias Bynens
81d2600236 feat(chromium): roll Chromium to r705776 (#5058)
This corresponds to Chromium 79.0.3942.0.

This roll includes:

- Support CSS media feature emulation in CDP
  https://chromium-review.googlesource.com/c/chromium/src/+/1821608
- Implement timezone ID verification
  https://chromium-review.googlesource.com/c/chromium/src/+/1822557
- Allow aria-hidden objects into the browser-side ax tree
  https://chromium-review.googlesource.com/c/chromium/src/+/1760862
- Remove pre-BlinkGenPropertyTrees codepaths (affects screenshot clipping)
  https://chromium-review.googlesource.com/c/chromium/src/+/1752244
- Terminate some asynchronous waiting commands on cross-process navigation
  https://chromium-review.googlesource.com/c/chromium/src/+/1766001/21#message-a6a61261b97ffca6ecb180c0a2303b538f7a6476

Per upstream Chromium changes, `page.screenshot` now clips elements to the viewport. This matches the clipping behavior of elements in inner scrollers (i.e., document and overflow scroll clipping now work the same).
2019-10-23 11:41:44 +02:00
Yury Semikhatsky
3773229ac2 fix: avoid unnecessary zero-delays in input code (#4934) 2019-10-21 08:29:56 +02:00
Mathias Bynens
11ff374ca3
chore: drop Node.js v6 support (#5045)
Node.js v6 was end-of-life'd in April, 2019, with AWS Lambda prohibiting updaets to the Node.js v6 runtime since June 30, 2019.

This makes it quite safe for us to remove the Node 6 support from the repository.
2019-10-16 17:00:20 +02:00
2kindsofcs
4c62e09a76 fix(comment): fix typo in Launcher.js (#4959) 2019-10-10 11:54:14 +01:00
Yury Semikhatsky
c2651c2b5c refactor: Pass FrameManager to NetworkManager constructor (#4907) 2019-09-04 18:11:58 -07:00
Pavel Feldman
73fd7ff822 feat(api): add element.select and element.evaluate for consistency (#4892) 2019-09-04 15:19:34 -07:00
Yury Semikhatsky
e0c8d46af1 fix: abort page.waitForRequest/Response when page closes (#4865)
We'd like to pass an abortion signal inside Helper.waitForEvent in order to interrupt it when browser/page closes. Several approaches have been considered:

1. Pass CDPSession instance as a another parameter to the helper method and listen to Disconnected event on it. It would introduce undesired dependency on the session object.
2. Listen to the CDPSession closure at the call sites (e.g. waitForRequest) and pass an abortion promise which would be fulfilled when such event is fired. The listeners would have to be removed from the session on successful completion of waitForEvent so we'd have to pass some kind of DisposablePromise which would be disposed during cleanup. Such parameter looked somewhat hairy.
3. Create DisconnectPromise on CDPSession. One potential risk with that is all chained promises would hang around until the event is fired which might inadvertently cause memory leaks. On the other hand, adding such promise to Promise.race will remove dependency as soon as the race is finished. So this is the approach we're taking with one tweak: the promise is created locally inside Page. 

Ideally the disconnectPromise would throw when the session is closed but it may lead to uncaught promise errors if all chained promises are resolved, to avoid that the promise is resolved with an Error and Helper.waitForEvent throws it later.

Fix #4733
2019-08-21 10:26:48 -07:00
Andrey Lushnikov
b6b29502eb
feat(chromium): roll Chromium to r686378 (#4841)
This roll includes:
- https://crrev.com/685152 - [DevTools] Do not debug worker shadow pages
- https://crrev.com/683391 - [CSP] Check inline script CSP in prepare-a-script
- https://chromium-review.googlesource.com/c/v8/v8/+/1707331 - Ignore returnByValue when serializing caught value in promise rejections.
2019-08-14 21:43:18 -07:00
mittermayr
f47ed1649c feat: update DeviceDescriptors to include 6.5 inch iPhone XR and similar (#4846) 2019-08-14 17:19:33 -07:00
Andrey Lushnikov
0e0a67916d
feat(chromium): roll Chromium to r682225 (#4844)
This roll includes:
- https://crrev.com/681997 - Turn on default SiteInstance by default.

The SiteInstance by default was breaking "devtools: true" option, so
there's a new feature we disable now by default.

This keeps pressuring us towards OOPIF support since that's an
inevitable future.
2019-08-14 15:25:55 -07:00
Darío Kondratiuk
b9b6ca1825 fix(network): be able to remove headers using request.continue (#4797)
I think this was a regression caused here https://github.com/GoogleChrome/puppeteer/pull/4265/files#diff-d2ac7cb061b0c51644d0a5d6140e3a32R446

Fix #4743
2019-08-05 15:26:17 -07:00
ylb
f4f21896d2 fix(page): fix page.screenshot method (#4786)
This patch fixes page.screenshot. so that it throws correct error message when options.clip.height===0
2019-08-01 17:09:39 -07:00
Andrey Lushnikov
ea28cccfe0 feat(page): introduce file chooser interception (#4653)
This patch introduces a page.waitForFileChooser() method
that adds a watchdog to wait for file chooser dialogs.

This lets Puppeteer users to capture file chooser requests
and fulfill/cancel them if necessary.

Fixes #2946
2019-07-22 21:30:49 -07:00
Yury Semikhatsky
2abaac10aa fix(launcher): make dumpio and pipe options work together (#4727)
Don't ignore stdout and stderr when using pipe for remote debugging and dumpio is true. In that case puppeteer process connects to the stdout/stderr streams of the child process and it will not hang.
2019-07-18 16:42:11 -07:00
Andrey Lushnikov
ad1802188d
fix(launcher): disable component extensions (#4704)
Chrome has a set of component extensions - e.g. CryptoTokenExtension
that helps with 2FA.

These extensions are loaded regardless of the `--disable-extensions`
flag we already pass. To disable these extensions, we need to pass additional
`--disable-component-extensions-with-background-pages` flag.

Fix #4300
2019-07-15 17:54:42 -07:00
Andrey Lushnikov
f8616ed54b
fix(pipe): handle 'error' events on pipes to avoid crashing (#4705)
Fix #4374
2019-07-15 17:53:27 -07:00
Andrey Lushnikov
c9977c8004
fix(page): properly create isolated worlds (#4707)
The following error:
```
Error: Protocol error (Page.createIsolatedWorld): No frame for given id found
```

was mentioned here:
https://github.com/FlowCrypt/flowcrypt-browser/pull/1799#issuecomment-508729602

This indeed seems to be a bug with our creation of isolated worlds.
2019-07-15 17:52:40 -07:00
Mohamed Akram
bafda9fbe7 fix(launcher): ensure PUPPETEER_EXECUTABLE_PATH is used if set (#4677)
Without this patch, Puppeteer exits with `Error: Unsupported platform: freebsd` on FreeBSD despite `PUPPETEER_EXECUTABLE_PATH` being set.
2019-07-14 16:37:57 -07:00
Yury Semikhatsky
715aad2d67 fix: proper parse Error message on Node 12 (#4698)
Check message prefix rather than strict equality when detecting circular JSON error. The message format has changed in Node 12 which broke the condition and failed a test.
2019-07-12 17:19:02 -07:00
Joel Einbinder
49b27956fe feat(keyboard): feature phone buttons (#4694)
I got a KaiOS phone with some awesome buttons on it I hadn't seen before on the web. This adds the following buttons to our default keyboard layout:

`SoftLeft`
`SoftRight`
`Camera`
`Call`
`EndCall`
`VolumeUp`
`VolumeDown`
2019-07-12 15:41:01 -07:00
Andrey Lushnikov
955e7cb6fc
feat(page): move page.pdf to protocol streams (#4587)
This lets transferring massive PDF files over the protocol.

Fix #4563
2019-06-14 22:36:06 -07:00
Andrey Lushnikov
62733a20d7
fix(connection): bump websocket max received message size to 256Mb (#4571)
This is the max message size that DevTools can emit over the DevTools
protocol: https://cs.chromium.org/chromium/src/content/browser/devtools/devtools_http_handler.cc?type=cs&q=kSendBufferSizeForDevTools&sq=package:chromium&g=0&l=83

Test is failing on firefox since Firefox crashes when allocating 100Mb string.

Fix #4543
2019-06-14 01:05:09 -07:00
Andrey Lushnikov
6860d73212
fix(network): align HTTP status codes with IANA (#4567)
Aline HTTP status texts with https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml
2019-06-10 21:04:45 -07:00
Takahiro Nishino
792264122c fix(page): fix A2 and A3 format size (#4569)
16.5 inch ~ 419.1mm
16.54 inch ~ 420.1mm

Fix #4568
2019-06-10 20:59:12 -07:00
Andrey Lushnikov
8df0b3ec3b
fix(network): allow fulfilling requests with status code 422 (#4566)
Migration onto fetch domain (2265974ce5)
changed the way we generate status texts: instead of relying on build-in
list of statuses, we tried to rely upon the [Chromium implementation](https://cs.chromium.org/chromium/src/net/http/http_status_code_list.h?sq=package:chromium&g=0).

Puppeteer's list is much more exhaustive - let's keep it.

Fix #4454
2019-06-10 17:39:58 -07:00
Joel Einbinder
b2f94909a1 fix(eval): be able to throw protocol like errors (#4551) 2019-06-10 17:35:40 -07:00
Philip Bergqvist
dd6fcfe77b fix(page): fix missing awaits in mouse.click (#4541)
Lack of await causes dangling promises, which lead to unhandled rejections that cannot be caught if these calls fail (e.g. if the page is closed). 

References #4536
2019-06-10 01:42:19 -07:00
david weil
7faf1c9030 fix(BrowserFetcher): fix httpRequest when using proxy against http HOST (#4558)
This patch avoids https-proxy-agent usage when target host protocol is http and proxy is specified.
This is a valid scenario for installer but was failing.

Fixes #4556
2019-06-10 01:11:01 -07:00
Andrey Lushnikov
90df69cf77
fix(page): do evaluations with one roundtrip (#4539)
This patch teaches page.evaluate to do 1 hop instead of 2 hops.
As a result, things such as `page.select` will not throw an
unfortunate exception when they schedule a navigation.

Fix #4537
2019-06-07 13:46:43 -07:00
Andrey Lushnikov
f79d0a3be8
fix: try running without --disable-gpu on windows (#4523)
It looks like https://crbug.com/737678 and https://crbug.com/729961
are fixed - so we shouldn't need the `--disable-gpu` flag on windows
headless.

Reference #1260
2019-06-05 19:03:09 -07:00
Adriel Codeco
ac611bacac fix(helper): fix Helper.installAsyncStackHooks method (#4478)
Using Error.captureStackTrace instead of new Error () to avoid UnhandledPromiseRejectionWarning
2019-05-31 19:39:32 -07:00
Andrey Lushnikov
76b24e64e8
feat(chromium): roll Chromium to r662092 (#4462)
This roll includes:
- [inspector_protocol:8ec18cf](8ec18cf088) Support STRING16 in the template when converting CBOR map keys
to protocol::Value.
- [inspector_protocol:37518ac](37518ac421) fix parsing of the last ASCII character

This fixes protocol handling of UTF8 in both V8 and Chromium.

Fixes #4443.
2019-05-22 21:21:45 +03:00
Jake Causon
90a1032300 fix(network): stringify response headers for intercepted requests (#4436)
Stringifying the headers was the behaviour before v1.15

References #4379
2019-05-20 11:05:32 +03:00
Sergio M
60249e0bc2 fix: Page.setContent working with unicode strings (#4433)
Fix `page.setContent` with unicode strings that exceeds the range of a 8-bit byte.

This patch implements decoding part of the [MDN's solution #4 for the "unicode problem"](https://developer.mozilla.org/en-US/docs/Web/API/WindowBase64/Base64_encoding_and_decoding#Solution_4_%E2%80%93_escaping_the_string_before_encoding_it).

Since we rely on node.js buffer to encode into base64, we don't have troubles with base64 encoding, so it is left as-is.

Fixes #4424
2019-05-18 17:15:16 +03:00
Joel Einbinder
a3cb16308c feat: root option in page.accessibility.snapshot() (#4318)
Going from `AXNode` -> `ElementHandle` is turning out to be controversial.

This patch instead adds a way to go from `ElementHandle` -> `AXNode`. If the API looks good, I'll add it into Firefox as well.

References #3641
2019-05-09 23:39:42 -07:00
Joel Einbinder
1516e0df21 feat(workers): create workers from service workers and shared workers (#4397)
This allows users to easily evaluate javascript inside service workers and shared workers by creating a Worker object for them.
2019-05-09 17:29:18 -07:00