This patch:
- adds experimental "transport" option to pptr.connect
- uses "transport" option to make sure Puppeteer-Web works with
Target.exposeDevToolsProtocol
Drive-by: add `browser.target()` to access browser target.
This patch introduces API to manage frame navigations.
As a drive-by, the `response.frame()` method is added as a shortcut
for `response.request().frame()`.
Fixes#2918.
This patch traces all public async methods and wraps them
in a helper method that tags the sync stack trace.
Later on, if the method call throws an exception, we add
a captured stack trace to the original stack trace with the "--ASYNC--"
heading.
An example of a stack trace:
```
Error: net::ERR_ABORTED at http://localhost:8907/empty.html
at navigate (/Users/lushnikov/prog/puppeteer/lib/Page.js:622:37)
at process._tickCallback (internal/process/next_tick.js:68:7)
-- ASYNC --
at Page.<anonymous> (/Users/lushnikov/prog/puppeteer/lib/helper.js:147:27)
at fit (/Users/lushnikov/prog/puppeteer/test/page.spec.js:546:18)
at process._tickCallback (internal/process/next_tick.js:68:7)
```
This patch:
- moves implementation of page.goto and page.waitForNavigation
into FrameManager. The defaultNavigationTimeout gets moved to
FrameManager as well.
- moves NavigatorWatcher into FrameManager to avoid circular dependency
References #2918
This patch unifies logic in response trackign in page.goto and
page.waitForNavigation.
As a drive-by, we now make sure that we return the right response
for the right frame. This will come handy for future frame navigation
API.
References #2918
Bundled version of Puppeteer should rely on native WebSocket.
Luckily, 'ws' module supports the same interface as the native
browser websockets. This patch switches WebSocketTransport to
use the browser-compliant interface of 'ws'.
After this patch, I was able to bundle Puppeteer for browser
using the following config in `package.json`:
```json
"browser": {
"./lib/BrowserFetcher.js": false,
"ws": "./lib/BrowserWebSocket",
"fs": false,
"child_process": false,
"rimraf": false,
"readline": false
}
```
where `./lib/BrowserWebSocket` is:
```js
module.exports = WebSocket;
```
and the bundling command is:
```sh
$ browserify -r ./index.js:puppeteer > ppweb.js
```
References #2119
Currently connection assumes that transport is a websocket
and tries to handle websocket-related errors.
This patch:
- moves ConnectionTransport interface to use callbacks instead
of events. This way it could be used in browser context as well.
- introduces WebSocketTransport that implements ConnectionTransport
interface for ws.
This is a preparation step for 2 things:
- exposing `transport` option in the `puppeteer.connect` method
- better support for `browserify`
References #2119
This patch removes all dynamic requires in Puppeteer. This should
make it much simpler to bundle puppeteer/puppeteer-core packages.
We used dynamic requires in a few places in lib/:
- BrowserFetcher was choosing between `http` and `https` based on some
runtime value. This was easy to fix with explicit `require`.
- BrowserFetcher and Launcher needed to know project root to store
chromium revisions and to read package name and chromium revision from
package.json. (projectRoot value would be different in node6).
Instead of doing a backwards logic to infer these
variables, we now pass them directly from `//index.js`.
With this patch, I was able to bundle Puppeteer using browserify and
the following config in `package.json`:
```json
"browser": {
"./lib/BrowserFetcher.js": false,
"ws": "./lib/BrowserWebSocket",
"fs": false,
"child_process": false,
"rimraf": false,
"readline": false
}
```
(where `lib/BrowserWebSocket.js` is a courtesy of @Janpot from
https://github.com/GoogleChrome/puppeteer/pull/2374/)
And command:
```sh
$ browserify -r puppeteer:./index.js > ppweb.js
```
References #2119
We had (and still have) a ton of pull requests to support
PUPPETEER_EXECUTABLE_PATH and PUPPETEER_CHROMIUM_REVISION in puppeteer launcher.
We were hesitant before since env variables are not scoped
and thus don't make a good interface for a library. Now, since we
determined `puppeteer-core` as a library and `puppeteer` as our end-user
product, it's safe to satisfy our user needs.
This patch:
- teaches PUPPETEER_EXECUTABLE_PATH and PUPPETEER_CHROMIUM_REVISION
env variables to control how Puppeteer launches browser
- makes sure these variables play no role in `puppeteer-core` package.
Before v1.7.0 we were creating Response objects either from protocol's
Network.Response struct, or from the data available in the
requestIntercepted.
With the recent chagens to the request interception logic, we can
always create response from Newtork.Response struct; this allows
us to simply create Response objects from protocol payload.
If referer is passed to the options object its value will be used as the referer instead of the value set by `Page.setExtraHTTPHeaders()`.
This is the correct way to set referer header: otherwise, the `referer` header will override all the document subrequests.
Fixes#3090.
Introduce an API to manage permissions per browser context:
- BrowserContext.overridePermissions(origin, permissions)
- BrowserContext.clearPermissionOverrides()
Fixes#846.
I ran TypeScript against our code with `strictNullChecks` on. Most of the errors generated are noise, because TypeScript doesn't understand how our `assert` method works. But some were legitimate bugs. They are fixed in this patch.
This patch:
- merges `ElementHandle` into `ExecutionContext` (for simplicity; there's no good reason to have them in separate files).
- removes the necessity to pass handle factory when creating `ExecutionContext`
This makes it easier to create execution contexts out of payloads.
References #1215
It turned out that almost any usecase requires helper methods to access
DOM inside the ExecutionContext.
Instead of exposing execution contexts as-is, we should introduce
IsolatedWorld as a first-class citizen that will hold execution contexts
inside.
This patch adds a new require, `puppeteer/Errors`, that
holds all the Puppeteer-specific error classes.
Currently, the only custom error class we use is `TimeoutError`. We'll
expand in future with `CrashError` and some others.
Fixes#1694.
When an evaluation causes a navigation, for example:
```js
await page.evaluate(() => window.reload());
```
sometimes we process the ExecutionContextDestroyed event before the ack from the evaluate. When we do get the ack from the evaluate, we try to build a JSHandle for it, and try to find the execution by id. But it is gone, and we throw an error. This patch switches createJSHandle to accept an ExecutionContext instead of just an id.
This bug was making the test `should throw a nice error after a navigation` flaky.
In accordance with its declared type, `Map<string, Function>`.
Currently, it is used as a plain old JS object. The compiler marks this
usage as an error when `noImplicitAny: true`. This change switches to
use the appropriate Map methods `has/get/set`.
Fixes#3000
This changes sendCharacter to use document.execCommand instead of sending a `'char'` event from the protocol. This is more aligned with how input would come in from emoji keyboards, and removes the 3ish byte limit on characters that can be sent which prevented larger emoji from being rendered correctly.
Emoji will still fail to type correctly if typing them into an iframe that is in shadow dom.
fixes#1096
This patch rolls Chromium to r579032. The patch includes:
- https://crrev.com/577366 - DevTools: report redirect responses only if response interception is enabled
- https://crrev.com/577212 - DevTools: intercept requests resulting from redirects
- https://crrev.com/578934 - DevTools: Add a protocol method to insertText
Interception Logic in DevTools protocol has changed regarding redirects;
this patch migrates interceptions to dispatch "request" events based on
requestWillBeSent event.
I have seen some flaky test failures where it would be nice to have run the tests with `DEBUG=puppeteer:error`. Instead of always running tests like that, I am redirecting `debugError` to the output category of the test. This is the same thing that we do for Chromium's stderr.
As a drive-by, I added an additional `debugError` where we were usually a try..finally pattern.
When process is spawned with the 'pipe' set for stdout and stderr, Node expects these streams to be actually dispatched. If we don't, write into pipe becomes blocking and chrome stalls.
We work around that via setting 'ignore' to the stdin, stdout and stderr when the pipe communication channel is used.
Chrome Headless used to open about:blank by default; however, this
was recently changed.
We should open starting page no matter what to keep the environment
predictable.
Unfortunately, disabling javascript in page prevents any microtasks
to be executed even from puppeteer-originating javascript. As a
result, the IntersectionObserver hack we use to conditionally
scroll into view doesn't work.
To workaround this, we start always scrolling before clicking if
page's javascript is disabled.
Fixes#2898
This roll includes:
- https://crrev.com/574785 - DevTools: allow tracing over the remote debugging pipe.
This fixes tracing over the remote debugging pipe.
Chrome DevTools shows anonymous scripts with yellow background and names
them with `debugger://VM<scriptId>` prefix.
This patch starts reporting the same debugger:// urls for anonymous
scripts in puppeteer's JS coverage. This might simplify debugging, e.g.
using `debugger;` statement to reveal the script in DevTools and later
matching it against the one in the coverage.
This patch:
- simplifies test reusing the `offscreenbuttons.html` asset
- aligns IntersectionObserver usage with the one we have for
`ElementHandle._scrollIntoViewIfNeeded`.
