Currently, the NavigatorWatcher ensures that all the frames in the
frame subtree reach the desired lifecycle events state.
For this to be fully correct, NavigatorWatcher should also check
navigation status whenever frames are detached.
In Blink, frames don't necesserily have execution context all the time.
DevTools Protocol precisely reports this situation, which results in
Puppeteer's frame.executionContext() being null occasionally.
However, from puppeteer point of view every frame will have at least a
default executions context, sooner or later:
- frame's execution context might be created naturally to run frame's
javascript
- if frame has no javascript, devtools protocol will issue execution
context creation
This patch builds up on this assumption and makes frame.executionContext()
to be a promise.
As a result, all the evaluations await for the execution context to be created first.
Fixes#827, #1325
BREAKING CHANGE: this patch changes frame.executionContext() method to return a promise.
To migrate onto a new behavior, await the context first before using it.
The SIGHUP signal is sent whenever the controlling terminal is closed.
On Windows, SIGHUP is emulated by libuv, and will be the only signal we
receive before the application will be terminated.
This patch starts handling SIGHUP in the same way we handle SIGTERM.
Fixes#1367.
SIGTERM signal is widely used to notify application that it will be shut down.
This patch starts listening to SIGTERM event to gracefully retire
chromium instance.
References #1047.
Currently, `elementHandle.screenshot` passes bounding box into the `page.captureScreenshot`
method.
This is wrong since `captureScreenshot` accepts viewport, not bounding box.
This patch uses layout metrics to convert bounding box into viewport.
Fixes#1315.
Currently, we wait only for the main frame to reach the desired
lifecycle state.
This patch starts waiting until all the frames reach the desired
lifecycle state.
Fixes#1173.
This patch:
- starts persisting lifecycle state for every frame
- migrates NavigationWatcher to rely on these lifecycle events
- refactors Page.goto to properly return navigation errors
Fixes#1218.
This patch starts asserting that all values are of type "string".
The alternative approach to cast values to strings
might yield a hard-to-debug errors.
Fixes#1276.
This roll brings in a bunch of important patches:
- crrev.com/512647 Changed headless browser profile dir to use Default profile path
- crrev.com/512760 DevTools: stop idleness detector when pending navigation commits
- crrev.com/512905 DevTools: introduce Page.getFrameTree
- crrev.com/513373 DevTools: report loaderId in the lifecycle events
- crrev.com/513419 DevTools: introduce Page.setLifecycleEventsEnabled
- crrev.com/513422 DevTools: return loaderId from Page.navigate
Fixes#921
BREAKING CHANGE:
Headless user profile structure is changing. Custom profiles set with --user-data-dir flag will no longer be read in Chrome 63 and will have to be recreated.
Alternatively, you can migrate old headless profile to a new structure. if you stored your profile in `<profile>` folder, you would run the following bash commands:
```bash
cd <profile>
mkdir Default
mv * Default
```
Full headless-dev PSA announcement: https://groups.google.com/a/chromium.org/forum/#!msg/headless-dev/asX8WgktXIE/zTUfmHDcAQAJ
This patch adds `Frame.select` method that does the same functionality as
former `Page.select`, but on a per-frame level.
The `Page.select` method becomes a shortcut for the ÷main frame's select.
Fixes#1139
This patch migrates puppeteer to support PlzNavigate chromium
project.
As a consequence of this patch, we no longer wait for both
requestWillBeSent and requestIntercepted events to happen. This should
resolve a ton of request interception bugs that "hanged" the loading.
Fixes#877.
Currently, NavigationWatcher listens to lifecycle events from Page
domain and security events from Security domain.
However, the events are dispatched from different processes in browser:
- Page's lifecycle events are dispatched from renderer process
- Security events are dispatched from browser process
This makes for the undefined order between events and results in
NavigationWatcher reporting different failuer messages, based on
the event order.
This patch stops relying on security errors in navigation watcher and
instead switches to request failure codes for the main resource.
Fixes#1195
Elements in shadow dom erroneously considered that they were detached
from document.
This patch starts using `Element.isConnected` instead of
`document.contains()` call.
Fixes#1061.
This roll includes:
- crrev.com/510651 that changes request interception methods in protocol
- s/Page.setRequestInterceptionEnabled/Page.setRequestInterception
BREAKING CHANGE
Page.setRequestInterceptionEnabled is renamed into
Page.setRequestInterception.
This patch adds "options" parameter to the `page.setContent` method. The
parameter is the same as a navigation parameter and allows to specify
maximum timeout to wait for resources to be loaded, as well as to
describe events that should be emitted before the setContent operation
would be considered successful.
Fixes#728.
This patch adds support to multiple events that could be passed inside
navigation methods:
- Page.goto
- Page.waitForNavigation
- Page.goForward
- Page.goBack
- Page.reload
Fixes#805
This patch adds a new `domcontentloaded` option to a bunch of navigation
methods:
- Page.goto
- Page.waitForNavigation
- Page.goBack
- Page.goForward
- Page.reload
Fixes#946.
This patch:
- migrates navigation watcher to use protocol-issued lifecycle events.
- removes `networkIdleTimeout` and `networkIdleInflight` options for
`page.goto` method
- adds a new `networkidle0` value to the waitUntil option of navigation
methods
References #728.
BREAKING CHANGE:
As an implication of this new approach, the `networkIdleTimeout` and
`networkIdleInflight` options are no longer supported. Interested
clients should implement the behavior themselves using the `request` and
`response` events.
BREAKING CHANGE:
This patch lets key names be code in addition to key. When specifying a code, the proper text is generated assuming a standard US keyboard layout. e.g Digit5 -> "5" or "%" depending on Shift.
* location is now specified. #777
* Using unknown key names now throws an error. #723
* Typing newlines now correctly presses enter. #681
feat(interception): Implement request.respond method
This patch implements a new Request.respond method. This
allows users to fulfill the intercepted request with a hand-crafted
response if they wish so.
References #1020.
Currently, JSHandle.jsonValue() is implemented as in-page JSON.stringify
call and consequent JSON.parse in node. This approach proved to be
unfortunate for automation purposes: if page author overrode the
Object.prototype.toJSON method, then it's harder for puppeteer to
interact with the page.
This patch switches JSHandle.jsonValue to use protocol serialization
that ignores toJSON property. THis also changes the `page.evaluate`
behavior since it is based on JSHandle.jsonValue().
Fixes#1003.
BREAKING CHANGE:
`page.evaluate` no longer calls toJSON when generating return value.
For the old behavior, do JSON.parse/JSON.stringify manually:
```js
const json = JSON.parse(await page.evaluate(() => JSON.stringify(obj)));
```
This patch:
- introduces Target class that represents any inspectable target, such as service worker or page
- emits events when targets come and go
- introduces target.page() to instantiate a page from a target
Fixes#386, fixes#443.
Similarly to the `request.response()` method, this patch adds
`request.failure()` method that returns error details for the failed
requests.
Fixes#901.
This patch:
- changes `browser.close` to terminate browser.
- introduces new `browser.disconnect` to disconnect from a browser without closing it
This patch: fixes#918, fixes#989
BREAKING CHANGE:
`browser.close()` will always close a browser, even if it was initialized with
`puppeteer.connect`. To disconnect from a remote browser, use `browser.disconnect()` instead.
This patch starts generating input events for `keyboard.down`, addressing bullet 2 of
#723. With this patch, there's no longer any difference between `keboard.press('a')` and
`keyboard.press('a', {text: 'a'})`.
BREAKING CHANGE:
`keyboard.down('a')` starts generating input event (wasn't the case before).
References #723
This patch:
- deprecates injectFile as it was confused with the addScriptTag
- accepts an options object in addScriptTag which supports properties url, path and content.
- accepts an options object in addStyleTag which supports properties url, path and content.
Fixes#949.
BREAKING CHANGE:
- the addStyleTag/addScriptTag have changed;
- the injectFile was removed in favor of (addStyleTag({path:}).
This patch introduces `Page.queryObjects` and
`ExecutionContext.queryObjects` methods to query JavaScript heap
for objects with a certain prototype.
Fixes#304.
This patch:
- introduces `helper.promisify` - a simple polyfill for the `util.promisify`. The
`util.promisify` could not be used due to Node6 compatibility issues.
- migrates all sync filesystem operations to the async replicas
Fixes#884.
The page.plainText is confusing: it's unclear what kind of text it
returns, textContent or innerText. It's also easily polyfillable and
doesn't seem to be used.
BREAKING CHANGE: the page.plainText is not existing any more.
Instead, use `page.evaluate(() => document.body.innerText)`.
This patch:
- updates JSHandle.toString to make a nicer description for primitives
- excludes JSHandle.toString from documentation to avoid its abuse
References #382
This patch moves resourceType to be all small-caps. This aligns
with our convention that all string constants should be smallcaps.
BREAKING CHANGE: this patch changes the constants of the
request.resourceType to be all small-caps.
This patch:
- adds `ElementHandle.boundingBox()` method to get bounding box of element relative
to the page
- adds `ElementHandle.screenshot()` method to capture a screenshot of an element
This patch starts using typescript to lint JSDoc annotations.
Note: this uses typescript's bleeding edge. We should migrate to stable once
it has all the necessary bugfixes.
References #65.
This patch:
- adds input methods to ElementHandle, such as ElementHandle.type and ElementHandle.press
- changes `page.type` to accept selector as the first argument
- removes `page.press` method. The `page.press` is rarely used and doesn't operate with selectors; if there's a need to press a button, `page.keyboard.press` should be used.
BREAKING CHANGE: `page.type` is changed, `page.press` is removed.
Fixes#241.
This patch:
- introduces ExecutionContext class that incapsulates javascript
execution context. An examples of execution contexts are workers and
frames
- introduces JSHandle that holds a references to the javascript
object in ExecutionContext
- inherits ElementHandle from JSHandle
Fixes#382.
This patch allows passing 0 to disable timeout for the following methods:
- page.goto
- page.waitForNavigation
- page.goForward
- page.goBack
Fixes#782.
This patch starts using taskkill program on windows to gracefully
terminate chrome.
Note: this slows down chrome shutdown on Windows in case of using
custom userDataDir. This is because chrome takes some time to shutdown
its operations and leave profile directory in a consistent state.
Fixes#839.
This patch introduces ConsoleMessage type and starts dispatching
it for the 'console' event.
BREAKING CHANGE: this breaks the api of the 'console' event.
Fixes#744.
This patch rolls chromium to r503964.
Note: since the plznavigate is not supported by puppeteer right now, the
patch also starts passing the `--disable-browser-side-navigation` flag.
This is a temporary work around for us.
References #877.
This lets the user pass `...args` into `page.waitFor`. It also clarifies that the docs that `options` is not optional if `...args` are specified.
Fixes#770
Since protocol ignores all HTTP headers that don't have string
value, this patch starts validating header key-values before
sending them over the protocol.
Fixes#713.
This patch:
- makes `browser.close()` return a promise that resolves when browser gets closed
- starts closing chrome gracefully if a custom `userDataDir` is supplied
Fixes#527
This patch:
- teaches `page.evaluate` to accept ElementHandles as parameters
- removes `ElementHandle.evaluate` method since it's not needed any
more
References #382
It turns out that [undefined, 1].join(',') results in ",1" instead
of "undefined,1". This causes a syntax error when trying to pass undefined
as a first argument to `page.evaluate` method.
Fixes#572.
Currently, navigation watcher throws exception if timeout
is exceeded.
Due to the way it is used in `page.navigate`, the promise
get's rejected before it is awaited, which is considered to
be "unhandled promise rejection".
Fixes#738
It's very bad to have 'unhandled promise rejection' that can't be
handled in user code. These errors will exit node process in a near
future.
This patch avoids 'unhandled promise rejection' while sending protocol
messages.
This patch:
- introduces `puppeteer:error` debug scope and starts using it for all
swalloed errors.
- makes sure that every `client.send` method is either awaited or its
errors are handled.
- starts return promises from Request.continue() and Request.abort().
- starts swallow errors from Request.contine() and Request.abort().
The last is the most important part of the patch. Since
`Request.continue()` might try to continue canceled request, we should
disregard the error.
Fixes#627.
This patch:
- adds `page.touchscreen` namespace, similar to `page.mouse` and `page.keyboard`.
- adds tapping to multiple layers:
- `page.touchscreen.tap`
- `page.tap` - convenience method which accepts selector
- `elementHandle.tap`
Fixes#568 and #569.
This patch:
- switches to objects instead of maps for headers (in Request, Response and
page.setExtraHTTPHeaders)
- converts all header names to lower case
Fixes#547, fixes#509
It turned out that either Network.requestIntercepted or
Network.requestWillBeSent occasionally report encoded URL.
This patch starts decoding URL's when generating request hash.
Fixes#558.
This patch rolls chromium to r496140. This includes the r496130 that
introduces multiple sessions for single target.
With this patch, it is possible to run puppeteer in headful mode
and open devtools over the automated pages without puppeteer losing
connection to the page.
This patch:
- adds a 'timeout' launcher option that constrains the time for chromium to launch.
- adds a 'handleSIGINT' launcher option that is `true` by default and that closes chrome instance
Fixes#363.
Fail gracefully when chromium failed to download
This patch changes both install.js and Launcher.js to inform how
chromium could be downloaded manually.
This patch:
- removes the `page.uploadFile` method
- adds `elementHandle.uploadFile` method.
Motivation: `elementHandle.uploadFile` is rarely used, so it doesn't worth it
to keep it on page.
This patch:
- rolls chromium to r494365
- starts using Runtime.evaluate(awaitPromise: true), with new semantic
we can avoid additional Runtime.awaitPromise call
- stops resolving promises for Console event
This patch:
- refactors Connection to use a single remote debugging URL instead of a
pair of port and browserTargetId
- introduces Puppeteer.connect() method to attach to already running
browser instance.
Fixes#238.
This patch:
- teaches request interception to ignore data URLs. Currently protocol
doesn't send interceptions for data URLs.
- teaches request interception to properly process URLs with hashes.
Currently `Network.requestIntercepted` sends url with a hash, whereas
`Network.requestWillBeSent` doesn't report hashes in its urls. @see
crbug.com/755456
- skips one more header that I spotted during debugging interception on
the realworld websites.
Fixes#258, #259.
This patch starts emitting 'error' event when page crashes.
'error' events have special treatment in node, so page crashes
become observable for users.
Fixes#262.
This patch:
- fixes multimap implementation to work properly in node
- moves ESTreeWalker from third-party into utils/doclint. ESTreeWalker
license is compliant with Apache2.0.
This patch:
- split browser launching logic from Browser into `lib/Launcher.js`
- introduce `puppeteer` namespace which currently has a single `launch`
method to start a browser
With this patch, the browser is no longer created with the `new
Browser(..)` command. Instead, it should be "launched" via the
`puppeteer.launch` method:
```js
const puppeteer = require('puppeteer');
puppeteer.launch().then(async browser => {
...
});
```
With this approach browser instance lifetime matches the lifetime of
actual browser process. This helps us:
- remove proxy streams, e.g. browser.stderr and browser.stdout
- cleanup browser class and make it possible to connect to remote
browser
- introduce events on the browser instance, e.g. 'page' event. In case
of lazy-launching browser, we should've launch browser when an event
listener is added, which is unneded comlpexity.
Mouse events are no longer racy. Enabling touch no longer converts all mouse events into touches. Promises in destroyed execution contexts are rejected immediately.
The issue #168 is a protocol inconsistency which happens only
in case of HTTPS error. This patch starts refering to the
upstream bug instead of puppeteer issue.
Closes#168.
This patch
- rolls chromium to 492629
- migrates connection establishing to use browser target. This migration means
that now we have a single websocket connection to browser (implemented
in Connection class). A connection to a particular target is
incapsulated in a new Session class.
This patch:
- changes interception API so that it better aligns with what we'd like to see
in #121
- fixes the issue with redirect interception
Fixes#217.
This patch starts using "Failed" command for request interception instead of
"Aborted".
The "Aborted" status also has a side-effect of cancelling the navigation, so
there will be no error on the page and form puppeteer's standpoint, the navigation
will never complete.
The `DEBUG=*page npm run unit` is too verbose due to events spamming
the console.
This patch starts tracing emitted events only if there are any
listeners.
This patch implements 'autoRepeat' functionality for `keyboard.down`.
With this patch, the subsequent calls to `keyboard.down` would generate
an event with 'autoRepeat` flag set to true.
Closes#157
This patch makes sure that request.text() doesn't try
to fetch response body from the backend until the request is
actually finished (finished or failed).
It turns out we're not receiving 'Network.requestWillBeSent' event
for every requestId.
This patch makes sure we don't dispatch `requestfinished` and
`requestfailed` events without passing actual request.
References #168
VolumeUp, VolumeDown, and VolumeMute were changed to AudioVolumeUp, AudioVolumeDown, and AudioVolumeMute
The media keys like MediaTrackNext were also missing, so I added them.
This patch:
- teaches page.uploadFile() to resolve given file paths against
current working directory. This aligns paths handling with all the
other methods
- moves page.uploadFile() under Frame
- changes test to use relative path for file upload
The Body class was inlined in the Request and Response classes.
This patch:
- removes the Body class
- adds Request.postData public property
- adds Response.buffer(), Response.text() and Response.json() methods
Fixes#106.
The page.waitForFunction method allows to wait for a general predicate.
The predicate will be continiously polled for in page, until
it either returns true or the timeout happens.
The polling parameter could be one of the following:
- 'raf' - to poll on every animation frame
- 'mutation' - to poll on every dom mutation
- <number> - to poll every X milliseconds
References #91
This patch:
- implements a basic public API coverage based on 'helper.tracePublicAPI' methods
- adds `npm run coverage` command which reports coverage after running all of the unit tests
References #50.
This patch:
- removes Body.arrayBuffer. This method is redundant since there's
already a Body.buffer() method
- removes Body.bodyUsed getter.
References #106
This patch:
- renames page.setHTTPHeaders into page.setExtraHTTPHeaders
- starts using Map instead of Object to align with other headers
arguments
Fixes#112.
This patch improves on page.evaluate to accept a string.
The string can have a trailing '//# sourceURL=' comment which would
name the evaluation to make stacks beautiful.
In order to make sourceURL comments possible, this patch:
- removes wrapping of the client function into `Promise.resolve()`
- stops passing `awaitPromise` parameter to `Runtime.evaluate`
- starts to await promise via the `Runtime.awaitPromise` if the return type of the evaluation
is promise
closes#118
This patch removes Header class and substitutes it with a simple
Map object.
The map is chosen over the vanilla object since it has explicit
order of headers which we'd like to preserve.
References #106.
This patchs makes sure DOM domain is never enabled. The reasoning behind this is that
DOM domain is heavy-weight: it generates a lot of traffic over the protocol.
Drive-by: use Page.addScriptToEvaluateOnNewDocument instead of Page.addScriptToEvaluateOnLoad
This patch starts waiting for double raf after every mouse event.
It looks like a good enough workaround to make sure mouse events
are delivered.
The single raf might not be enough because browser might 'return' an already-prepared
frame for us without actually dispatching events.
Closes#122
This patch renames page.evaluateOnInitialized into
page.evaluateOnNewDocument to better align with the protocol and with
what the method is actually doing.
Fixes#119.
This patch removes the InterceptedRequest.isHandled method. There's
always a single request interceptor who owns all the intercepted
requests and who know if they were handled.
References #115.
This patch stops serializing console API arguments unless there are
listeners of the 'console' event in puppeteer.
This saves quite a lot CPU cycles.
Fixes#117.
Refactor Frame.waitForSelector to make room for Frame.waitForFunction
implementation.
This patch:
- removes AwaitedElement class which proved to be confusing, and
introduces a more straight-forward WaitTask.
- refactors the mutation observer to return true in case of successful
waiting or false in case of timeout.
References #91
This patch:
- introduces helper.addEventListener/helper.removeEventListeners
to simplify event management
- moves NavigatorWatchdog over to the helper.addEventListener to
stop leaking event listeners
This patch:
- adds Mouse class which holds mouse state and implements mouse primitives,
such as moving, button down and button up.
- implements high-level mouse api, such as `page.click` and `page.hover`.
References #40, References #89
This patch:
- introduces page.waitForSelector to wait for the selector to appear
- introduces polymorphic page.waitFor method, which accepts
either string (and in this case is a shortcut for page.waitForSelector)
or number (and in this case it's a promisified timeout).
References #91.
This patch implements timeout option for page.waitFor. The function
will throw if the selector doesn't appear during timeout milliseconds
of waittime.
References #89, #91.
This patch adds a 'visible' option to the Page.waitFor method, making
it possible to wait for the element to become actually visible.
References #89, #91.
This patch:
- gets rid of `page.emulate` and `page.emulatedDevices`
methods. Instead, it is suggested to use `page.setViewport()`
and `page.setUserAgent()` methods.
- moves DeviceDescriptors to the top level of the puppeteer so that
it is convenient to require them.
- improves on documentation to describe the suggested emulation
approach.
References #88.
This patch converts lib/DevicesDescriptors from a devtools front-end
format into a puppeteer format.
This patch does this via introducing a scripts utils/fetch_devices.js
which grabs devices from upstream of DevTools Front-end and
converts them into puppeteer devices.
References #88.
Currently, it's impossible to do screenshots in parallel.
This patch:
- makes all screenshot tasks sequential inside one browser
- starts activating target before taking screenshot
- adds a test to make sure it's possible to take screenshots across
tabs
- starts waiting for a proper page closing after each test. This might
finally solve the ECONNRESET issues in tests.
References #89
This patch:
- resolves pending callbacks in connection once it gets closed
- fixes one of Page.screenshot() tests to wait for all the screenshots
- starts handling ECONNRESET error in test server to avoid throwing with no
good reason
This patch introduces Page.waitForNavigation which allows to wait
for render-initiated navigation.
This patch also does a nice refactoring, replacing Navigator with NavigatorWatcher which
is not a part of a page state.
References #89
This patch re-introduces the DEBUG module to expose some of
the puppeteer's internals.
Currently, only the protocol message communication is exposed under
the 'puppeteer:protocol' namespace.
This patch:
- renames Page.printToPDF into page.pdf
- adds a 'path' option to the page.pdf options instead of a separate
`filePath` parameter
- improves on the documentation for the `page.pdf`
References #39.
