This patch:
- switches to objects instead of maps for headers (in Request, Response and
page.setExtraHTTPHeaders)
- converts all header names to lower case
Fixes#547, fixes#509
It turned out that either Network.requestIntercepted or
Network.requestWillBeSent occasionally report encoded URL.
This patch starts decoding URL's when generating request hash.
Fixes#558.
This patch rolls chromium to r496140. This includes the r496130 that
introduces multiple sessions for single target.
With this patch, it is possible to run puppeteer in headful mode
and open devtools over the automated pages without puppeteer losing
connection to the page.
This patch:
- adds a 'timeout' launcher option that constrains the time for chromium to launch.
- adds a 'handleSIGINT' launcher option that is `true` by default and that closes chrome instance
Fixes#363.
Fail gracefully when chromium failed to download
This patch changes both install.js and Launcher.js to inform how
chromium could be downloaded manually.
This patch:
- removes the `page.uploadFile` method
- adds `elementHandle.uploadFile` method.
Motivation: `elementHandle.uploadFile` is rarely used, so it doesn't worth it
to keep it on page.
This patch:
- rolls chromium to r494365
- starts using Runtime.evaluate(awaitPromise: true), with new semantic
we can avoid additional Runtime.awaitPromise call
- stops resolving promises for Console event
This patch:
- refactors Connection to use a single remote debugging URL instead of a
pair of port and browserTargetId
- introduces Puppeteer.connect() method to attach to already running
browser instance.
Fixes#238.
This patch:
- teaches request interception to ignore data URLs. Currently protocol
doesn't send interceptions for data URLs.
- teaches request interception to properly process URLs with hashes.
Currently `Network.requestIntercepted` sends url with a hash, whereas
`Network.requestWillBeSent` doesn't report hashes in its urls. @see
crbug.com/755456
- skips one more header that I spotted during debugging interception on
the realworld websites.
Fixes#258, #259.
This patch starts emitting 'error' event when page crashes.
'error' events have special treatment in node, so page crashes
become observable for users.
Fixes#262.
This patch:
- fixes multimap implementation to work properly in node
- moves ESTreeWalker from third-party into utils/doclint. ESTreeWalker
license is compliant with Apache2.0.
This patch:
- split browser launching logic from Browser into `lib/Launcher.js`
- introduce `puppeteer` namespace which currently has a single `launch`
method to start a browser
With this patch, the browser is no longer created with the `new
Browser(..)` command. Instead, it should be "launched" via the
`puppeteer.launch` method:
```js
const puppeteer = require('puppeteer');
puppeteer.launch().then(async browser => {
...
});
```
With this approach browser instance lifetime matches the lifetime of
actual browser process. This helps us:
- remove proxy streams, e.g. browser.stderr and browser.stdout
- cleanup browser class and make it possible to connect to remote
browser
- introduce events on the browser instance, e.g. 'page' event. In case
of lazy-launching browser, we should've launch browser when an event
listener is added, which is unneded comlpexity.
Mouse events are no longer racy. Enabling touch no longer converts all mouse events into touches. Promises in destroyed execution contexts are rejected immediately.
The issue #168 is a protocol inconsistency which happens only
in case of HTTPS error. This patch starts refering to the
upstream bug instead of puppeteer issue.
Closes#168.
This patch
- rolls chromium to 492629
- migrates connection establishing to use browser target. This migration means
that now we have a single websocket connection to browser (implemented
in Connection class). A connection to a particular target is
incapsulated in a new Session class.
This patch:
- changes interception API so that it better aligns with what we'd like to see
in #121
- fixes the issue with redirect interception
Fixes#217.
This patch starts using "Failed" command for request interception instead of
"Aborted".
The "Aborted" status also has a side-effect of cancelling the navigation, so
there will be no error on the page and form puppeteer's standpoint, the navigation
will never complete.
The `DEBUG=*page npm run unit` is too verbose due to events spamming
the console.
This patch starts tracing emitted events only if there are any
listeners.
This patch implements 'autoRepeat' functionality for `keyboard.down`.
With this patch, the subsequent calls to `keyboard.down` would generate
an event with 'autoRepeat` flag set to true.
Closes#157
This patch makes sure that request.text() doesn't try
to fetch response body from the backend until the request is
actually finished (finished or failed).
It turns out we're not receiving 'Network.requestWillBeSent' event
for every requestId.
This patch makes sure we don't dispatch `requestfinished` and
`requestfailed` events without passing actual request.
References #168
VolumeUp, VolumeDown, and VolumeMute were changed to AudioVolumeUp, AudioVolumeDown, and AudioVolumeMute
The media keys like MediaTrackNext were also missing, so I added them.
This patch:
- teaches page.uploadFile() to resolve given file paths against
current working directory. This aligns paths handling with all the
other methods
- moves page.uploadFile() under Frame
- changes test to use relative path for file upload
The Body class was inlined in the Request and Response classes.
This patch:
- removes the Body class
- adds Request.postData public property
- adds Response.buffer(), Response.text() and Response.json() methods
Fixes#106.
The page.waitForFunction method allows to wait for a general predicate.
The predicate will be continiously polled for in page, until
it either returns true or the timeout happens.
The polling parameter could be one of the following:
- 'raf' - to poll on every animation frame
- 'mutation' - to poll on every dom mutation
- <number> - to poll every X milliseconds
References #91
This patch:
- implements a basic public API coverage based on 'helper.tracePublicAPI' methods
- adds `npm run coverage` command which reports coverage after running all of the unit tests
References #50.
This patch:
- removes Body.arrayBuffer. This method is redundant since there's
already a Body.buffer() method
- removes Body.bodyUsed getter.
References #106
This patch:
- renames page.setHTTPHeaders into page.setExtraHTTPHeaders
- starts using Map instead of Object to align with other headers
arguments
Fixes#112.
This patch improves on page.evaluate to accept a string.
The string can have a trailing '//# sourceURL=' comment which would
name the evaluation to make stacks beautiful.
In order to make sourceURL comments possible, this patch:
- removes wrapping of the client function into `Promise.resolve()`
- stops passing `awaitPromise` parameter to `Runtime.evaluate`
- starts to await promise via the `Runtime.awaitPromise` if the return type of the evaluation
is promise
closes#118
This patch removes Header class and substitutes it with a simple
Map object.
The map is chosen over the vanilla object since it has explicit
order of headers which we'd like to preserve.
References #106.
This patchs makes sure DOM domain is never enabled. The reasoning behind this is that
DOM domain is heavy-weight: it generates a lot of traffic over the protocol.
Drive-by: use Page.addScriptToEvaluateOnNewDocument instead of Page.addScriptToEvaluateOnLoad
This patch starts waiting for double raf after every mouse event.
It looks like a good enough workaround to make sure mouse events
are delivered.
The single raf might not be enough because browser might 'return' an already-prepared
frame for us without actually dispatching events.
Closes#122
This patch renames page.evaluateOnInitialized into
page.evaluateOnNewDocument to better align with the protocol and with
what the method is actually doing.
Fixes#119.
This patch removes the InterceptedRequest.isHandled method. There's
always a single request interceptor who owns all the intercepted
requests and who know if they were handled.
References #115.
This patch stops serializing console API arguments unless there are
listeners of the 'console' event in puppeteer.
This saves quite a lot CPU cycles.
Fixes#117.
Refactor Frame.waitForSelector to make room for Frame.waitForFunction
implementation.
This patch:
- removes AwaitedElement class which proved to be confusing, and
introduces a more straight-forward WaitTask.
- refactors the mutation observer to return true in case of successful
waiting or false in case of timeout.
References #91
This patch:
- introduces helper.addEventListener/helper.removeEventListeners
to simplify event management
- moves NavigatorWatchdog over to the helper.addEventListener to
stop leaking event listeners
This patch:
- adds Mouse class which holds mouse state and implements mouse primitives,
such as moving, button down and button up.
- implements high-level mouse api, such as `page.click` and `page.hover`.
References #40, References #89
This patch:
- introduces page.waitForSelector to wait for the selector to appear
- introduces polymorphic page.waitFor method, which accepts
either string (and in this case is a shortcut for page.waitForSelector)
or number (and in this case it's a promisified timeout).
References #91.
This patch implements timeout option for page.waitFor. The function
will throw if the selector doesn't appear during timeout milliseconds
of waittime.
References #89, #91.
This patch adds a 'visible' option to the Page.waitFor method, making
it possible to wait for the element to become actually visible.
References #89, #91.
This patch:
- gets rid of `page.emulate` and `page.emulatedDevices`
methods. Instead, it is suggested to use `page.setViewport()`
and `page.setUserAgent()` methods.
- moves DeviceDescriptors to the top level of the puppeteer so that
it is convenient to require them.
- improves on documentation to describe the suggested emulation
approach.
References #88.
This patch converts lib/DevicesDescriptors from a devtools front-end
format into a puppeteer format.
This patch does this via introducing a scripts utils/fetch_devices.js
which grabs devices from upstream of DevTools Front-end and
converts them into puppeteer devices.
References #88.
Currently, it's impossible to do screenshots in parallel.
This patch:
- makes all screenshot tasks sequential inside one browser
- starts activating target before taking screenshot
- adds a test to make sure it's possible to take screenshots across
tabs
- starts waiting for a proper page closing after each test. This might
finally solve the ECONNRESET issues in tests.
References #89
This patch:
- resolves pending callbacks in connection once it gets closed
- fixes one of Page.screenshot() tests to wait for all the screenshots
- starts handling ECONNRESET error in test server to avoid throwing with no
good reason
This patch introduces Page.waitForNavigation which allows to wait
for render-initiated navigation.
This patch also does a nice refactoring, replacing Navigator with NavigatorWatcher which
is not a part of a page state.
References #89
This patch re-introduces the DEBUG module to expose some of
the puppeteer's internals.
Currently, only the protocol message communication is exposed under
the 'puppeteer:protocol' namespace.
This patch:
- renames Page.printToPDF into page.pdf
- adds a 'path' option to the page.pdf options instead of a separate
`filePath` parameter
- improves on the documentation for the `page.pdf`
References #39.
This patch introduces a general Documentation.diff method, which
produces a diff of two documentations.
With this, the patch teaches documentation linter to lint method arguments.
References #14.
This patch remove remoteDebuggingPort option. Instead, browser
is launched with '--remote-debugging-port=0' flag, letting browser
to pick any port. The puppeteer reads the port number from the
browser's stderr stream.
This change cuts average browser start time from 300ms to 250ms
on my machine. This happens since puppeteer doesn't have to probe
network once every 100ms, waiting for the remote debugging server to
instantiate.
Fixes#21.
This patch changes Page.navigate API:
- Page.navigate now resolves to the main page response
- Page.navigate throws errors if there's no main page response,
e.g. in case of SSL errors, max navigation timeout,
or invalid url.
This patch also adds httpsServer with a self-signed certificates
for the testing purposes.
Fixes#10.
This patch adds browser.stdout and browser.stderr streams.
These streams allow to get the browser instance output, e.g.
```js
browser.stderr.pipe(process.stdout);
```
This line within `injectFile` wasn't doing much of anything:
```js
let expression = fs.readFile(filePath, 'utf8', (err, data) => callback({err, data}));
```
* That's fixed.
* A path error in examples/features.js is fixed.
* Test added for injectFile.
This patch:
- drops Request dependency on NetworkManager
- drops Response dependency on NetworkManager
- drops requestIds from Request and Response objects
This patch implements NetworkManager, which encapsulates all the
interaction with Network domain.
The NetworkManager also uses partial implementation of Request and
Response classes, defined in the Fetch API specification.
References #26
This patch:
- Changes network idle promise to wait for 2 or fewer network requests for at least idleTime (defaults to 5s) before resolving.
- Adds timer cleanup to failure navigation case.
- Adds handling of webSocketClosed.
- Ignores unrecognized requestIds to avoid negative inflight requests.
References #10
This patch introduces the 'request' event which is fired when
page has initiated a request.
The event dispatches an instance of Request class.
References #26.
This patch does a step towards Fetch API:
- implements Request object to some extend. The Request object will be
sent in RequestWillBeSent event.
- implements InterceptedRequest which extends from Request and allows
for request modification. The InterceptedRequest does not
conform to Fetch API spec - there seems to be nothing related to
amending in-flight request.
- adds test to make sure that request can change headers.
References #26
This patch implements Basic Input api:
- Page.focus(selector) - focuses element with selector
- Page.click(selector) - clicks element with selector
- Page.type(text) - types text into a focused element
Fixed#43.
This patch:
- reformats codebase to use 2-spaces instead of 4. This will
align the project with other codebases (e.g. DevTools and Lighthouse)
- enables eslint indentation checking
References #19
It turned out that the tip-of-tree chromium fails to start on Travis
because of two reasons:
- inability to run LinuxSUIDSandbox
- libnss3 library version being too small
Both problems happen because of the outdated "trusty" distribution
which is used on travis.
This patch:
- reverts the previous patch 9e6f779. Instead of introducing such
a weird "api", both phantom_shim/runner.js and test/test.js no
explicitly pass '--no-sandbox' flag to the browser
- updates the libnss3 lib on travis
Fixes#33
This patch implements FrameManager which is responsible for maintaining
the frame tree. FrameManager is quite basic: it sends FrameAttached,
FrameDetached and FrameNavigated events, and can report mainFrame and
all frames.
The next step would be moving certain Page API's to the Frame. For
example, such method as Page.evaluate, Page.navigate and others should
be available on Frame object as well.
References #4
Page.screenshot operates the global state of the page. In case of
multiple Page.screenshot() commands running in parallel with different
clipping rects, they interfere with each other.
This patch makes Page.screenshot() commands run sequencially
even though they were called in parallel.
Fixes#15.
This patch refactors Page.screenshot api, accoring to the discussion
in #5:
- Page.screenshot accepts single optional options object
- Page.saveScreenshot is removed
- Page.screenshot assumes 'png' screenshot if no type is set and no
'path' property is given
Fixes#5.
This patch introduces a Dialog class and a new 'dialog'
event instead of the 'alert', 'beforeunload', 'confirm' and
'prompt' events and 'Page.handleDialog' method.
Fixes#2.
This patch makes Page.evaluate await promise if one is
returned by the evaluated code.
This makes the Page.evaluateAsync unneeded, so the patch
removes it.
Fixes#11.
This patch removes the Page.setBlockedURLs method. The
functionality is trivially implementable with the request
interception (see examples/loadurlwithoutcss.js).
Fixes#1.
This patch:
- introduces Request class.
- implements Page.setRequestInterceptor method. The method
allows to install a callback which will be called for every request
with a |Request| object as a single parameter. The callback is free
to override certain request's properties and then either continue or
abort it.
- implements request interception api for phantom-shim and unskips the
module/webpage/abort-network-request.js phantomjs test
References #1
This patch drops the chrome-remote-interface dependency and
introduces Connection class which handles all the communication
with remote target.
Closes#3
It turns page.size() and page.setSize() methods are slightly
confusing since there multiple different sizes (layout size,
content size, viewport size..)
This patch renames Page.{size,setSize} methods into
Page.{viewportSize,setViewportSize} methods to avoid confusion.
This patch adds docs/api.md file which contains API description
for the puppeteer API. This patch adds the API outline, which
doesn't not have explanations and samples.
This patch introduces check_availability.js utility which looks for
available chromium binaries for different revisions and platforms.
This patch also re-factors the chromium downloader scripts so that
it can operate different platforms.
This patch implements Puppeteer's Page.setBlockedURLs method.
This is a less agile alternative to the phantom's request.abort()
api.
This patch also adds a loadurlwithoutcss.js example re-implementation
to illustrate the usage of Page.setBlockedURLs.
This patch implements:
- puppeteer's Page.printToPDF method. The method's defaults
are similar to phantom's defaults for the paperSize property.
- phantom's render into pdf file.
The PhantomJS has a similar callback called onInitialized. This
callback passes control to the automation script when the page
gets initialized.
To precisely implement this functionality atop of puppeteer,
and since puppeteer controller script lives in a separate process to
the page, we need an ability to pause page at the moment of
initialization. For now, we are not able to do this.
However, oftentimes clients want to evaluate certain code in
page at the point of page initialization. This patch implements
this capability with the Page.evaluateOnInitilized method call.
This patch also re-implements phantom's unrandomize.js example
with the puppeteer API. This is serves an illustration purpose
for the page.evaluateOnInitilized callback.
With this patch, page has two methods to include javascript:
- Page.addScriptTag(url) which is similar to phantom's includeJs.
- Page.injectFile(filePath) which is similar to phantom's injectJs.
