Commit Graph

263 Commits

Author SHA1 Message Date
Andrey Lushnikov
7cef3d7ce7 Teach page.pdf to respect printBackground argument (#307)
References #301
2017-08-16 15:08:31 -07:00
Jonathan Garbee
da53218107 Fix doc on pdf method. Minor addition to PDF path option docs. (#292) 2017-08-16 11:11:40 -07:00
Andrey Lushnikov
0a190ebec9 Rename 'remoteDebuggingURL' into 'browserWSEndpoint' (#272) 2017-08-16 01:10:55 -07:00
Andrey Lushnikov
44c15220f9 Rename ElementHandle.release into ElementHandle.dispose (#284)
The ElementHandle.release feels related to ElementHandle.click.
The dispose is a more clear name.
2017-08-16 00:49:20 -07:00
JoelEinbinder
3c2aaaaeb0 Readme fixup part 1 (#278) 2017-08-15 23:04:16 -07:00
Aleksey
56eab00fa7 Roll chromium to r494365 (#271)
This patch:
- rolls chromium to r494365
- starts using Runtime.evaluate(awaitPromise: true), with new semantic
  we can avoid additional Runtime.awaitPromise call
- stops resolving promises for Console event
2017-08-15 17:35:59 -07:00
Andrey Lushnikov
af89e893e7 Implement basic element handles (#248)
This patch implements basic element handles which a backed with remote objects.

Fixes #111
2017-08-15 14:54:02 -07:00
Andrey Lushnikov
a424f5613a Introduce Puppeteer.connect method (#264)
This patch:
- refactors Connection to use a single remote debugging URL instead of a
  pair of port and browserTargetId
- introduces Puppeteer.connect() method to attach to already running
  browser instance.

Fixes #238.
2017-08-15 14:29:42 -07:00
Andrey Lushnikov
96309a207c Fix request interception corner cases (#261)
This patch:
- teaches request interception to ignore data URLs. Currently protocol
  doesn't send interceptions for data URLs.
- teaches request interception to properly process URLs with hashes.
  Currently `Network.requestIntercepted` sends url with a hash, whereas
  `Network.requestWillBeSent` doesn't report hashes in its urls. @see
  crbug.com/755456
- skips one more header that I spotted during debugging interception on
  the realworld websites.

Fixes #258, #259.
2017-08-15 13:55:48 -07:00
Andrey Lushnikov
24764df4c4 Emit 'error' event when page crashes (#266)
This patch starts emitting 'error' event when page crashes.
'error' events have special treatment in node, so page crashes
become observable for users.

Fixes #262.
2017-08-15 11:13:05 -07:00
Andrey Lushnikov
0a1294c7ee general cleanup (#256)
This patch:
- fixes multimap implementation to work properly in node
- moves ESTreeWalker from third-party into utils/doclint. ESTreeWalker
  license is compliant with Apache2.0.
2017-08-14 21:16:59 -07:00
Andrey Lushnikov
13e8580a34 Use puppeteer.launch instead of browser constructor (#255)
This patch:
- split browser launching logic from Browser into `lib/Launcher.js`
- introduce `puppeteer` namespace which currently has a single `launch`
  method to start a browser

With this patch, the browser is no longer created with the `new
Browser(..)` command. Instead, it should be "launched" via the
`puppeteer.launch` method:

```js
const puppeteer = require('puppeteer');
puppeteer.launch().then(async browser => {
  ...
});
```

With this approach browser instance lifetime matches the lifetime of
actual browser process. This helps us:
- remove proxy streams, e.g. browser.stderr and browser.stdout
- cleanup browser class and make it possible to connect to remote
  browser
- introduce events on the browser instance, e.g. 'page' event. In case
  of lazy-launching browser, we should've launch browser when an event
  listener is added, which is unneded comlpexity.
2017-08-14 18:08:06 -07:00
Andrey Lushnikov
4269f6a1bb Roll chromium to r493985 (#251)
This patch:
- rolls chromium to r493985
- migrates Browser.version() puppeteer method to use newly added
  Browser.getVersion() protocol method

References #227
2017-08-12 22:04:39 -07:00
Andrey Lushnikov
3ee4951506 [DEBUG] add "session" namespace to trace target protocol messages (#249) 2017-08-11 17:47:33 -07:00
Jonathan Garbee
56b0d8722b Default value for property on dialog (#247) 2017-08-11 17:24:46 -07:00
Andrey Lushnikov
4551afc6dc Introduce new interception API (#242)
This patch introduces new interception API, via killing InterceptedRequest and giving the `abort` and `continue` methods to the Request object.
2017-08-11 17:24:31 -07:00
Andrey Lushnikov
dbac6788ae Plumb default prompt value to the dialog (#243)
This patch:
- plumbs default prompt value to the dialog
- enables one more phantom test
2017-08-11 12:17:43 -07:00
JoelEinbinder
497baf0198 Roll chromium to r493673 (#240)
Mouse events are no longer racy. Enabling touch no longer converts all mouse events into touches. Promises in destroyed execution contexts are rejected immediately.
2017-08-11 01:21:02 -07:00
Andrey Lushnikov
b8dbd28b8c Refer to the upstream bug (#239)
The issue #168 is a protocol inconsistency which happens only
in case of HTTPS error. This patch starts refering to the
upstream bug instead of puppeteer issue.

Closes #168.
2017-08-11 01:07:33 -07:00
Andrey Lushnikov
6347a049ba Rename Page.setInPageCallback into Page.addBinding (#236)
This seems to be a much better name which is actually used for a similar
purposes in chromium/v8.
2017-08-10 21:44:49 -07:00
Pavel Feldman
3a7741a172 Introduce page.emulate convenience wrapper (#231)
This patch:
- introduces Page.emulate convenience wrapper
- adds a screenshot-fullPage.js example
2017-08-10 18:42:30 -07:00
Andrey Lushnikov
e1d8a3781e Rename Page.navigate into Page.goto (#232)
Fixes #228.
2017-08-10 00:02:10 -07:00
Andrey Lushnikov
fe06c896eb Roll chromium to 492629 (#230)
This patch
- rolls chromium to 492629
- migrates connection establishing to use browser target. This migration means
  that now we have a single websocket connection to browser (implemented
  in Connection class). A connection to a particular target is
  incapsulated in a new Session class.
2017-08-09 16:14:00 -07:00
Andrey Lushnikov
34b0095c10 Make interception work with redirects (#218)
This patch:
- changes interception API so that it better aligns with what we'd like to see
  in #121
- fixes the issue with redirect interception

Fixes #217.
2017-08-07 17:48:52 -07:00
Andrey Lushnikov
e1c5b8d244 Aborted network requests should use protocol's "Failed" status (#215)
This patch starts using "Failed" command for request interception instead of
"Aborted".

The "Aborted" status also has a side-effect of cancelling the navigation, so
there will be no error on the page and form puppeteer's standpoint, the navigation
will never complete.
2017-08-07 15:13:17 -07:00
Andrey Lushnikov
7888a37a3d Page.pdf should accept case-insensetive page format (#211) 2017-08-04 01:01:10 -07:00
Andrey Lushnikov
00196e6cdc Fix setContent to actually await the inner evaluation (#209) 2017-08-04 00:23:04 -07:00
Pavel Feldman
82fbb268db Do not count inflight requests explicitly - use set. (#202) 2017-08-03 16:30:55 -07:00
Andrey Lushnikov
20ba447689 [DEBUG] Trace only those events which have listeners. (#201)
The `DEBUG=*page npm run unit` is too verbose due to events spamming
the console.

This patch starts tracing emitted events only if there are any
listeners.
2017-08-03 15:20:31 -07:00
Andrey Lushnikov
31224392bb Kill page.$ and page.$$ functions (#197)
page.$ and page.$$ conflicts with our ideas about element handles.

This patch removes functions in favor of future implementation.

References #111
2017-08-03 13:35:31 -07:00
Andrey Lushnikov
6c163122bf Update default viewport size to be 800px by 600px (#194)
This patch updates default viewport size to be 800px by 600px.
This comes handy when observing scripts running in non-headless mode.
2017-08-02 15:47:00 -07:00
JoelEinbinder
984e011cf0 Move path option into tracing.start (#192)
This consolidates all the tracing options in the tracing.start()
2017-08-02 15:41:05 -07:00
JoelEinbinder
c32df08e4c Update jsdoc from Tracing patch (#189)
This fixes the comments @a1ph on #181 about JSDOC.
2017-08-02 15:02:41 -07:00
JoelEinbinder
5d647bf1fa Use const a lot fo places (#188) 2017-08-02 12:06:47 -07:00
JoelEinbinder
bd72e40e73 Introduce page.tracing (#181)
This patch introduces page.tracing, which allows to start and stop
trace recording for a page. The trace could be then saved to file.
2017-08-02 10:45:11 -07:00
Andrey Lushnikov
a43c315214 Implement browser 'ignoreHTTPSErrors' option. (#177)
This patch implements Browser 'ignoreHTTPSErrors' option.

Fixes #84.
2017-08-01 15:17:57 -07:00
JoelEinbinder
2acfec0989 Slow mode (#173)
This patch introduces a `slowMo` browser option which
slows down puppeteer operations. Comes handy for debugging
scripts.
2017-07-31 18:47:56 -07:00
JoelEinbinder
fc70ab8f21 Add delay option to input methods (#171)
Add delay option to input methods for the keyboard and mouse.

Closes #156
2017-07-31 18:18:15 -07:00
JoelEinbinder
bfc40b2ee6 Specify repeat property on repeated keypresses (#172)
This patch implements 'autoRepeat' functionality for `keyboard.down`.
With this patch, the subsequent calls to `keyboard.down` would generate
an event with 'autoRepeat` flag set to true.

Closes #157
2017-07-31 12:05:46 -07:00
JoelEinbinder
b474a2d0d9 Mark options objects as optional (#170) 2017-07-30 13:46:56 -07:00
Andrey Lushnikov
c013a531cf Refactor EmulationManager
This patch refactors EmulationManager so that it is a normal
class with a state.
2017-07-29 19:19:37 -07:00
Andrey Lushnikov
445dce46f6 response.text() should wait for request to finish
This patch makes sure that request.text() doesn't try
to fetch response body from the backend until the request is
actually finished (finished or failed).
2017-07-29 18:48:30 -07:00
Andrey Lushnikov
67f4264162 Never sent 'requestfinished' event without passing actual request
It turns out we're not receiving 'Network.requestWillBeSent' event
for every requestId.

This patch makes sure we don't dispatch `requestfinished` and
`requestfailed` events without passing actual request.

References #168
2017-07-29 18:34:47 -07:00
Andrey Lushnikov
72991c16eb [DEBUG] chnage direction of arrow in protocol's SEND and RECEIVE logs
They currently feel backwards.
2017-07-29 17:43:24 -07:00
Andrey Lushnikov
adf35952fc [DEBUG] More descriptive event reporting
This patch improves DEBUG reporting so that circular event
arguments are shown with some information.
2017-07-29 13:02:56 -07:00
Pavel Feldman
7fdf800a39 Simplify frame manager, do not fetch resources upfront (#159)
This patch:
- gets rid of Page.getResourceTree() protocol method
- rolls chromium to 490379

Fixes #127
2017-07-28 16:52:38 -07:00
Andrey Lushnikov
bd767002bb Fix navigation to about:blank
This patch fixes navigation to about:blank url.
2017-07-28 16:44:51 -07:00
JoelEinbinder
8780fcb662 Event coverage and debugging (#160)
This patch introduces event coverage for DEBUG module and 
API coverage.

Closes #50.
2017-07-28 11:45:05 -07:00
JoelEinbinder
3c75767288 Fix media keys to use new key strings and codes (#164)
VolumeUp, VolumeDown, and VolumeMute were changed to AudioVolumeUp, AudioVolumeDown, and AudioVolumeMute

The media keys like MediaTrackNext were also missing, so I added them.
2017-07-28 11:38:30 -07:00
Andrey Lushnikov
d26e2399f2 Resolve paths against CWD in page.uploadFile() method
This patch:
- teaches page.uploadFile() to resolve given file paths against
  current working directory. This aligns paths handling with all the
  other methods
- moves page.uploadFile() under Frame
- changes test to use relative path for file upload
2017-07-28 00:06:57 -07:00
Andrey Lushnikov
ba37a4f82d Get rid of Body class
The Body class was inlined in the Request and Response classes.
This patch:
- removes the Body class
- adds Request.postData public property
- adds Response.buffer(), Response.text() and Response.json() methods

Fixes #106.
2017-07-27 23:11:24 -07:00
Andrey Lushnikov
b9e3cce5fd Fail navigation when main resource fails to load
This patch fails navigation when the main resource fails to load.

Fixes #148.
2017-07-27 17:54:39 -07:00
Andrey Lushnikov
bd898b7f56 Implement function as a part of a page.waitFor shortcut
This patch adds a function as a possible argument to
page.waitFor shortcut.

Fixes #91.
2017-07-27 17:09:28 -07:00
Andrey Lushnikov
ff5ed1c738 Implement page.waitForFunction method
The page.waitForFunction method allows to wait for a general predicate.
The predicate will be continiously polled for in page, until
it either returns true or the timeout happens.

The polling parameter could be one of the following:
- 'raf' - to poll on every animation frame
- 'mutation' - to poll on every dom mutation
- <number> - to poll every X milliseconds

References #91
2017-07-27 16:45:14 -07:00
JoelEinbinder
a2e0d27fb6 Implement public API coverage
This patch:
- implements a basic public API coverage based on 'helper.tracePublicAPI' methods
- adds `npm run coverage` command which reports coverage after running all of the unit tests

References #50.
2017-07-27 16:16:37 -07:00
Andrey Lushnikov
c26d2c8271 Remove redundant methods from Body class
This patch:
- removes Body.arrayBuffer. This method is redundant since there's
  already a Body.buffer() method
- removes Body.bodyUsed getter.

References #106
2017-07-27 14:43:14 -07:00
Andrey Lushnikov
e39d8602b0 Rename page.setHTTPHeaders into page.setExtraHTTPHeaders
This patch:
- renames page.setHTTPHeaders into page.setExtraHTTPHeaders
- starts using Map instead of Object to align with other headers
  arguments

Fixes #112.
2017-07-27 12:51:18 -07:00
JoelEinbinder
bbde8fd1c2 page.evaluate takes a string in addition to function (#135)
This patch improves on page.evaluate to accept a string.
The string can have a trailing '//# sourceURL=' comment which would
name the evaluation to make stacks beautiful.

In order to make sourceURL comments possible, this patch:
- removes wrapping of the client function into `Promise.resolve()`
- stops passing `awaitPromise` parameter to `Runtime.evaluate`
- starts to await promise via the `Runtime.awaitPromise` if the return type of the evaluation
  is promise

closes #118
2017-07-27 12:23:41 -07:00
Andrey Lushnikov
480547955c Drop Headers class in favor of a Map object
This patch removes Header class and substitutes it with a simple
Map object.

The map is chosen over the vanilla object since it has explicit
order of headers which we'd like to preserve.

References #106.
2017-07-27 09:35:44 -07:00
Pavel Feldman
585f815548 Don't use Emulation.resetPageScaleFactor (#147)
Instead of using Emulation.resetPageScaleFactor, this patch starts using clip's scale for
fullPage screenshots.
2017-07-26 15:28:44 -07:00
Pavel Feldman
5068185d03 Don't ever enable DOM domain. (#145)
This patchs makes sure DOM domain is never enabled. The reasoning behind this is that
DOM domain is heavy-weight: it generates a lot of traffic over the protocol.

Drive-by: use Page.addScriptToEvaluateOnNewDocument instead of Page.addScriptToEvaluateOnLoad
2017-07-26 15:19:43 -07:00
JoelEinbinder
2e6751d6f5 Trace only string method names (#141) 2017-07-26 11:48:40 -07:00
JoelEinbinder
9363a15cce Small fixes to jsdoc (#140)
References #65.
2017-07-26 09:58:03 -07:00
Alexei Filippov
83f4e43564 Clean up Browser annotations. (#138) 2017-07-25 20:39:50 -07:00
JoelEinbinder
aa28eb80f0 Wait for double raf after all mouse events. (#123)
This patch starts waiting for double raf after every mouse event.
It looks like a good enough workaround to make sure mouse events
are delivered.

The single raf might not be enough because browser might 'return' an already-prepared
frame for us without actually dispatching events.

Closes #122
2017-07-25 17:03:13 -07:00
JoelEinbinder
1a50403090 fix linter (#136)
This patch fixes broken travis, caused by impatient merge of f455e3a1e
2017-07-25 16:47:49 -07:00
Pavel Feldman
f455e3a1e4 Remove NetworkManager usage from NavigatorWatcher (#134) 2017-07-25 16:06:53 -07:00
JoelEinbinder
64968862db Implement mouse.click(x,y) (#131)
Closes #107
2017-07-25 16:05:23 -07:00
Pavel Feldman
49e363c121 Mess with lusha's waitFor for fun. (#133) 2017-07-25 15:57:31 -07:00
Pavel Feldman
4adf11427a Unconditionally update frame payload on navigation only. (#130) 2017-07-25 14:58:10 -07:00
Pavel Feldman
eca0d7fe53 Simplify keyboard modifiers handling (#129) 2017-07-25 14:35:03 -07:00
Pavel Feldman
0007809305 Move inner evaluate helpers into frame class. (#128) 2017-07-25 14:30:04 -07:00
JoelEinbinder
4f5f1f6d77 Move missing methods from page onto frame. (#125)
Closes #113
2017-07-25 11:37:46 -07:00
Andrey Lushnikov
b564029589 Improve documentation for the page.waitFor method.
Fixes #109.
2017-07-25 08:46:11 -07:00
JoelEinbinder
877f06aacb Remove userAgent getter (#126)
If needed, the user agent could be fetched via querying
`navigator.userAgent` in page.

Closes #110
2017-07-25 02:01:10 -07:00
JoelEinbinder
615381c70e Remove browser.closePage. (#124)
Remove browser.closePage. Closes #116
2017-07-25 00:44:13 -07:00
Andrey Lushnikov
d4c5aee5a8 Rename page.evaluateOnInitialized into page.evaluateOnNewDocument
This patch renames page.evaluateOnInitialized into
page.evaluateOnNewDocument to better align with the protocol and with
what the method is actually doing.

Fixes #119.
2017-07-25 00:17:56 -07:00
Andrey Lushnikov
5acd711b20 Remove InterceptedRequest.isHandled() method
This patch removes the InterceptedRequest.isHandled method. There's
always a single request interceptor who owns all the intercepted
requests and who know if they were handled.

References #115.
2017-07-24 23:37:28 -07:00
Andrey Lushnikov
efa33c37b3 Remove page.httpHeaders() getter
Remove the page.httpHeaders() method since it always returns the last
value of page.setHTTPHeaders().

References #115.
2017-07-24 23:35:10 -07:00
Andrey Lushnikov
5549ad0282 Do not serialize remote objects unless needed
This patch stops serializing console API arguments unless there are
listeners of the 'console' event in puppeteer.

This saves quite a lot CPU cycles.

Fixes #117.
2017-07-24 21:43:54 -07:00
Andrey Lushnikov
236acec228 Remove Frame.isMainFrame()
This patch removes redundant frame.isMainFrame() method. The suggested
way to check for main frame is:

```js
frame === page.mainFrame()
```

Fixes #120.
2017-07-24 21:25:54 -07:00
JoelEinbinder
a24cec20f9 Fix clicking offscreen horizontally (#105)
This patch starts clicking in the center of the element area which is
visible in a viewport.

Fixes #103.
2017-07-24 16:14:32 -07:00
Andrey Lushnikov
0a3125434e Refactor Frame.waitForSelector method
Refactor Frame.waitForSelector to make room for Frame.waitForFunction
implementation.
This patch:
- removes AwaitedElement class which proved to be confusing, and
  introduces a more straight-forward WaitTask.
- refactors the mutation observer to return true in case of successful
  waiting or false in case of timeout.

References #91
2017-07-24 10:07:43 -07:00
JoelEinbinder
fdaaa2c0e6 Inject file with sourceURL (#102)
This patch starts adding a sourceURL trailing comment to make stack traces
readable.
2017-07-23 09:56:35 -07:00
Andrey Lushnikov
c4904b4e10 Do not leak event listeners on navigation
This patch:
- introduces helper.addEventListener/helper.removeEventListeners
  to simplify event management
- moves NavigatorWatchdog over to the helper.addEventListener to
  stop leaking event listeners
2017-07-22 17:46:39 -07:00
Andrey Lushnikov
5757bc18f2 Rename 'maxTime' option of Page.navigate into 'timeout'
The motivation behind this rename is to name all the 'timeout' options
across methods similarly.

References #39.
2017-07-22 16:32:57 -07:00
JoelEinbinder
98ee35655f Mouse (#101)
This patch:
- adds Mouse class which holds mouse state and implements mouse primitives,
such as moving, button down and button up.
- implements high-level mouse api, such as `page.click` and `page.hover`.

References #40, References #89
2017-07-21 20:29:31 -07:00
JoelEinbinder
eb2cb67b0e Remove keyboard.type and keyboard.press (#98)
This patch removes keyboard.type and keyboard.press methods. The motivation
behind this is to keep only low-level API in the `keyboard` namespace.
2017-07-21 20:00:09 -07:00
Andrey Lushnikov
dc032b42b9 Introduce polymorphic page.waitFor method
This patch:
- introduces page.waitForSelector to wait for the selector to appear
- introduces polymorphic page.waitFor method, which accepts
either string (and in this case is a shortcut for page.waitForSelector)
or number (and in this case it's a promisified timeout).

References #91.
2017-07-21 12:54:04 -07:00
Andrey Lushnikov
1f954fa7ed Implement timeout option for page.waitFor
This patch implements timeout option for page.waitFor. The function
will throw if the selector doesn't appear during timeout milliseconds
of waittime.

References #89, #91.
2017-07-21 12:07:48 -07:00
Andrey Lushnikov
aba61de905 Make helper.getExceptionMessage synchronous
We now have description of an exception, no need for a roundtrip
to the backend.
2017-07-21 11:57:25 -07:00
Andrey Lushnikov
52de75742b Implement visible option for Page.waitFor method
This patch adds a 'visible' option to the Page.waitFor method, making
it possible to wait for the element to become actually visible.

References #89, #91.
2017-07-21 10:01:19 -07:00
Andrey Lushnikov
139b9e9b6d Get rid of page.emulate() / page.emulatedDevices() methods
This patch:
- gets rid of `page.emulate` and `page.emulatedDevices`
  methods. Instead, it is suggested to use `page.setViewport()`
  and `page.setUserAgent()` methods.
- moves DeviceDescriptors to the top level of the puppeteer so that
  it is convenient to require them.
- improves on documentation to describe the suggested emulation
  approach.

References #88.
2017-07-20 23:53:06 -07:00
Andrey Lushnikov
76ac3bded5 Convert DevicesDescriptors into puppeteer format
This patch converts lib/DevicesDescriptors from a devtools front-end
format into a puppeteer format.

This patch does this via introducing a scripts utils/fetch_devices.js
which grabs devices from upstream of DevTools Front-end and
converts them into puppeteer devices.

References #88.
2017-07-20 17:49:15 -07:00
Andrey Lushnikov
42edc3108a [emulation] fix touch emulation
This patch:
- fixes touch emulation
- adds tests to cover basic Page.emulate

References #88.
2017-07-20 12:22:34 -07:00
Andrey Lushnikov
4af0911b90 Use more good flags to run chrome instance.
The switches are mostly borrowed from chromedriver launching code.
2017-07-20 12:20:10 -07:00
Andrey Lushnikov
a981594b1d Revert node's scrolling into view in page.click()
This change was not intentional in e33a8f818c
2017-07-20 00:53:14 -07:00
Andrey Lushnikov
e33a8f818c Support HEADLESS env variable for unit tests
This patch makes it possible to run unit tests in non-headless
mode with the following command:

```
HEADLESS=false npm run unit
```
2017-07-19 19:36:22 -07:00
Andrey Lushnikov
a63a0198de Implement waitFor which survives navigation (#99)
This patch implements page.waitFor method which survives navigation.

References #89.
2017-07-19 19:04:51 -07:00
JoelEinbinder
febd747c5b Inroduce page.press (#96)
This patch:
- introduces page.press() method
- adds more input tests

References #89
2017-07-19 14:43:07 -07:00