Commit Graph

245 Commits

Author SHA1 Message Date
Andrey Lushnikov
b8dbd28b8c Refer to the upstream bug (#239)
The issue #168 is a protocol inconsistency which happens only
in case of HTTPS error. This patch starts refering to the
upstream bug instead of puppeteer issue.

Closes #168.
2017-08-11 01:07:33 -07:00
Andrey Lushnikov
6347a049ba Rename Page.setInPageCallback into Page.addBinding (#236)
This seems to be a much better name which is actually used for a similar
purposes in chromium/v8.
2017-08-10 21:44:49 -07:00
Pavel Feldman
3a7741a172 Introduce page.emulate convenience wrapper (#231)
This patch:
- introduces Page.emulate convenience wrapper
- adds a screenshot-fullPage.js example
2017-08-10 18:42:30 -07:00
Andrey Lushnikov
e1d8a3781e Rename Page.navigate into Page.goto (#232)
Fixes #228.
2017-08-10 00:02:10 -07:00
Andrey Lushnikov
fe06c896eb Roll chromium to 492629 (#230)
This patch
- rolls chromium to 492629
- migrates connection establishing to use browser target. This migration means
  that now we have a single websocket connection to browser (implemented
  in Connection class). A connection to a particular target is
  incapsulated in a new Session class.
2017-08-09 16:14:00 -07:00
Andrey Lushnikov
34b0095c10 Make interception work with redirects (#218)
This patch:
- changes interception API so that it better aligns with what we'd like to see
  in #121
- fixes the issue with redirect interception

Fixes #217.
2017-08-07 17:48:52 -07:00
Andrey Lushnikov
e1c5b8d244 Aborted network requests should use protocol's "Failed" status (#215)
This patch starts using "Failed" command for request interception instead of
"Aborted".

The "Aborted" status also has a side-effect of cancelling the navigation, so
there will be no error on the page and form puppeteer's standpoint, the navigation
will never complete.
2017-08-07 15:13:17 -07:00
Andrey Lushnikov
7888a37a3d Page.pdf should accept case-insensetive page format (#211) 2017-08-04 01:01:10 -07:00
Andrey Lushnikov
00196e6cdc Fix setContent to actually await the inner evaluation (#209) 2017-08-04 00:23:04 -07:00
Pavel Feldman
82fbb268db Do not count inflight requests explicitly - use set. (#202) 2017-08-03 16:30:55 -07:00
Andrey Lushnikov
20ba447689 [DEBUG] Trace only those events which have listeners. (#201)
The `DEBUG=*page npm run unit` is too verbose due to events spamming
the console.

This patch starts tracing emitted events only if there are any
listeners.
2017-08-03 15:20:31 -07:00
Andrey Lushnikov
31224392bb Kill page.$ and page.$$ functions (#197)
page.$ and page.$$ conflicts with our ideas about element handles.

This patch removes functions in favor of future implementation.

References #111
2017-08-03 13:35:31 -07:00
Andrey Lushnikov
6c163122bf Update default viewport size to be 800px by 600px (#194)
This patch updates default viewport size to be 800px by 600px.
This comes handy when observing scripts running in non-headless mode.
2017-08-02 15:47:00 -07:00
JoelEinbinder
984e011cf0 Move path option into tracing.start (#192)
This consolidates all the tracing options in the tracing.start()
2017-08-02 15:41:05 -07:00
JoelEinbinder
c32df08e4c Update jsdoc from Tracing patch (#189)
This fixes the comments @a1ph on #181 about JSDOC.
2017-08-02 15:02:41 -07:00
JoelEinbinder
5d647bf1fa Use const a lot fo places (#188) 2017-08-02 12:06:47 -07:00
JoelEinbinder
bd72e40e73 Introduce page.tracing (#181)
This patch introduces page.tracing, which allows to start and stop
trace recording for a page. The trace could be then saved to file.
2017-08-02 10:45:11 -07:00
Andrey Lushnikov
a43c315214 Implement browser 'ignoreHTTPSErrors' option. (#177)
This patch implements Browser 'ignoreHTTPSErrors' option.

Fixes #84.
2017-08-01 15:17:57 -07:00
JoelEinbinder
2acfec0989 Slow mode (#173)
This patch introduces a `slowMo` browser option which
slows down puppeteer operations. Comes handy for debugging
scripts.
2017-07-31 18:47:56 -07:00
JoelEinbinder
fc70ab8f21 Add delay option to input methods (#171)
Add delay option to input methods for the keyboard and mouse.

Closes #156
2017-07-31 18:18:15 -07:00
JoelEinbinder
bfc40b2ee6 Specify repeat property on repeated keypresses (#172)
This patch implements 'autoRepeat' functionality for `keyboard.down`.
With this patch, the subsequent calls to `keyboard.down` would generate
an event with 'autoRepeat` flag set to true.

Closes #157
2017-07-31 12:05:46 -07:00
JoelEinbinder
b474a2d0d9 Mark options objects as optional (#170) 2017-07-30 13:46:56 -07:00
Andrey Lushnikov
c013a531cf Refactor EmulationManager
This patch refactors EmulationManager so that it is a normal
class with a state.
2017-07-29 19:19:37 -07:00
Andrey Lushnikov
445dce46f6 response.text() should wait for request to finish
This patch makes sure that request.text() doesn't try
to fetch response body from the backend until the request is
actually finished (finished or failed).
2017-07-29 18:48:30 -07:00
Andrey Lushnikov
67f4264162 Never sent 'requestfinished' event without passing actual request
It turns out we're not receiving 'Network.requestWillBeSent' event
for every requestId.

This patch makes sure we don't dispatch `requestfinished` and
`requestfailed` events without passing actual request.

References #168
2017-07-29 18:34:47 -07:00
Andrey Lushnikov
72991c16eb [DEBUG] chnage direction of arrow in protocol's SEND and RECEIVE logs
They currently feel backwards.
2017-07-29 17:43:24 -07:00
Andrey Lushnikov
adf35952fc [DEBUG] More descriptive event reporting
This patch improves DEBUG reporting so that circular event
arguments are shown with some information.
2017-07-29 13:02:56 -07:00
Pavel Feldman
7fdf800a39 Simplify frame manager, do not fetch resources upfront (#159)
This patch:
- gets rid of Page.getResourceTree() protocol method
- rolls chromium to 490379

Fixes #127
2017-07-28 16:52:38 -07:00
Andrey Lushnikov
bd767002bb Fix navigation to about:blank
This patch fixes navigation to about:blank url.
2017-07-28 16:44:51 -07:00
JoelEinbinder
8780fcb662 Event coverage and debugging (#160)
This patch introduces event coverage for DEBUG module and 
API coverage.

Closes #50.
2017-07-28 11:45:05 -07:00
JoelEinbinder
3c75767288 Fix media keys to use new key strings and codes (#164)
VolumeUp, VolumeDown, and VolumeMute were changed to AudioVolumeUp, AudioVolumeDown, and AudioVolumeMute

The media keys like MediaTrackNext were also missing, so I added them.
2017-07-28 11:38:30 -07:00
Andrey Lushnikov
d26e2399f2 Resolve paths against CWD in page.uploadFile() method
This patch:
- teaches page.uploadFile() to resolve given file paths against
  current working directory. This aligns paths handling with all the
  other methods
- moves page.uploadFile() under Frame
- changes test to use relative path for file upload
2017-07-28 00:06:57 -07:00
Andrey Lushnikov
ba37a4f82d Get rid of Body class
The Body class was inlined in the Request and Response classes.
This patch:
- removes the Body class
- adds Request.postData public property
- adds Response.buffer(), Response.text() and Response.json() methods

Fixes #106.
2017-07-27 23:11:24 -07:00
Andrey Lushnikov
b9e3cce5fd Fail navigation when main resource fails to load
This patch fails navigation when the main resource fails to load.

Fixes #148.
2017-07-27 17:54:39 -07:00
Andrey Lushnikov
bd898b7f56 Implement function as a part of a page.waitFor shortcut
This patch adds a function as a possible argument to
page.waitFor shortcut.

Fixes #91.
2017-07-27 17:09:28 -07:00
Andrey Lushnikov
ff5ed1c738 Implement page.waitForFunction method
The page.waitForFunction method allows to wait for a general predicate.
The predicate will be continiously polled for in page, until
it either returns true or the timeout happens.

The polling parameter could be one of the following:
- 'raf' - to poll on every animation frame
- 'mutation' - to poll on every dom mutation
- <number> - to poll every X milliseconds

References #91
2017-07-27 16:45:14 -07:00
JoelEinbinder
a2e0d27fb6 Implement public API coverage
This patch:
- implements a basic public API coverage based on 'helper.tracePublicAPI' methods
- adds `npm run coverage` command which reports coverage after running all of the unit tests

References #50.
2017-07-27 16:16:37 -07:00
Andrey Lushnikov
c26d2c8271 Remove redundant methods from Body class
This patch:
- removes Body.arrayBuffer. This method is redundant since there's
  already a Body.buffer() method
- removes Body.bodyUsed getter.

References #106
2017-07-27 14:43:14 -07:00
Andrey Lushnikov
e39d8602b0 Rename page.setHTTPHeaders into page.setExtraHTTPHeaders
This patch:
- renames page.setHTTPHeaders into page.setExtraHTTPHeaders
- starts using Map instead of Object to align with other headers
  arguments

Fixes #112.
2017-07-27 12:51:18 -07:00
JoelEinbinder
bbde8fd1c2 page.evaluate takes a string in addition to function (#135)
This patch improves on page.evaluate to accept a string.
The string can have a trailing '//# sourceURL=' comment which would
name the evaluation to make stacks beautiful.

In order to make sourceURL comments possible, this patch:
- removes wrapping of the client function into `Promise.resolve()`
- stops passing `awaitPromise` parameter to `Runtime.evaluate`
- starts to await promise via the `Runtime.awaitPromise` if the return type of the evaluation
  is promise

closes #118
2017-07-27 12:23:41 -07:00
Andrey Lushnikov
480547955c Drop Headers class in favor of a Map object
This patch removes Header class and substitutes it with a simple
Map object.

The map is chosen over the vanilla object since it has explicit
order of headers which we'd like to preserve.

References #106.
2017-07-27 09:35:44 -07:00
Pavel Feldman
585f815548 Don't use Emulation.resetPageScaleFactor (#147)
Instead of using Emulation.resetPageScaleFactor, this patch starts using clip's scale for
fullPage screenshots.
2017-07-26 15:28:44 -07:00
Pavel Feldman
5068185d03 Don't ever enable DOM domain. (#145)
This patchs makes sure DOM domain is never enabled. The reasoning behind this is that
DOM domain is heavy-weight: it generates a lot of traffic over the protocol.

Drive-by: use Page.addScriptToEvaluateOnNewDocument instead of Page.addScriptToEvaluateOnLoad
2017-07-26 15:19:43 -07:00
JoelEinbinder
2e6751d6f5 Trace only string method names (#141) 2017-07-26 11:48:40 -07:00
JoelEinbinder
9363a15cce Small fixes to jsdoc (#140)
References #65.
2017-07-26 09:58:03 -07:00
Alexei Filippov
83f4e43564 Clean up Browser annotations. (#138) 2017-07-25 20:39:50 -07:00
JoelEinbinder
aa28eb80f0 Wait for double raf after all mouse events. (#123)
This patch starts waiting for double raf after every mouse event.
It looks like a good enough workaround to make sure mouse events
are delivered.

The single raf might not be enough because browser might 'return' an already-prepared
frame for us without actually dispatching events.

Closes #122
2017-07-25 17:03:13 -07:00
JoelEinbinder
1a50403090 fix linter (#136)
This patch fixes broken travis, caused by impatient merge of f455e3a1e
2017-07-25 16:47:49 -07:00
Pavel Feldman
f455e3a1e4 Remove NetworkManager usage from NavigatorWatcher (#134) 2017-07-25 16:06:53 -07:00
JoelEinbinder
64968862db Implement mouse.click(x,y) (#131)
Closes #107
2017-07-25 16:05:23 -07:00
Pavel Feldman
49e363c121 Mess with lusha's waitFor for fun. (#133) 2017-07-25 15:57:31 -07:00
Pavel Feldman
4adf11427a Unconditionally update frame payload on navigation only. (#130) 2017-07-25 14:58:10 -07:00
Pavel Feldman
eca0d7fe53 Simplify keyboard modifiers handling (#129) 2017-07-25 14:35:03 -07:00
Pavel Feldman
0007809305 Move inner evaluate helpers into frame class. (#128) 2017-07-25 14:30:04 -07:00
JoelEinbinder
4f5f1f6d77 Move missing methods from page onto frame. (#125)
Closes #113
2017-07-25 11:37:46 -07:00
Andrey Lushnikov
b564029589 Improve documentation for the page.waitFor method.
Fixes #109.
2017-07-25 08:46:11 -07:00
JoelEinbinder
877f06aacb Remove userAgent getter (#126)
If needed, the user agent could be fetched via querying
`navigator.userAgent` in page.

Closes #110
2017-07-25 02:01:10 -07:00
JoelEinbinder
615381c70e Remove browser.closePage. (#124)
Remove browser.closePage. Closes #116
2017-07-25 00:44:13 -07:00
Andrey Lushnikov
d4c5aee5a8 Rename page.evaluateOnInitialized into page.evaluateOnNewDocument
This patch renames page.evaluateOnInitialized into
page.evaluateOnNewDocument to better align with the protocol and with
what the method is actually doing.

Fixes #119.
2017-07-25 00:17:56 -07:00
Andrey Lushnikov
5acd711b20 Remove InterceptedRequest.isHandled() method
This patch removes the InterceptedRequest.isHandled method. There's
always a single request interceptor who owns all the intercepted
requests and who know if they were handled.

References #115.
2017-07-24 23:37:28 -07:00
Andrey Lushnikov
efa33c37b3 Remove page.httpHeaders() getter
Remove the page.httpHeaders() method since it always returns the last
value of page.setHTTPHeaders().

References #115.
2017-07-24 23:35:10 -07:00
Andrey Lushnikov
5549ad0282 Do not serialize remote objects unless needed
This patch stops serializing console API arguments unless there are
listeners of the 'console' event in puppeteer.

This saves quite a lot CPU cycles.

Fixes #117.
2017-07-24 21:43:54 -07:00
Andrey Lushnikov
236acec228 Remove Frame.isMainFrame()
This patch removes redundant frame.isMainFrame() method. The suggested
way to check for main frame is:

```js
frame === page.mainFrame()
```

Fixes #120.
2017-07-24 21:25:54 -07:00
JoelEinbinder
a24cec20f9 Fix clicking offscreen horizontally (#105)
This patch starts clicking in the center of the element area which is
visible in a viewport.

Fixes #103.
2017-07-24 16:14:32 -07:00
Andrey Lushnikov
0a3125434e Refactor Frame.waitForSelector method
Refactor Frame.waitForSelector to make room for Frame.waitForFunction
implementation.
This patch:
- removes AwaitedElement class which proved to be confusing, and
  introduces a more straight-forward WaitTask.
- refactors the mutation observer to return true in case of successful
  waiting or false in case of timeout.

References #91
2017-07-24 10:07:43 -07:00
JoelEinbinder
fdaaa2c0e6 Inject file with sourceURL (#102)
This patch starts adding a sourceURL trailing comment to make stack traces
readable.
2017-07-23 09:56:35 -07:00
Andrey Lushnikov
c4904b4e10 Do not leak event listeners on navigation
This patch:
- introduces helper.addEventListener/helper.removeEventListeners
  to simplify event management
- moves NavigatorWatchdog over to the helper.addEventListener to
  stop leaking event listeners
2017-07-22 17:46:39 -07:00
Andrey Lushnikov
5757bc18f2 Rename 'maxTime' option of Page.navigate into 'timeout'
The motivation behind this rename is to name all the 'timeout' options
across methods similarly.

References #39.
2017-07-22 16:32:57 -07:00
JoelEinbinder
98ee35655f Mouse (#101)
This patch:
- adds Mouse class which holds mouse state and implements mouse primitives,
such as moving, button down and button up.
- implements high-level mouse api, such as `page.click` and `page.hover`.

References #40, References #89
2017-07-21 20:29:31 -07:00
JoelEinbinder
eb2cb67b0e Remove keyboard.type and keyboard.press (#98)
This patch removes keyboard.type and keyboard.press methods. The motivation
behind this is to keep only low-level API in the `keyboard` namespace.
2017-07-21 20:00:09 -07:00
Andrey Lushnikov
dc032b42b9 Introduce polymorphic page.waitFor method
This patch:
- introduces page.waitForSelector to wait for the selector to appear
- introduces polymorphic page.waitFor method, which accepts
either string (and in this case is a shortcut for page.waitForSelector)
or number (and in this case it's a promisified timeout).

References #91.
2017-07-21 12:54:04 -07:00
Andrey Lushnikov
1f954fa7ed Implement timeout option for page.waitFor
This patch implements timeout option for page.waitFor. The function
will throw if the selector doesn't appear during timeout milliseconds
of waittime.

References #89, #91.
2017-07-21 12:07:48 -07:00
Andrey Lushnikov
aba61de905 Make helper.getExceptionMessage synchronous
We now have description of an exception, no need for a roundtrip
to the backend.
2017-07-21 11:57:25 -07:00
Andrey Lushnikov
52de75742b Implement visible option for Page.waitFor method
This patch adds a 'visible' option to the Page.waitFor method, making
it possible to wait for the element to become actually visible.

References #89, #91.
2017-07-21 10:01:19 -07:00
Andrey Lushnikov
139b9e9b6d Get rid of page.emulate() / page.emulatedDevices() methods
This patch:
- gets rid of `page.emulate` and `page.emulatedDevices`
  methods. Instead, it is suggested to use `page.setViewport()`
  and `page.setUserAgent()` methods.
- moves DeviceDescriptors to the top level of the puppeteer so that
  it is convenient to require them.
- improves on documentation to describe the suggested emulation
  approach.

References #88.
2017-07-20 23:53:06 -07:00
Andrey Lushnikov
76ac3bded5 Convert DevicesDescriptors into puppeteer format
This patch converts lib/DevicesDescriptors from a devtools front-end
format into a puppeteer format.

This patch does this via introducing a scripts utils/fetch_devices.js
which grabs devices from upstream of DevTools Front-end and
converts them into puppeteer devices.

References #88.
2017-07-20 17:49:15 -07:00
Andrey Lushnikov
42edc3108a [emulation] fix touch emulation
This patch:
- fixes touch emulation
- adds tests to cover basic Page.emulate

References #88.
2017-07-20 12:22:34 -07:00
Andrey Lushnikov
4af0911b90 Use more good flags to run chrome instance.
The switches are mostly borrowed from chromedriver launching code.
2017-07-20 12:20:10 -07:00
Andrey Lushnikov
a981594b1d Revert node's scrolling into view in page.click()
This change was not intentional in e33a8f818c
2017-07-20 00:53:14 -07:00
Andrey Lushnikov
e33a8f818c Support HEADLESS env variable for unit tests
This patch makes it possible to run unit tests in non-headless
mode with the following command:

```
HEADLESS=false npm run unit
```
2017-07-19 19:36:22 -07:00
Andrey Lushnikov
a63a0198de Implement waitFor which survives navigation (#99)
This patch implements page.waitFor method which survives navigation.

References #89.
2017-07-19 19:04:51 -07:00
JoelEinbinder
febd747c5b Inroduce page.press (#96)
This patch:
- introduces page.press() method
- adds more input tests

References #89
2017-07-19 14:43:07 -07:00
JoelEinbinder
71f8c76f04 Rename keyboard.hold and release to up and down (#95)
References #89
2017-07-19 14:27:01 -07:00
Andrey Lushnikov
21af495b65 Move screenshot task chain in Browser
Currently, it's impossible to do screenshots in parallel.
This patch:
- makes all screenshot tasks sequential inside one browser
- starts activating target before taking screenshot
- adds a test to make sure it's possible to take screenshots across
  tabs
- starts waiting for a proper page closing after each test. This might
  finally solve the ECONNRESET issues in tests.

References #89
2017-07-19 14:15:16 -07:00
Andrey Lushnikov
2e94f9f67b Resolve pending callbacks when connection closes
This patch:
- resolves pending callbacks in connection once it gets closed
- fixes one of Page.screenshot() tests to wait for all the screenshots
- starts handling ECONNRESET error in test server to avoid throwing with no
  good reason
2017-07-19 01:00:02 -07:00
Andrey Lushnikov
55acae40fd Introduce DEBUG module which traces public API calls
This patch improves on DEBUG module to trace all puppeteer's
public API calls.

References #89.
2017-07-18 21:06:03 -07:00
Pavel Feldman
f154d537f7 Introduce page.goBack/page.goForward (#93)
This patch introduces page.goBack/page.goForward methods
to navigate the page history.

References #89.
2017-07-18 19:11:37 -07:00
Pavel Feldman
98c3894c84 Introduce Page.waitForNavigation (#94)
This patch introduces Page.waitForNavigation which allows to wait
for render-initiated navigation.

This patch also does a nice refactoring, replacing Navigator with NavigatorWatcher which
is not a part of a page state.

References #89
2017-07-18 18:54:24 -07:00
Pavel Feldman
bef9982687 Simplify waitFor code (#92) 2017-07-18 12:16:19 -07:00
JoelEinbinder
28265fc313 Fix frame.waitFor to work with complex selectors (#90)
The frame.waitFor didn't account for multi-level selectors.
This patch fixes this, executing document.querySelector on every DOM mutation.
2017-07-18 02:10:02 -07:00
Andrey Lushnikov
6c1c3a0c45 Return remote object description for unserializable objects
This patch starts handling unserializable objects by returning their
description.

Fixes #86.
2017-07-18 01:34:43 -07:00
Andrey Lushnikov
2ca08b032b Rename Page's 'consolemessage' event into 'console'
This patch:
- renames 'consolemessage' event into 'console'
- improves on 'console' event documentation

References #39.
2017-07-17 20:38:11 -07:00
Andrey Lushnikov
1a97d8b3c2 Teach 'consolemessage' event to send all the arguments
This patch fixes 'consolemessage' event so that it passes
over all the arguments of console API call.
2017-07-17 20:22:45 -07:00
JoelEinbinder
117a128b42 Introduce Page.$ and Page.$$ (#75)
This patch introduces Page.$ and Page.$$ methods which are
aliases for `document.querySelector` and `document.querySelectorAll`. 

Fixes #78.
2017-07-17 18:56:56 -07:00
JoelEinbinder
bf7698e8f8 Intorduce Page.keyboard (#74)
Introduce page.keyboard to provide low-level access to the keyboard.
2017-07-17 18:49:52 -07:00
Pavel Feldman
895f69d17a Add emulation for named devices. (#72)
This patch introduces page emulation, making it possible to emulate different devices.
2017-07-17 18:13:04 -07:00
Andrey Lushnikov
0414dfa98b Introduce DEBUG module
This patch re-introduces the DEBUG module to expose some of
the puppeteer's internals.

Currently, only the protocol message communication is exposed under
the 'puppeteer:protocol' namespace.
2017-07-17 15:15:07 -07:00
Pavel Feldman
4581ada210 Roll chromium to r486981. (#87)
This patch rolls chromium to r486981 and prepares
for the introduction of device emulation.
2017-07-17 12:15:06 -07:00
Andrey Lushnikov
b2d2bf822a Rename Page.printToPDF into page.pdf
This patch:
- renames Page.printToPDF into page.pdf
- adds a 'path' option to the page.pdf options instead of a separate
  `filePath` parameter
- improves on the documentation for the `page.pdf`

References #39.
2017-07-17 10:37:33 -07:00
Aleksey
ac75455983 Fixed page.waitFor (#81)
The original implementation didn't account for frames.
2017-07-14 15:55:38 -07:00