Commit Graph

641 Commits

Author SHA1 Message Date
Andrey Lushnikov
2becd156d0
feat(Chromium): roll Chromium to r518818 (#1455)
This roll includes:
- crrev.com/518362 DevTools: override referrer policy for the request if
  referer header is overriden

Fixes #1373.
2017-11-23 07:42:48 +03:00
Trent Willis
770c17b2ea feat(Frame): implement frame.content and frame.setContent methods (#1447)
This refactors the page.content and page.setContent methods to be defined on the Frame class. This allows access from the Page still but also on all frames.

Fixes #754
2017-11-23 05:44:33 +03:00
Andrey Lushnikov
88eaede5ad
fix(page): fix "timeout: 0" to actually disable any navigation timeout (#1435)
Since non-promise values always win the `Promise.race`, we shouldn't
return `null` for timeout promise in NavigationWatcher.

Instead, we can return a promise that never resolved. It should be
GC'd later with the navigation watcher itself.

Fixes #1417.
2017-11-21 08:21:25 +03:00
Andrey Lushnikov
cafd040bf2
fix(Network): be prepared to miss requestWillBeSent events (#1420)
With the addition of `browser.targets()` api, we now can connect to
in-flight targets.

For Puppeteer, it means that it can "miss" certain events happenning
while it wasn't attached to the target.

This patch:
- fixes this problem with NetworkManager, preparing it for the
  missed `requestWillBeSent` event.
- adds a new test to ensure that not a single unhandled promise
  rejection has happened during test execution.

Fixes #1363.
2017-11-20 15:59:07 -08:00
Andrey Lushnikov
ea70ac9003
fix(Page.goto): check navigation completeness on frame detachment (#1419)
Currently, the NavigatorWatcher ensures that all the frames in the
frame subtree reach the desired lifecycle events state.

For this to be fully correct, NavigatorWatcher should also check
navigation status whenever frames are detached.
2017-11-20 14:47:11 -08:00
Andrey Lushnikov
6512ce768d
fix(Frame): postpone evaluations until execution context gets created (#1415)
In Blink, frames don't necesserily have execution context all the time.
DevTools Protocol precisely reports this situation, which results in
Puppeteer's frame.executionContext() being null occasionally.

However, from puppeteer point of view every frame will have at least a
default executions context, sooner or later:

- frame's execution context might be created naturally to run frame's
  javascript
- if frame has no javascript, devtools protocol will issue execution
  context creation

This patch builds up on this assumption and makes frame.executionContext()
to be a promise.
As a result, all the evaluations await for the execution context to be created first.

Fixes #827, #1325

BREAKING CHANGE: this patch changes frame.executionContext() method to return a promise.
To migrate onto a new behavior, await the context first before using it.
2017-11-18 16:27:52 -08:00
Andrey Lushnikov
90ca007367
fix(Launcher): handle SIGHUP signal (#1405)
The SIGHUP signal is sent whenever the controlling terminal is closed.

On Windows, SIGHUP is emulated by libuv, and will be the only signal we
receive before the application will be terminated.

This patch starts handling SIGHUP in the same way we handle SIGTERM.

Fixes #1367.
2017-11-16 19:28:32 -08:00
Octavian Cioaca
a350e89381 chore(Browser): fix jsdoc in Browser.create method (#1403) 2017-11-16 16:28:52 -08:00
Andrey Lushnikov
cc0c461ea8
fix(Launcher): handle SIGTERM by default (#1402)
SIGTERM signal is widely used to notify application that it will be shut down.

This patch starts listening to SIGTERM event to gracefully retire
chromium instance.

References #1047.
2017-11-16 14:26:32 -08:00
Christopher Dieringer
b0b22199f9 fix(ElementHandle.screenshot): account for scroll (#1323)
Currently, `elementHandle.screenshot` passes bounding box into the `page.captureScreenshot` 
method.
This is wrong since `captureScreenshot` accepts viewport, not bounding box.

This patch uses layout metrics to convert bounding box into viewport.

Fixes #1315.
2017-11-10 16:02:52 -08:00
Andrey Lushnikov
2c8f658815
fix(Navigation): wait for lifecycle events for the frame subtree (#1356)
Currently, we wait only for the main frame to reach the desired
lifecycle state.

This patch starts waiting until all the frames reach the desired
lifecycle state.

Fixes #1173.
2017-11-10 15:44:14 -08:00
JoelEinbinder
f8d19e79e7 fix(Page.waitForSelector): "visible" option should check parent visibility (#1354)
This patch starts checking for boundingClientRect to make sure that the
element is visible.
2017-11-10 15:44:01 -08:00
Andrey Lushnikov
44d1e834a4
fix(Navigation): correctly wait for navigation in Page.goto (#1355)
This patch:
- starts persisting lifecycle state for every frame
- migrates NavigationWatcher to rely on these lifecycle events
- refactors Page.goto to properly return navigation errors

Fixes #1218.
2017-11-10 15:33:14 -08:00
Alix Axel
7d18275fb9 Test: add tests to cover clicking checkbox (#1125)
This patch adds tests to make sure clicking both checkbox and its label
works.

These are regression tests to cover the upstream fix, rolled in #1299.
2017-11-07 20:38:22 -08:00
Andrey Lushnikov
cbe3dc58a2
Revert "feat(Page): teach Page.setContent to wait for resources to load (#1152)" (#1312)
This reverts commit 80ee469429.

Lifecycle events are not ready yet to support the setContent scenario.
The prerequisite for this is
https://chromium-review.googlesource.com/c/chromium/src/+/747805 that
might not land soon due to technical concerns.
2017-11-07 14:18:05 -08:00
JoelEinbinder
b58d319926 fix(Page.click): throw a meaningful message for invisible elements (#1309)
This patch starts throwing a meaningful error message when trying to click the hidden
node.

References #1294
2017-11-07 13:54:40 -08:00
JoelEinbinder
3cb0f1af34 fix: make exposeFunction work for frames (#1295)
This patch fixes page.exposeFunction method so that it
adds exposed binding to all existing frames.

Fixes #690
2017-11-07 13:26:23 -08:00
Schelkun
77f585298f feat(Page): introduce Page bringToFront method (#1252)
This patch introduces Page.bringToFront method to activate tabs.

Fixes #1244
2017-11-07 13:17:36 -08:00
Robert Sköld
cc5e8a9bd4 fix(Page.type): Add assertion to page.type() (#1301)
This patch starts throwing a nice error when `page.type` is called
with non-existent selector.
2017-11-07 03:07:51 -08:00
Xingan Wang
8e445734c6 feat(Frame): reject error for addScriptTag/addStyleTag (#1287)
fix #1221
2017-11-06 22:04:40 -08:00
Andrey Lushnikov
5e154dc835
fix(Page.select): assert all values are of type "string". (#1280)
This patch starts asserting that all values are of type "string".

The alternative approach to cast values to strings
might yield a hard-to-debug errors.

Fixes #1276.
2017-11-03 19:20:12 -07:00
Adi Prasetyo
03fefb53f8 feat(browser): add browser.disconnected event (#960)
This patch adds a 'disconnected' event for the browser. The event is issued
when the "browser" loses connection to the chrome process.

References #952
2017-11-03 18:46:17 -07:00
Andrey Lushnikov
88ba52a363
refactor: use Page.getFrameTree instead of Page.getResourceTree (#1275)
This patch starts using Page.getFrameTree instead of
Page.getResourceTree.

The resource tree is experimental, whereas the frame tree is not.
2017-11-03 18:37:21 -07:00
Xingan Wang
e0f5b93923 feat(Frame): addStyleTag and addScriptTag now return ElementHandles.
This patch teaches `page.addStyleTag` and `page.addScriptTag` to return ElementHandles
to the added tags.

Fixes #1179.
2017-11-03 00:05:38 -07:00
Andrey Lushnikov
ac55e69676
feat(Chromium): roll chromium to r513435. (#1259)
This roll brings in a bunch of important patches:
- crrev.com/512647 Changed headless browser profile dir to use Default profile path
- crrev.com/512760 DevTools: stop idleness detector when pending navigation commits
- crrev.com/512905 DevTools: introduce Page.getFrameTree
- crrev.com/513373 DevTools: report loaderId in the lifecycle events
- crrev.com/513419 DevTools: introduce Page.setLifecycleEventsEnabled
- crrev.com/513422 DevTools: return loaderId from Page.navigate

Fixes #921 

BREAKING CHANGE:

Headless user profile structure is changing. Custom profiles set with --user-data-dir flag will no longer be read in Chrome 63 and will have to be recreated.

Alternatively, you can migrate old headless profile to a new structure. if you stored your profile in `<profile>` folder, you would run the following bash commands:

```bash
cd <profile>
mkdir Default
mv * Default
```

Full headless-dev PSA announcement: https://groups.google.com/a/chromium.org/forum/#!msg/headless-dev/asX8WgktXIE/zTUfmHDcAQAJ
2017-11-02 13:26:21 -07:00
Suvrat Jain
fede2643ac feat(Frame): introduce Frame.select
This patch adds `Frame.select` method that does the same functionality as
former `Page.select`, but on a per-frame level.

The `Page.select` method becomes a shortcut for the ÷main frame's select.

Fixes #1139
2017-11-01 22:06:04 -07:00
Andrey Lushnikov
f5bb333cd0
fix: support PlzNavigate in puppeteer. (#1239)
This patch migrates puppeteer to support PlzNavigate chromium
project.

As a consequence of this patch, we no longer wait for both
requestWillBeSent and requestIntercepted events to happen. This should
resolve a ton of request interception bugs that "hanged" the loading.

Fixes #877.
2017-11-01 14:04:10 -07:00
Andrey Lushnikov
b9266c74f8
feat(Page): rename Page.getMetrics into Page.metrics (#1240)
BREAKING CHANGE: method page.getMetrics() got renamed into
page.metrics().
2017-11-01 13:39:31 -07:00
Andrey Lushnikov
9f071bf411
fix(Navigation): do not race with security error for navigation (#1237)
Currently, NavigationWatcher listens to lifecycle events from Page
domain and security events from Security domain.

However, the events are dispatched from different processes in browser:
- Page's lifecycle events are dispatched from renderer process
- Security events are dispatched from browser process

This makes for the undefined order between events and results in
NavigationWatcher reporting different failuer messages, based on
the event order.

This patch stops relying on security errors in navigation watcher and
instead switches to request failure codes for the main resource.

Fixes #1195
2017-11-01 13:28:00 -07:00
AlexChung1995
e70f98ddb9 feat(Page.select): return selected options from Page.select (#1099)
This patch teaches Page.select to return an array of actually selected options.
If no option is selected, an empty array will be returned.
2017-10-31 21:47:52 -07:00
Manoj Patel
9f19641a3d chore: Fix typo in comment for Events (#1234) 2017-10-31 16:10:18 -07:00
Andrey Lushnikov
c9a26e11f1
fix(ElementHandle): teach ElementHandle to work with shadowdom (#1227)
Elements in shadow dom erroneously considered that they were detached
from document.

This patch starts using `Element.isConnected` instead of
`document.contains()` call.

Fixes #1061.
2017-10-31 12:02:16 -07:00
Adam Stankiewicz
5ffbd0d221 feat(ElementHandle): add ElementHandle.$ and ElementHandle.$$ (#1151)
This patch adds `ElementHandle.$` and `ElementHandle.$$` methods to query nested
elements.

Fixes #508
2017-10-27 02:08:58 -07:00
JoelEinbinder
655b94db47 fix(touch): requestAnimationFrame before sending tap (#1142)
This patch adds a work-around for the upstream bug, issuing double-raf before emulating touch event.

References #673
2017-10-26 14:53:50 -07:00
Andrey Lushnikov
ab9b34cd5d fix(Launcher): force-kill chrome on process exit and interruption (#1155)
This patch starts force-killing chromium regardless of custom userdata
directory when the node process exits.

References #1047
2017-10-24 16:05:12 -07:00
Andrey Lushnikov
b9ab6fe4bb fix(interception): Disable newtork caching when intercepting (#1154)
Request interception might not work properly if caching is enabled.
2017-10-24 15:14:25 -07:00
Andrey Lushnikov
ce005d480c feat(Chromium): Roll chromium to r511134 (#1153)
This roll includes:
- crrev.com/510651 that changes request interception methods in protocol
- s/Page.setRequestInterceptionEnabled/Page.setRequestInterception

BREAKING CHANGE

Page.setRequestInterceptionEnabled is renamed into
Page.setRequestInterception.
2017-10-24 14:45:03 -07:00
Andrey Lushnikov
80ee469429 feat(Page): teach Page.setContent to wait for resources to load (#1152)
This patch adds "options" parameter to the `page.setContent` method. The
parameter is the same as a navigation parameter and allows to specify
maximum timeout to wait for resources to be loaded, as well as to
describe events that should be emitted before the setContent operation
would be considered successful.

Fixes #728.
2017-10-24 13:57:39 -07:00
Andrey Lushnikov
f38c8bb17b feat(Page.goto): support multiple events in waitUntil option (#1147)
This patch adds support to multiple events that could be passed inside
navigation methods:
- Page.goto
- Page.waitForNavigation
- Page.goForward
- Page.goBack
- Page.reload

Fixes #805
2017-10-24 10:05:15 -07:00
Andrey Lushnikov
ef369e5d5d feat(Page.goto): add a 'domcontentloaded' option to the Page.goto (#1145)
This patch adds a new `domcontentloaded` option to a bunch of navigation
methods:
- Page.goto
- Page.waitForNavigation
- Page.goBack
- Page.goForward
- Page.reload

Fixes #946.
2017-10-24 00:35:18 -07:00
Andrey Lushnikov
ce8a952044 refactor: migrate NavigatorWatcher to lifecycle events (#1141)
This patch:
- migrates navigation watcher to use protocol-issued lifecycle events.
- removes `networkIdleTimeout` and `networkIdleInflight` options for
  `page.goto` method
- adds a new `networkidle0` value to the waitUntil option of navigation
  methods

References #728.

BREAKING CHANGE:

As an implication of this new approach, the `networkIdleTimeout` and
`networkIdleInflight` options are no longer supported. Interested
clients should implement the behavior themselves using the `request` and
`response` events.
2017-10-23 18:10:59 -07:00
JoelEinbinder
126ab7b90e feat(keyboard): Accept codes (#1116)
BREAKING CHANGE:

This patch lets key names be code in addition to key. When specifying a code, the proper text is generated assuming a standard US keyboard layout. e.g Digit5 -> "5" or "%" depending on Shift.

* location is now specified. #777

* Using unknown key names now throws an error. #723

* Typing newlines now correctly presses enter. #681
2017-10-23 12:43:45 -07:00
Andrey Lushnikov
bcc969ccc4 feat(interception): Implement request.mockResponse method (#1064)
feat(interception): Implement request.respond method

This patch implements a new Request.respond method. This
allows users to fulfill the intercepted request with a hand-crafted
response if they wish so.

References #1020.
2017-10-20 16:55:15 -07:00
Andrey Lushnikov
5c92ba222a fix(page.evaluate): jsonValue should ignore toJSON property (#1098)
Currently, JSHandle.jsonValue() is implemented as in-page JSON.stringify
call and consequent JSON.parse in node. This approach proved to be
unfortunate for automation purposes: if page author overrode the
Object.prototype.toJSON method, then it's harder for puppeteer to
interact with the page.

This patch switches JSHandle.jsonValue to use protocol serialization
that ignores toJSON property. THis also changes the `page.evaluate`
behavior since it is based on JSHandle.jsonValue().

Fixes #1003.

BREAKING CHANGE:
`page.evaluate` no longer calls toJSON when generating return value.
For the old behavior, do JSON.parse/JSON.stringify manually:

```js
const json = JSON.parse(await page.evaluate(() => JSON.stringify(obj)));
```
2017-10-20 10:45:49 -07:00
Andrey Lushnikov
4f64dfd993 feat(Request): allow aborting intercepted requests with custom reasons (#1080)
This patch adds optional parameter to the `request.abort()` method
that specifies abort reason.

References #1020.
2017-10-18 00:26:48 -07:00
Christian Davis
b3857bd690 fix(page): page.close() assert that connection is not closed (#1038) 2017-10-17 19:24:34 -07:00
JoelEinbinder
32398d11bd feat(Browser): introduce Browser.pages() (#554)
This patch:
- introduces Target class that represents any inspectable target, such as service worker or page
- emits events when targets come and go
- introduces target.page() to instantiate a page from a target

Fixes #386, fixes #443.
2017-10-17 19:14:57 -07:00
Andrey Lushnikov
273c733237 feat(Request): introduce Request.failure() method (#1065)
Similarly to the `request.response()` method, this patch adds
`request.failure()` method that returns error details for the failed
requests.

Fixes #901.
2017-10-17 17:48:04 -07:00
JoelEinbinder
2b7951473d feat(Browser): make browser.close() to always terminate remote browser
This patch:
- changes `browser.close` to terminate browser.
- introduces new `browser.disconnect` to disconnect from a browser without closing it

This patch: fixes #918, fixes #989 

BREAKING CHANGE:
`browser.close()` will always close a browser, even if it was initialized with
`puppeteer.connect`. To disconnect from a remote browser, use `browser.disconnect()` instead.
2017-10-17 15:35:00 -07:00
JoelEinbinder
6a8865cd85 feat(keyboard): make keyboard.down generate input event by default (#1016)
This patch starts generating input events for `keyboard.down`, addressing bullet 2 of
#723. With this patch, there's no longer any difference between `keboard.press('a')` and
`keyboard.press('a', {text: 'a'})`.

BREAKING CHANGE:
`keyboard.down('a')` starts generating input event (wasn't the case before).

References #723
2017-10-13 15:22:55 -07:00
Andrey Lushnikov
a02347e3ef feat(Page): implement page.setOfflineMode (#1032)
This patch implements page.setOfflineMode method.

Fixes #63.
2017-10-13 14:41:39 -07:00
Christian Davis
8511db957f feat(page): waitForSelector hidden option (#967)
This patch adds a 'hidden' option for the `page.waitForSelector` method.
2017-10-13 09:11:11 -07:00
JoelEinbinder
8f430fb23d fix(ElementHandle.boundingBox): account for margin, padding, and border in element.boxModel (#1017)
This patch starts relying on border quad to compute element's bounding box.

Fixes #1010
2017-10-12 17:53:59 -07:00
Sam Verschueren
c310f139a0 feat(Frame): Support options in addScriptTag and addStyleTag (#996)
This patch:
- deprecates injectFile as it was confused with the addScriptTag
- accepts an options object in addScriptTag which supports properties url, path and content.
- accepts an options object in addStyleTag which supports properties url, path and content.

Fixes #949.

BREAKING CHANGE:
- the addStyleTag/addScriptTag have changed;
- the injectFile was removed in favor of (addStyleTag({path:}).
2017-10-12 01:26:44 -07:00
JoelEinbinder
0426e3c068 Roll chromium to 508268 (#1019)
This gets us:

- Plural metrics names https://crrev.com/507885
- TargetInfoChanged on URL changes https://crrev.com/508049
- Sending proper resource types for requests https://crrev.com/667504

Fixes #734
2017-10-12 01:17:06 -07:00
JoelEinbinder
c893bf3684 fix(input) Send .code with keyboard events (#1015)
This patch starts sending proper `.code` field with emulated keyboard events.

References #777.
2017-10-11 18:09:43 -07:00
Andrey Lushnikov
23c0ba0727 feat(Page): introduce Page.queryObjects (#1005)
This patch introduces `Page.queryObjects` and
`ExecutionContext.queryObjects` methods to query JavaScript heap
for objects with a certain prototype.

Fixes #304.
2017-10-11 14:41:20 -07:00
Barry vd. Heuvel
3f9f0f44ff feat(Page.pdf): add 'A6' page format
This patch adds 'A6' page format option to the Page.pdf method.
2017-10-11 13:18:14 -07:00
Gleb Azarov
ff08e45785 fix: Remove synchronous operation with file system. (#879)
This patch:
- introduces `helper.promisify` - a simple polyfill for the `util.promisify`. The 
  `util.promisify` could not be used due to Node6 compatibility issues.
- migrates all sync filesystem operations to the async replicas

Fixes #884.
2017-10-11 00:55:48 -07:00
Andrey Lushnikov
7a8aa73466 feat(Page): introduce Page.$$eval method (#1006)
This patch adds a `Page.$$eval` method that runs `document.querySelectorAll`
and passes resulting array to the page function.

Fixes #625.
2017-10-10 23:23:14 -07:00
Andrey Lushnikov
464b6a9616 fix(Page.getMetrics): do not report a few metrics (#1001)
A few of the reported metrics in the Page.getMetrics are already
reported with lifecycle events.

This patch excludes these metrics for now.
2017-10-10 15:22:14 -07:00
Alexei Filippov
b82d3197ed feat(Page): Support Page.getMetrics and metrics event. (#939)
Provides access to the current page performance metrics.
Allows to push page metrics from the page JavaScript with console.timeStamp()

Fixes #309
2017-10-10 14:50:38 -07:00
Andrey Lushnikov
1fbf2a8a46 feat(Page): kill the page.plainText method (#994)
The page.plainText is confusing: it's unclear what kind of text it
returns, textContent or innerText. It's also easily polyfillable and
doesn't seem to be used.

BREAKING CHANGE: the page.plainText is not existing any more.
Instead, use `page.evaluate(() => document.body.innerText)`.
2017-10-10 10:54:51 -07:00
Andrey Lushnikov
079db90066 fix(JSHandle.toString): clearer description for primitives (#993)
This patch:
- updates JSHandle.toString to make a nicer description for primitives
- excludes JSHandle.toString from documentation to avoid its abuse

References #382
2017-10-10 10:54:20 -07:00
Andrey Lushnikov
c3fb367148 fix(Request): convert resourceType to all small-caps (#990)
This patch moves resourceType to be all small-caps. This aligns
with our convention that all string constants should be smallcaps.

BREAKING CHANGE: this patch changes the constants of the
request.resourceType to be all small-caps.
2017-10-10 10:53:37 -07:00
Eli Sherer
7e28dbafb5 feat(ElementHandle): add EH.boundingBox and EH.screenshot
This patch:
- adds `ElementHandle.boundingBox()` method to get bounding box of element relative
  to the page
- adds `ElementHandle.screenshot()` method to capture a screenshot of an element
2017-10-09 23:14:09 -07:00
JoelEinbinder
e59172de83 chore: Use Typescript to lint JSDoc annotations (#986)
This patch starts using typescript to lint JSDoc annotations.

Note: this uses typescript's bleeding edge. We should migrate to stable once
it has all the necessary bugfixes.

References #65.
2017-10-09 22:31:40 -07:00
Vse Mozhet Byt
f1aa18af4e feat(Launcher): add devtools option (#953)
This patch adds a `devtools` option to the launcher that adds the `--auto-open-devtools-for-tabs`
argument to the launched chrome.

Fixes #864.
2017-10-09 17:25:25 -07:00
Andrey Lushnikov
e6af6e19d6 feat(console): dispatch JSHandles as console arguments (#975)
This patch starts dispatching JSHandle instances as console arguments.

BREAKING CHANGE: this changes the API of the ConsoleMessage.

Fixes #324.
2017-10-09 17:01:01 -07:00
JoelEinbinder
52f92c9891 fix(input): clicking an element should take into account frame position (#971) 2017-10-09 13:23:36 -07:00
JoelEinbinder
a7672acb85 fix: return undefined when Page.evaluate encounters circular JSON (#983) 2017-10-09 12:30:05 -07:00
Christian Davis
3ecd98d634 fix(Page.select): synthesized events should bubble
This patch fixes `Page.select` to synthesize bubbling events.
2017-10-07 10:27:38 -07:00
JoelEinbinder
0af0d7dba5 feat(Input): Add keyboard methods to elementHandle (#801)
This patch:
- adds input methods to ElementHandle, such as ElementHandle.type and ElementHandle.press
- changes `page.type` to accept selector as the first argument
- removes `page.press` method. The `page.press` is rarely used and doesn't operate with selectors; if there's a need to press a button, `page.keyboard.press` should be used.

BREAKING CHANGE: `page.type` is changed, `page.press` is removed.

Fixes #241.
2017-10-07 00:28:24 -07:00
Andrey Lushnikov
0d0f9b7984 feat(JSHandles): introduce JSHandles (#943)
This patch:
- introduces ExecutionContext class that incapsulates javascript
  execution context. An examples of execution contexts are workers and
  frames
- introduces JSHandle that holds a references to the javascript
  object in ExecutionContext
- inherits ElementHandle from JSHandle

Fixes #382.
2017-10-06 15:35:02 -07:00
Benoît Zugmeyer
c225b93037 feat(Launcher): Allow environment variables definition when launching chromium (#912)
This patch adds `env` option to the `puppeteer.launch` method to define custom environment
variables to the launched chrome.
2017-10-05 14:34:35 -07:00
Sam Verschueren
97e40e6823 feat(Frame): Add addStyleTag API to frame - fixes #892 (#947)
This patch adds `Page.addStyleTag` and `Frame.addStyleTag` methods to the API.

Fixes #892.
2017-10-04 13:42:26 -07:00
mizdra
215b3497c7 fix(Page): remove event listeners to avoid MaxListenersExceededWarning
This patch starts to properly clean up event listeners on a failed navigation (Page.goto).
2017-09-30 00:50:13 -07:00
Adi Prasetyo
53531c9a92 feat(page): Allow Page.goto's timeout to be 0 to disable timeout (#887)
This patch allows passing 0 to disable timeout for the following methods:

- page.goto
- page.waitForNavigation
- page.goForward
- page.goBack

Fixes #782.
2017-09-30 00:29:38 -07:00
Pavel Feldman
a6cf8237b8 feat(Launcher): introduce appMode option
The patch introduces a new appMode option to the launcher. This is an experimental
option, thus we don't document it.
2017-09-29 15:16:32 -07:00
Andrey Lushnikov
c46c41d89c fix(Launcher): Launcher should handle chrome process errors (#863)
Fixes #750
2017-09-29 12:21:24 -07:00
Andrey Lushnikov
cb280c526d fix(Launcher): Terminate chrome gracefully on Windows (#876)
This patch starts using taskkill program on windows to gracefully
terminate chrome.

Note: this slows down chrome shutdown on Windows in case of using
custom userDataDir. This is because chrome takes some time to shutdown
its operations and leave profile directory in a consistent state.

Fixes #839.
2017-09-29 11:41:11 -07:00
Andrey Lushnikov
f6255029bd feat(Console): Introduce ConsoleMessage type (#909)
This patch introduces ConsoleMessage type and starts dispatching
it for the 'console' event.

BREAKING CHANGE: this breaks the api of the 'console' event.

Fixes #744.
2017-09-29 11:27:22 -07:00
Andrey Lushnikov
cfece3451d fix(Network): Do not attempt to normalize malformed URLs. (#875)
This patch avoids throwing 'url malformed' error during generating
request hash for request interception.

Fixes #869.
2017-09-29 08:29:19 +09:00
Andrey Lushnikov
72161c475d [roll] Roll chromium to r503964 (#878)
This patch rolls chromium to r503964.

Note: since the plznavigate is not supported by puppeteer right now, the
patch also starts passing the `--disable-browser-side-navigation` flag.
This is a temporary work around for us.

References #877.
2017-09-27 10:37:31 +09:00
Alix Axel
45f264024b [api] Introduce Page.select method (#779)
This patch adds `Page.select` method to select
values in a `select` tag.
2017-09-25 18:23:34 +09:00
JoelEinbinder
e2cad568d6 page.waitFor should send ...args to page.waitForFunction (#775)
This lets the user pass `...args` into `page.waitFor`. It also clarifies that the docs that `options` is not optional if `...args` are specified.

Fixes #770
2017-09-15 14:28:15 -07:00
Andrey Lushnikov
d562db3140 Assert that all extra HTTP header values are strings (#781)
Since protocol ignores all HTTP headers that don't have string
value, this patch starts validating header key-values before
sending them over the protocol.

Fixes #713.
2017-09-14 19:08:48 -07:00
Andrey Lushnikov
86b05dadd5 disable extensions and google translate by default (#774)
This patch teaches `puppeteer.launch` to run chromium with disabled
chrome extensions and google translate services.

References #760
2017-09-14 19:07:22 -07:00
JoelEinbinder
f398e69dbb [api] Launcher: Close gracefully when a userDataDir is specified (#700)
This patch:
- makes `browser.close()` return a promise that resolves when browser gets closed
- starts closing chrome gracefully if a custom `userDataDir` is supplied

Fixes #527
2017-09-13 21:27:14 -07:00
Andrey Lushnikov
d7e673645a [api] add Puppeteer.executablePath() method (#771)
This patch adds Puppeteer.executablePath() method to query the path
of bundled chromium.

Fixes #745
2017-09-13 17:39:18 -07:00
Andrey Lushnikov
9292a56eaf [api] Teach page.evaluate to accept element handles as parameters (#725)
This patch:
- teaches `page.evaluate` to accept ElementHandles as parameters
- removes `ElementHandle.evaluate` method since it's not needed any
  more

References #382
2017-09-11 19:20:02 -07:00
Andrey Lushnikov
a71f287c6b Fix evaluation in case of first "undefind" argument (#732)
It turns out that [undefined, 1].join(',') results in ",1" instead
of "undefined,1". This causes a syntax error when trying to pass undefined
as a first argument to `page.evaluate` method.

Fixes #572.
2017-09-11 16:59:38 -07:00
Andrey Lushnikov
e5c17eecb9 Carefully manage unhandled rejections for navigation (#741)
Currently, navigation watcher throws exception if timeout
is exceeded.

Due to the way it is used in `page.navigate`, the promise
get's rejected before it is awaited, which is considered to
be "unhandled promise rejection".

Fixes #738
2017-09-11 16:43:37 -07:00
Andrey Lushnikov
0db6165d73 [api] Implement page.authenticate method (#729)
This patch implements `page.authenticate` which should cover all
cases of HTTP authentication.

Fixes #426.
2017-09-11 16:32:13 -07:00
Andrey Lushnikov
0bea42bd8c Do not leave dangling promises when sending messages over protocol (#742)
It's very bad to have 'unhandled promise rejection' that can't be
handled in user code. These errors will exit node process in a near
future.

This patch avoids 'unhandled promise rejection' while sending protocol
messages.

This patch:
- introduces `puppeteer:error` debug scope and starts using it for all
  swalloed errors.
- makes sure that every `client.send` method is either awaited or its
  errors are handled.
- starts return promises from Request.continue() and Request.abort().
- starts swallow errors from Request.contine() and Request.abort().

The last is the most important part of the patch. Since
`Request.continue()` might try to continue canceled request, we should
disregard the error.

Fixes #627.
2017-09-11 16:21:51 -07:00
JoelEinbinder
64124df62f [api] add touchScreen.tap (#639)
This patch:
- adds `page.touchscreen` namespace, similar to `page.mouse` and `page.keyboard`.
- adds tapping to multiple layers:
  - `page.touchscreen.tap`
  - `page.tap` - convenience method which accepts selector
  - `elementHandle.tap`

Fixes #568 and #569.
2017-09-01 19:03:51 -07:00
JoelEinbinder
42fa3af659 Remove promise jsdoc (#641)
This patch removes unnecessary "promise" return types in jsdoc for async functions.
2017-08-31 17:58:07 -07:00
Andrey Lushnikov
62ececb1c7 Implement page.$eval (#638)
This patch:
- implements page.$eval and frame.$eval
- drops elementHandle.attribute() method in favor of the page.$eval

References #625
2017-08-31 15:38:01 -07:00
Andrey Lushnikov
66676c0235 Kill child process group (#621) 2017-08-30 14:16:03 -07:00
Andrey Lushnikov
c9b0a6c060 Fix launcher to not leak event listeners (#605)
This patch fixes launcher to not leak event listeners.
2017-08-29 22:49:50 -07:00
Andrey Lushnikov
11ce8b249f Implement Request.resourceType getter (#590)
This patch plumbs `resourceType` parameter of RequestWillBeSent
and requestIntercepted methods.
2017-08-29 16:27:59 -07:00
JoelEinbinder
8f74cc8a90 [api] Add "step" option to mouse.move method (#601)
This patch adds "step" option to the mouse.move method, that optionally tweens mouse movement over multiple steps.

References #423.
2017-08-29 14:13:38 -07:00
Batiste Bieler
77600c6c5e Implement ElementHandle.attribute() method (#543)
This patch implements ElementHandle.attribute() method to fetch a value of
element's attribute.
2017-08-29 11:24:32 -07:00
Andrey Lushnikov
8f66f7339e Fix launcher in case of empty arguments (#582)
The regression was introduced in
d5327e6a0f
2017-08-28 14:59:41 -07:00
JoelEinbinder
d5327e6a0f [api] Allow a custom userDataDir to be specified (#555)
This patch adds a `userDataDir` option to the `puppeteer.launch()` method.

Fixes #411.
2017-08-28 12:14:20 -07:00
Andrey Lushnikov
66912a7277 [api] use object instead of map for headers (#567)
This patch:

- switches to objects instead of maps for headers (in Request, Response and
page.setExtraHTTPHeaders)
- converts all header names to lower case

Fixes #547, fixes #509
2017-08-28 12:09:24 -07:00
Andrey Lushnikov
e921a1cc21 Fix Request Interception (#565)
It turned out that either Network.requestIntercepted or
Network.requestWillBeSent occasionally report encoded URL.

This patch starts decoding URL's when generating request hash.

Fixes #558.
2017-08-27 12:41:09 -07:00
JoelEinbinder
2c4dfbfd88 Fix node6 support for Object.entries and URL object (#548)
This patch introduces a polyfill for Object.entries which is missing in Node 6
2017-08-25 15:13:59 -07:00
JoelEinbinder
0791774faa Cookies (#314)
Introduce cookies API. 

Fixes #53.
2017-08-24 12:21:46 -07:00
JoelEinbinder
a1414de42f Implement page.setJavaScriptEnabled method
Fixes #462.
2017-08-23 14:08:56 -07:00
Dan Schuman
8fe2477850 Enable MutationObserver attributes (#499)
This patch fixes page.waitForSelector to resolve on
attribute changes. 

Fixes #474.
2017-08-23 13:25:40 -07:00
JoelEinbinder
ac109dba6d Kill Chrome quickly on Windows (#488)
This patch start killing chrome on windows with taskkill command.
2017-08-23 11:55:33 -07:00
Andrey Lushnikov
151d512ae2 Implement page.$$ method (#463)
This patch implements page.$$ which runs document.querySelectorAll
in page and returns results as an array of ElementHandle instances.

Fixes #384.
2017-08-22 22:56:55 -07:00
Andrey Lushnikov
ce08fc77b1 Roll chromium to r496140 (#465)
This patch rolls chromium to r496140. This includes the r496130 that
introduces multiple sessions for single target.

With this patch, it is possible to run puppeteer in headful mode
and open devtools over the automated pages without puppeteer losing
connection to the page.
2017-08-22 14:15:02 -07:00
JoelEinbinder
11e3343b3d Put chrome profiles into a temporary directory (#439)
This patch:
- puts chrome profiles into a temporary directory
- starts using `fs.mkdtemp` to create temp profile directories
2017-08-21 20:12:16 -07:00
Eric Bidelman
1f9b4fb4c8 Change let into const (#457)
This patch:
- changes `let` into `const` throughout codebase
- adds eslint check to prefer const over let
2017-08-21 16:39:04 -07:00
Eric Bidelman
5d6d3e0a81 cleanup: Use ES6 default params (#447) 2017-08-21 16:32:39 -07:00
Andrey Lushnikov
271fd09379 Launcher: add timeout to chrome launching (#434)
This patch:
- adds a 'timeout' launcher option that constrains the time for chromium to launch.
- adds a 'handleSIGINT' launcher option that is `true` by default and that closes chrome instance

Fixes #363.
2017-08-21 15:43:36 -07:00
Andrey Lushnikov
afd90123be waitTask should survive cross-process navigation (#435)
In case of cross-process navigation, we receive a context
which is immediately getting destroyed.

Fixes #396.
2017-08-21 13:34:26 -07:00
Andrey Lushnikov
c1731dd5d7 Fail gracefully when chromium failed to download (#430)
Fail gracefully when chromium failed to download

This patch changes both install.js and Launcher.js to inform how
chromium could be downloaded manually.
2017-08-21 13:34:10 -07:00
John Resig
598f439a32 Add Page.content() method. Fixes #406. (#419) 2017-08-21 09:02:30 -07:00
Andrey Lushnikov
fcb8ab9cd7 Add troubleshooting.md (#428)
This patch adds troubleshooting.md and starts referring to it if puppeteer 
fails to launch chromium.
2017-08-20 19:46:13 -07:00
Andrey Lushnikov
6cc274135c Introduce page.emulateMedia method (#383)
Emulating `screen` media would force `page.pdf()` to print with `screen`
media.

Fixes #312
2017-08-18 16:49:02 -07:00
Andrey Lushnikov
d71feb63d0 Determine failed navigation based on Frame.unreachedURL (#365)
This patch starts using frame.unreachedURL property from
frameNavigated event to determine if the page navigation
was successful.
2017-08-17 23:24:16 -07:00
Andrey Lushnikov
ae14c1eee4 Use randomized folder names for user profile directories (#364)
This will make it much harder for puppeteer browsers to collide
on profile directories.

Fixes #360
2017-08-17 23:18:08 -07:00
Pavel Feldman
127f3966d7 Allow capturing transparent screenshots (#354)
This patch adds `omitBackground` option to the `page.screenshot` method.

Fixes #344
2017-08-17 21:11:39 -07:00
JoelEinbinder
1ca7849017 Mute audio in headless, closes #350 (#355)
This patch adds the `--mute-audio` flag if chromium is launched in headless
mode.
2017-08-17 20:54:16 -07:00
JoelEinbinder
5f0afd5547 Move uploadFiles onto ElementHandle (#348)
This patch:
- removes the `page.uploadFile` method
- adds `elementHandle.uploadFile` method.

Motivation: `elementHandle.uploadFile` is rarely used, so it doesn't worth it
to keep it on page.
2017-08-17 14:53:37 -07:00
Andrey Lushnikov
7cef3d7ce7 Teach page.pdf to respect printBackground argument (#307)
References #301
2017-08-16 15:08:31 -07:00
Jonathan Garbee
da53218107 Fix doc on pdf method. Minor addition to PDF path option docs. (#292) 2017-08-16 11:11:40 -07:00
Andrey Lushnikov
0a190ebec9 Rename 'remoteDebuggingURL' into 'browserWSEndpoint' (#272) 2017-08-16 01:10:55 -07:00
Andrey Lushnikov
44c15220f9 Rename ElementHandle.release into ElementHandle.dispose (#284)
The ElementHandle.release feels related to ElementHandle.click.
The dispose is a more clear name.
2017-08-16 00:49:20 -07:00
JoelEinbinder
3c2aaaaeb0 Readme fixup part 1 (#278) 2017-08-15 23:04:16 -07:00
Aleksey
56eab00fa7 Roll chromium to r494365 (#271)
This patch:
- rolls chromium to r494365
- starts using Runtime.evaluate(awaitPromise: true), with new semantic
  we can avoid additional Runtime.awaitPromise call
- stops resolving promises for Console event
2017-08-15 17:35:59 -07:00
Andrey Lushnikov
af89e893e7 Implement basic element handles (#248)
This patch implements basic element handles which a backed with remote objects.

Fixes #111
2017-08-15 14:54:02 -07:00
Andrey Lushnikov
a424f5613a Introduce Puppeteer.connect method (#264)
This patch:
- refactors Connection to use a single remote debugging URL instead of a
  pair of port and browserTargetId
- introduces Puppeteer.connect() method to attach to already running
  browser instance.

Fixes #238.
2017-08-15 14:29:42 -07:00
Andrey Lushnikov
96309a207c Fix request interception corner cases (#261)
This patch:
- teaches request interception to ignore data URLs. Currently protocol
  doesn't send interceptions for data URLs.
- teaches request interception to properly process URLs with hashes.
  Currently `Network.requestIntercepted` sends url with a hash, whereas
  `Network.requestWillBeSent` doesn't report hashes in its urls. @see
  crbug.com/755456
- skips one more header that I spotted during debugging interception on
  the realworld websites.

Fixes #258, #259.
2017-08-15 13:55:48 -07:00
Andrey Lushnikov
24764df4c4 Emit 'error' event when page crashes (#266)
This patch starts emitting 'error' event when page crashes.
'error' events have special treatment in node, so page crashes
become observable for users.

Fixes #262.
2017-08-15 11:13:05 -07:00
Andrey Lushnikov
0a1294c7ee general cleanup (#256)
This patch:
- fixes multimap implementation to work properly in node
- moves ESTreeWalker from third-party into utils/doclint. ESTreeWalker
  license is compliant with Apache2.0.
2017-08-14 21:16:59 -07:00
Andrey Lushnikov
13e8580a34 Use puppeteer.launch instead of browser constructor (#255)
This patch:
- split browser launching logic from Browser into `lib/Launcher.js`
- introduce `puppeteer` namespace which currently has a single `launch`
  method to start a browser

With this patch, the browser is no longer created with the `new
Browser(..)` command. Instead, it should be "launched" via the
`puppeteer.launch` method:

```js
const puppeteer = require('puppeteer');
puppeteer.launch().then(async browser => {
  ...
});
```

With this approach browser instance lifetime matches the lifetime of
actual browser process. This helps us:
- remove proxy streams, e.g. browser.stderr and browser.stdout
- cleanup browser class and make it possible to connect to remote
  browser
- introduce events on the browser instance, e.g. 'page' event. In case
  of lazy-launching browser, we should've launch browser when an event
  listener is added, which is unneded comlpexity.
2017-08-14 18:08:06 -07:00
Andrey Lushnikov
4269f6a1bb Roll chromium to r493985 (#251)
This patch:
- rolls chromium to r493985
- migrates Browser.version() puppeteer method to use newly added
  Browser.getVersion() protocol method

References #227
2017-08-12 22:04:39 -07:00
Andrey Lushnikov
3ee4951506 [DEBUG] add "session" namespace to trace target protocol messages (#249) 2017-08-11 17:47:33 -07:00
Jonathan Garbee
56b0d8722b Default value for property on dialog (#247) 2017-08-11 17:24:46 -07:00
Andrey Lushnikov
4551afc6dc Introduce new interception API (#242)
This patch introduces new interception API, via killing InterceptedRequest and giving the `abort` and `continue` methods to the Request object.
2017-08-11 17:24:31 -07:00
Andrey Lushnikov
dbac6788ae Plumb default prompt value to the dialog (#243)
This patch:
- plumbs default prompt value to the dialog
- enables one more phantom test
2017-08-11 12:17:43 -07:00
JoelEinbinder
497baf0198 Roll chromium to r493673 (#240)
Mouse events are no longer racy. Enabling touch no longer converts all mouse events into touches. Promises in destroyed execution contexts are rejected immediately.
2017-08-11 01:21:02 -07:00
Andrey Lushnikov
b8dbd28b8c Refer to the upstream bug (#239)
The issue #168 is a protocol inconsistency which happens only
in case of HTTPS error. This patch starts refering to the
upstream bug instead of puppeteer issue.

Closes #168.
2017-08-11 01:07:33 -07:00
Andrey Lushnikov
6347a049ba Rename Page.setInPageCallback into Page.addBinding (#236)
This seems to be a much better name which is actually used for a similar
purposes in chromium/v8.
2017-08-10 21:44:49 -07:00
Pavel Feldman
3a7741a172 Introduce page.emulate convenience wrapper (#231)
This patch:
- introduces Page.emulate convenience wrapper
- adds a screenshot-fullPage.js example
2017-08-10 18:42:30 -07:00
Andrey Lushnikov
e1d8a3781e Rename Page.navigate into Page.goto (#232)
Fixes #228.
2017-08-10 00:02:10 -07:00
Andrey Lushnikov
fe06c896eb Roll chromium to 492629 (#230)
This patch
- rolls chromium to 492629
- migrates connection establishing to use browser target. This migration means
  that now we have a single websocket connection to browser (implemented
  in Connection class). A connection to a particular target is
  incapsulated in a new Session class.
2017-08-09 16:14:00 -07:00
Andrey Lushnikov
34b0095c10 Make interception work with redirects (#218)
This patch:
- changes interception API so that it better aligns with what we'd like to see
  in #121
- fixes the issue with redirect interception

Fixes #217.
2017-08-07 17:48:52 -07:00
Andrey Lushnikov
e1c5b8d244 Aborted network requests should use protocol's "Failed" status (#215)
This patch starts using "Failed" command for request interception instead of
"Aborted".

The "Aborted" status also has a side-effect of cancelling the navigation, so
there will be no error on the page and form puppeteer's standpoint, the navigation
will never complete.
2017-08-07 15:13:17 -07:00
Andrey Lushnikov
7888a37a3d Page.pdf should accept case-insensetive page format (#211) 2017-08-04 01:01:10 -07:00
Andrey Lushnikov
00196e6cdc Fix setContent to actually await the inner evaluation (#209) 2017-08-04 00:23:04 -07:00
Pavel Feldman
82fbb268db Do not count inflight requests explicitly - use set. (#202) 2017-08-03 16:30:55 -07:00
Andrey Lushnikov
20ba447689 [DEBUG] Trace only those events which have listeners. (#201)
The `DEBUG=*page npm run unit` is too verbose due to events spamming
the console.

This patch starts tracing emitted events only if there are any
listeners.
2017-08-03 15:20:31 -07:00
Andrey Lushnikov
31224392bb Kill page.$ and page.$$ functions (#197)
page.$ and page.$$ conflicts with our ideas about element handles.

This patch removes functions in favor of future implementation.

References #111
2017-08-03 13:35:31 -07:00
Andrey Lushnikov
6c163122bf Update default viewport size to be 800px by 600px (#194)
This patch updates default viewport size to be 800px by 600px.
This comes handy when observing scripts running in non-headless mode.
2017-08-02 15:47:00 -07:00
JoelEinbinder
984e011cf0 Move path option into tracing.start (#192)
This consolidates all the tracing options in the tracing.start()
2017-08-02 15:41:05 -07:00
JoelEinbinder
c32df08e4c Update jsdoc from Tracing patch (#189)
This fixes the comments @a1ph on #181 about JSDOC.
2017-08-02 15:02:41 -07:00
JoelEinbinder
5d647bf1fa Use const a lot fo places (#188) 2017-08-02 12:06:47 -07:00
JoelEinbinder
bd72e40e73 Introduce page.tracing (#181)
This patch introduces page.tracing, which allows to start and stop
trace recording for a page. The trace could be then saved to file.
2017-08-02 10:45:11 -07:00
Andrey Lushnikov
a43c315214 Implement browser 'ignoreHTTPSErrors' option. (#177)
This patch implements Browser 'ignoreHTTPSErrors' option.

Fixes #84.
2017-08-01 15:17:57 -07:00
JoelEinbinder
2acfec0989 Slow mode (#173)
This patch introduces a `slowMo` browser option which
slows down puppeteer operations. Comes handy for debugging
scripts.
2017-07-31 18:47:56 -07:00
JoelEinbinder
fc70ab8f21 Add delay option to input methods (#171)
Add delay option to input methods for the keyboard and mouse.

Closes #156
2017-07-31 18:18:15 -07:00
JoelEinbinder
bfc40b2ee6 Specify repeat property on repeated keypresses (#172)
This patch implements 'autoRepeat' functionality for `keyboard.down`.
With this patch, the subsequent calls to `keyboard.down` would generate
an event with 'autoRepeat` flag set to true.

Closes #157
2017-07-31 12:05:46 -07:00
JoelEinbinder
b474a2d0d9 Mark options objects as optional (#170) 2017-07-30 13:46:56 -07:00
Andrey Lushnikov
c013a531cf Refactor EmulationManager
This patch refactors EmulationManager so that it is a normal
class with a state.
2017-07-29 19:19:37 -07:00
Andrey Lushnikov
445dce46f6 response.text() should wait for request to finish
This patch makes sure that request.text() doesn't try
to fetch response body from the backend until the request is
actually finished (finished or failed).
2017-07-29 18:48:30 -07:00
Andrey Lushnikov
67f4264162 Never sent 'requestfinished' event without passing actual request
It turns out we're not receiving 'Network.requestWillBeSent' event
for every requestId.

This patch makes sure we don't dispatch `requestfinished` and
`requestfailed` events without passing actual request.

References #168
2017-07-29 18:34:47 -07:00
Andrey Lushnikov
72991c16eb [DEBUG] chnage direction of arrow in protocol's SEND and RECEIVE logs
They currently feel backwards.
2017-07-29 17:43:24 -07:00
Andrey Lushnikov
adf35952fc [DEBUG] More descriptive event reporting
This patch improves DEBUG reporting so that circular event
arguments are shown with some information.
2017-07-29 13:02:56 -07:00
Pavel Feldman
7fdf800a39 Simplify frame manager, do not fetch resources upfront (#159)
This patch:
- gets rid of Page.getResourceTree() protocol method
- rolls chromium to 490379

Fixes #127
2017-07-28 16:52:38 -07:00
Andrey Lushnikov
bd767002bb Fix navigation to about:blank
This patch fixes navigation to about:blank url.
2017-07-28 16:44:51 -07:00
JoelEinbinder
8780fcb662 Event coverage and debugging (#160)
This patch introduces event coverage for DEBUG module and 
API coverage.

Closes #50.
2017-07-28 11:45:05 -07:00
JoelEinbinder
3c75767288 Fix media keys to use new key strings and codes (#164)
VolumeUp, VolumeDown, and VolumeMute were changed to AudioVolumeUp, AudioVolumeDown, and AudioVolumeMute

The media keys like MediaTrackNext were also missing, so I added them.
2017-07-28 11:38:30 -07:00
Andrey Lushnikov
d26e2399f2 Resolve paths against CWD in page.uploadFile() method
This patch:
- teaches page.uploadFile() to resolve given file paths against
  current working directory. This aligns paths handling with all the
  other methods
- moves page.uploadFile() under Frame
- changes test to use relative path for file upload
2017-07-28 00:06:57 -07:00
Andrey Lushnikov
ba37a4f82d Get rid of Body class
The Body class was inlined in the Request and Response classes.
This patch:
- removes the Body class
- adds Request.postData public property
- adds Response.buffer(), Response.text() and Response.json() methods

Fixes #106.
2017-07-27 23:11:24 -07:00
Andrey Lushnikov
b9e3cce5fd Fail navigation when main resource fails to load
This patch fails navigation when the main resource fails to load.

Fixes #148.
2017-07-27 17:54:39 -07:00
Andrey Lushnikov
bd898b7f56 Implement function as a part of a page.waitFor shortcut
This patch adds a function as a possible argument to
page.waitFor shortcut.

Fixes #91.
2017-07-27 17:09:28 -07:00
Andrey Lushnikov
ff5ed1c738 Implement page.waitForFunction method
The page.waitForFunction method allows to wait for a general predicate.
The predicate will be continiously polled for in page, until
it either returns true or the timeout happens.

The polling parameter could be one of the following:
- 'raf' - to poll on every animation frame
- 'mutation' - to poll on every dom mutation
- <number> - to poll every X milliseconds

References #91
2017-07-27 16:45:14 -07:00
JoelEinbinder
a2e0d27fb6 Implement public API coverage
This patch:
- implements a basic public API coverage based on 'helper.tracePublicAPI' methods
- adds `npm run coverage` command which reports coverage after running all of the unit tests

References #50.
2017-07-27 16:16:37 -07:00
Andrey Lushnikov
c26d2c8271 Remove redundant methods from Body class
This patch:
- removes Body.arrayBuffer. This method is redundant since there's
  already a Body.buffer() method
- removes Body.bodyUsed getter.

References #106
2017-07-27 14:43:14 -07:00
Andrey Lushnikov
e39d8602b0 Rename page.setHTTPHeaders into page.setExtraHTTPHeaders
This patch:
- renames page.setHTTPHeaders into page.setExtraHTTPHeaders
- starts using Map instead of Object to align with other headers
  arguments

Fixes #112.
2017-07-27 12:51:18 -07:00
JoelEinbinder
bbde8fd1c2 page.evaluate takes a string in addition to function (#135)
This patch improves on page.evaluate to accept a string.
The string can have a trailing '//# sourceURL=' comment which would
name the evaluation to make stacks beautiful.

In order to make sourceURL comments possible, this patch:
- removes wrapping of the client function into `Promise.resolve()`
- stops passing `awaitPromise` parameter to `Runtime.evaluate`
- starts to await promise via the `Runtime.awaitPromise` if the return type of the evaluation
  is promise

closes #118
2017-07-27 12:23:41 -07:00
Andrey Lushnikov
480547955c Drop Headers class in favor of a Map object
This patch removes Header class and substitutes it with a simple
Map object.

The map is chosen over the vanilla object since it has explicit
order of headers which we'd like to preserve.

References #106.
2017-07-27 09:35:44 -07:00
Pavel Feldman
585f815548 Don't use Emulation.resetPageScaleFactor (#147)
Instead of using Emulation.resetPageScaleFactor, this patch starts using clip's scale for
fullPage screenshots.
2017-07-26 15:28:44 -07:00
Pavel Feldman
5068185d03 Don't ever enable DOM domain. (#145)
This patchs makes sure DOM domain is never enabled. The reasoning behind this is that
DOM domain is heavy-weight: it generates a lot of traffic over the protocol.

Drive-by: use Page.addScriptToEvaluateOnNewDocument instead of Page.addScriptToEvaluateOnLoad
2017-07-26 15:19:43 -07:00
JoelEinbinder
2e6751d6f5 Trace only string method names (#141) 2017-07-26 11:48:40 -07:00
JoelEinbinder
9363a15cce Small fixes to jsdoc (#140)
References #65.
2017-07-26 09:58:03 -07:00
Alexei Filippov
83f4e43564 Clean up Browser annotations. (#138) 2017-07-25 20:39:50 -07:00
JoelEinbinder
aa28eb80f0 Wait for double raf after all mouse events. (#123)
This patch starts waiting for double raf after every mouse event.
It looks like a good enough workaround to make sure mouse events
are delivered.

The single raf might not be enough because browser might 'return' an already-prepared
frame for us without actually dispatching events.

Closes #122
2017-07-25 17:03:13 -07:00
JoelEinbinder
1a50403090 fix linter (#136)
This patch fixes broken travis, caused by impatient merge of f455e3a1e
2017-07-25 16:47:49 -07:00
Pavel Feldman
f455e3a1e4 Remove NetworkManager usage from NavigatorWatcher (#134) 2017-07-25 16:06:53 -07:00
JoelEinbinder
64968862db Implement mouse.click(x,y) (#131)
Closes #107
2017-07-25 16:05:23 -07:00
Pavel Feldman
49e363c121 Mess with lusha's waitFor for fun. (#133) 2017-07-25 15:57:31 -07:00
Pavel Feldman
4adf11427a Unconditionally update frame payload on navigation only. (#130) 2017-07-25 14:58:10 -07:00
Pavel Feldman
eca0d7fe53 Simplify keyboard modifiers handling (#129) 2017-07-25 14:35:03 -07:00
Pavel Feldman
0007809305 Move inner evaluate helpers into frame class. (#128) 2017-07-25 14:30:04 -07:00
JoelEinbinder
4f5f1f6d77 Move missing methods from page onto frame. (#125)
Closes #113
2017-07-25 11:37:46 -07:00
Andrey Lushnikov
b564029589 Improve documentation for the page.waitFor method.
Fixes #109.
2017-07-25 08:46:11 -07:00
JoelEinbinder
877f06aacb Remove userAgent getter (#126)
If needed, the user agent could be fetched via querying
`navigator.userAgent` in page.

Closes #110
2017-07-25 02:01:10 -07:00
JoelEinbinder
615381c70e Remove browser.closePage. (#124)
Remove browser.closePage. Closes #116
2017-07-25 00:44:13 -07:00
Andrey Lushnikov
d4c5aee5a8 Rename page.evaluateOnInitialized into page.evaluateOnNewDocument
This patch renames page.evaluateOnInitialized into
page.evaluateOnNewDocument to better align with the protocol and with
what the method is actually doing.

Fixes #119.
2017-07-25 00:17:56 -07:00
Andrey Lushnikov
5acd711b20 Remove InterceptedRequest.isHandled() method
This patch removes the InterceptedRequest.isHandled method. There's
always a single request interceptor who owns all the intercepted
requests and who know if they were handled.

References #115.
2017-07-24 23:37:28 -07:00
Andrey Lushnikov
efa33c37b3 Remove page.httpHeaders() getter
Remove the page.httpHeaders() method since it always returns the last
value of page.setHTTPHeaders().

References #115.
2017-07-24 23:35:10 -07:00
Andrey Lushnikov
5549ad0282 Do not serialize remote objects unless needed
This patch stops serializing console API arguments unless there are
listeners of the 'console' event in puppeteer.

This saves quite a lot CPU cycles.

Fixes #117.
2017-07-24 21:43:54 -07:00
Andrey Lushnikov
236acec228 Remove Frame.isMainFrame()
This patch removes redundant frame.isMainFrame() method. The suggested
way to check for main frame is:

```js
frame === page.mainFrame()
```

Fixes #120.
2017-07-24 21:25:54 -07:00
JoelEinbinder
a24cec20f9 Fix clicking offscreen horizontally (#105)
This patch starts clicking in the center of the element area which is
visible in a viewport.

Fixes #103.
2017-07-24 16:14:32 -07:00
Andrey Lushnikov
0a3125434e Refactor Frame.waitForSelector method
Refactor Frame.waitForSelector to make room for Frame.waitForFunction
implementation.
This patch:
- removes AwaitedElement class which proved to be confusing, and
  introduces a more straight-forward WaitTask.
- refactors the mutation observer to return true in case of successful
  waiting or false in case of timeout.

References #91
2017-07-24 10:07:43 -07:00
JoelEinbinder
fdaaa2c0e6 Inject file with sourceURL (#102)
This patch starts adding a sourceURL trailing comment to make stack traces
readable.
2017-07-23 09:56:35 -07:00
Andrey Lushnikov
c4904b4e10 Do not leak event listeners on navigation
This patch:
- introduces helper.addEventListener/helper.removeEventListeners
  to simplify event management
- moves NavigatorWatchdog over to the helper.addEventListener to
  stop leaking event listeners
2017-07-22 17:46:39 -07:00
Andrey Lushnikov
5757bc18f2 Rename 'maxTime' option of Page.navigate into 'timeout'
The motivation behind this rename is to name all the 'timeout' options
across methods similarly.

References #39.
2017-07-22 16:32:57 -07:00
JoelEinbinder
98ee35655f Mouse (#101)
This patch:
- adds Mouse class which holds mouse state and implements mouse primitives,
such as moving, button down and button up.
- implements high-level mouse api, such as `page.click` and `page.hover`.

References #40, References #89
2017-07-21 20:29:31 -07:00
JoelEinbinder
eb2cb67b0e Remove keyboard.type and keyboard.press (#98)
This patch removes keyboard.type and keyboard.press methods. The motivation
behind this is to keep only low-level API in the `keyboard` namespace.
2017-07-21 20:00:09 -07:00
Andrey Lushnikov
dc032b42b9 Introduce polymorphic page.waitFor method
This patch:
- introduces page.waitForSelector to wait for the selector to appear
- introduces polymorphic page.waitFor method, which accepts
either string (and in this case is a shortcut for page.waitForSelector)
or number (and in this case it's a promisified timeout).

References #91.
2017-07-21 12:54:04 -07:00
Andrey Lushnikov
1f954fa7ed Implement timeout option for page.waitFor
This patch implements timeout option for page.waitFor. The function
will throw if the selector doesn't appear during timeout milliseconds
of waittime.

References #89, #91.
2017-07-21 12:07:48 -07:00
Andrey Lushnikov
aba61de905 Make helper.getExceptionMessage synchronous
We now have description of an exception, no need for a roundtrip
to the backend.
2017-07-21 11:57:25 -07:00
Andrey Lushnikov
52de75742b Implement visible option for Page.waitFor method
This patch adds a 'visible' option to the Page.waitFor method, making
it possible to wait for the element to become actually visible.

References #89, #91.
2017-07-21 10:01:19 -07:00
Andrey Lushnikov
139b9e9b6d Get rid of page.emulate() / page.emulatedDevices() methods
This patch:
- gets rid of `page.emulate` and `page.emulatedDevices`
  methods. Instead, it is suggested to use `page.setViewport()`
  and `page.setUserAgent()` methods.
- moves DeviceDescriptors to the top level of the puppeteer so that
  it is convenient to require them.
- improves on documentation to describe the suggested emulation
  approach.

References #88.
2017-07-20 23:53:06 -07:00
Andrey Lushnikov
76ac3bded5 Convert DevicesDescriptors into puppeteer format
This patch converts lib/DevicesDescriptors from a devtools front-end
format into a puppeteer format.

This patch does this via introducing a scripts utils/fetch_devices.js
which grabs devices from upstream of DevTools Front-end and
converts them into puppeteer devices.

References #88.
2017-07-20 17:49:15 -07:00
Andrey Lushnikov
42edc3108a [emulation] fix touch emulation
This patch:
- fixes touch emulation
- adds tests to cover basic Page.emulate

References #88.
2017-07-20 12:22:34 -07:00
Andrey Lushnikov
4af0911b90 Use more good flags to run chrome instance.
The switches are mostly borrowed from chromedriver launching code.
2017-07-20 12:20:10 -07:00
Andrey Lushnikov
a981594b1d Revert node's scrolling into view in page.click()
This change was not intentional in e33a8f818c
2017-07-20 00:53:14 -07:00
Andrey Lushnikov
e33a8f818c Support HEADLESS env variable for unit tests
This patch makes it possible to run unit tests in non-headless
mode with the following command:

```
HEADLESS=false npm run unit
```
2017-07-19 19:36:22 -07:00
Andrey Lushnikov
a63a0198de Implement waitFor which survives navigation (#99)
This patch implements page.waitFor method which survives navigation.

References #89.
2017-07-19 19:04:51 -07:00
JoelEinbinder
febd747c5b Inroduce page.press (#96)
This patch:
- introduces page.press() method
- adds more input tests

References #89
2017-07-19 14:43:07 -07:00
JoelEinbinder
71f8c76f04 Rename keyboard.hold and release to up and down (#95)
References #89
2017-07-19 14:27:01 -07:00
Andrey Lushnikov
21af495b65 Move screenshot task chain in Browser
Currently, it's impossible to do screenshots in parallel.
This patch:
- makes all screenshot tasks sequential inside one browser
- starts activating target before taking screenshot
- adds a test to make sure it's possible to take screenshots across
  tabs
- starts waiting for a proper page closing after each test. This might
  finally solve the ECONNRESET issues in tests.

References #89
2017-07-19 14:15:16 -07:00
Andrey Lushnikov
2e94f9f67b Resolve pending callbacks when connection closes
This patch:
- resolves pending callbacks in connection once it gets closed
- fixes one of Page.screenshot() tests to wait for all the screenshots
- starts handling ECONNRESET error in test server to avoid throwing with no
  good reason
2017-07-19 01:00:02 -07:00
Andrey Lushnikov
55acae40fd Introduce DEBUG module which traces public API calls
This patch improves on DEBUG module to trace all puppeteer's
public API calls.

References #89.
2017-07-18 21:06:03 -07:00
Pavel Feldman
f154d537f7 Introduce page.goBack/page.goForward (#93)
This patch introduces page.goBack/page.goForward methods
to navigate the page history.

References #89.
2017-07-18 19:11:37 -07:00
Pavel Feldman
98c3894c84 Introduce Page.waitForNavigation (#94)
This patch introduces Page.waitForNavigation which allows to wait
for render-initiated navigation.

This patch also does a nice refactoring, replacing Navigator with NavigatorWatcher which
is not a part of a page state.

References #89
2017-07-18 18:54:24 -07:00
Pavel Feldman
bef9982687 Simplify waitFor code (#92) 2017-07-18 12:16:19 -07:00
JoelEinbinder
28265fc313 Fix frame.waitFor to work with complex selectors (#90)
The frame.waitFor didn't account for multi-level selectors.
This patch fixes this, executing document.querySelector on every DOM mutation.
2017-07-18 02:10:02 -07:00
Andrey Lushnikov
6c1c3a0c45 Return remote object description for unserializable objects
This patch starts handling unserializable objects by returning their
description.

Fixes #86.
2017-07-18 01:34:43 -07:00
Andrey Lushnikov
2ca08b032b Rename Page's 'consolemessage' event into 'console'
This patch:
- renames 'consolemessage' event into 'console'
- improves on 'console' event documentation

References #39.
2017-07-17 20:38:11 -07:00
Andrey Lushnikov
1a97d8b3c2 Teach 'consolemessage' event to send all the arguments
This patch fixes 'consolemessage' event so that it passes
over all the arguments of console API call.
2017-07-17 20:22:45 -07:00
JoelEinbinder
117a128b42 Introduce Page.$ and Page.$$ (#75)
This patch introduces Page.$ and Page.$$ methods which are
aliases for `document.querySelector` and `document.querySelectorAll`. 

Fixes #78.
2017-07-17 18:56:56 -07:00
JoelEinbinder
bf7698e8f8 Intorduce Page.keyboard (#74)
Introduce page.keyboard to provide low-level access to the keyboard.
2017-07-17 18:49:52 -07:00
Pavel Feldman
895f69d17a Add emulation for named devices. (#72)
This patch introduces page emulation, making it possible to emulate different devices.
2017-07-17 18:13:04 -07:00
Andrey Lushnikov
0414dfa98b Introduce DEBUG module
This patch re-introduces the DEBUG module to expose some of
the puppeteer's internals.

Currently, only the protocol message communication is exposed under
the 'puppeteer:protocol' namespace.
2017-07-17 15:15:07 -07:00
Pavel Feldman
4581ada210 Roll chromium to r486981. (#87)
This patch rolls chromium to r486981 and prepares
for the introduction of device emulation.
2017-07-17 12:15:06 -07:00
Andrey Lushnikov
b2d2bf822a Rename Page.printToPDF into page.pdf
This patch:
- renames Page.printToPDF into page.pdf
- adds a 'path' option to the page.pdf options instead of a separate
  `filePath` parameter
- improves on the documentation for the `page.pdf`

References #39.
2017-07-17 10:37:33 -07:00
Aleksey
ac75455983 Fixed page.waitFor (#81)
The original implementation didn't account for frames.
2017-07-14 15:55:38 -07:00
Andrey Lushnikov
560b817d7f Make page.url() return URL synchronously.
page.url() is just a shortcut to page.mainFrame().url().

References #39.
2017-07-14 14:05:27 -07:00
Andrey Lushnikov
34a96a8462 Rename navigation option 'waitFor' into 'waitUntil'.
This patch renames navigation option 'waitFor' into 'waitUntil' to
avoid confusion with 'page.waitFor' call.

References #39.
2017-07-14 13:59:36 -07:00
Andrey Lushnikov
76415e1be5 [doc] distinct naming for functions which are evaluated in page context
This patch renames all the function parameters which are executed in
page context into 'pageFunction'.
2017-07-14 13:51:22 -07:00
Andrey Lushnikov
235001bab3 [doclint] validate return statements
This patch teaches doclint to validate presence of return statements.
It also adds a bunch of documentation to the api.md.

References #14.
2017-07-14 13:04:15 -07:00
JoelEinbinder
4bd855c66b Use getBoundingClientRect instead of DOM.BoxModel (#76)
This patch re-implements `page.click()` and `page.focus()` using
`page.evaluate`
2017-07-13 18:32:34 -07:00
Andrey Lushnikov
64ebcdba9f Do not close readline after reading remote debugging port
Closing readline interface results in closing underlying
stream. We don't want this.
2017-07-12 19:15:21 -07:00
Andrey Lushnikov
38e6f53cc7 [doclint] add linting for class properties
This patch:
- adds linting for class properties
- adds documentation for the missing class properties

References #14.
2017-07-12 09:45:08 -07:00
Andrey Lushnikov
60621b8815 Fix Click unit tests on Mac OS X
This patch brings screenDPI back in clicking equation, which was
regressed in afb096cfd7.
2017-07-11 22:35:22 -07:00
Aleksey
b77ae4b77c Properly cleanup FrameManager._frameIdToExecutionContextId. (#73) 2017-07-11 21:52:09 -07:00
Andrey Lushnikov
b0a2cf662e Roll chromium to r485143 2017-07-11 19:38:20 -07:00
Andrey Lushnikov
7765760186 [doclint] Start linting method arguments
This patch introduces a general Documentation.diff method, which
produces a diff of two documentations.

With this, the patch teaches documentation linter to lint method arguments.

References #14.
2017-07-11 16:43:41 -07:00
Pavel Feldman
afb096cfd7 Disable DOM domain after each operation. (#71)
It's very heavy to have DOM domain enabled. This patch keeps DOM domain disabled after each use.
2017-07-11 16:25:25 -07:00
Andrey Lushnikov
279cd4c9fb Remote Browser's remoteDebuggingPort option
This patch remove remoteDebuggingPort option. Instead, browser
is launched with '--remote-debugging-port=0' flag, letting browser
to pick any port. The puppeteer reads the port number from the
browser's stderr stream.

This change cuts average browser start time from 300ms to 250ms
on my machine. This happens since puppeteer doesn't have to probe
network once every 100ms, waiting for the remote debugging server to
instantiate.

Fixes #21.
2017-07-11 08:30:41 -07:00
Andrey Lushnikov
d120e7e426 properly cleanup exit process listener as browser closes 2017-07-11 08:11:16 -07:00
Andrey Lushnikov
b6d0ebfbf2 Run headless chromium with --hide-scrollbars
This patch runs chromium in headless mode with the --hide-scrollbars
flag so that it doesn't spoil screenshots.

Fixes #68.
2017-07-10 18:58:29 -07:00
Andrey Lushnikov
50d9c186b5 Change Page.navigate to return main resource response
This patch changes Page.navigate API:
- Page.navigate now resolves to the main page response
- Page.navigate throws errors if there's no main page response,
  e.g. in case of SSL errors, max navigation timeout,
  or invalid url.

This patch also adds httpsServer with a self-signed certificates
for the testing purposes.

Fixes #10.
2017-07-10 18:50:06 -07:00
JoelEinbinder
da0cde1b45 Implement Page.uploadFile (#61)
This patch implements `Page.uploadFile` method to support file upload inputs.
2017-07-10 11:21:46 -07:00
JoelEinbinder
739c1566a9 Add tests for all public methods on Page (#62) 2017-07-10 10:09:59 -07:00
Andrea Cardaci
ff4a389274 Fix watchdog timer cleanup (#64)
Fix a bug in f7cd004 which prevents the cleanup of the watchdog timer, thus keeping the Node.js process alive for this._maxTime msec no matter what.
2017-07-08 08:54:28 -07:00
Andrey Lushnikov
f7cd0048af Cleanup navigator logic
This patch cleans up navigator logic. (This is a result of a pair
programming with @pavelfeldman).
2017-07-07 15:46:05 -07:00
Aleksey
090ecfa6b9 Added frame.waitFor(selector) (#59)
This patch adds `frame.waitFor(selector)` method.

Fixes #42.
2017-07-08 01:39:02 +03:00
Andrey Lushnikov
34f043d821 Implement browser.stdout and browser.stderr streams
This patch adds browser.stdout and browser.stderr streams.

These streams allow to get the browser instance output, e.g.

```js
browser.stderr.pipe(process.stdout);
```
2017-07-07 20:49:17 +03:00
Andrey Lushnikov
c8664319ed Make browser._ensureChromeIsRunning idempotent
The _ensureChromeIsRunning should launch chrome only once
and return the same promise for all its clients.
2017-07-06 18:20:01 -07:00
Paul Irish
6c7ae41ae6 fix page.injectFile and add test. (#52)
This line within `injectFile` wasn't doing much of anything: 

```js
let expression = fs.readFile(filePath, 'utf8', (err, data) => callback({err, data}));
```

* That's fixed.
* A path error in examples/features.js is fixed.
* Test added for injectFile.
2017-07-07 02:09:23 +03:00
Andrey Lushnikov
19a8d74d3c Implement Body interface for both Request and Response
This patch partially implements Fetch API's [Body](https://developer.mozilla.org/en-US/docs/Web/API/Body)
interface for both Request and Response.

Fixes #26.
2017-07-06 12:17:01 -07:00
Andrey Lushnikov
64fed38c60 Rename Page.Events.Error into Page.Events.PageError
The 'error' event has a special treatment in Node:
https://nodejs.org/api/events.html#events_error_events

To avoid this, this patch renames the 'error' event into
the 'pageerror'.
2017-07-06 11:22:32 -07:00
Andrey Lushnikov
19e882f505 fix switch statement indentation in FrameManager 2017-07-06 10:46:19 -07:00
Andrey Lushnikov
773a09a8cf Add support for unserializable values in Page.evaluate
This patch adds support for unserializable return values in
Page.evaluate.

Currently, these values are:
- NaN
- Infinity
- -Infinity
- -0

Fixes #51.
2017-07-06 10:41:01 -07:00
Andrey Lushnikov
62111fbc08 Fix style after 9faecc2e 2017-07-05 19:07:33 -07:00
Andrey Lushnikov
9faecc2e67 Fix Page.setContent method
The patch fixes an unfortunate bug in Page.setContent method and
adds a test to cover it.

References #50.
2017-07-05 18:09:17 -07:00
Andrey Lushnikov
4fb7f7754c add a more clear message for protocol errors 2017-07-05 17:21:42 -07:00
Andrey Lushnikov
0d2c32e8cd Roll chromium to r484159 2017-07-05 17:02:58 -07:00
Paul Irish
2cd60c9e1e network manager: remove duplicated method (#48) 2017-06-30 08:26:36 -07:00
Andrey Lushnikov
9c138e9dba Support redirects in NewtorkManager
This patch adds support to redirects so that they will be reported
as a separate requests.

References #26.
2017-06-29 18:57:07 -07:00
Andrey Lushnikov
7f74daf62c Simplify NetworkManager's request/response management
This patch:
- drops Request dependency on NetworkManager
- drops Response dependency on NetworkManager
- drops requestIds from Request and Response objects
2017-06-29 18:18:06 -07:00
Andrey Lushnikov
3dfa688920 Simplify lib/Navigator.js 2017-06-29 12:28:05 -07:00
Andrey Lushnikov
4aa74cc7f8 cleanup Page.js from network leftovers after a35a21d 2017-06-29 12:00:19 -07:00
Andrey Lushnikov
a35a21dfaf Implement NetworkManager
This patch implements NetworkManager, which encapsulates all the
interaction with Network domain.

The NetworkManager also uses partial implementation of Request and
Response classes, defined in the Fetch API specification.

References #26
2017-06-29 11:49:56 -07:00
Patrick Hulce
4b0b81fd9b Add better network idle definition (#38)
This patch:
- Changes network idle promise to wait for 2 or fewer network requests for at least idleTime (defaults to 5s) before resolving.
- Adds timer cleanup to failure navigation case.
- Adds handling of webSocketClosed.
- Ignores unrecognized requestIds to avoid negative inflight requests.

References #10
2017-06-28 14:39:37 -07:00
Andrey Lushnikov
d5be1a6436 Introduce Page.Events.Request
This patch introduces the 'request' event which is fired when
page has initiated a request.

The event dispatches an instance of Request class.

References #26.
2017-06-28 01:42:42 -07:00
Andrey Lushnikov
7b59a89695 Implement Request object
This patch does a step towards Fetch API:
- implements Request object to some extend. The Request object will be
  sent in RequestWillBeSent event.
- implements InterceptedRequest which extends from Request and allows
  for request modification. The InterceptedRequest does not
  conform to Fetch API spec - there seems to be nothing related to
  amending in-flight request.
- adds test to make sure that request can change headers.

References #26
2017-06-28 01:41:42 -07:00
Andrey Lushnikov
5ed71fcb8f Test the 'networkidle' navigation logic
This patch adds a test to verify that navigation properly waits for the
network to become idle.

References #10
2017-06-27 22:02:46 -07:00
JoelEinbinder
d5a91650ae Implement Basic input API
This patch implements Basic Input api:
- Page.focus(selector) - focuses element with selector
- Page.click(selector) - clicks element with selector
- Page.type(text) - types text into a focused element

Fixed #43.
2017-06-27 18:27:22 -07:00
Andrey Lushnikov
3d90ea38a9 Implement Frame.evaluate
This patch implements Frame.evaluate method.

References #4.
2017-06-27 14:57:14 -07:00
Andrey Lushnikov
819fa355f4 Convert var's to let's
This patch:
- converts all var's to let's throughout the codebase
- enforces the let's over var's via the eslint rule
2017-06-22 14:58:39 -07:00
Pavel Feldman
cf35524285 Foramt JSDocs for 2 spaces 2017-06-21 14:11:52 -07:00
Pavel Feldman
437a93b26e Reformat code using 2 spaces 2017-06-21 14:11:52 -07:00
Pavel Feldman
bc0655b587 Extract basic in-flight counting navigator 2017-06-21 14:11:52 -07:00
Pavel Feldman
4761f13740 Inline helper eval functions 2017-06-21 14:11:52 -07:00
Pavel Feldman
84bc09bce1 Revert "Reformat codebase into 2-spaces"
This reverts commit d0d1ee303e41fe4ba762a031b78c3894edac52df.
2017-06-21 14:11:52 -07:00
Andrey Lushnikov
448ac4ce64 Reformat codebase into 2-spaces
This patch:
- reformats codebase to use 2-spaces instead of 4. This will
  align the project with other codebases (e.g. DevTools and Lighthouse)
- enables eslint indentation checking

References #19
2017-06-21 14:11:52 -07:00
Andrey Lushnikov
aaefec7868 Fix Travis-Ci tests
It turned out that the tip-of-tree chromium fails to start on Travis
because of two reasons:
- inability to run LinuxSUIDSandbox
- libnss3 library version being too small

Both problems happen because of the outdated "trusty" distribution
which is used on travis.

This patch:
- reverts the previous patch 9e6f779. Instead of introducing such
  a weird "api", both phantom_shim/runner.js and test/test.js no
  explicitly pass '--no-sandbox' flag to the browser
- updates the libnss3 lib on travis

Fixes #33
2017-06-21 14:11:52 -07:00
Andrey Lushnikov
7874108fda Fix running on Travis-Ci 2017-06-21 14:11:52 -07:00
Andrey Lushnikov
eb02af93b6 Improve error reporting when chrome fails to start
This patch starts throwing exception and dumping chromium stderr when
the browser fails to start.

References #33
2017-06-21 14:11:52 -07:00
Andrey Lushnikov
175963182e Implement FrameManager
This patch implements FrameManager which is responsible for maintaining
the frame tree. FrameManager is quite basic: it sends FrameAttached,
FrameDetached and FrameNavigated events, and can report mainFrame and
all frames.

The next step would be moving certain Page API's to the Frame. For
example, such method as Page.evaluate, Page.navigate and others should
be available on Frame object as well.

References #4
2017-06-21 14:11:52 -07:00
Paul Irish
a66480a416 Fix ESLint failures (#34)
References #33
2017-06-21 14:11:52 -07:00
Pavel Feldman
8e0a3ac6d9 Make InPageCallback async (#24)
Fixes #20
2017-06-21 14:11:52 -07:00
Pavel Feldman
9ad4938fcb Inline helper evaluate functions 2017-06-21 14:11:52 -07:00
Andrey Lushnikov
868814ac7f Implement fullPage screenshots
This patch adds a 'fullPage' option to the Page.screenshot
method.

Fixes #6.
2017-06-16 22:34:29 -07:00
Andrey Lushnikov
dbb374d4af Fix racy condition in case of multiple parallel screenshots
Page.screenshot operates the global state of the page. In case of
multiple Page.screenshot() commands running in parallel with different
clipping rects, they interfere with each other.

This patch makes Page.screenshot() commands run sequencially
even though they were called in parallel.

Fixes #15.
2017-06-16 20:20:36 -07:00
Andrey Lushnikov
3b0bc0802d Refactor Page.screenshot() api
This patch refactors Page.screenshot api, accoring to the discussion
in #5:
- Page.screenshot accepts single optional options object
- Page.saveScreenshot is removed
- Page.screenshot assumes 'png' screenshot if no type is set and no
  'path' property is given

Fixes #5.
2017-06-16 17:15:24 -07:00
Andrey Lushnikov
6bed8c62b3 Pass integers to the Emulation.setVisibleSize
Integers are required in the Emulation.setVisibleSize. This patch
fixes the screenshot clipRect so that it never tries to pass
float values.
2017-06-16 15:43:09 -07:00
Andrey Lushnikov
163e14345d Remove Page.handleDialog method
This is a left-over after refactoring in f62cfc3b.

References #2.
2017-06-16 11:24:47 -07:00
Andrey Lushnikov
632b90efae Page.Events.Error should throw an proper error
This patch:
- renames Page.Events.Exception in Page.Events.Error
- dispatches an error which has a page stack as its message
2017-06-16 11:21:44 -07:00
Andrey Lushnikov
2066da9ec7 Page.evaluate should reject in case of evaluation error
This patch makes Page.evaluate to throw an error if there
was an error in evaluated code.
2017-06-16 10:34:13 -07:00
Andrey Lushnikov
ff2c3bbca9 Cleanup Page class
This patch removes unneeded dependency from Page
to Browser.
2017-06-15 21:35:31 -07:00
Andrey Lushnikov
f62cfc3b34 Refactor JavaScript dialog API
This patch introduces a Dialog class and a new 'dialog'
event instead of the 'alert', 'beforeunload', 'confirm' and
'prompt' events and 'Page.handleDialog' method.

Fixes #2.
2017-06-15 21:22:41 -07:00
Andrey Lushnikov
14a75a83ea Merge Page.evaluate and Page.evaluateAsync together
This patch makes Page.evaluate await promise if one is
returned by the evaluated code.

This makes the Page.evaluateAsync unneeded, so the patch
removes it.

Fixes #11.
2017-06-15 14:56:40 -07:00
Andrey Lushnikov
50976c7f29 Implement Request.setHeader() method
This patch implements Request.setHeader() method to override
request headers.

References #8.
2017-06-15 08:37:06 -07:00
Andrey Lushnikov
5ba6621cde Remove the Page.setBlockedURLs method
This patch removes the Page.setBlockedURLs method. The
functionality is trivially implementable with the request
interception (see examples/loadurlwithoutcss.js).

Fixes #1.
2017-06-15 08:26:55 -07:00
Andrey Lushnikov
e274c26e8b Implement Page.setRequestInterceptor
This patch:
- introduces Request class.
- implements Page.setRequestInterceptor method. The method
  allows to install a callback which will be called for every request
  with a |Request| object as a single parameter. The callback is free
  to override certain request's properties and then either continue or
  abort it.
- implements request interception api for phantom-shim and unskips the
  module/webpage/abort-network-request.js phantomjs test

References #1
2017-06-15 08:26:50 -07:00
Andrey Lushnikov
9bdf9ed5de Fix jsdoc after 67932b8
This patch gets rid of "CDP" type in jsdoc annotations.
2017-06-14 18:37:42 -07:00
Andrey Lushnikov
67932b87c1 Drop the chrome-remote-interface dependency
This patch drops the chrome-remote-interface dependency and
introduces Connection class which handles all the communication
with remote target.

Closes #3
2017-06-14 15:45:59 -07:00
Andrey Lushnikov
dbffc3c35c Await promises returned from the inPageCallback
This patch teaches puppeteer to await promises returned
from the inPageCallback.
2017-06-14 08:21:56 -07:00
Andrey Lushnikov
ec414eb774 Rename page.size() and page.setSize() into page.viewportSize()
It turns page.size() and page.setSize() methods are slightly
confusing since there multiple different sizes (layout size,
content size, viewport size..)

This patch renames Page.{size,setSize} methods into
Page.{viewportSize,setViewportSize} methods to avoid confusion.
2017-06-14 07:41:26 -07:00
Andrey Lushnikov
7136b3aaff Improve inPageCallback handling
This patch improves inPageCallback handling so that it doesn't
retain any variables.
2017-06-13 20:09:50 -07:00
Andrey Lushnikov
549535e674 Add docs/api.md
This patch adds docs/api.md file which contains API description
for the puppeteer API. This patch adds the API outline, which
doesn't not have explanations and samples.
2017-06-13 00:28:39 -07:00
Andrey Lushnikov
ac3e76fb22 Cleanup typechecker errors
This patch cleans up different issues which were spotted by the
typechecker.
2017-06-11 23:10:33 -07:00
Andrey Lushnikov
1f51384918 Introduce Eslint to validate style
This patch introduces eslint and fixes multiple minor code
style issues.
2017-06-11 01:32:59 -07:00
Andrey Lushnikov
2b50d8cc32 Introduce utils/check_availability.js utility
This patch introduces check_availability.js utility which looks for
available chromium binaries for different revisions and platforms.

This patch also re-factors the chromium downloader scripts so that
it can operate different platforms.
2017-06-08 11:22:21 -07:00
Andrey Lushnikov
e05fd8c7b9 minor code cleanup 2017-05-14 23:44:35 -07:00
Andrey Lushnikov
245984905e Implement Puppeteer's setBlockedURLs method
This patch implements Puppeteer's Page.setBlockedURLs method.
This is a less agile alternative to the phantom's request.abort()
api.

This patch also adds a loadurlwithoutcss.js example re-implementation
to illustrate the usage of Page.setBlockedURLs.
2017-05-14 22:28:19 -07:00
Andrey Lushnikov
2f74644cb8 Puppeteer's Page.navigate should not throw with invalid URL.
This patch makes puppeteer's Page.navigate to just return
'false' when given an invalid URL.
2017-05-14 22:17:57 -07:00
Andrey Lushnikov
91e54378a3 Implement printing to PDF
This patch implements:
- puppeteer's Page.printToPDF method. The method's defaults
  are similar to phantom's defaults for the paperSize property.
- phantom's render into pdf file.
2017-05-14 22:00:39 -07:00
Andrey Lushnikov
bdf895bed6 Support Page.{Alert,Confirm,Prompt,BeforeUnload} events
This patch adds Page events to fire when javascript dialogs
get opened.
2017-05-13 13:44:24 -07:00
Andrey Lushnikov
939038bb08 Convert event names to small caps
This patch takes inspiration from DOM events and makes all the
events small-caps.
2017-05-13 12:04:30 -07:00
Andrey Lushnikov
15b36b1cf0 Drop unneeded Page Event's prefixes.
This patch drops 'Page.Event.' prefix in every puppeteer's page
event. This makes it convenient to subscribe to events by their
string value.
2017-05-13 11:12:06 -07:00
Andrey Lushnikov
8a8076c15b Rename Page.Events.ConsoleMessageAdded -> Page.Events.ConsoleMessage
This patch:
- renames ConsoleMessageAdded into ConsoleMessage for the sake of
  clarity
- adds a test to cover basic functionality
2017-05-13 11:05:54 -07:00
Andrey Lushnikov
e8af69e5bb Make in-page callback survive navigations
This patch makes in-page callbacks survive navigations via
running in-page harness code on page load.
2017-05-12 17:55:29 -07:00
Andrey Lushnikov
58ad5dd823 fix implementation of Page.injectFile method 2017-05-12 14:45:47 -07:00
Andrey Lushnikov
c3a3bfe1d2 Implement Page.evaluateOnInitilized method
The PhantomJS has a similar callback called onInitialized. This
callback passes control to the automation script when the page
gets initialized.

To precisely implement this functionality atop of puppeteer,
and since puppeteer controller script lives in a separate process to
the page, we need an ability to pause page at the moment of
initialization. For now, we are not able to do this.

However, oftentimes clients want to evaluate certain code in
page at the point of page initialization. This patch implements
this capability with the Page.evaluateOnInitilized method call.

This patch also re-implements phantom's unrandomize.js example
with the puppeteer API. This is serves an illustration purpose
for the page.evaluateOnInitilized callback.
2017-05-12 14:13:40 -07:00
Andrey Lushnikov
52e2633f32 Introduce Page.saveScreenshot
Page.saveScreenshot uses filepath to infer screenshot type (either
PNG of JPG) and saves the screenshot to file system.
2017-05-12 10:50:06 -07:00
Andrey Lushnikov
2de672ed1b Introduce Page.injectFile method
With this patch, page has two methods to include javascript:
- Page.addScriptTag(url) which is similar to phantom's includeJs.
- Page.injectFile(filePath) which is similar to phantom's injectJs.
2017-05-12 10:19:01 -07:00
Andrey Lushnikov
e991b0e20a Implement puppeteer's Page.addScript 2017-05-12 10:01:22 -07:00
Andrey Lushnikov
2cda8c18d1 Puppeteer: staging commit. 2017-05-11 00:06:41 -07:00