Commit Graph

635 Commits

Author SHA1 Message Date
Andrey Lushnikov
7765760186 [doclint] Start linting method arguments
This patch introduces a general Documentation.diff method, which
produces a diff of two documentations.

With this, the patch teaches documentation linter to lint method arguments.

References #14.
2017-07-11 16:43:41 -07:00
Pavel Feldman
afb096cfd7 Disable DOM domain after each operation. (#71)
It's very heavy to have DOM domain enabled. This patch keeps DOM domain disabled after each use.
2017-07-11 16:25:25 -07:00
Andrey Lushnikov
279cd4c9fb Remote Browser's remoteDebuggingPort option
This patch remove remoteDebuggingPort option. Instead, browser
is launched with '--remote-debugging-port=0' flag, letting browser
to pick any port. The puppeteer reads the port number from the
browser's stderr stream.

This change cuts average browser start time from 300ms to 250ms
on my machine. This happens since puppeteer doesn't have to probe
network once every 100ms, waiting for the remote debugging server to
instantiate.

Fixes #21.
2017-07-11 08:30:41 -07:00
Andrey Lushnikov
d120e7e426 properly cleanup exit process listener as browser closes 2017-07-11 08:11:16 -07:00
Andrey Lushnikov
b6d0ebfbf2 Run headless chromium with --hide-scrollbars
This patch runs chromium in headless mode with the --hide-scrollbars
flag so that it doesn't spoil screenshots.

Fixes #68.
2017-07-10 18:58:29 -07:00
Andrey Lushnikov
50d9c186b5 Change Page.navigate to return main resource response
This patch changes Page.navigate API:
- Page.navigate now resolves to the main page response
- Page.navigate throws errors if there's no main page response,
  e.g. in case of SSL errors, max navigation timeout,
  or invalid url.

This patch also adds httpsServer with a self-signed certificates
for the testing purposes.

Fixes #10.
2017-07-10 18:50:06 -07:00
JoelEinbinder
da0cde1b45 Implement Page.uploadFile (#61)
This patch implements `Page.uploadFile` method to support file upload inputs.
2017-07-10 11:21:46 -07:00
JoelEinbinder
739c1566a9 Add tests for all public methods on Page (#62) 2017-07-10 10:09:59 -07:00
Andrea Cardaci
ff4a389274 Fix watchdog timer cleanup (#64)
Fix a bug in f7cd004 which prevents the cleanup of the watchdog timer, thus keeping the Node.js process alive for this._maxTime msec no matter what.
2017-07-08 08:54:28 -07:00
Andrey Lushnikov
f7cd0048af Cleanup navigator logic
This patch cleans up navigator logic. (This is a result of a pair
programming with @pavelfeldman).
2017-07-07 15:46:05 -07:00
Aleksey
090ecfa6b9 Added frame.waitFor(selector) (#59)
This patch adds `frame.waitFor(selector)` method.

Fixes #42.
2017-07-08 01:39:02 +03:00
Andrey Lushnikov
34f043d821 Implement browser.stdout and browser.stderr streams
This patch adds browser.stdout and browser.stderr streams.

These streams allow to get the browser instance output, e.g.

```js
browser.stderr.pipe(process.stdout);
```
2017-07-07 20:49:17 +03:00
Andrey Lushnikov
c8664319ed Make browser._ensureChromeIsRunning idempotent
The _ensureChromeIsRunning should launch chrome only once
and return the same promise for all its clients.
2017-07-06 18:20:01 -07:00
Paul Irish
6c7ae41ae6 fix page.injectFile and add test. (#52)
This line within `injectFile` wasn't doing much of anything: 

```js
let expression = fs.readFile(filePath, 'utf8', (err, data) => callback({err, data}));
```

* That's fixed.
* A path error in examples/features.js is fixed.
* Test added for injectFile.
2017-07-07 02:09:23 +03:00
Andrey Lushnikov
19a8d74d3c Implement Body interface for both Request and Response
This patch partially implements Fetch API's [Body](https://developer.mozilla.org/en-US/docs/Web/API/Body)
interface for both Request and Response.

Fixes #26.
2017-07-06 12:17:01 -07:00
Andrey Lushnikov
64fed38c60 Rename Page.Events.Error into Page.Events.PageError
The 'error' event has a special treatment in Node:
https://nodejs.org/api/events.html#events_error_events

To avoid this, this patch renames the 'error' event into
the 'pageerror'.
2017-07-06 11:22:32 -07:00
Andrey Lushnikov
19e882f505 fix switch statement indentation in FrameManager 2017-07-06 10:46:19 -07:00
Andrey Lushnikov
773a09a8cf Add support for unserializable values in Page.evaluate
This patch adds support for unserializable return values in
Page.evaluate.

Currently, these values are:
- NaN
- Infinity
- -Infinity
- -0

Fixes #51.
2017-07-06 10:41:01 -07:00
Andrey Lushnikov
62111fbc08 Fix style after 9faecc2e 2017-07-05 19:07:33 -07:00
Andrey Lushnikov
9faecc2e67 Fix Page.setContent method
The patch fixes an unfortunate bug in Page.setContent method and
adds a test to cover it.

References #50.
2017-07-05 18:09:17 -07:00
Andrey Lushnikov
4fb7f7754c add a more clear message for protocol errors 2017-07-05 17:21:42 -07:00
Andrey Lushnikov
0d2c32e8cd Roll chromium to r484159 2017-07-05 17:02:58 -07:00
Paul Irish
2cd60c9e1e network manager: remove duplicated method (#48) 2017-06-30 08:26:36 -07:00
Andrey Lushnikov
9c138e9dba Support redirects in NewtorkManager
This patch adds support to redirects so that they will be reported
as a separate requests.

References #26.
2017-06-29 18:57:07 -07:00
Andrey Lushnikov
7f74daf62c Simplify NetworkManager's request/response management
This patch:
- drops Request dependency on NetworkManager
- drops Response dependency on NetworkManager
- drops requestIds from Request and Response objects
2017-06-29 18:18:06 -07:00
Andrey Lushnikov
3dfa688920 Simplify lib/Navigator.js 2017-06-29 12:28:05 -07:00
Andrey Lushnikov
4aa74cc7f8 cleanup Page.js from network leftovers after a35a21d 2017-06-29 12:00:19 -07:00
Andrey Lushnikov
a35a21dfaf Implement NetworkManager
This patch implements NetworkManager, which encapsulates all the
interaction with Network domain.

The NetworkManager also uses partial implementation of Request and
Response classes, defined in the Fetch API specification.

References #26
2017-06-29 11:49:56 -07:00
Patrick Hulce
4b0b81fd9b Add better network idle definition (#38)
This patch:
- Changes network idle promise to wait for 2 or fewer network requests for at least idleTime (defaults to 5s) before resolving.
- Adds timer cleanup to failure navigation case.
- Adds handling of webSocketClosed.
- Ignores unrecognized requestIds to avoid negative inflight requests.

References #10
2017-06-28 14:39:37 -07:00
Andrey Lushnikov
d5be1a6436 Introduce Page.Events.Request
This patch introduces the 'request' event which is fired when
page has initiated a request.

The event dispatches an instance of Request class.

References #26.
2017-06-28 01:42:42 -07:00
Andrey Lushnikov
7b59a89695 Implement Request object
This patch does a step towards Fetch API:
- implements Request object to some extend. The Request object will be
  sent in RequestWillBeSent event.
- implements InterceptedRequest which extends from Request and allows
  for request modification. The InterceptedRequest does not
  conform to Fetch API spec - there seems to be nothing related to
  amending in-flight request.
- adds test to make sure that request can change headers.

References #26
2017-06-28 01:41:42 -07:00
Andrey Lushnikov
5ed71fcb8f Test the 'networkidle' navigation logic
This patch adds a test to verify that navigation properly waits for the
network to become idle.

References #10
2017-06-27 22:02:46 -07:00
JoelEinbinder
d5a91650ae Implement Basic input API
This patch implements Basic Input api:
- Page.focus(selector) - focuses element with selector
- Page.click(selector) - clicks element with selector
- Page.type(text) - types text into a focused element

Fixed #43.
2017-06-27 18:27:22 -07:00
Andrey Lushnikov
3d90ea38a9 Implement Frame.evaluate
This patch implements Frame.evaluate method.

References #4.
2017-06-27 14:57:14 -07:00
Andrey Lushnikov
819fa355f4 Convert var's to let's
This patch:
- converts all var's to let's throughout the codebase
- enforces the let's over var's via the eslint rule
2017-06-22 14:58:39 -07:00
Pavel Feldman
cf35524285 Foramt JSDocs for 2 spaces 2017-06-21 14:11:52 -07:00
Pavel Feldman
437a93b26e Reformat code using 2 spaces 2017-06-21 14:11:52 -07:00
Pavel Feldman
bc0655b587 Extract basic in-flight counting navigator 2017-06-21 14:11:52 -07:00
Pavel Feldman
4761f13740 Inline helper eval functions 2017-06-21 14:11:52 -07:00
Pavel Feldman
84bc09bce1 Revert "Reformat codebase into 2-spaces"
This reverts commit d0d1ee303e41fe4ba762a031b78c3894edac52df.
2017-06-21 14:11:52 -07:00
Andrey Lushnikov
448ac4ce64 Reformat codebase into 2-spaces
This patch:
- reformats codebase to use 2-spaces instead of 4. This will
  align the project with other codebases (e.g. DevTools and Lighthouse)
- enables eslint indentation checking

References #19
2017-06-21 14:11:52 -07:00
Andrey Lushnikov
aaefec7868 Fix Travis-Ci tests
It turned out that the tip-of-tree chromium fails to start on Travis
because of two reasons:
- inability to run LinuxSUIDSandbox
- libnss3 library version being too small

Both problems happen because of the outdated "trusty" distribution
which is used on travis.

This patch:
- reverts the previous patch 9e6f779. Instead of introducing such
  a weird "api", both phantom_shim/runner.js and test/test.js no
  explicitly pass '--no-sandbox' flag to the browser
- updates the libnss3 lib on travis

Fixes #33
2017-06-21 14:11:52 -07:00
Andrey Lushnikov
7874108fda Fix running on Travis-Ci 2017-06-21 14:11:52 -07:00
Andrey Lushnikov
eb02af93b6 Improve error reporting when chrome fails to start
This patch starts throwing exception and dumping chromium stderr when
the browser fails to start.

References #33
2017-06-21 14:11:52 -07:00
Andrey Lushnikov
175963182e Implement FrameManager
This patch implements FrameManager which is responsible for maintaining
the frame tree. FrameManager is quite basic: it sends FrameAttached,
FrameDetached and FrameNavigated events, and can report mainFrame and
all frames.

The next step would be moving certain Page API's to the Frame. For
example, such method as Page.evaluate, Page.navigate and others should
be available on Frame object as well.

References #4
2017-06-21 14:11:52 -07:00
Paul Irish
a66480a416 Fix ESLint failures (#34)
References #33
2017-06-21 14:11:52 -07:00
Pavel Feldman
8e0a3ac6d9 Make InPageCallback async (#24)
Fixes #20
2017-06-21 14:11:52 -07:00
Pavel Feldman
9ad4938fcb Inline helper evaluate functions 2017-06-21 14:11:52 -07:00
Andrey Lushnikov
868814ac7f Implement fullPage screenshots
This patch adds a 'fullPage' option to the Page.screenshot
method.

Fixes #6.
2017-06-16 22:34:29 -07:00
Andrey Lushnikov
dbb374d4af Fix racy condition in case of multiple parallel screenshots
Page.screenshot operates the global state of the page. In case of
multiple Page.screenshot() commands running in parallel with different
clipping rects, they interfere with each other.

This patch makes Page.screenshot() commands run sequencially
even though they were called in parallel.

Fixes #15.
2017-06-16 20:20:36 -07:00
Andrey Lushnikov
3b0bc0802d Refactor Page.screenshot() api
This patch refactors Page.screenshot api, accoring to the discussion
in #5:
- Page.screenshot accepts single optional options object
- Page.saveScreenshot is removed
- Page.screenshot assumes 'png' screenshot if no type is set and no
  'path' property is given

Fixes #5.
2017-06-16 17:15:24 -07:00
Andrey Lushnikov
6bed8c62b3 Pass integers to the Emulation.setVisibleSize
Integers are required in the Emulation.setVisibleSize. This patch
fixes the screenshot clipRect so that it never tries to pass
float values.
2017-06-16 15:43:09 -07:00
Andrey Lushnikov
163e14345d Remove Page.handleDialog method
This is a left-over after refactoring in f62cfc3b.

References #2.
2017-06-16 11:24:47 -07:00
Andrey Lushnikov
632b90efae Page.Events.Error should throw an proper error
This patch:
- renames Page.Events.Exception in Page.Events.Error
- dispatches an error which has a page stack as its message
2017-06-16 11:21:44 -07:00
Andrey Lushnikov
2066da9ec7 Page.evaluate should reject in case of evaluation error
This patch makes Page.evaluate to throw an error if there
was an error in evaluated code.
2017-06-16 10:34:13 -07:00
Andrey Lushnikov
ff2c3bbca9 Cleanup Page class
This patch removes unneeded dependency from Page
to Browser.
2017-06-15 21:35:31 -07:00
Andrey Lushnikov
f62cfc3b34 Refactor JavaScript dialog API
This patch introduces a Dialog class and a new 'dialog'
event instead of the 'alert', 'beforeunload', 'confirm' and
'prompt' events and 'Page.handleDialog' method.

Fixes #2.
2017-06-15 21:22:41 -07:00
Andrey Lushnikov
14a75a83ea Merge Page.evaluate and Page.evaluateAsync together
This patch makes Page.evaluate await promise if one is
returned by the evaluated code.

This makes the Page.evaluateAsync unneeded, so the patch
removes it.

Fixes #11.
2017-06-15 14:56:40 -07:00
Andrey Lushnikov
50976c7f29 Implement Request.setHeader() method
This patch implements Request.setHeader() method to override
request headers.

References #8.
2017-06-15 08:37:06 -07:00
Andrey Lushnikov
5ba6621cde Remove the Page.setBlockedURLs method
This patch removes the Page.setBlockedURLs method. The
functionality is trivially implementable with the request
interception (see examples/loadurlwithoutcss.js).

Fixes #1.
2017-06-15 08:26:55 -07:00
Andrey Lushnikov
e274c26e8b Implement Page.setRequestInterceptor
This patch:
- introduces Request class.
- implements Page.setRequestInterceptor method. The method
  allows to install a callback which will be called for every request
  with a |Request| object as a single parameter. The callback is free
  to override certain request's properties and then either continue or
  abort it.
- implements request interception api for phantom-shim and unskips the
  module/webpage/abort-network-request.js phantomjs test

References #1
2017-06-15 08:26:50 -07:00
Andrey Lushnikov
9bdf9ed5de Fix jsdoc after 67932b8
This patch gets rid of "CDP" type in jsdoc annotations.
2017-06-14 18:37:42 -07:00
Andrey Lushnikov
67932b87c1 Drop the chrome-remote-interface dependency
This patch drops the chrome-remote-interface dependency and
introduces Connection class which handles all the communication
with remote target.

Closes #3
2017-06-14 15:45:59 -07:00
Andrey Lushnikov
dbffc3c35c Await promises returned from the inPageCallback
This patch teaches puppeteer to await promises returned
from the inPageCallback.
2017-06-14 08:21:56 -07:00
Andrey Lushnikov
ec414eb774 Rename page.size() and page.setSize() into page.viewportSize()
It turns page.size() and page.setSize() methods are slightly
confusing since there multiple different sizes (layout size,
content size, viewport size..)

This patch renames Page.{size,setSize} methods into
Page.{viewportSize,setViewportSize} methods to avoid confusion.
2017-06-14 07:41:26 -07:00
Andrey Lushnikov
7136b3aaff Improve inPageCallback handling
This patch improves inPageCallback handling so that it doesn't
retain any variables.
2017-06-13 20:09:50 -07:00
Andrey Lushnikov
549535e674 Add docs/api.md
This patch adds docs/api.md file which contains API description
for the puppeteer API. This patch adds the API outline, which
doesn't not have explanations and samples.
2017-06-13 00:28:39 -07:00
Andrey Lushnikov
ac3e76fb22 Cleanup typechecker errors
This patch cleans up different issues which were spotted by the
typechecker.
2017-06-11 23:10:33 -07:00
Andrey Lushnikov
1f51384918 Introduce Eslint to validate style
This patch introduces eslint and fixes multiple minor code
style issues.
2017-06-11 01:32:59 -07:00
Andrey Lushnikov
2b50d8cc32 Introduce utils/check_availability.js utility
This patch introduces check_availability.js utility which looks for
available chromium binaries for different revisions and platforms.

This patch also re-factors the chromium downloader scripts so that
it can operate different platforms.
2017-06-08 11:22:21 -07:00
Andrey Lushnikov
e05fd8c7b9 minor code cleanup 2017-05-14 23:44:35 -07:00
Andrey Lushnikov
245984905e Implement Puppeteer's setBlockedURLs method
This patch implements Puppeteer's Page.setBlockedURLs method.
This is a less agile alternative to the phantom's request.abort()
api.

This patch also adds a loadurlwithoutcss.js example re-implementation
to illustrate the usage of Page.setBlockedURLs.
2017-05-14 22:28:19 -07:00
Andrey Lushnikov
2f74644cb8 Puppeteer's Page.navigate should not throw with invalid URL.
This patch makes puppeteer's Page.navigate to just return
'false' when given an invalid URL.
2017-05-14 22:17:57 -07:00
Andrey Lushnikov
91e54378a3 Implement printing to PDF
This patch implements:
- puppeteer's Page.printToPDF method. The method's defaults
  are similar to phantom's defaults for the paperSize property.
- phantom's render into pdf file.
2017-05-14 22:00:39 -07:00
Andrey Lushnikov
bdf895bed6 Support Page.{Alert,Confirm,Prompt,BeforeUnload} events
This patch adds Page events to fire when javascript dialogs
get opened.
2017-05-13 13:44:24 -07:00
Andrey Lushnikov
939038bb08 Convert event names to small caps
This patch takes inspiration from DOM events and makes all the
events small-caps.
2017-05-13 12:04:30 -07:00
Andrey Lushnikov
15b36b1cf0 Drop unneeded Page Event's prefixes.
This patch drops 'Page.Event.' prefix in every puppeteer's page
event. This makes it convenient to subscribe to events by their
string value.
2017-05-13 11:12:06 -07:00
Andrey Lushnikov
8a8076c15b Rename Page.Events.ConsoleMessageAdded -> Page.Events.ConsoleMessage
This patch:
- renames ConsoleMessageAdded into ConsoleMessage for the sake of
  clarity
- adds a test to cover basic functionality
2017-05-13 11:05:54 -07:00
Andrey Lushnikov
e8af69e5bb Make in-page callback survive navigations
This patch makes in-page callbacks survive navigations via
running in-page harness code on page load.
2017-05-12 17:55:29 -07:00
Andrey Lushnikov
58ad5dd823 fix implementation of Page.injectFile method 2017-05-12 14:45:47 -07:00
Andrey Lushnikov
c3a3bfe1d2 Implement Page.evaluateOnInitilized method
The PhantomJS has a similar callback called onInitialized. This
callback passes control to the automation script when the page
gets initialized.

To precisely implement this functionality atop of puppeteer,
and since puppeteer controller script lives in a separate process to
the page, we need an ability to pause page at the moment of
initialization. For now, we are not able to do this.

However, oftentimes clients want to evaluate certain code in
page at the point of page initialization. This patch implements
this capability with the Page.evaluateOnInitilized method call.

This patch also re-implements phantom's unrandomize.js example
with the puppeteer API. This is serves an illustration purpose
for the page.evaluateOnInitilized callback.
2017-05-12 14:13:40 -07:00
Andrey Lushnikov
52e2633f32 Introduce Page.saveScreenshot
Page.saveScreenshot uses filepath to infer screenshot type (either
PNG of JPG) and saves the screenshot to file system.
2017-05-12 10:50:06 -07:00
Andrey Lushnikov
2de672ed1b Introduce Page.injectFile method
With this patch, page has two methods to include javascript:
- Page.addScriptTag(url) which is similar to phantom's includeJs.
- Page.injectFile(filePath) which is similar to phantom's injectJs.
2017-05-12 10:19:01 -07:00
Andrey Lushnikov
e991b0e20a Implement puppeteer's Page.addScript 2017-05-12 10:01:22 -07:00
Andrey Lushnikov
2cda8c18d1 Puppeteer: staging commit. 2017-05-11 00:06:41 -07:00