Commit Graph

229 Commits

Author SHA1 Message Date
Andrey Lushnikov
508a78d99b
test: prohibit focused tests/suites on bots (#1686) 2017-12-28 15:41:57 -08:00
Andrey Lushnikov
be438c59c1
chore(testrunner): support first-class test debugging (#1606)
This patch teaches testrunner to override both timeout and
parallel execution option if there's attached inspector.
2017-12-15 14:25:06 -08:00
Andrey Lushnikov
a3a3774926
test: nicer matchers output (#1605)
This patch improves matchers to provide more information once
some of them fail.
2017-12-15 14:14:27 -08:00
Andrey Lushnikov
a5db6d412c
test: migrate test.js to support concurrent test execution (#1531)
This patch migrates tests so that they can be run concurrently.

- By default, tests still run in one thread.
- To run tests in 4 parallel threads, run node test/test.js -j 4 or npm run unit -- -j 4
- Environment variable PPTR_PARALLEL_TESTS could be set to override default parallelization

Every test gets passed in a state. State is set up in the beforeAll and beforeEach callbacks,
and should be teared down in the afterAll and afterEach callbacks.

By default, state has a parallelIndex variable initialized that defines the thread index that runs the execution.
2017-12-12 13:34:21 -08:00
Andrey Lushnikov
fc1f15e251
test(testrunner): fix the 'timeout: 0' test runner setting (#1561) 2017-12-08 16:44:24 -08:00
Andrey Lushnikov
391d1abaa7
chore: generalize node6 transpilation (#1560)
This patch unifies node6 transpilation:
- instead of generating multiple top-level directories, prefixed with
  `node6-`, all transpiled code gets placed under single `node6/` folder
- transpilation doesn't change require paths of transpiled modules any
  more
2017-12-08 15:14:28 -08:00
Andrey Lushnikov
9a5086847c
refactor: move ChromiumDownloader under lib/ (#1554)
This patch:
- renames ChromiumDownloader into just Downloader (this is in
  preparation for different products download)
- moves Downloader from utils/ to lib/. This unifies all of the
  production-critical code in the lib/.

Drive-by: make Downloader a regular class.
2017-12-08 13:39:13 -08:00
Andrey Lushnikov
e6725e15af
tests: drop jasmine test runner (#1519)
This patch introduces a tiny test runner to run puppeteer tests.
The test runner is self-container and allows parallel (wrt IO) test execution.
It will also allow us to split tests into multiple files if necessary.

Comparing to the jasmine, the testrunner supports parallel execution, properly
handles "unhandled promise rejection" event and signals.

Comparing to ava/jest, the testrunner doesn't run multiple node processes,
which makes it simpler but sufficient for our goals.
2017-12-07 16:37:22 -08:00
Allan Kimmer Jensen
c55f4e3268 feat(installer): Support HTTP mirror (#1372)
This patch adds support for HTTP mirror for ChromiumDownloader.

This fixes #1371
2017-12-04 13:45:21 -08:00
Thomas Reggi
b28adbb438 fix(Downloader): better handling for HTTPS proxy (#1508)
This patch starts using `rejectUnauthorized` option set to false to improve chances of
downloading chromium behind proxy.
2017-12-04 01:13:53 -08:00
Andrey Lushnikov
6512ce768d
fix(Frame): postpone evaluations until execution context gets created (#1415)
In Blink, frames don't necesserily have execution context all the time.
DevTools Protocol precisely reports this situation, which results in
Puppeteer's frame.executionContext() being null occasionally.

However, from puppeteer point of view every frame will have at least a
default executions context, sooner or later:

- frame's execution context might be created naturally to run frame's
  javascript
- if frame has no javascript, devtools protocol will issue execution
  context creation

This patch builds up on this assumption and makes frame.executionContext()
to be a promise.
As a result, all the evaluations await for the execution context to be created first.

Fixes #827, #1325

BREAKING CHANGE: this patch changes frame.executionContext() method to return a promise.
To migrate onto a new behavior, await the context first before using it.
2017-11-18 16:27:52 -08:00
Andrey Lushnikov
a467d35a5f chore(doclint): do not attempt to lint non-documentation markdown 2017-11-07 03:33:25 -08:00
r3dDoX
945a826a0b feat(install): add environment variable to overwrite host part of url (#958)
This patch adds PUPPETEER_DOWNLOAD_HOST env variable that allows to specify a download host. Useful for downloading from mirrors.

Fixes #951
2017-10-21 18:22:13 -07:00
Sashiyama Yoshiki
eee168a745 chore(downloader): fix typo in error exception message (#1076) 2017-10-17 21:33:16 -07:00
JoelEinbinder
32398d11bd feat(Browser): introduce Browser.pages() (#554)
This patch:
- introduces Target class that represents any inspectable target, such as service worker or page
- emits events when targets come and go
- introduces target.page() to instantiate a page from a target

Fixes #386, fixes #443.
2017-10-17 19:14:57 -07:00
Andrey Lushnikov
079db90066 fix(JSHandle.toString): clearer description for primitives (#993)
This patch:
- updates JSHandle.toString to make a nicer description for primitives
- excludes JSHandle.toString from documentation to avoid its abuse

References #382
2017-10-10 10:54:20 -07:00
JoelEinbinder
e59172de83 chore: Use Typescript to lint JSDoc annotations (#986)
This patch starts using typescript to lint JSDoc annotations.

Note: this uses typescript's bleeding edge. We should migrate to stable once
it has all the necessary bugfixes.

References #65.
2017-10-09 22:31:40 -07:00
Andrey Lushnikov
0d0f9b7984 feat(JSHandles): introduce JSHandles (#943)
This patch:
- introduces ExecutionContext class that incapsulates javascript
  execution context. An examples of execution contexts are workers and
  frames
- introduces JSHandle that holds a references to the javascript
  object in ExecutionContext
- inherits ElementHandle from JSHandle

Fixes #382.
2017-10-06 15:35:02 -07:00
JoelEinbinder
d87480b609 Node6: Remove parentheses around the body of async arrow functions (#942) 2017-10-02 18:25:11 -07:00
Andrey Lushnikov
dc4c8786e3 chore(doclint): fix doclint tests (#941)
Last commit 017429eef1 broke doclint
tests. Try bots didn't catch this because they were not running doclint
tests.

This patch:
- fixes doclint tests
- starts running doclint tests on travis
2017-10-02 15:14:57 -07:00
Andrey Lushnikov
017429eef1 chore(doclint): exclude constructors by default (#938)
Our API does not expose any classes; thus all the constructors
should be excluded from the API.
2017-10-02 14:28:51 -07:00
Andrey Lushnikov
6c9a99477b chore(doclint): support classes inheritance (#935)
This patch:
- gives meaningful names to doclint tests
- supports classes inheritance in documentation linter. When class A
  extends class B, all methods of class B are added to documentation of
  class A.

This is a prerequisite for Object Handles: ElementHandle will be
extending ObjectHandle.

References #382
2017-10-02 13:38:44 -07:00
Andrey Lushnikov
8bcf550bb6 chore(doclint): add basic tests for documentation parsers (#934)
This patch adds basic tests to verify javascript and markdown
documentation parsers.
2017-10-02 11:45:00 -07:00
Andrey Lushnikov
41fd4b529e chore(doclint): move doclint tests expectations into test folders (#932)
This makes for much easier work with doclint tests.
2017-10-02 11:37:16 -07:00
Andrey Lushnikov
f6255029bd feat(Console): Introduce ConsoleMessage type (#909)
This patch introduces ConsoleMessage type and starts dispatching
it for the 'console' event.

BREAKING CHANGE: this breaks the api of the 'console' event.

Fixes #744.
2017-09-29 11:27:22 -07:00
JoelEinbinder
f398e69dbb [api] Launcher: Close gracefully when a userDataDir is specified (#700)
This patch:
- makes `browser.close()` return a promise that resolves when browser gets closed
- starts closing chrome gracefully if a custom `userDataDir` is supplied

Fixes #527
2017-09-13 21:27:14 -07:00
JoelEinbinder
64124df62f [api] add touchScreen.tap (#639)
This patch:
- adds `page.touchscreen` namespace, similar to `page.mouse` and `page.keyboard`.
- adds tapping to multiple layers:
  - `page.touchscreen.tap`
  - `page.tap` - convenience method which accepts selector
  - `elementHandle.tap`

Fixes #568 and #569.
2017-09-01 19:03:51 -07:00
Abdulaziz Homaily
e95fb96279 Throw an error when Chromium download is failed (#619)
This patch teaches ChromiumDownloader to throw an error if
downloading failed.
2017-09-01 09:47:57 -07:00
JoelEinbinder
42fa3af659 Remove promise jsdoc (#641)
This patch removes unnecessary "promise" return types in jsdoc for async functions.
2017-08-31 17:58:07 -07:00
JoelEinbinder
cd81944e66 Test node6 on travis (#551)
This patch starts running all puppeteer tests with node6 on travis-ci.
2017-08-25 19:28:49 -07:00
JoelEinbinder
2c4dfbfd88 Fix node6 support for Object.entries and URL object (#548)
This patch introduces a polyfill for Object.entries which is missing in Node 6
2017-08-25 15:13:59 -07:00
JoelEinbinder
9212863b92 Node 6 support (#484)
This patch:
- introduces a transpiler which substitutes async/await logic with
generators.
- starts using the transpiler to generate a node6-compatible version of puppeteer
- introduces a runtime-check to decide which version of code to use

Fixes #316.
2017-08-24 12:20:05 -07:00
Paul Hawxby
a00ba6a3d9 Adding proxy support for ChromiumDownloader (#455)
This patch teaches puppeteer to download chromium over the
HTTP proxy, if any.

Fixes #448.
2017-08-23 08:33:29 -07:00
JoelEinbinder
a526e9f9fb Stricter eslint rules (#472)
Accept stricter eslint rules.
2017-08-22 14:18:07 -07:00
Eric Bidelman
1f9b4fb4c8 Change let into const (#457)
This patch:
- changes `let` into `const` throughout codebase
- adds eslint check to prefer const over let
2017-08-21 16:39:04 -07:00
Andrey Lushnikov
c1731dd5d7 Fail gracefully when chromium failed to download (#430)
Fail gracefully when chromium failed to download

This patch changes both install.js and Launcher.js to inform how
chromium could be downloaded manually.
2017-08-21 13:34:10 -07:00
Andrey Lushnikov
af89e893e7 Implement basic element handles (#248)
This patch implements basic element handles which a backed with remote objects.

Fixes #111
2017-08-15 14:54:02 -07:00
Andrey Lushnikov
a424f5613a Introduce Puppeteer.connect method (#264)
This patch:
- refactors Connection to use a single remote debugging URL instead of a
  pair of port and browserTargetId
- introduces Puppeteer.connect() method to attach to already running
  browser instance.

Fixes #238.
2017-08-15 14:29:42 -07:00
Andrey Lushnikov
0a1294c7ee general cleanup (#256)
This patch:
- fixes multimap implementation to work properly in node
- moves ESTreeWalker from third-party into utils/doclint. ESTreeWalker
  license is compliant with Apache2.0.
2017-08-14 21:16:59 -07:00
Andrey Lushnikov
13e8580a34 Use puppeteer.launch instead of browser constructor (#255)
This patch:
- split browser launching logic from Browser into `lib/Launcher.js`
- introduce `puppeteer` namespace which currently has a single `launch`
  method to start a browser

With this patch, the browser is no longer created with the `new
Browser(..)` command. Instead, it should be "launched" via the
`puppeteer.launch` method:

```js
const puppeteer = require('puppeteer');
puppeteer.launch().then(async browser => {
  ...
});
```

With this approach browser instance lifetime matches the lifetime of
actual browser process. This helps us:
- remove proxy streams, e.g. browser.stderr and browser.stdout
- cleanup browser class and make it possible to connect to remote
  browser
- introduce events on the browser instance, e.g. 'page' event. In case
  of lazy-launching browser, we should've launch browser when an event
  listener is added, which is unneded comlpexity.
2017-08-14 18:08:06 -07:00
Andrey Lushnikov
4551afc6dc Introduce new interception API (#242)
This patch introduces new interception API, via killing InterceptedRequest and giving the `abort` and `continue` methods to the Request object.
2017-08-11 17:24:31 -07:00
Andrey Lushnikov
fe06c896eb Roll chromium to 492629 (#230)
This patch
- rolls chromium to 492629
- migrates connection establishing to use browser target. This migration means
  that now we have a single websocket connection to browser (implemented
  in Connection class). A connection to a particular target is
  incapsulated in a new Session class.
2017-08-09 16:14:00 -07:00
JoelEinbinder
5d647bf1fa Use const a lot fo places (#188) 2017-08-02 12:06:47 -07:00
JoelEinbinder
bd72e40e73 Introduce page.tracing (#181)
This patch introduces page.tracing, which allows to start and stop
trace recording for a page. The trace could be then saved to file.
2017-08-02 10:45:11 -07:00
Andrey Lushnikov
337315c5fe [doclint] remove preprocessor's gen:copy and gen:paste commands
These commands proved to be over-complicating the documentation source.
We should keep documentation source as simple to edit as possible to
make it friendly to contributions.

This patch keeps the gen:version command as it is non-invasive.
2017-07-31 21:21:45 -07:00
Andrey Lushnikov
4ee8eb8afc [doclint] introduce preprocessor's gen:copy and gen:paste tasks
The two tasks allow to copy text from one part of document to another.
This comes handy in organizing the documentation for our shortcut
methods, which should be exactly the same as the original methods.

The tasks work like this:
- the gen:copy(id) task saves a part of document under the name 'ID'.
- the gen:paste(id) task pastes text saved with id 'ID'

This patch also fixes a bunch of links in documentation, as well as
migrating `api.md` to use the two tasks.
2017-07-31 04:14:41 -07:00
Andrey Lushnikov
3ada7e1adb [doclint] Implement simple markdown preprocessor
This patch implements simple markdown preprocessor. The goal
is to generate certain parts of markdown, such as:
- puppeteer version
- chromium revision
- table-of-contents
- copy/paste parts of documentation (for shortcut methods)
2017-07-31 02:06:27 -07:00
Andrey Lushnikov
73a99c6e0d [doclint] do not use util.promisify
util.promisify is available since node 8. This patch re-implements
the method so that it works in node 7.
2017-07-31 01:39:39 -07:00
Andrey Lushnikov
75a8d7b0c3 [doclint] Prepare doclint for more checks
This patch refactors doclint so that more checks and more generators
could be added.

This patch:
- Introduces 'Source' class, which holds file content in-memory and
  allows it to be updated.
- Introduces 'Message' class - which is a pair of a text and a type.
  Messages could have either 'error' type or 'warning' type.
2017-07-31 00:10:59 -07:00
Andrey Lushnikov
91785d97f3 Add missing LICENSE headers to the source files
This patch:
- adds missing LICENSE headers to the source files
- set's puppeteer version to 0.1.0
- set's repository field in package.json
2017-07-28 01:09:26 -07:00
Andrey Lushnikov
0a3125434e Refactor Frame.waitForSelector method
Refactor Frame.waitForSelector to make room for Frame.waitForFunction
implementation.
This patch:
- removes AwaitedElement class which proved to be confusing, and
  introduces a more straight-forward WaitTask.
- refactors the mutation observer to return true in case of successful
  waiting or false in case of timeout.

References #91
2017-07-24 10:07:43 -07:00
JoelEinbinder
98ee35655f Mouse (#101)
This patch:
- adds Mouse class which holds mouse state and implements mouse primitives,
such as moving, button down and button up.
- implements high-level mouse api, such as `page.click` and `page.hover`.

References #40, References #89
2017-07-21 20:29:31 -07:00
Andrey Lushnikov
794f9bb82a [doclint] move all the lint checks in the lint.js
This patch cleans up Documentation.js file by moving all the checks
into the lint.js

Drive-by: fix the Page.waitFor test.
2017-07-21 14:21:17 -07:00
Andrey Lushnikov
dc032b42b9 Introduce polymorphic page.waitFor method
This patch:
- introduces page.waitForSelector to wait for the selector to appear
- introduces polymorphic page.waitFor method, which accepts
either string (and in this case is a shortcut for page.waitForSelector)
or number (and in this case it's a promisified timeout).

References #91.
2017-07-21 12:54:04 -07:00
Andrey Lushnikov
0960dc38d1 Automatically generate table-of-contents for markdown
This patch teaches doclint to regenerate table of contents
automatically whenever it's needed.

This patch:
- splits lint.js into lint.js and cli.js
- teaches cli.js to generate table-of-contents
- removes the test for table-of-contents errors from doclint
- adds a test for doclint failing to parse object destructuring in
  method parameters.
2017-07-21 11:12:05 -07:00
Andrey Lushnikov
52de75742b Implement visible option for Page.waitFor method
This patch adds a 'visible' option to the Page.waitFor method, making
it possible to wait for the element to become actually visible.

References #89, #91.
2017-07-21 10:01:19 -07:00
Andrey Lushnikov
76ac3bded5 Convert DevicesDescriptors into puppeteer format
This patch converts lib/DevicesDescriptors from a devtools front-end
format into a puppeteer format.

This patch does this via introducing a scripts utils/fetch_devices.js
which grabs devices from upstream of DevTools Front-end and
converts them into puppeteer devices.

References #88.
2017-07-20 17:49:15 -07:00
Andrey Lushnikov
a63a0198de Implement waitFor which survives navigation (#99)
This patch implements page.waitFor method which survives navigation.

References #89.
2017-07-19 19:04:51 -07:00
Andrey Lushnikov
21af495b65 Move screenshot task chain in Browser
Currently, it's impossible to do screenshots in parallel.
This patch:
- makes all screenshot tasks sequential inside one browser
- starts activating target before taking screenshot
- adds a test to make sure it's possible to take screenshots across
  tabs
- starts waiting for a proper page closing after each test. This might
  finally solve the ECONNRESET issues in tests.

References #89
2017-07-19 14:15:16 -07:00
Pavel Feldman
98c3894c84 Introduce Page.waitForNavigation (#94)
This patch introduces Page.waitForNavigation which allows to wait
for render-initiated navigation.

This patch also does a nice refactoring, replacing Navigator with NavigatorWatcher which
is not a part of a page state.

References #89
2017-07-18 18:54:24 -07:00
JoelEinbinder
117a128b42 Introduce Page.$ and Page.$$ (#75)
This patch introduces Page.$ and Page.$$ methods which are
aliases for `document.querySelector` and `document.querySelectorAll`. 

Fixes #78.
2017-07-17 18:56:56 -07:00
JoelEinbinder
bf7698e8f8 Intorduce Page.keyboard (#74)
Introduce page.keyboard to provide low-level access to the keyboard.
2017-07-17 18:49:52 -07:00
Pavel Feldman
895f69d17a Add emulation for named devices. (#72)
This patch introduces page emulation, making it possible to emulate different devices.
2017-07-17 18:13:04 -07:00
Andrey Lushnikov
3b5dbe2308 [doclint] be more permissive with syntax
This patch adds a helpful message if the 'returns' statement is somehow
mistyped.

References #14.
2017-07-15 10:22:07 -07:00
Andrey Lushnikov
6eac22dd87 [doclint] lint events
This patch:
- adds event linting to the doclint
- improves `api.md` to add events and more information about
  classes reported by events.

References #14.
2017-07-14 23:52:13 -07:00
Andrey Lushnikov
235001bab3 [doclint] validate return statements
This patch teaches doclint to validate presence of return statements.
It also adds a bunch of documentation to the api.md.

References #14.
2017-07-14 13:04:15 -07:00
Andrey Lushnikov
67d109862a [doclint] parse getters as properties
From the api standpoint, there's no difference between property
and a getter.

References #14.
2017-07-13 23:12:38 -07:00
Andrey Lushnikov
45c20c7dad [doclint] introduce class members
This patch unifies Documentation.Method with Documentation.Property,
making it possible to verify sorting order across methods and
properties.

References #14.
2017-07-13 23:02:43 -07:00
Andrey Lushnikov
56619baa64 [doclint] simplify error reporting
This patch simplifies error reporting, making it straight-forward.
It also adds a few tests to make doclint more robust.
2017-07-13 15:52:49 -07:00
Andrey Lushnikov
c468c451c5 [doclint] move doclint testing to golden
This patch:
- move doclint testing to golden
- refactor doclint/lint.js to drop specs and everything
2017-07-13 11:17:02 -07:00
Andrey Lushnikov
d99031ba46 [doclint] Move doclint under utils/
This patch:
- moves doclint under utils/ folder
- adds tests to verify doclint basic functionality

This patch also drops the jasmine as a spec runner for the doclint
checks. It turned out it's hard to customize jasmine's behavior,
so instead this patch implements a dummy spec runner.

The dummy spec runner allows us:
- to format messages however we want (the custom jasmine reporter would
  also allow us to do this)
- to avoid `beforeAll` functions which pollute global to pass
  initialized variables over to specs

References #14
2017-07-13 00:28:52 -07:00
Andrey Lushnikov
b3d38a1b8d [Downloader] Fix Downloader.downloadedRevisions methods
The methods should take into account that there's no downloads
folder.

This was broken in 20fb1f6b9b
2017-07-12 14:54:48 -07:00
Andrey Lushnikov
20fb1f6b9b [Downloader] Remove previous chromium revisions after roll
Currently, after the chromium was rolled to a new revision, the
previsious revision is kept downloaded in the puppeteer folder.

Since every revision is ~70Mb, it quickly piles up.

This patch removes previous downloaded chromium revisions after
a new one is successfully installed.
2017-07-12 14:48:42 -07:00
Andrey Lushnikov
819fa355f4 Convert var's to let's
This patch:
- converts all var's to let's throughout the codebase
- enforces the let's over var's via the eslint rule
2017-06-22 14:58:39 -07:00
Pavel Feldman
437a93b26e Reformat code using 2 spaces 2017-06-21 14:11:52 -07:00
Pavel Feldman
84bc09bce1 Revert "Reformat codebase into 2-spaces"
This reverts commit d0d1ee303e.
2017-06-21 14:11:52 -07:00
Andrey Lushnikov
448ac4ce64 Reformat codebase into 2-spaces
This patch:
- reformats codebase to use 2-spaces instead of 4. This will
  align the project with other codebases (e.g. DevTools and Lighthouse)
- enables eslint indentation checking

References #19
2017-06-21 14:11:52 -07:00
Andrey Lushnikov
1f51384918 Introduce Eslint to validate style
This patch introduces eslint and fixes multiple minor code
style issues.
2017-06-11 01:32:59 -07:00
Andrey Lushnikov
2b50d8cc32 Introduce utils/check_availability.js utility
This patch introduces check_availability.js utility which looks for
available chromium binaries for different revisions and platforms.

This patch also re-factors the chromium downloader scripts so that
it can operate different platforms.
2017-06-08 11:22:21 -07:00