`testRunner.run()` might have 4 different outcomes:
- `ok` - all non-skipped tests passed
- `failed` - some tests failed or timed out
- `terminated` - process received SIGHUP/SIGINT while testrunner was running tests. This happens on CI's under certain circumstances, e.g. when
VM is getting re-scheduled.
- `crashed` - testrunner terminated test execution due to either `UnhandledPromiseRejection` or
some of the hooks (`beforeEach/afterEach/beforeAll/afterAll`) failures.
As an implication, there are 2 new test results: `terminated` and `crashed`.
All possible test results are:
- `ok` - test worked just fine
- `skipped` - test was skipped with `xit`
- `timedout` - test timed out
- `failed` - test threw an exception while running
- `terminated` - testrunner got terminated while running this test
- `crashed` - some `beforeEach` / `afterEach` hook corresponding to this
test timed out of threw an exception.
This patch changes a few parts of the testrunner API:
- `testRunner.run()` now returns an object `{result: string,
terminationError?: Error, terminationMessage?: string}`
- the same object is dispatched via `testRunner.on('finished')` event
- `testRunner.on('terminated')` got removed
- tests now might have `crashed` and `terminated` results
- `testRunner.on('teststarted')` dispatched before running all related
`beforeEach` hooks, and `testRunner.on('testfinished')` dispatched after
running all related `afterEach` hooks.
This patch:
- updates Flakiness Dashboard format to define version per-build
and to pass COMMIT information
- drops the README.md generation - we'll move on to a designated flakiness
dashboard viewer
- fix `FLAKINESS_DASHBOARD_BUILD_URL` to point to a task instead of a build
- do not pretty-print `dashboard.json` when serializing flakiness results
- filter out 'COVERAGE' test(s) so that they don't add up to `dashboard.json` payload. These are useless
- validate certain important options of flakiness dashboard
- more logging to STDOUT to actually say which repo and what branch is getting used
- enhance commit message with a build URL
- use a more compact format for JSON. For 100 runs of 700 tests it yields 21MB json instead of 23MB.
- bump default builds number to 100
This patch introduces a dashboard that records test results and
uploads them to https://github.com/aslushnikov/puppeteer-flakiness-dashboard
Since many bots might push results in parallel, each bot pushes
results to its own git branch.
FlakinessDashboard also generates a simple README.md with a flakiness
summary. If this proves to be not enough, we can build a website that
fetches flakiness data and renders it nicely.
This patch teaches TestRunner to support async suite
descriptions. This is needed to require tests using ES6 dynamic
imports:
```js
const t = new TestRunner();
await t.describe('tests', async () => {
(await import('./some.spec.js')).addTests(t);
(await import('./other.spec.js')).addTests(t);
});
```
This patch adds new TestRunner options:
- `disableTimeoutWhenInspectorIsEnabled` - disable test timeout if
testrunner detects enabled inspector.
- `breakOnFailure` - if testrunner should terminate test running on
first test failure
This patch improves the logic for test runner termination.
With this patch:
- TestRunner runs all afterEach/afterAll hooks when a
termination happens, properly terminating browser instances
- TestRunner cleans up all dangling timeout timers so that node.js
process is not retained and is free to exit
These getters are introduced as a more convenient substitute for
a `require('puppeteer/Errors')` and
`require('puppeteer/DeviceDescriptors')`.
This way we can make cross-browser story nicer - a single require
of `puppeteer` or `puppeteer-firefox` fully defines Puppeteer
environment.
This patch makes sure header overrides in request interception are
functioning as expected.
Drive-by: teach test server to use utf-8 charset header for text files.
This patch:
- implements Response.buffer() and other methods
- splits out relevant tests into a separate test suites
- implements `testServer.enableGzip()` method to optionally gzip
certain routes in tests
- adds tests to make sure `Response.text()` returns expected results
for binary and compressed responses.
This patch:
* unifies assets between tests
* enables a few puppeteer tests on Puppeteer-Firefox
Drive-by: beautify failing output of `expect.toEqual` matcher.
References #3889
This patch:
- introduces new testRunner methods `addTestDSL` and `addSuiteDSL`
to add annotated test / suite.
- introduces new test/suite declaration methods: `it_fails_ffox` and
`describe_fails_ffox`. These are equal to `it`/`describe` for chromium
tests and to `xit`/`xdescribe` for firefox.
- marks all unsupported tests with `it_fails_ffox`
- adds a new command-line flag `'--firefox-status'` to `//test/test.js`.
This flag dumps current amount of tests that are intentionally skipped
for Firefox.
End goal: get rid of all `it_fails_ffox` and `describe_fails_ffox`
tests.
Drive-By: remove cookie tests "afterEach" hook that was removing
cookies - it's not needed any more since every test is run in a
designated browser context.
References #3889
Introduce `//lib/api.js` that declares a list of publicly exposed
classes.
The `//lib/api.js` list superceedes dynamic `helper.tracePublicAPI()` calls
and is used in the following places:
- [ASYNC STACKS]: generate "async stacks" for publicy exposed API in `//index.js`
- [COVERAGE]: move coverage support from `//lib/helper` to `//test/utils`
- [DOCLINT]: get rid of 'exluded classes' hardcoded list
This will help us to re-use our coverage and doclint infrastructure
for Puppeteer-Firefox.
Drive-By: it turns out we didn't run coverage for `SecurityDetails`
class, so we lack coverage for a few methods there. These are excluded
for now, sanity tests will be added in a follow-up.
This patch splits out `IsolatedWorld` class from Frame.
The `IsolatedWorld` abstraction is an execution context
with a designated set of DOM wrappers.
References #2671
This patch teaches `page.setContent` to await resources in
the new document.
**NOTE**: This patch changes behavior: currently, `page.setContent`
awaits the `"domcontentloaded"` event; with this patch, we can now await
other lifecycle events, and switched default to the `"load"` event.
The change is justified since current behavior made `page.setContent`
unusable for its main designated usecases, pushing our client
to use [dataURL workaround](https://github.com/GoogleChrome/puppeteer/issues/728#issuecomment-334301491).
Fixes#728
This patch adds a new utility - `utils/bisect.js` - that accepts
a range of Chromium revisions and a pptr script and bisects the
range to figure when the script breaks.
The Puppeteer Script, given to the tool, should be exiting
with non-zero code to signify malfunctioning.
Example:
```
$ node utils/bisect.js --good 577361 --bad 599821 a.js
```
Also, new Chrome now exposes a new type in its protocol - binary.
It becomes a raw C++ array once used through C++ bindings, but for
us it's still a base64 string.
This adds `page.accessibility.snapshot()`. It serializes and returns the accessibility tree for the page. By default, uninteresting nodes are filtered out of the snapshot.
fixes#2033
This patch:
- adds experimental "transport" option to pptr.connect
- uses "transport" option to make sure Puppeteer-Web works with
Target.exposeDevToolsProtocol
Drive-by: add `browser.target()` to access browser target.
This patch:
- adds "browser" field to the package.json with default
bundling options.
- introduces "bundle" and "unit-bundle" commands to
create bundle and test bundle
- starts running bundle tests on Travis Node 8 bots
Fixes#2374.
Currently connection assumes that transport is a websocket
and tries to handle websocket-related errors.
This patch:
- moves ConnectionTransport interface to use callbacks instead
of events. This way it could be used in browser context as well.
- introduces WebSocketTransport that implements ConnectionTransport
interface for ws.
This is a preparation step for 2 things:
- exposing `transport` option in the `puppeteer.connect` method
- better support for `browserify`
References #2119
When we merge commits to master, Travis kicks job to build a new commit
and to publish new version of puppeteer@next.
If two commits are landed in almost the same time, then travis starts
two parallel jobs to build each commit. This race condition results
in the incorrect puppeteer@next revision.
This patch teaches apply_next_version.js to verify if current HEAD
is matching upstream HEAD. If it doesn't, the predeploy hook fails
which (hopefully) aborts deployment.
Fixes#2925.
One of our checks makes sure all links from README.md to API.md
point to the last-released version of the API.
This sometimes doesn't work: when we refer to a section
in api.md that is just added, we should be able to reference
the "master" version of the api.md
This patch:
- teaches the doclint check to keep links to tip-of-tree version
of api.md in README.md intact.
- starts refering to tip-of-tree version of api.md in `puppeter-core` section
This patch adds a new require, `puppeteer/Errors`, that
holds all the Puppeteer-specific error classes.
Currently, the only custom error class we use is `TimeoutError`. We'll
expand in future with `CrashError` and some others.
Fixes#1694.
This patch:
- updates `utils/fetch_devices.js` script to format UAs for Chrome UAs
and to add iPhone 6/7/8 as separate devices.
- re-generates `DeviceDescriptors.js` with the new script
Fixes#2730.
Since Node 10, `console.assert` no longer throws an AssertionError.
(This is generally good since it aligns Node.js with Browsers.)
This patch migrates all usages of `console.assert` in our codebase.
- All the `lib/` and testing code is migrated onto a handmade `assert`
function. This is to make Puppeteer transpilation / bundling easier.
- All the tooling is switched to use Node's `assert` module.
Fixes#2547.
This patch drops the markdown-toc module and instead rolls out
our own simple markdown table-of-contents generator.
As a side effect, it fixes links to `page.$` and `page.$$`.
This patch allows logging the output of the Chromium process to be enabled in tests by passing in the environment variable `DUMPIO=true`.
Additionally, the `stderr` of the Chromium process will always be logged in the the "Output" section of failing page tests.
Scrollbars look different on different platforms, so must be made invisible in tests. As a drive-by, xdescribe was broken with the new test runner.
References #2524
Previously protocol.d.ts was generated on `npm run tsc`. This was inconvenient because it meant that vscode checking was wrong until type checking was run manually, and was inefficient because it necessarily regenerated the types even if no new Chromium was downloaded. This patch generates the types when npm install is run from the github checkout, assuming a new Chromium revision was downloaded.
SourceFactory was meant to cache Sources so that they could be used
in different preprocessor tasks.
This turned out to be over-engineering. This patch kills the layer.
Last release v1.3.0 had an error in the documentation, claiming
it wasn't released.
This patch makes sure we have a little bit of automation in place
to save us from this in future.
This uses the `/json/protocol` endpoint to generate type definitions for the protocol.
Currently it is lacking protocol events and commands, but I will add those later.
This patch introduces `BrowserFetcher` class that manages
downloaded versions of products.
This patch:
- shapes Downloader API to be minimal yet usable for our needs. This
includes removing such methods as `Downloader.supportedPlatforms` and
`Downloader.defaultRevision`.
- makes most of the fs-related methods in Downloader async. The only
exception is the `Downloader.revisionInfo`: it has stay sync due to the
`pptr.executablePath()` method being sync.
- updates `install.js` and `utils/check_availability.js` to use new API
- finally, renames `Downloader` into `BrowserFetcher`
Fixes#1748.
feat: expose raw devtools protocol connection
This patch introduces `target.createCDPSession` method that
allows directly communicating with the target over the
Chrome DevTools Protocol.
Fixes#31.
This patch adds two new methods to the `page.coverage` namespace:
- `page.coverage.startCSSCoverage()` - to initiate css coverage
- `page.coverage.stopCSSCoverage()` - to stop css coverage
The coverage format is consistent with the JavaScript coverage.
This patch introduces a new `page.coverage` namespace with two methods:
- `page.coverage.startJSCoverage` to initiate JavaScript coverage
recording
- `page.coverage.stopJSCoverage` to stop JavaScript coverage and get
results
This patch migrates tests so that they can be run concurrently.
- By default, tests still run in one thread.
- To run tests in 4 parallel threads, run node test/test.js -j 4 or npm run unit -- -j 4
- Environment variable PPTR_PARALLEL_TESTS could be set to override default parallelization
Every test gets passed in a state. State is set up in the beforeAll and beforeEach callbacks,
and should be teared down in the afterAll and afterEach callbacks.
By default, state has a parallelIndex variable initialized that defines the thread index that runs the execution.
This patch unifies node6 transpilation:
- instead of generating multiple top-level directories, prefixed with
`node6-`, all transpiled code gets placed under single `node6/` folder
- transpilation doesn't change require paths of transpiled modules any
more
This patch:
- renames ChromiumDownloader into just Downloader (this is in
preparation for different products download)
- moves Downloader from utils/ to lib/. This unifies all of the
production-critical code in the lib/.
Drive-by: make Downloader a regular class.
This patch introduces a tiny test runner to run puppeteer tests.
The test runner is self-container and allows parallel (wrt IO) test execution.
It will also allow us to split tests into multiple files if necessary.
Comparing to the jasmine, the testrunner supports parallel execution, properly
handles "unhandled promise rejection" event and signals.
Comparing to ava/jest, the testrunner doesn't run multiple node processes,
which makes it simpler but sufficient for our goals.
In Blink, frames don't necesserily have execution context all the time.
DevTools Protocol precisely reports this situation, which results in
Puppeteer's frame.executionContext() being null occasionally.
However, from puppeteer point of view every frame will have at least a
default executions context, sooner or later:
- frame's execution context might be created naturally to run frame's
javascript
- if frame has no javascript, devtools protocol will issue execution
context creation
This patch builds up on this assumption and makes frame.executionContext()
to be a promise.
As a result, all the evaluations await for the execution context to be created first.
Fixes#827, #1325
BREAKING CHANGE: this patch changes frame.executionContext() method to return a promise.
To migrate onto a new behavior, await the context first before using it.
This patch:
- introduces Target class that represents any inspectable target, such as service worker or page
- emits events when targets come and go
- introduces target.page() to instantiate a page from a target
Fixes#386, fixes#443.
This patch:
- updates JSHandle.toString to make a nicer description for primitives
- excludes JSHandle.toString from documentation to avoid its abuse
References #382
This patch starts using typescript to lint JSDoc annotations.
Note: this uses typescript's bleeding edge. We should migrate to stable once
it has all the necessary bugfixes.
References #65.
This patch:
- introduces ExecutionContext class that incapsulates javascript
execution context. An examples of execution contexts are workers and
frames
- introduces JSHandle that holds a references to the javascript
object in ExecutionContext
- inherits ElementHandle from JSHandle
Fixes#382.
Last commit 017429eef1 broke doclint
tests. Try bots didn't catch this because they were not running doclint
tests.
This patch:
- fixes doclint tests
- starts running doclint tests on travis
This patch:
- gives meaningful names to doclint tests
- supports classes inheritance in documentation linter. When class A
extends class B, all methods of class B are added to documentation of
class A.
This is a prerequisite for Object Handles: ElementHandle will be
extending ObjectHandle.
References #382
This patch introduces ConsoleMessage type and starts dispatching
it for the 'console' event.
BREAKING CHANGE: this breaks the api of the 'console' event.
Fixes#744.
This patch:
- makes `browser.close()` return a promise that resolves when browser gets closed
- starts closing chrome gracefully if a custom `userDataDir` is supplied
Fixes#527
This patch:
- adds `page.touchscreen` namespace, similar to `page.mouse` and `page.keyboard`.
- adds tapping to multiple layers:
- `page.touchscreen.tap`
- `page.tap` - convenience method which accepts selector
- `elementHandle.tap`
Fixes#568 and #569.
This patch:
- introduces a transpiler which substitutes async/await logic with
generators.
- starts using the transpiler to generate a node6-compatible version of puppeteer
- introduces a runtime-check to decide which version of code to use
Fixes#316.
Fail gracefully when chromium failed to download
This patch changes both install.js and Launcher.js to inform how
chromium could be downloaded manually.
This patch:
- refactors Connection to use a single remote debugging URL instead of a
pair of port and browserTargetId
- introduces Puppeteer.connect() method to attach to already running
browser instance.
Fixes#238.
This patch:
- fixes multimap implementation to work properly in node
- moves ESTreeWalker from third-party into utils/doclint. ESTreeWalker
license is compliant with Apache2.0.
This patch:
- split browser launching logic from Browser into `lib/Launcher.js`
- introduce `puppeteer` namespace which currently has a single `launch`
method to start a browser
With this patch, the browser is no longer created with the `new
Browser(..)` command. Instead, it should be "launched" via the
`puppeteer.launch` method:
```js
const puppeteer = require('puppeteer');
puppeteer.launch().then(async browser => {
...
});
```
With this approach browser instance lifetime matches the lifetime of
actual browser process. This helps us:
- remove proxy streams, e.g. browser.stderr and browser.stdout
- cleanup browser class and make it possible to connect to remote
browser
- introduce events on the browser instance, e.g. 'page' event. In case
of lazy-launching browser, we should've launch browser when an event
listener is added, which is unneded comlpexity.
This patch
- rolls chromium to 492629
- migrates connection establishing to use browser target. This migration means
that now we have a single websocket connection to browser (implemented
in Connection class). A connection to a particular target is
incapsulated in a new Session class.
These commands proved to be over-complicating the documentation source.
We should keep documentation source as simple to edit as possible to
make it friendly to contributions.
This patch keeps the gen:version command as it is non-invasive.
The two tasks allow to copy text from one part of document to another.
This comes handy in organizing the documentation for our shortcut
methods, which should be exactly the same as the original methods.
The tasks work like this:
- the gen:copy(id) task saves a part of document under the name 'ID'.
- the gen:paste(id) task pastes text saved with id 'ID'
This patch also fixes a bunch of links in documentation, as well as
migrating `api.md` to use the two tasks.
This patch implements simple markdown preprocessor. The goal
is to generate certain parts of markdown, such as:
- puppeteer version
- chromium revision
- table-of-contents
- copy/paste parts of documentation (for shortcut methods)
This patch refactors doclint so that more checks and more generators
could be added.
This patch:
- Introduces 'Source' class, which holds file content in-memory and
allows it to be updated.
- Introduces 'Message' class - which is a pair of a text and a type.
Messages could have either 'error' type or 'warning' type.
Refactor Frame.waitForSelector to make room for Frame.waitForFunction
implementation.
This patch:
- removes AwaitedElement class which proved to be confusing, and
introduces a more straight-forward WaitTask.
- refactors the mutation observer to return true in case of successful
waiting or false in case of timeout.
References #91
This patch:
- adds Mouse class which holds mouse state and implements mouse primitives,
such as moving, button down and button up.
- implements high-level mouse api, such as `page.click` and `page.hover`.
References #40, References #89
This patch:
- introduces page.waitForSelector to wait for the selector to appear
- introduces polymorphic page.waitFor method, which accepts
either string (and in this case is a shortcut for page.waitForSelector)
or number (and in this case it's a promisified timeout).
References #91.
This patch teaches doclint to regenerate table of contents
automatically whenever it's needed.
This patch:
- splits lint.js into lint.js and cli.js
- teaches cli.js to generate table-of-contents
- removes the test for table-of-contents errors from doclint
- adds a test for doclint failing to parse object destructuring in
method parameters.
This patch adds a 'visible' option to the Page.waitFor method, making
it possible to wait for the element to become actually visible.
References #89, #91.
This patch converts lib/DevicesDescriptors from a devtools front-end
format into a puppeteer format.
This patch does this via introducing a scripts utils/fetch_devices.js
which grabs devices from upstream of DevTools Front-end and
converts them into puppeteer devices.
References #88.
Currently, it's impossible to do screenshots in parallel.
This patch:
- makes all screenshot tasks sequential inside one browser
- starts activating target before taking screenshot
- adds a test to make sure it's possible to take screenshots across
tabs
- starts waiting for a proper page closing after each test. This might
finally solve the ECONNRESET issues in tests.
References #89
This patch introduces Page.waitForNavigation which allows to wait
for render-initiated navigation.
This patch also does a nice refactoring, replacing Navigator with NavigatorWatcher which
is not a part of a page state.
References #89
