Commit Graph

48 Commits

Author SHA1 Message Date
Andrey Lushnikov
ea28cccfe0 feat(page): introduce file chooser interception (#4653)
This patch introduces a page.waitForFileChooser() method
that adds a watchdog to wait for file chooser dialogs.

This lets Puppeteer users to capture file chooser requests
and fulfill/cancel them if necessary.

Fixes #2946
2019-07-22 21:30:49 -07:00
Andrey Lushnikov
955e7cb6fc
feat(page): move page.pdf to protocol streams (#4587)
This lets transferring massive PDF files over the protocol.

Fix #4563
2019-06-14 22:36:06 -07:00
Adriel Codeco
ac611bacac fix(helper): fix Helper.installAsyncStackHooks method (#4478)
Using Error.captureStackTrace instead of new Error () to avoid UnhandledPromiseRejectionWarning
2019-05-31 19:39:32 -07:00
Vse Mozhet Byt
854b1c0912 feat(executioncontext): support bigints transferring (#4016)
Refs: https://chromedevtools.github.io/devtools-protocol/tot/Runtime#type-UnserializableValue
2019-03-15 10:20:48 -07:00
Joel Einbinder
02b2451db5 fix: check if async error has a stack (#4017) 2019-03-04 17:32:54 -08:00
Andrey Lushnikov
62da2366c6
chore: introduce //lib/api.js (#3835)
Introduce `//lib/api.js` that declares a list of publicly exposed
classes.

The `//lib/api.js` list superceedes dynamic `helper.tracePublicAPI()` calls
and is used in the following places:
- [ASYNC STACKS]: generate "async stacks" for publicy exposed API in `//index.js`
- [COVERAGE]: move coverage support from `//lib/helper` to `//test/utils`
- [DOCLINT]: get rid of 'exluded classes' hardcoded list

This will help us to re-use our coverage and doclint infrastructure
for Puppeteer-Firefox.

Drive-By: it turns out we didn't run coverage for `SecurityDetails`
class, so we lack coverage for a few methods there. These are excluded
for now, sanity tests will be added in a follow-up.
2019-01-25 23:21:14 -05:00
Andrey Lushnikov
4e9e3bc614
refactor: consolidate all events in Events.js (#3772)
This will allow us to break all dependency cycles that were forcing
us to put many things in a single file (e.g. ExecutionContext and
ElementHandle).
2019-01-14 19:57:05 -08:00
Joel Einbinder
29a2438534 chore(types): Upgrade TypeScript to 3.2.2 (#3754) 2019-01-10 22:56:39 -08:00
Joel Einbinder
6ac66c3547 feat: browser.waitForTarget (#3356)
This adds `browser.waitForTarget` and `browserContext.waitForTarget`. It also fixes a flaky test that was incorrectly expecting targets to appear instantly.
2018-10-09 14:16:53 -07:00
Andrey Lushnikov
0b9d8a6271
feat: async stacks for all "async" public methods (#3262)
This patch traces all public async methods and wraps them
in a helper method that tags the sync stack trace.

Later on, if the method call throws an exception, we add
a captured stack trace to the original stack trace with the "--ASYNC--"
heading.

An example of a stack trace:

```
Error: net::ERR_ABORTED at http://localhost:8907/empty.html
    at navigate (/Users/lushnikov/prog/puppeteer/lib/Page.js:622:37)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  -- ASYNC --
    at Page.<anonymous> (/Users/lushnikov/prog/puppeteer/lib/helper.js:147:27)
    at fit (/Users/lushnikov/prog/puppeteer/test/page.spec.js:546:18)
    at process._tickCallback (internal/process/next_tick.js:68:7)
```
2018-09-19 13:54:58 -07:00
Andrey Lushnikov
d547b9d24a
fix(browser): browser closing/disconnecting should abort navigations (#3245)
Fixes #2721.
2018-09-14 19:44:54 +01:00
Andrey Lushnikov
f0beabd22a
chore: drop DEBUG for public API calls (#3246)
This proved to be useless: we've never used it since we
released the initial version of puppeteer more than a year ago.
2018-09-14 18:23:30 +01:00
Andrey Lushnikov
f230722ff0
refactor: avoid dynamic requires in lib/ folder (#3208)
This patch removes all dynamic requires in Puppeteer. This should
make it much simpler to bundle puppeteer/puppeteer-core packages.

We used dynamic requires in a few places in lib/:
- BrowserFetcher was choosing between `http` and `https` based on some
  runtime value. This was easy to fix with explicit `require`.
- BrowserFetcher and Launcher needed to know project root to store
  chromium revisions and to read package name and chromium revision from
  package.json. (projectRoot value would be different in node6).
  Instead of doing a backwards logic to infer these
  variables, we now pass them directly from `//index.js`.

With this patch, I was able to bundle Puppeteer using browserify and
the following config in `package.json`:

```json
  "browser": {
    "./lib/BrowserFetcher.js": false,
    "ws": "./lib/BrowserWebSocket",
    "fs": false,
    "child_process": false,
    "rimraf": false,
    "readline": false
  }
```

(where `lib/BrowserWebSocket.js` is a courtesy of @Janpot from
https://github.com/GoogleChrome/puppeteer/pull/2374/)

And command:

```sh
$ browserify -r puppeteer:./index.js > ppweb.js
```

References #2119
2018-09-06 20:33:41 +01:00
Andrey Lushnikov
28d92116b7
feat(puppeteer): support convenience env variables (#3190)
We had (and still have) a ton of pull requests to support
PUPPETEER_EXECUTABLE_PATH and PUPPETEER_CHROMIUM_REVISION in puppeteer launcher.

We were hesitant before since env variables are not scoped
and thus don't make a good interface for a library. Now, since we
determined `puppeteer-core` as a library and `puppeteer` as our end-user
product, it's safe to satisfy our user needs.

This patch:
- teaches PUPPETEER_EXECUTABLE_PATH and PUPPETEER_CHROMIUM_REVISION
  env variables to control how Puppeteer launches browser
- makes sure these variables play no role in `puppeteer-core` package.
2018-09-05 22:59:14 +01:00
Andrey Lushnikov
204c7ec8c4
feat: introduce puppeteer/Errors (#3056)
This patch adds a new require, `puppeteer/Errors`, that
holds all the Puppeteer-specific error classes.

Currently, the only custom error class we use is `TimeoutError`. We'll
expand in future with `CrashError` and some others.

Fixes #1694.
2018-08-09 16:51:12 -07:00
Matthew Shirley
726c8dc046 feat(page): introduce waitForRequest and waitForResponse methods (#2776)
This patch introduces `page.waitForRequest` and `page.waitForResponse` helper methods.

Fixes #2362
2018-07-12 14:36:31 -07:00
Georgii Dolzhykov
9498b1057b fix(helpers): support thrown strings and numbers in getExceptionMessage (#2715)
Of course, strings aren't proper errors, but scripts sometimes do that, and Puppeteer loses such error messages.
2018-06-14 10:23:13 -07:00
Andrey Lushnikov
0b94fa70eb
chore: stop using console.assert everywhere (#2646)
Since Node 10, `console.assert` no longer throws an AssertionError.
(This is generally good since it aligns Node.js with Browsers.)

This patch migrates all usages of `console.assert` in our codebase.
- All the `lib/` and testing code is migrated onto a handmade `assert`
function. This is to make Puppeteer transpilation / bundling easier.
- All the tooling is switched to use Node's `assert` module.

Fixes #2547.
2018-05-31 16:53:51 -07:00
JoelEinbinder
8c54f41552 chore(types): generate a d.ts file for protocol types (#2325)
This uses the `/json/protocol` endpoint to generate type definitions for the protocol.

Currently it is lacking protocol events and commands, but I will add those later.
2018-04-06 18:20:48 -07:00
JoelEinbinder
2370618819 chore(types): upgrade to TypeScript 2.8.1 (#2304)
This converts `externs.d.ts` to export a global namespace instead of a UMD global.

See: https://github.com/Microsoft/TypeScript/issues/22969

Fixes #2279.
2018-04-04 14:06:21 -07:00
Andrey Lushnikov
a363a733b7
feat: Introduce BrowserFetcher class (#1983)
This patch introduces `BrowserFetcher` class that manages
downloaded versions of products.

This patch:
- shapes Downloader API to be minimal yet usable for our needs. This
  includes removing such methods as `Downloader.supportedPlatforms` and
  `Downloader.defaultRevision`.
- makes most of the fs-related methods in Downloader async. The only
  exception is the `Downloader.revisionInfo`: it has stay sync due to the
  `pptr.executablePath()` method being sync.
- updates `install.js` and `utils/check_availability.js` to use new API
- finally, renames `Downloader` into `BrowserFetcher`

Fixes #1748.
2018-02-07 12:31:53 -05:00
Andrey Lushnikov
5368051610
feat: expose raw devtools protocol connection (#1770)
feat: expose raw devtools protocol connection

This patch introduces `target.createCDPSession` method that
allows directly communicating with the target over the
Chrome DevTools Protocol.

Fixes #31.
2018-01-10 19:33:22 -08:00
Gleb Azarov
ff08e45785 fix: Remove synchronous operation with file system. (#879)
This patch:
- introduces `helper.promisify` - a simple polyfill for the `util.promisify`. The 
  `util.promisify` could not be used due to Node6 compatibility issues.
- migrates all sync filesystem operations to the async replicas

Fixes #884.
2017-10-11 00:55:48 -07:00
JoelEinbinder
e59172de83 chore: Use Typescript to lint JSDoc annotations (#986)
This patch starts using typescript to lint JSDoc annotations.

Note: this uses typescript's bleeding edge. We should migrate to stable once
it has all the necessary bugfixes.

References #65.
2017-10-09 22:31:40 -07:00
Andrey Lushnikov
e6af6e19d6 feat(console): dispatch JSHandles as console arguments (#975)
This patch starts dispatching JSHandle instances as console arguments.

BREAKING CHANGE: this changes the API of the ConsoleMessage.

Fixes #324.
2017-10-09 17:01:01 -07:00
Andrey Lushnikov
0d0f9b7984 feat(JSHandles): introduce JSHandles (#943)
This patch:
- introduces ExecutionContext class that incapsulates javascript
  execution context. An examples of execution contexts are workers and
  frames
- introduces JSHandle that holds a references to the javascript
  object in ExecutionContext
- inherits ElementHandle from JSHandle

Fixes #382.
2017-10-06 15:35:02 -07:00
Andrey Lushnikov
a71f287c6b Fix evaluation in case of first "undefind" argument (#732)
It turns out that [undefined, 1].join(',') results in ",1" instead
of "undefined,1". This causes a syntax error when trying to pass undefined
as a first argument to `page.evaluate` method.

Fixes #572.
2017-09-11 16:59:38 -07:00
Andrey Lushnikov
0bea42bd8c Do not leave dangling promises when sending messages over protocol (#742)
It's very bad to have 'unhandled promise rejection' that can't be
handled in user code. These errors will exit node process in a near
future.

This patch avoids 'unhandled promise rejection' while sending protocol
messages.

This patch:
- introduces `puppeteer:error` debug scope and starts using it for all
  swalloed errors.
- makes sure that every `client.send` method is either awaited or its
  errors are handled.
- starts return promises from Request.continue() and Request.abort().
- starts swallow errors from Request.contine() and Request.abort().

The last is the most important part of the patch. Since
`Request.continue()` might try to continue canceled request, we should
disregard the error.

Fixes #627.
2017-09-11 16:21:51 -07:00
JoelEinbinder
42fa3af659 Remove promise jsdoc (#641)
This patch removes unnecessary "promise" return types in jsdoc for async functions.
2017-08-31 17:58:07 -07:00
Andrey Lushnikov
66912a7277 [api] use object instead of map for headers (#567)
This patch:

- switches to objects instead of maps for headers (in Request, Response and
page.setExtraHTTPHeaders)
- converts all header names to lower case

Fixes #547, fixes #509
2017-08-28 12:09:24 -07:00
JoelEinbinder
2c4dfbfd88 Fix node6 support for Object.entries and URL object (#548)
This patch introduces a polyfill for Object.entries which is missing in Node 6
2017-08-25 15:13:59 -07:00
Eric Bidelman
1f9b4fb4c8 Change let into const (#457)
This patch:
- changes `let` into `const` throughout codebase
- adds eslint check to prefer const over let
2017-08-21 16:39:04 -07:00
Aleksey
56eab00fa7 Roll chromium to r494365 (#271)
This patch:
- rolls chromium to r494365
- starts using Runtime.evaluate(awaitPromise: true), with new semantic
  we can avoid additional Runtime.awaitPromise call
- stops resolving promises for Console event
2017-08-15 17:35:59 -07:00
Andrey Lushnikov
fe06c896eb Roll chromium to 492629 (#230)
This patch
- rolls chromium to 492629
- migrates connection establishing to use browser target. This migration means
  that now we have a single websocket connection to browser (implemented
  in Connection class). A connection to a particular target is
  incapsulated in a new Session class.
2017-08-09 16:14:00 -07:00
Andrey Lushnikov
20ba447689 [DEBUG] Trace only those events which have listeners. (#201)
The `DEBUG=*page npm run unit` is too verbose due to events spamming
the console.

This patch starts tracing emitted events only if there are any
listeners.
2017-08-03 15:20:31 -07:00
Andrey Lushnikov
adf35952fc [DEBUG] More descriptive event reporting
This patch improves DEBUG reporting so that circular event
arguments are shown with some information.
2017-07-29 13:02:56 -07:00
JoelEinbinder
8780fcb662 Event coverage and debugging (#160)
This patch introduces event coverage for DEBUG module and 
API coverage.

Closes #50.
2017-07-28 11:45:05 -07:00
Andrey Lushnikov
ff5ed1c738 Implement page.waitForFunction method
The page.waitForFunction method allows to wait for a general predicate.
The predicate will be continiously polled for in page, until
it either returns true or the timeout happens.

The polling parameter could be one of the following:
- 'raf' - to poll on every animation frame
- 'mutation' - to poll on every dom mutation
- <number> - to poll every X milliseconds

References #91
2017-07-27 16:45:14 -07:00
JoelEinbinder
a2e0d27fb6 Implement public API coverage
This patch:
- implements a basic public API coverage based on 'helper.tracePublicAPI' methods
- adds `npm run coverage` command which reports coverage after running all of the unit tests

References #50.
2017-07-27 16:16:37 -07:00
JoelEinbinder
bbde8fd1c2 page.evaluate takes a string in addition to function (#135)
This patch improves on page.evaluate to accept a string.
The string can have a trailing '//# sourceURL=' comment which would
name the evaluation to make stacks beautiful.

In order to make sourceURL comments possible, this patch:
- removes wrapping of the client function into `Promise.resolve()`
- stops passing `awaitPromise` parameter to `Runtime.evaluate`
- starts to await promise via the `Runtime.awaitPromise` if the return type of the evaluation
  is promise

closes #118
2017-07-27 12:23:41 -07:00
JoelEinbinder
2e6751d6f5 Trace only string method names (#141) 2017-07-26 11:48:40 -07:00
Andrey Lushnikov
5549ad0282 Do not serialize remote objects unless needed
This patch stops serializing console API arguments unless there are
listeners of the 'console' event in puppeteer.

This saves quite a lot CPU cycles.

Fixes #117.
2017-07-24 21:43:54 -07:00
Andrey Lushnikov
c4904b4e10 Do not leak event listeners on navigation
This patch:
- introduces helper.addEventListener/helper.removeEventListeners
  to simplify event management
- moves NavigatorWatchdog over to the helper.addEventListener to
  stop leaking event listeners
2017-07-22 17:46:39 -07:00
Andrey Lushnikov
aba61de905 Make helper.getExceptionMessage synchronous
We now have description of an exception, no need for a roundtrip
to the backend.
2017-07-21 11:57:25 -07:00
Andrey Lushnikov
55acae40fd Introduce DEBUG module which traces public API calls
This patch improves on DEBUG module to trace all puppeteer's
public API calls.

References #89.
2017-07-18 21:06:03 -07:00
Andrey Lushnikov
6c1c3a0c45 Return remote object description for unserializable objects
This patch starts handling unserializable objects by returning their
description.

Fixes #86.
2017-07-18 01:34:43 -07:00
Andrey Lushnikov
1a97d8b3c2 Teach 'consolemessage' event to send all the arguments
This patch fixes 'consolemessage' event so that it passes
over all the arguments of console API call.
2017-07-17 20:22:45 -07:00
Andrey Lushnikov
3d90ea38a9 Implement Frame.evaluate
This patch implements Frame.evaluate method.

References #4.
2017-06-27 14:57:14 -07:00