Commit Graph

94 Commits

Author SHA1 Message Date
Jack Franklin
a614bc45aa
chore: migrate src/Connection to TypeScript (#5694)
* chore: migrate `src/Connection` to TypeScript

This commit migrates `src/Connection` to TypeScript. It also changes its
exports to be ESM because TypeScript's support for exporting values to
use as types via CommonJS is poor (by design) and so rather than battle
that it made more sense to migrate the file to ESM.

The good news is that TypeScript is still outputting to `lib/` as
CommonJS, so the fact that we author in ESM is actually not a breaking
change at all.

So going forwards we will:

* migrate TS files to use ESM for importing and exporting
* continue to output to `lib/` as CommonJS
* continue to use CommonJS requires when in a `src/*.js` file

I'd also like to split `Connection.ts` into two; I think the
`CDPSession` class belongs in its own file, but I will do that in
another PR to avoid this one becoming bigger than it already is.

I also turned off `@typescript-eslint/no-use-before-define` as I don't
think it was adding value and Puppeteer's codebase seems to have a style
of declaring helper functions at the bottom which is fine by me.

Finally, I updated the DocLint tool so it knows of expected method
mismatches. It was either that or come up with a smart way to support
TypeScript generics in DocLint and given we don't want to use DocLint
that much longer that didn't feel worth it.

* Fix params being required
2020-04-21 09:20:25 +01:00
Jack Franklin
3e4c8c9d0d
chore(typescript): migrate src/Dialog (#5639)
This PR changes `src/Dialog.js` to `src/Dialog.ts` and rewrites
accordingly. Most of the changes are straight forward; the only
interesting one from a TS point of view is the `DialogType` enum. I
expose it again as `Dialog.Type` to avoid a breaking change.

This PR also exposed some bugs with our ESLint TypeScript settings and
applying the overrides, so I fixed those too.

I also updated our DocLint tool to work on TS source files over JS lib
files if they exist. This is the minimal change to keep the existing doc
system working as we're working on moving away from this system longer
term.
2020-04-16 14:59:28 +01:00
Jack Franklin
a9f6a266b9
chore: Log reminder about tsc if DocLint fails locally (#5652)
I lost some time debugging before realising that I needed to run tsc. I
don't really want to put `npm run tsc` before this command else we'll
run tsc multiple times on each CI build, so I think this message is
suitable.

Travis defines `process.env.TRAVIS` and if that exists we don't want to
log this as on CI we're guaranteed to have an up to date `lib/`
directory.
2020-04-16 14:40:04 +01:00
Jack Franklin
88d843d4f0
feat(TypeScript): move DeviceDescriptors to TS (#5595)
This commit moves `src/DeviceDescriptors` to be authored in TypeScript. This file was chosen due to its simplicity so that we can focus on getting a mixed JS/TS codebase playing nicely before migrating the more complex files.

The file itself was a bit odd: although the array of devices was exported via `module.exports` that was never referenced by any consumers; each device was also exported via `module.exports[name] = device` and that is how it's consumed. The Puppeteer docs suggest using it like so:

```js
puppeteer.devices['iPhone 6']
```

So instead of exporting the array and then setting a bunch of properties on that, we instead define the array and export an object of keys where each key is a device. This is a breaking change (see the footer for details).

Rather than export an object I'd much rather export a Map, but that would be a larger breaking change and I'm keen to avoid those for the time being.

Note that we have to use special TypeScript specific syntax for the export that enables it to work in a CommonJS codebase [1] but again I'd rather this than move to ESM at this time. TypeScript still outputs CommonJS into `lib/` as you would expect.

BREAKING CHANGE: We no longer export an array of devices, so any users relying on doing:

```js
puppeter.devices.forEach(...)
```

…will now see a breakage. The fix is to use `Object.{keys/entries/values}` to iterate instead.

[1]: https://www.typescriptlang.org/docs/handbook/modules.html#export--and-import--require
2020-04-14 11:55:29 +02:00
Jack Franklin
0bcc5a7ad8
chore: migrate remaining tests to Mocha (#5616)
This commit updates all the non-Puppeteer unit tests to run using Mocha and then deletes the custom test runner framework from this repository. The documentation has also been updated.
2020-04-09 20:12:32 +02:00
Jack Franklin
efe561e112
chore: fix DocLint method diffing (#5594)
Our logic around missing methods wasn't quite right; if there is no set of missing methods for a class it _is_ an error and we still need to report it, we don't want to `continue`.
2020-04-06 12:36:28 +02:00
Jack Franklin
4ee2c43f06
chore: fix Page.emulateMedia doclint failure (#5584)
This is expected as we now alias `emulateMedia` in `index.js` which isn't a file checked by DocLint. We alias there to avoid having the function overriden by the `asyncInstallHooks` code.

This commit updates doclint to know about methods that we expect it will find are missing and in that case just skip over them. We should only do this for methods where we plan to deprecate them or we have to define them in an odd way to work around some problem (and if that's the case long term we should fix that problem so we can define them as normal).

Co-authored-by: Mathias Bynens <mathias@qiwi.be>
2020-04-06 10:38:05 +02:00
Jack Franklin
88446df724
chore: fix missed src/ vs lib/ documentation (#5591)
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
2020-04-06 10:32:42 +02:00
Jack Franklin
6522e4f524
chore: Use expect for assertions (#5581)
Rather than use our own custom expect library, we can use expect from npm [1], which has an API almost identical to the one Puppeteer has, but with more options, better diffing, and is used by many in the community as it's the default assertions library that comes with Jest.

It's also thoroughly documented [2].

[1]: https://www.npmjs.com/package/expect
[2]: https://jestjs.io/docs/en/expect
2020-04-03 13:22:55 +02:00
Jack Franklin
4b0fd8bc30
chore: upgrade to TypeScript 3.7 (#5562)
TypeScript seems to struggle to understand `Promise.all` when the items in the array return different types. If we were authoring in TS we could fix this with TS generics (`Promise.all<OurTypeHere>(...)`) but for now we can typecast the result. We'll fix this properly when we author in TS.
2020-03-31 18:42:32 +02:00
Jack Franklin
5e8d79bf98
chore: upgrade to TypeScript 3.5 (#5556)
TS 3.5 got much stricter on writing changes to objects with varied types [1] so we have to do a bit of typecasting work to convince TS about the types of keys and values that we are setting.

Longer term we should think about a better data structure that avoids us having to jump through some hoops but for now I think this is a reasonable step to get us onto 3.5.

Same story regarding bindings on `window`: the easiest fix is to cast `window` to `any` for the code that adds to it. I'm sure we can come up with a more type-safe way of doing this in the future.

[1]: https://github.com/Microsoft/TypeScript/wiki/Breaking-Changes#fixes-to-unsound-writes-to-indexed-access-types
2020-03-31 10:48:09 +02:00
Mathias Bynens
eddb23b521
chore: update URLs (#5185) 2019-11-26 13:12:25 +01:00
Mathias Bynens
11ff374ca3
chore: drop Node.js v6 support (#5045)
Node.js v6 was end-of-life'd in April, 2019, with AWS Lambda prohibiting updaets to the Node.js v6 runtime since June 30, 2019.

This makes it quite safe for us to remove the Node 6 support from the repository.
2019-10-16 17:00:20 +02:00
Andrey Lushnikov
e3abb0aa32
feat(puppeteer): introduce puppeteer.errors and puppeteer.devices (#4312)
These getters are introduced as a more convenient substitute for
a `require('puppeteer/Errors')` and
`require('puppeteer/DeviceDescriptors')`.

This way we can make cross-browser story nicer - a single require
of `puppeteer` or `puppeteer-firefox` fully defines Puppeteer
environment.
2019-04-19 15:33:06 -07:00
Andrey Lushnikov
080b80975f
docs(contributing): add instructions to build Chromium revisions (#4246) 2019-04-04 15:34:41 -07:00
Andrey Lushnikov
0adffcc2cb
docs: add table-of-contents to troubleshooting (#4234)
Drive-by: teach our table-of-contents generator to ignore comments
inside fenced blocks and to de-linkify titles.
2019-04-02 19:08:22 -07:00
Joel Einbinder
5cca7c0cae fix(types): don't report events as class members (#3874) 2019-01-31 17:53:16 -08:00
Joel Einbinder
f2c968fdb8 chore(types): generate our own d.ts file from api.md (#3744)
Generate `//index.d.ts` file with precise typescript definitions for all of the
Puppeteer API.
2019-01-28 15:12:45 -08:00
Andrey Lushnikov
62da2366c6
chore: introduce //lib/api.js (#3835)
Introduce `//lib/api.js` that declares a list of publicly exposed
classes.

The `//lib/api.js` list superceedes dynamic `helper.tracePublicAPI()` calls
and is used in the following places:
- [ASYNC STACKS]: generate "async stacks" for publicy exposed API in `//index.js`
- [COVERAGE]: move coverage support from `//lib/helper` to `//test/utils`
- [DOCLINT]: get rid of 'exluded classes' hardcoded list

This will help us to re-use our coverage and doclint infrastructure
for Puppeteer-Firefox.

Drive-By: it turns out we didn't run coverage for `SecurityDetails`
class, so we lack coverage for a few methods there. These are excluded
for now, sanity tests will be added in a follow-up.
2019-01-25 23:21:14 -05:00
Andrey Lushnikov
4ac00caf9d
refactor: split out DOMWorld from Frame (#3780)
This patch splits out `IsolatedWorld` class from Frame.
The `IsolatedWorld` abstraction is an execution context
with a designated set of DOM wrappers.

References #2671
2019-01-15 17:21:23 -08:00
Andrey Lushnikov
4e9e3bc614
refactor: consolidate all events in Events.js (#3772)
This will allow us to break all dependency cycles that were forcing
us to put many things in a single file (e.g. ExecutionContext and
ElementHandle).
2019-01-14 19:57:05 -08:00
Joel Einbinder
a0cbaf39ab chore(types): lint the api docs with typescript (#3577) 2018-11-21 14:49:08 -08:00
Andrey Lushnikov
927d0f443b
feat(page): support waitUntil option for page.setContent (#3557)
This patch teaches `page.setContent` to await resources in
the new document.

**NOTE**: This patch changes behavior: currently, `page.setContent`
awaits the `"domcontentloaded"` event; with this patch, we can now await
other lifecycle events, and switched default to the `"load"` event.

The change is justified since current behavior made `page.setContent`
unusable for its main designated usecases, pushing our client
to use [dataURL workaround](https://github.com/GoogleChrome/puppeteer/issues/728#issuecomment-334301491).

Fixes #728
2018-11-20 15:32:46 -08:00
Andrey Lushnikov
eb7bd9d7d3
test: setup sandbox on linux (#3530)
Drop all the `--no-sandbox` bits from tests and infrastructure. Instead, configure
Travis to enable user namespace clone.
2018-11-12 23:26:16 -08:00
Joel Einbinder
9ba3261571 feat(accessibility): snapshot the accessibility tree (#3470)
This adds `page.accessibility.snapshot()`. It serializes and returns the accessibility tree for the page. By default, uninteresting nodes are filtered out of the snapshot.

fixes #2033
2018-11-01 18:54:51 -07:00
Andrey Lushnikov
1b2c8113ba
refactor: move Connection to use ConnectionTransport (#3217)
Currently connection assumes that transport is a websocket
and tries to handle websocket-related errors.

This patch:
- moves ConnectionTransport interface to use callbacks instead
  of events. This way it could be used in browser context as well.
- introduces WebSocketTransport that implements ConnectionTransport
  interface for ws.

This is a preparation step for 2 things:
- exposing `transport` option in the `puppeteer.connect` method
- better support for `browserify`

References #2119
2018-09-07 21:36:16 +01:00
Andrey Lushnikov
7db4f0f798
chore: preserve links to master version of API from README.md (#3197)
One of our checks makes sure all links from README.md to API.md
point to the last-released version of the API.

This sometimes doesn't work: when we refer to a section
in api.md that is just added, we should be able to reference
the "master" version of the api.md

This patch:
- teaches the doclint check to keep links to tip-of-tree version
  of api.md in README.md intact.
- starts refering to tip-of-tree version of api.md in `puppeter-core` section
2018-09-05 20:33:04 +01:00
Andrey Lushnikov
204c7ec8c4
feat: introduce puppeteer/Errors (#3056)
This patch adds a new require, `puppeteer/Errors`, that
holds all the Puppeteer-specific error classes.

Currently, the only custom error class we use is `TimeoutError`. We'll
expand in future with `CrashError` and some others.

Fixes #1694.
2018-08-09 16:51:12 -07:00
Andrey Lushnikov
1be7545b70
chore: enhance markdown preprocessor (#3050)
Use preprocessor to make sure all the links in out README.md point
to the last-released version of documentation.

Fixes #3038.
2018-08-08 18:20:20 -07:00
Andrey Lushnikov
9904da262e
chore(doclint): remove old unused table-of-contents generator (#2659)
We no longer use markdown-toc to generate table-of-contents.
2018-06-01 13:59:31 -07:00
Andrey Lushnikov
0b94fa70eb
chore: stop using console.assert everywhere (#2646)
Since Node 10, `console.assert` no longer throws an AssertionError.
(This is generally good since it aligns Node.js with Browsers.)

This patch migrates all usages of `console.assert` in our codebase.
- All the `lib/` and testing code is migrated onto a handmade `assert`
function. This is to make Puppeteer transpilation / bundling easier.
- All the tooling is switched to use Node's `assert` module.

Fixes #2547.
2018-05-31 16:53:51 -07:00
Andrey Lushnikov
754df58d4e
docs(api.md): fix table-of-contents (#2636)
This patch drops the markdown-toc module and instead rolls out
our own simple markdown table-of-contents generator.

As a side effect, it fixes links to `page.$` and `page.$$`.
2018-05-31 14:21:43 -07:00
Andrey Lushnikov
0820d48f80
chore: link to the latest-released API from the README.md (#2449)
This patch adds a preprocessor command to link to the latest-released
API from the README.md.

Fixes #1923.
2018-04-25 18:18:08 -07:00
Andrey Lushnikov
13a41495aa
chore(doclint): remove SourceFactory (#2447)
SourceFactory was meant to cache Sources so that they could be used
in different preprocessor tasks.

This turned out to be over-engineering. This patch kills the layer.
2018-04-25 18:07:20 -07:00
Andrey Lushnikov
6d19db4df1
chore: enhance preprocessor commands to automate releases. (#2446)
Last release v1.3.0 had an error in the documentation, claiming
it wasn't released.

This patch makes sure we have a little bit of automation in place
to save us from this in future.
2018-04-25 17:11:45 -07:00
JoelEinbinder
2370618819 chore(types): upgrade to TypeScript 2.8.1 (#2304)
This converts `externs.d.ts` to export a global namespace instead of a UMD global.

See: https://github.com/Microsoft/TypeScript/issues/22969

Fixes #2279.
2018-04-04 14:06:21 -07:00
Andrey Lushnikov
e8a085ccfb
feat(appMode): support pipes for appMode (#2032)
This patch starts using pipes as a transport to
the browser instance instead of websocket.
2018-02-14 17:51:29 -08:00
Drew Diamantoukos
b275e66594 chore: Spelling and Markdown Consistency (#1998)
- Adding missing language tags to markdown code blocks.
- Fixed various spelling mistakes my IDE complained to me about.
2018-02-08 22:59:46 -05:00
Andrey Lushnikov
a363a733b7
feat: Introduce BrowserFetcher class (#1983)
This patch introduces `BrowserFetcher` class that manages
downloaded versions of products.

This patch:
- shapes Downloader API to be minimal yet usable for our needs. This
  includes removing such methods as `Downloader.supportedPlatforms` and
  `Downloader.defaultRevision`.
- makes most of the fs-related methods in Downloader async. The only
  exception is the `Downloader.revisionInfo`: it has stay sync due to the
  `pptr.executablePath()` method being sync.
- updates `install.js` and `utils/check_availability.js` to use new API
- finally, renames `Downloader` into `BrowserFetcher`

Fixes #1748.
2018-02-07 12:31:53 -05:00
Andrey Lushnikov
5368051610
feat: expose raw devtools protocol connection (#1770)
feat: expose raw devtools protocol connection

This patch introduces `target.createCDPSession` method that
allows directly communicating with the target over the
Chrome DevTools Protocol.

Fixes #31.
2018-01-10 19:33:22 -08:00
Andrey Lushnikov
24354a4879
feat: implement CSS Coverage (#1714)
This patch adds two new methods to the `page.coverage` namespace:
- `page.coverage.startCSSCoverage()` - to initiate css coverage
- `page.coverage.stopCSSCoverage()` - to stop css coverage

The coverage format is consistent with the JavaScript coverage.
2018-01-03 18:21:40 -08:00
Andrey Lushnikov
d062381978
feat: Implement JavaScript Coverage (#1673)
This patch introduces a new `page.coverage` namespace with two methods:
- `page.coverage.startJSCoverage` to initiate JavaScript coverage
  recording
- `page.coverage.stopJSCoverage` to stop JavaScript coverage and get
  results
2018-01-02 19:53:53 -08:00
Andrey Lushnikov
9a5086847c
refactor: move ChromiumDownloader under lib/ (#1554)
This patch:
- renames ChromiumDownloader into just Downloader (this is in
  preparation for different products download)
- moves Downloader from utils/ to lib/. This unifies all of the
  production-critical code in the lib/.

Drive-by: make Downloader a regular class.
2017-12-08 13:39:13 -08:00
Andrey Lushnikov
e6725e15af
tests: drop jasmine test runner (#1519)
This patch introduces a tiny test runner to run puppeteer tests.
The test runner is self-container and allows parallel (wrt IO) test execution.
It will also allow us to split tests into multiple files if necessary.

Comparing to the jasmine, the testrunner supports parallel execution, properly
handles "unhandled promise rejection" event and signals.

Comparing to ava/jest, the testrunner doesn't run multiple node processes,
which makes it simpler but sufficient for our goals.
2017-12-07 16:37:22 -08:00
Andrey Lushnikov
6512ce768d
fix(Frame): postpone evaluations until execution context gets created (#1415)
In Blink, frames don't necesserily have execution context all the time.
DevTools Protocol precisely reports this situation, which results in
Puppeteer's frame.executionContext() being null occasionally.

However, from puppeteer point of view every frame will have at least a
default executions context, sooner or later:

- frame's execution context might be created naturally to run frame's
  javascript
- if frame has no javascript, devtools protocol will issue execution
  context creation

This patch builds up on this assumption and makes frame.executionContext()
to be a promise.
As a result, all the evaluations await for the execution context to be created first.

Fixes #827, #1325

BREAKING CHANGE: this patch changes frame.executionContext() method to return a promise.
To migrate onto a new behavior, await the context first before using it.
2017-11-18 16:27:52 -08:00
Andrey Lushnikov
a467d35a5f chore(doclint): do not attempt to lint non-documentation markdown 2017-11-07 03:33:25 -08:00
JoelEinbinder
32398d11bd feat(Browser): introduce Browser.pages() (#554)
This patch:
- introduces Target class that represents any inspectable target, such as service worker or page
- emits events when targets come and go
- introduces target.page() to instantiate a page from a target

Fixes #386, fixes #443.
2017-10-17 19:14:57 -07:00
Andrey Lushnikov
079db90066 fix(JSHandle.toString): clearer description for primitives (#993)
This patch:
- updates JSHandle.toString to make a nicer description for primitives
- excludes JSHandle.toString from documentation to avoid its abuse

References #382
2017-10-10 10:54:20 -07:00
JoelEinbinder
e59172de83 chore: Use Typescript to lint JSDoc annotations (#986)
This patch starts using typescript to lint JSDoc annotations.

Note: this uses typescript's bleeding edge. We should migrate to stable once
it has all the necessary bugfixes.

References #65.
2017-10-09 22:31:40 -07:00
Andrey Lushnikov
0d0f9b7984 feat(JSHandles): introduce JSHandles (#943)
This patch:
- introduces ExecutionContext class that incapsulates javascript
  execution context. An examples of execution contexts are workers and
  frames
- introduces JSHandle that holds a references to the javascript
  object in ExecutionContext
- inherits ElementHandle from JSHandle

Fixes #382.
2017-10-06 15:35:02 -07:00