The `Launcher` class was serving two purposes:
1. Launch browsers
2. Connect to browsers
Number 1) only needs to be done in Node land, but 2) is agnostic; in a
browser version of Puppeteer we'll need the ability to connect over a
websocket to send commands back and forth.
As part of the agnostification work we needed to split the `Launcher` up
so that the connection part can be made agnostic. Additionally, I
removed dependencies on `https`, `http` and `URL` from Node, instead
leaning on fetch (via `node-fetch` if in Node land) and the browser
`URL` API (which was added to Node in Node 10).
This commit updates some miscellaneous dependencies to their latest
versions (with no other changes required) and also removes the `browser`
section, which was used by Browserify for the now long gone
Puppeteer-Web package that we used to publish.
This patch sets up commitlint to enforce the Conventional Commits format. This check runs with the other lint checks as part of npm run lint, and a Git commit hook is set up via Husky for automated local checks.
Issue: #6482
In `src/common` we now use `fs.promises.X` which we can dynamically
`import`. In a browser environment this code will never run because it's
gated on `isNode` (in a future PR we will add tree-shaking to the bundle
step such that this code is eliminated). By using `import`, we ensure
TypeScript still can track types and give good type information.
In `src/node` we continue to use `util.promisify` but that's not a
concern as that code explicitly is never run in the browser.
* roll Chromium to version 86, r800071
* roll Chrome DevTools protocol version to 0.0.799653
* fix HTTPRequest.continue after
* CDP: accept post data in the binary form in Fetch.continueRequest.
https://chromium-review.googlesource.com/c/chromium/src/+/2315239
* update new-docs
This corresponds to Chromium 85.0.4182.0.
This roll includes:
- Enable SameSiteByDefaultCookies and CookiesWithoutSameSiteMustBeSecure
https://crrev.com/c/2231445
- [FlexNG] Enable FlexNG by default
https://crrev.com/c/2216595Closes#6151.
* chore: vendor Mitt into src/common/third-party
As discussed in #6203 we need to vendor our common dependencies in so
that when we ship an ESM build all imports point to file paths and do
not rely on Node resolution (e.g. a browser does not understand `import
mitt from 'mitt'`).
* chore: enforce file extensions on imports
To make our output agnostic it should include file extensions in the
output, as per the ESM spec. It's a bit odd for Node packages but makes
it easier to publish a browser build.
* feat(chromium): roll Chromium to r768783
* fix: update unit test for crrev:2135046
* chore: update devtools-protocol revision
Co-authored-by: Changhao Han <changhaohan@chromium.org>
* chore: Use devtools-protocol package
Rather than maintain our own protocol we can instead use the devtools-protocol package and pin it to the version of Chromium that Puppeteer is shipping with.
The only changes are naming changes between the bespoke protocol that Puppeteer created and the devtools-protocol one.
Now the async hooks helper is gone api.ts was only used by the coverage
tools and by doclint.
DocLint is nearing the end of its lifespan with the TSDoc work, so I
focused on how best to define a list of modules for the coverage
tooling. They define an object of classes, and the path to that module.
They need the full path because we also check if the module exports any
events that need to be emitted - the coverage tool asserts that the
emitting of those events is also tested.
It's not _great_ that DocLint relies on a constant defined in the
coverage utils, but it should only be this way for a short period of
time and no one is actively working on DocLint (bar the effort to remove
it) so I don't think this is worth worrying about.
This change also broke the DocLint tests; based on the fact that DocLint is on its way out it doesn't feel worth fixing the tests, so this commit also removes them.
If you want to run TypeScript only to verify that it's typechecking correctly, this command is quicker as it doesn't output CJS and ESM to disk. Useful for checking during development.
* chore: Don't store revisions in `package.json`
It's quite messy to have to require the `package.json` file in multiple
places purely to find out what revision of a given browser we want to
use. We can also achieve better type safety by placing it in an actual
source file.
This commit makes that change and also tidies up our reliance on
`package.json` within the source code generally; we now only use it to
find the location of the Puppeteer root such that we know where to
install downloaded browsers to.
To avoid using `package.json` to parse the name of the module, we also
now explicitly have an entry point for the Puppeteer module and the
Puppeter Core module. This will make it easier in the future to ship
less code as part of core (e.g. core never needs to download a browser,
so why ship that code?). Core can also then not have any revisions based
info contained in it.
The test install script has also been updated to ensure that
puppeteer-core can be installed correctly too.
Finally, the `install` script has been moved to TypeScript for nicer
typechecking and safety. The functionality of it has not changed.
* chore(agnostic): ship CJS and ESM builds
For our work to enable Puppeteer in other environments (e.g. a browser)
we need to ship an ESM build. This commit changes our config to ship to
`lib/cjs` and `lib/esm` accordingly. The majority of our code stays the
same, with one small fix for the CJS build to ensure that we ship a
version that lets you `require('puppeteer')` rather than have to
`require('puppeteer').default`. We do this with the `cjs-entry.js` which
is what the `main` field in our `package.json` points to.
We also swap to `read-pkg-up` to find the `package.json` file. This is
because the folder structure of `lib/` does not match `src/` now we ship
to `cjs` and `esm`, so you cannot rely on exact paths. This module works
up from the file to find the nearest `package.json` so it will always
find Puppeteer's `package.json`.
Note that we *do not* point any users to the ESM build. We happen to
ship those files so people who know about them can get at them but it's
not expected (nor will we actively support) that people will rely on
them. The CommonJS build is considered our main build.
We may make breaking changes to the structure of the ESM build which we
will do without requiring new major versions. For example the ESM build
currently ships all files that the CJS build does, but given we are
working on the ESM build being able to run in the browser this may
change over time.
Long term once the Node versions catch up we can ditch CJS and ship
exclusively ESM but we are not there yet.
This CL migrates all the tests to TypeScript. The main benefits of this is that we start consuming our TypeScript definitions and therefore find errors in them. The act of migrating found some bugs in our definitions and now we can be sure to avoid them going forwards.
You'll notice the addition of some `TODO`s in the code; I didn't want this CL to get any bigger than it already is but I intend to follow those up once this lands. It's mostly figuring out how to extend the `expect` types with our `toBeGolden` helpers and some other slight confusions with types that the tests exposed.
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
Without the API-* dependencies pinned different versions may be
installed on local machines vs CI. One of the checks we do is to check
that the checked in docs matches what is generated on CI. Therefore we
need to ensure devs locally run the exact version that CI runs such that
they generate the same output. So in this case we pin to a particular
version of the dependencies.
As far as I can tell these became irrelevant as of v1.15 which added
`puppeteer.errors` and `puppeteer.devices [1]. This is a breaking change
but one that's easily mitigated. We've said that we don't consider
changes to our folder/file structure a breaking change, but we can't
really do that if we have these two top level files that we've
documented.
[1]: e3abb0aa32 (diff-522b24108d7446af4c59873472a90444)
It was causing some infra issues when trying to migrate tests to
TypeScript (that's WIP in another branch that I'll have up soon). It's
unusual to have the entire src in TS except for the main file, which
then reaches into the compiled `lib` directory for the files it needs.
Much better is to move the entry point into TypeScript itself and update
the `main` entry in our `package.json` to point to the compiled output.
This also has the advantange of hooking up all the TS type defs that we
are shipping and will make that process easier too, along with making it
easier to port our tests to TypeScript.
`api.ts` is a list of all our modules which is used to install a helper
and by DocLint. It's not that useful for our dep graph because it
literally requires the majority of modules so it just clutters up the diagram.
`--max-depth` stopped the chart including our own modules. What we want
instead is the `do-not-follow` option to make it go to infinite depth in
our code but stop at the top level of a node module.
* chore: ensure new-docs are up to date
This adds a Travis check that if we re-generate the docs we get the
exact output (by checking if the Git tree is dirty).
We do the dirty check by using `git status --porcelain`, seeing how many
lines that outputs, and using that as the exit code (taking only the
first 255 lines to avoid invalid exit codes). `--porcelain` makes the
output be empty if the repo is not dirty in anyway which translates into
an exit code of 0.
We can't use `git diff-index --quiet HEAD` as it exits with 0 if there are
untracked files in the repo but we want that to cause a failure.
I've had misleading type errors due to left over builds or tests
behaving oddly. We should just strip the entire folder out before
building again so there's no left over artefacts that could cause
issues.
The `|| true` in the command is so if `rm` errors because the folder
doesn't exist, it doesn't exit with an error code.
