With `nodejs@15.0.1`, install puppeteer with `https_proxy` set causes an error like:
```
> puppeteer@5.4.1 install node_modules/puppeteer
> node install.js
ERROR: Failed to set up Chromium r809590! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.
TypeError [ERR_INVALID_PROTOCOL]: Protocol "https:" not supported. Expected "http:"
at new NodeError (node:internal/errors:258:15)
at new ClientRequest (node:_http_client:155:11)
at Object.request (node:https:313:10)
at httpRequest (node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:488:17)
at downloadFile (node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:357:21)
at BrowserFetcher.download (node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:239:19)
at async downloadBrowser (node_modules/puppeteer/lib/cjs/puppeteer/node/install.js:48:5) {
code: 'ERR_INVALID_PROTOCOL'
}
```
The related issue is at https://github.com/TooTallNate/node-agent-base/pull/47, from package `agent-base` under `https-proxy-agent`
And the version bump is for `Refactor to TypeScript` is here: https://github.com/TooTallNate/node-https-proxy-agent/compare/4.0.0...5.0.0
`isNode` checks for the node version in which the process is running and it checks for the node key inside the versions. This is exactly the same as using `process.version` as they are set in the native code and `process.version` is just a shorthand for `process.versions.node`.
Remove the redundant rule for dependency graph as the `src/index.ts` is not required. This was introduced in 64c9c709, but the filestructure has changed from since then and it doesn't work as `src/index.ts` is replaced with `src/node.ts` and `src/web.ts` as per the use case.
This patch makes it so that we only refer to specific Node.js versions like `10.19.0` when we really mean to use exactly that version. Usually we just want to test the latest version in a major range, e.g. `10`.
It also adds an inline comment that makes it easier to update the versions across the file in the future (e.g. when Node.js 10 goes EOL in April 2021).
Issue: #6561
The npm dist tags are not worth the hassle, and are arguably harmful to the Web. Developers should feel encouraged to always test the latest available Puppeteer & Chromium versions, as opposed to any specific version.
Issue: #6482
This patch makes it easier to see exactly how to use the registerCustomQueryHandler API without having to follow the trail of breadcrumbs throughout the docs.
As long as we follow Conventional Commits for our commit messages (which is now enforced since #6483), we can automate the maintenance of this new changelog, which enables us to later automate the release process altogether.
This patch also eliminates the versioning decision-making process by automating it based on our commit messages.
Issue: #6482
* chore: run unit tests on node 10.15
We saw in https://github.com/puppeteer/puppeteer/issues/6548 that the
`fs.promises` module was experimental in Node <10.17 and as such we
introduced issues for users on 10.15.
Until we can drop Node v10 (it's EOL is 30-04-20201
https://github.com/nodejs/Release#release-schedule) we should run our
tests on an old Node 10 to avoid regressing in this area.
* chore: helper for importing fs safely
This PR updates the socket transport code to swap between a Node web
socket transport or a web one based on the `isNode` environment. It also
adds unit tests to the browser tests that show we can connect in a
browser.
* chore(agnostification): common/helper.ts
The `readProtocolStream` method uses `fs` only if you want to write to a
file. So we gate at the start of the function and ensure that if we got
given a path we are not in a Node environment.
They are generating a lot of noise in PRs. This commit removes them from git, but updates CI to generate them - to ensure there are no errors when generating the new documentation.
Further commits will:
1. Introduce linting to enforce methods are documented.
2. Generate previews of the new docs via GitHub actions or similar.
The `Puppeteer` class had two concerns:
* connect to an existing browser
* launch a new browser
The first of those concerns is needed in all environments, but the
second is only needed in Node.
https://github.com/puppeteer/puppeteer/pull/6484 landing enabled us to
pull the `Puppeteer` class apart into two:
1. `Puppeteer` which hosts the behaviour for connecting to existing
browsers.
2. `PuppeteerNode`, which extends `Puppeteer` and also adds the ability
to launch a new browser.
This is a non-breaking change, because Node users will still get an
instance of a class with all the methods they expect, but it'll be a
`PuppeteerNode` rather than `Puppeteer`. I don't expect this to cause
people any issues.
We also now have new files that are effectively the entry points for
Puppeteer:
* `node.ts`: the main entry point for Puppeteer on Node.
* `web.ts`: the main entry point for Puppeteer on the web.
* `node-puppeteer-core.ts`: for those using puppeteer-core (which only
exists in Node, not on the web).
Launching headless with a relative `userDataDir` hangs on Windows. Fix by calling `path.resolve` (idempotent) to add an absolute path instead in `defaultArgs`.
Issues: #3453
The `Launcher` class was serving two purposes:
1. Launch browsers
2. Connect to browsers
Number 1) only needs to be done in Node land, but 2) is agnostic; in a
browser version of Puppeteer we'll need the ability to connect over a
websocket to send commands back and forth.
As part of the agnostification work we needed to split the `Launcher` up
so that the connection part can be made agnostic. Additionally, I
removed dependencies on `https`, `http` and `URL` from Node, instead
leaning on fetch (via `node-fetch` if in Node land) and the browser
`URL` API (which was added to Node in Node 10).
This commit updates some miscellaneous dependencies to their latest
versions (with no other changes required) and also removes the `browser`
section, which was used by Browserify for the now long gone
Puppeteer-Web package that we used to publish.
This patch sets up commitlint to enforce the Conventional Commits format. This check runs with the other lint checks as part of npm run lint, and a Git commit hook is set up via Husky for automated local checks.
Issue: #6482
* chore(agnostify): Create Node and Web initializer.
This PR splits `initialize.ts` into two files, one for web, and one for
Node. The Node initializer requires much more information as it needs to
know which browser(s) to download and where to store them, whereas the
web one does not.
A future PR that I'm working on will tidy up `src/common/Puppeteer.ts`
(as it contains a lot of Node specific logic around downloading and
installing browsers), but this change enables us to stop the browser
bundle attempting to use the `pkg-dir` dependency, which wouldn't work
within a browser, as well as keeping the size of the PRs down and
avoiding one mammoth PR.
This commit adds a new built-in handler for querying by accessible name and role (#6307).
Support for waitForSelector will be added in a follow-up commit.
