This PR updates the socket transport code to swap between a Node web
socket transport or a web one based on the `isNode` environment. It also
adds unit tests to the browser tests that show we can connect in a
browser.
* chore(agnostification): common/helper.ts
The `readProtocolStream` method uses `fs` only if you want to write to a
file. So we gate at the start of the function and ensure that if we got
given a path we are not in a Node environment.
They are generating a lot of noise in PRs. This commit removes them from git, but updates CI to generate them - to ensure there are no errors when generating the new documentation.
Further commits will:
1. Introduce linting to enforce methods are documented.
2. Generate previews of the new docs via GitHub actions or similar.
The `Puppeteer` class had two concerns:
* connect to an existing browser
* launch a new browser
The first of those concerns is needed in all environments, but the
second is only needed in Node.
https://github.com/puppeteer/puppeteer/pull/6484 landing enabled us to
pull the `Puppeteer` class apart into two:
1. `Puppeteer` which hosts the behaviour for connecting to existing
browsers.
2. `PuppeteerNode`, which extends `Puppeteer` and also adds the ability
to launch a new browser.
This is a non-breaking change, because Node users will still get an
instance of a class with all the methods they expect, but it'll be a
`PuppeteerNode` rather than `Puppeteer`. I don't expect this to cause
people any issues.
We also now have new files that are effectively the entry points for
Puppeteer:
* `node.ts`: the main entry point for Puppeteer on Node.
* `web.ts`: the main entry point for Puppeteer on the web.
* `node-puppeteer-core.ts`: for those using puppeteer-core (which only
exists in Node, not on the web).
Launching headless with a relative `userDataDir` hangs on Windows. Fix by calling `path.resolve` (idempotent) to add an absolute path instead in `defaultArgs`.
Issues: #3453
The `Launcher` class was serving two purposes:
1. Launch browsers
2. Connect to browsers
Number 1) only needs to be done in Node land, but 2) is agnostic; in a
browser version of Puppeteer we'll need the ability to connect over a
websocket to send commands back and forth.
As part of the agnostification work we needed to split the `Launcher` up
so that the connection part can be made agnostic. Additionally, I
removed dependencies on `https`, `http` and `URL` from Node, instead
leaning on fetch (via `node-fetch` if in Node land) and the browser
`URL` API (which was added to Node in Node 10).
This commit updates some miscellaneous dependencies to their latest
versions (with no other changes required) and also removes the `browser`
section, which was used by Browserify for the now long gone
Puppeteer-Web package that we used to publish.
This patch sets up commitlint to enforce the Conventional Commits format. This check runs with the other lint checks as part of npm run lint, and a Git commit hook is set up via Husky for automated local checks.
Issue: #6482
* chore(agnostify): Create Node and Web initializer.
This PR splits `initialize.ts` into two files, one for web, and one for
Node. The Node initializer requires much more information as it needs to
know which browser(s) to download and where to store them, whereas the
web one does not.
A future PR that I'm working on will tidy up `src/common/Puppeteer.ts`
(as it contains a lot of Node specific logic around downloading and
installing browsers), but this change enables us to stop the browser
bundle attempting to use the `pkg-dir` dependency, which wouldn't work
within a browser, as well as keeping the size of the PRs down and
avoiding one mammoth PR.
This commit adds a new built-in handler for querying by accessible name and role (#6307).
Support for waitForSelector will be added in a follow-up commit.
In `src/common` we now use `fs.promises.X` which we can dynamically
`import`. In a browser environment this code will never run because it's
gated on `isNode` (in a future PR we will add tree-shaking to the bundle
step such that this code is eliminated). By using `import`, we ensure
TypeScript still can track types and give good type information.
In `src/node` we continue to use `util.promisify` but that's not a
concern as that code explicitly is never run in the browser.
This commit changes the internal representation of query handlers to contain Puppeteer-level code instead of page functions.
The interface `CustomQueryHandler` is introduced for user-defined query handlers. When a `CustomQueryHandler` is registered using `registerCustomQueryHandler` a corresponding Puppeteer-level handler is created through `makeQueryHandler` by wrapping the page functions as appropriate.
The internal query handlers (defined by the interface `QueryHandler`) contain two new functions: `waitFor` and `queryAllArray`.
- `waitFor` allows page-based handlers to make use of the `WaitTask`-backed implementation in `DOMWorld`, whereas purely Puppeteer-based handlers can define an alternative approach instead.
- `queryAllArray` is similar to `queryAll` but with a slightly different interface; it returns a `JSHandle` to an array with the results as opposed to an array of `ElementHandle`. It is used by `$$eval`.
After this change, we can introduce built-in query handlers that are not executed in the page context (#6307).
The logic for waitForXPath and waitForSelector is currently very tightly coupled. This commit tries to untangle that relationship. This is the first step towards introducing built-in query handlers that are not executed in the page context (#6307).
Debug message clarifying the problem like `Issue 1060080: Event Network.loadingFinished is not sent when a frame is loaded from another domain`: https://bugs.chromium.org/p/chromium/issues/detail?id=1060080).
It can help users to identify problem with OOPIF easier without digging into the CDP protocol implementation like in the bug `1060080` mentioned above.
To reproduce:
1. Run `DEBUG="puppeteer:frame" NODE_PATH=../ node examples/oopif.js`.
2. Verify the output contains the debug message:
`
puppeteer:frame The frame '...' moved to another session. Out-of-proccess
iframes (OOPIF) are not supported by Puppeteer yet.
https://github.com/puppeteer/puppeteer/issues/2548
`
* roll Chromium to version 86, r800071
* roll Chrome DevTools protocol version to 0.0.799653
* fix HTTPRequest.continue after
* CDP: accept post data in the binary form in Fetch.continueRequest.
https://chromium-review.googlesource.com/c/chromium/src/+/2315239
* update new-docs
prior command installs stable version of chrome (RUN command), but after that, the docs is referring to unstable version.
added correct commands for troubleshooting in linux environment.
By adding support for an environment variable `PUPPETEER_DOWNLOAD_PATH` it is possible to support downloading the browser binaries into a folder outside the `node_modules` folder. This makes it possible to preserve previously downloaded binaries in order to skip downloading them again.
