This PR implements automatic detection of the Firefox product when the `.connect()` method is used. This partially undoes the breaking change in https://github.com/puppeteer/puppeteer/pull/8520 but it's also a breaking change on its own since we don't accept an explicit product name anymore (it does not look like it was used anyway).
This commit tidies up the quite confusing state of all the various types
required to launch a browser. As we saw when upgrading DevTools
(https://source.chromium.org/chromium/chromium/src/+/master:third_party/devtools-frontend/src/test/conductor/hooks.ts;l=77),
we had to define our launch options variable like so:
```ts
const opts: puppeteer.LaunchOptions & puppeteer.ChromeArgOptions & puppeteer.BrowserOptions = {
...
};
```
This commit fixes that by introducing `AllNodeLaunchOptions`, which is
defined as the intersection of all the above types.
Additionally, the types defined as `ChromeArgOptions` are actually used
for launching both Chrome and Firefox, so I have renamed them to
`BrowserArgOptions`, and therefore this change is breaking because
anyone using `ChromeArgOptions` in their types will need to switch.
BREAKING CHANGE: renamed type `ChromeArgOptions` to `BrowserLaunchArgumentOptions`
BREAKING CHANGE: renamed type `BrowserOptions` to `BrowserConnectOptions`
The `Puppeteer` class had two concerns:
* connect to an existing browser
* launch a new browser
The first of those concerns is needed in all environments, but the
second is only needed in Node.
https://github.com/puppeteer/puppeteer/pull/6484 landing enabled us to
pull the `Puppeteer` class apart into two:
1. `Puppeteer` which hosts the behaviour for connecting to existing
browsers.
2. `PuppeteerNode`, which extends `Puppeteer` and also adds the ability
to launch a new browser.
This is a non-breaking change, because Node users will still get an
instance of a class with all the methods they expect, but it'll be a
`PuppeteerNode` rather than `Puppeteer`. I don't expect this to cause
people any issues.
We also now have new files that are effectively the entry points for
Puppeteer:
* `node.ts`: the main entry point for Puppeteer on Node.
* `web.ts`: the main entry point for Puppeteer on the web.
* `node-puppeteer-core.ts`: for those using puppeteer-core (which only
exists in Node, not on the web).
The `Launcher` class was serving two purposes:
1. Launch browsers
2. Connect to browsers
Number 1) only needs to be done in Node land, but 2) is agnostic; in a
browser version of Puppeteer we'll need the ability to connect over a
websocket to send commands back and forth.
As part of the agnostification work we needed to split the `Launcher` up
so that the connection part can be made agnostic. Additionally, I
removed dependencies on `https`, `http` and `URL` from Node, instead
leaning on fetch (via `node-fetch` if in Node land) and the browser
`URL` API (which was added to Node in Node 10).
* chore(agnostify): Create Node and Web initializer.
This PR splits `initialize.ts` into two files, one for web, and one for
Node. The Node initializer requires much more information as it needs to
know which browser(s) to download and where to store them, whereas the
web one does not.
A future PR that I'm working on will tidy up `src/common/Puppeteer.ts`
(as it contains a lot of Node specific logic around downloading and
installing browsers), but this change enables us to stop the browser
bundle attempting to use the `pkg-dir` dependency, which wouldn't work
within a browser, as well as keeping the size of the PRs down and
avoiding one mammoth PR.
This commit changes the internal representation of query handlers to contain Puppeteer-level code instead of page functions.
The interface `CustomQueryHandler` is introduced for user-defined query handlers. When a `CustomQueryHandler` is registered using `registerCustomQueryHandler` a corresponding Puppeteer-level handler is created through `makeQueryHandler` by wrapping the page functions as appropriate.
The internal query handlers (defined by the interface `QueryHandler`) contain two new functions: `waitFor` and `queryAllArray`.
- `waitFor` allows page-based handlers to make use of the `WaitTask`-backed implementation in `DOMWorld`, whereas purely Puppeteer-based handlers can define an alternative approach instead.
- `queryAllArray` is similar to `queryAll` but with a slightly different interface; it returns a `JSHandle` to an array with the results as opposed to an array of `ElementHandle`. It is used by `$$eval`.
After this change, we can introduce built-in query handlers that are not executed in the page context (#6307).
Also took the opportunity to pull out the PDF types into their own file
to clear up `Page.ts` slightly and give the PDF code a more natural
place to live.
* chore: enforce file extensions on imports
To make our output agnostic it should include file extensions in the
output, as per the ESM spec. It's a bit odd for Node packages but makes
it easier to publish a browser build.
* chore: remove `installAsyncStackHooks` helper
This code was written when browsers/Node didn't support errors in async
functions very well. They now do a much better job of this, so we can
lose the additonal complexity from our codebase and leave it to the host
environment :)
* lazy launcher is private
* remove async stack test
On a getter function you don't use `@returns` (as that's just for
methods). Instead we can just add to the general remarks to explain what
the property is providing.
* chore(docs): reduce warnings when generating docs
This is a bunch of small miscellaneous fixes that reduce the amount of
warnings logged when generating our new docs. The long term goal is to
get this list down to 0 warnings, but I'll do it in multiple PRs.
* satisfy doclint
* chore: Don't store revisions in `package.json`
It's quite messy to have to require the `package.json` file in multiple
places purely to find out what revision of a given browser we want to
use. We can also achieve better type safety by placing it in an actual
source file.
This commit makes that change and also tidies up our reliance on
`package.json` within the source code generally; we now only use it to
find the location of the Puppeteer root such that we know where to
install downloaded browsers to.
To avoid using `package.json` to parse the name of the module, we also
now explicitly have an entry point for the Puppeteer module and the
Puppeter Core module. This will make it easier in the future to ship
less code as part of core (e.g. core never needs to download a browser,
so why ship that code?). Core can also then not have any revisions based
info contained in it.
The test install script has also been updated to ensure that
puppeteer-core can be installed correctly too.
Finally, the `install` script has been moved to TypeScript for nicer
typechecking and safety. The functionality of it has not changed.
* chore(agnostic): ship CJS and ESM builds
For our work to enable Puppeteer in other environments (e.g. a browser)
we need to ship an ESM build. This commit changes our config to ship to
`lib/cjs` and `lib/esm` accordingly. The majority of our code stays the
same, with one small fix for the CJS build to ensure that we ship a
version that lets you `require('puppeteer')` rather than have to
`require('puppeteer').default`. We do this with the `cjs-entry.js` which
is what the `main` field in our `package.json` points to.
We also swap to `read-pkg-up` to find the `package.json` file. This is
because the folder structure of `lib/` does not match `src/` now we ship
to `cjs` and `esm`, so you cannot rely on exact paths. This module works
up from the file to find the nearest `package.json` so it will always
find Puppeteer's `package.json`.
Note that we *do not* point any users to the ESM build. We happen to
ship those files so people who know about them can get at them but it's
not expected (nor will we actively support) that people will rely on
them. The CommonJS build is considered our main build.
We may make breaking changes to the structure of the ESM build which we
will do without requiring new major versions. For example the ESM build
currently ships all files that the CJS build does, but given we are
working on the ESM build being able to run in the browser this may
change over time.
Long term once the Node versions catch up we can ditch CJS and ship
exclusively ESM but we are not there yet.
This CL migrates all the tests to TypeScript. The main benefits of this is that we start consuming our TypeScript definitions and therefore find errors in them. The act of migrating found some bugs in our definitions and now we can be sure to avoid them going forwards.
You'll notice the addition of some `TODO`s in the code; I didn't want this CL to get any bigger than it already is but I intend to follow those up once this lands. It's mostly figuring out how to extend the `expect` types with our `toBeGolden` helpers and some other slight confusions with types that the tests exposed.
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
These files will be used by both the web and node versions of Puppeteer.
Another name for this might be "core" but I don't want to cause
confusion with the puppeteer-core package that we publish at the moment.
