This PR aims to vastly improve our TS types and how we ship them.
Our previous attempt at shipping TypeScript was unfortunately flawed for
many reasons when compared to the @types/puppeteer package:
* It only worked if you needed the default export. If you wanted to
import a type that Puppeteer uses, you'd have to do `import type X
from 'puppeteer/lib/...'`. This is not something we want to encourage
because that means our internal file structure becomes almost public
API.
* It gave absolutely no help to CommonJS users in JS files because it
would warn people they needed to do `const pptr =
require('puppeteer').default, which is not correct.
* I found a bug in the `evaluate` types which mean't you couldn't
override the types to provide more info, and TS would insist the types
were all `unknown`.
The goal of this PR is to support:
1. In a `ts` file, `import puppeteer from 'puppeteer'`
1. In a `ts` file, `import type {ElementHandle} from 'puppeteer'`
1. In a `ts` file, referencing a type as `puppeteer.ElementHandle`
1. In a `ts` file, you can get good type inference when running
`foo.evaluate(x => x.clientHeight)`.
1. In a `js` file using CJS, you can do `const puppeteer =
require('puppeteer')` and get good type help from VSCode.
To test this I created a new empty repository with two test files in,
one `.ts` file with this in:
https://gist.github.com/jackfranklin/22ba2f390f97c7312cd70025a2096fc8,
and a `js` file with this in:
https://gist.github.com/jackfranklin/06bed136fdb22419cb7a8a9a4d4ef32f.
These files included enough code to check that the types were behaving
as I expected.
The fix for our types was to make use of API Extractor, which we already
use for our docs, to "rollup" all the disparate type files that TS
generates into one large `types.d.ts` which contains all the various
types that we define, such as:
```ts
export declare class ElementHandle {...}
export type EvaluateFn ...
```
If we then update our `package.json` `types` field to point to that file
in `lib/types.d.ts`, this then allows a developer to write:
```
import type {ElementHandle} from 'puppeteer'
```
And get the correct type definitions. However, what the `types.d.ts`
file doesn't do out of the box is declare the default export, so
importing Puppeteer's default export to call a method such as `launch`
on it will get you an error.
That's where the `script/add-default-export-to-types.ts` comes in. It
appends the following to the auto-generated `types.d.ts` file:
```ts
declare const puppeteer: PuppeteerNode;
export = puppeteer;
```
This tells TypeScript what the default export is, and by using the
`export =` syntax, we make sure TS understands both in a TS ESM
environment and in a JS CJS environment.
Now the `build` step, which is run by GitHub Actions when we release,
will generate the `.d.ts` file and then extend it with the default
export code.
To ensure that I was generating a valid package, I created a new
repository locally with the two code samples linked in Gists above. I
then ran:
```
npm init -y
npm install --save-dev typescript
npx tsc --init
```
Which gives me a base to test from. In Puppeteer, I ran `npm pack`,
which packs the module into a tar that's almost identical to what would
be published, so I can be confident that the .d.ts files in there are
what would be published.
I then installed it:
```
npm install --save-dev ../../puppeteer/puppeteer-7.0.1-post.tgz
```
And then reloaded VSCode in my dummy project. By deliberately making
typos and hovering over the code, I could confirm that all the goals
listed above were met, and this seems like a vast improvement on our
types.
This PR updates our `files` list to be more specific; rather than include everything from `lib`, we include just `.js`, `.d.ts`, and their equivalent sourcemaps. This prevents noisy meta-files like `.tsbuildinfo` sneaking into the package, which are no use to anyone.
In all my excitement about shipping types, we forgot to export the
actual `d.ts` file that makes it all happen... :(
I didn't pick this up locally because I was testing with `npm link`,
which does a symbolic link and therefore doesn't mirror the list of
files that make it into the published package.
To test this change, I made the change and ran `npm pack` to generate a
tar. I then created a new empty directory and did `npm init -y` followed
by `npm install path/to/puppeteer-7.tgz`.
I then initialised TypeScript, and wrote this:
```js
import puppeteer from 'puppeteer';
async function run() {
const browser = await puppeteer.launch()
const page = await browser.newPage()
await page.goto('https://foo.com')
}
run();
```
As I typed I got autocompletions and if I were to make an error, I'd get
a TypeScript error.
There is follow up work to be done because this unfortunately does not
work if you use `require` rather than ES Modules. @AviVahl suggested a
change in https://github.com/puppeteer/puppeteer/issues/6807 but I
cannot get it to work, so I'd like to do more investigation here. In the
mean time, whatever the contents of `cjs-entry.d.ts`, we should
definitely be exporting it as part of the module, so this PR is still
good to land.
This corresponds to Chromium 90.0.4403.0
This roll includes:
- Cut screenshot by ViewPort size, not position (crrev.com/c/2643792)
BREAKING CHANGE:
- `page.screenshot` cuts screenshot content by the ViewPort size, not ViewPort position.
This corresponds to Chromium 89.0.4389.0.
This roll includes:
- Add `SameParty` attribute to cookies
https://crrev.com/c/2598846
- Anchor `target=_blank` implies `rel=noopener`
https://crrev.com/c/1630010
- Don’t expect ignored elements in the AXTree
https://crrev.com/c/2505362
BREAKING CHANGE: The built-in `aria/` selector query handler doesn’t return ignored elements anymore.
Issue: #6758
Introduce the source-map-support package and require it for mocha running unit tests.
Turn on the sourceMap option for tsconfig.base.json so that the sourceMappingURL= line is emitted in the generated files.
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
With `nodejs@15.0.1`, install puppeteer with `https_proxy` set causes an error like:
```
> puppeteer@5.4.1 install node_modules/puppeteer
> node install.js
ERROR: Failed to set up Chromium r809590! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.
TypeError [ERR_INVALID_PROTOCOL]: Protocol "https:" not supported. Expected "http:"
at new NodeError (node:internal/errors:258:15)
at new ClientRequest (node:_http_client:155:11)
at Object.request (node:https:313:10)
at httpRequest (node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:488:17)
at downloadFile (node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:357:21)
at BrowserFetcher.download (node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:239:19)
at async downloadBrowser (node_modules/puppeteer/lib/cjs/puppeteer/node/install.js:48:5) {
code: 'ERR_INVALID_PROTOCOL'
}
```
The related issue is at https://github.com/TooTallNate/node-agent-base/pull/47, from package `agent-base` under `https-proxy-agent`
And the version bump is for `Refactor to TypeScript` is here: https://github.com/TooTallNate/node-https-proxy-agent/compare/4.0.0...5.0.0
Remove the redundant rule for dependency graph as the `src/index.ts` is not required. This was introduced in 64c9c709, but the filestructure has changed from since then and it doesn't work as `src/index.ts` is replaced with `src/node.ts` and `src/web.ts` as per the use case.
As long as we follow Conventional Commits for our commit messages (which is now enforced since #6483), we can automate the maintenance of this new changelog, which enables us to later automate the release process altogether.
This patch also eliminates the versioning decision-making process by automating it based on our commit messages.
Issue: #6482
* chore(agnostification): common/helper.ts
The `readProtocolStream` method uses `fs` only if you want to write to a
file. So we gate at the start of the function and ensure that if we got
given a path we are not in a Node environment.
They are generating a lot of noise in PRs. This commit removes them from git, but updates CI to generate them - to ensure there are no errors when generating the new documentation.
Further commits will:
1. Introduce linting to enforce methods are documented.
2. Generate previews of the new docs via GitHub actions or similar.
The `Launcher` class was serving two purposes:
1. Launch browsers
2. Connect to browsers
Number 1) only needs to be done in Node land, but 2) is agnostic; in a
browser version of Puppeteer we'll need the ability to connect over a
websocket to send commands back and forth.
As part of the agnostification work we needed to split the `Launcher` up
so that the connection part can be made agnostic. Additionally, I
removed dependencies on `https`, `http` and `URL` from Node, instead
leaning on fetch (via `node-fetch` if in Node land) and the browser
`URL` API (which was added to Node in Node 10).
This commit updates some miscellaneous dependencies to their latest
versions (with no other changes required) and also removes the `browser`
section, which was used by Browserify for the now long gone
Puppeteer-Web package that we used to publish.
This patch sets up commitlint to enforce the Conventional Commits format. This check runs with the other lint checks as part of npm run lint, and a Git commit hook is set up via Husky for automated local checks.
Issue: #6482
In `src/common` we now use `fs.promises.X` which we can dynamically
`import`. In a browser environment this code will never run because it's
gated on `isNode` (in a future PR we will add tree-shaking to the bundle
step such that this code is eliminated). By using `import`, we ensure
TypeScript still can track types and give good type information.
In `src/node` we continue to use `util.promisify` but that's not a
concern as that code explicitly is never run in the browser.
* roll Chromium to version 86, r800071
* roll Chrome DevTools protocol version to 0.0.799653
* fix HTTPRequest.continue after
* CDP: accept post data in the binary form in Fetch.continueRequest.
https://chromium-review.googlesource.com/c/chromium/src/+/2315239
* update new-docs
This corresponds to Chromium 85.0.4182.0.
This roll includes:
- Enable SameSiteByDefaultCookies and CookiesWithoutSameSiteMustBeSecure
https://crrev.com/c/2231445
- [FlexNG] Enable FlexNG by default
https://crrev.com/c/2216595Closes#6151.
