* chore: migrate to Mitt as the EventEmitter
This commit moves us to using Mitt [1] for the event emitter in
Puppeteer. This removes our dependency to Node's EventEmitter which is
part of a larger stream of work to enable a Puppeteer-web version that
doesn't depend on Node.
There are no large breaking changes as we support the main methods that
EventEmitter had, but it also provides some methods that Puppeteer
didn't use. Technically end users could depend on this but it's
unlikely.
[1]: https://github.com/developit/mitt
This script generated an `index.d.ts` file but that file was never
commited to git nor included in Puppeteer when we ship. As of right now
people who want TS types can install from the DefinitelyTyped repo and
we are working on shipping types from Puppeteer itself.
Therefore this script is not adding any value and can be removed.
This corresponds to Chromium 83.0.4103.0.
This roll includes:
- Enable SameSiteByDefaultCookies and CookiesWithoutSameSiteMustBeSecure https://crrev.com/c/2122809
Sticking to a specific version of TS rather than a ^ version so we know
everyone is on the exact same version. Think that's preferable for such
an important dependency.
* Warn when given unsupported product name.
Fixes#5844.
This change means when a user launches Puppeteer with a product name
that is not supported (which at the time of this commit means it's not
`firefox` or `chrome) we will warn them about it.
Decided on just a warning vs an error because the current behaviour is
that we fallback to launching Chrome and I don't think this warrants a
breaking change.
* chore: remove src/externs.d.ts
It defined global types that we don't want to use, and instead we move
to using interfaces that we import and reference just like with any
other interface.
This means other than Protocol (which I think is fine to leave as is),
there are no other magic global types and you have to import any types
or interfaces that you want.
* chore: update how we track coverage during unit tests
The old method of tracking coverage was causing issues. If a test failed
on CI, that test's failure would be lost because the test failing would
in turn cause the coverage to fail, but the `process.exit(1)` in the
coverage code caused Mocha to not output anything useful.
Instead the coverage checker now:
* tracks the coverage in memory in a Map (this hasn't changed)
* after all tests, writes that to disk in test/coverage.json (which is
gitignored)
* we then run a single Mocha test that asserts every method was called.
This means if the test run fails, the build will fail and give the error
about that test run, and that output won't be lost when the coverage
then fails too.
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
* chore: enforce src/protocol.d.ts is in sync
On CI we run `npm run compare-protocol-d-ts` which checks that the file
on disk is up to date with the protocol we fetch from the browser.
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
We don't support it and v3 shipped without including puppeteer-web in the browser. People are welcome to manually use Browserify to try to get Puppeteer running in a browser but it ultimately isn't our primary focus right now.
Getting puppeteer-core able to run in a browser is something we'll be looking at in the future so we'll revisit this soon.
* chore: add test for npm package installing correctly
This command packs up the module and installs it again to check we're
correctly bundling everything we need to allow users to do a fresh
install.
* install realpath
The codebase was incredibly inconsistent with the use of spacing around
curly braces, e.g.:
```
// this?
const a = {b: 1}
// or?
const a = { b: 1 }
```
This extended into import statements also. Google's styleguide is no
spacing, so we're going with that.
Rather than a denylist (`.npmignore`) we can instead use an allowlist
via the `files` option in `package.json`. This makes it much harder to
accidentally include files or folders in the build as you have to
explicitly list the files that will be included.
Fixes#5648.
The change to the install script to require TypeScript works fine when
installing from npm (because on npm the `lib` directory with the
compiled code already exists) but doesn't if you install from a GitHub
URL. By default it seems npm uses the `files` list when you install from
GitHub which means it's missing a bunch of files that we need to
compile.
Additionally by default when installing from a GitHub URL npm doesn't
install the dependencies which is an issue for us when we need to
compile TypeScript.
The fix is to create a `prepare` script that runs TypeScript if
required. From the npm docs [1]:
> `prepare`: Run both BEFORE the package is packed and published, on
> local npm install without any arguments, and when installing git
> dependencies
And from the npm docs on install [2], it confirms that if a package has
a `prepare` script it is run when installing from GitHub:
> As with regular git dependencies, dependencies and devDependencies
> will be installed if the package has a prepare script, before the
> package is done installing.
Despite having the `prepare` script we still need the TypeScript check
in `install.js` to satisfy the 3rd scenario below where we need to force
a compile:
* If I'm a user installing `puppeteer@X` from npm, the module is
published with the `lib/` directory of compiled code, so I'm set.
* If I'm a user installing Puppeteer from GitHub, the `prepare` script
will run TypeScript for me so I'm set.
* If I'm a developer working on Puppeteer, the `prepare` script also
runs but _after_ `npm install` which means `install.js` fails as it
requires `./lib/helper.js`. So in `install.js` we call
`compileTypeScriptIfRequired` to catch this case.
[1]: https://docs.npmjs.com/misc/scripts
[2]: https://docs.npmjs.com/cli/install
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
Fixes#5660.
This commit adds linting for `*.ts` files and loads up the recommended
list of TS rules from the ESLint TypeScript plugin. We can adjust the
exact rules overtime, but starting with the recommended list seems
sensible.
extract-zip removed support for callbacks and instead uses promises. Moreover, it has TypeScript support which allows us to remove the @types/extract-zip package.
This update allows downstream users to remove their installation of mkdirp, which uses a vulnerable version of minimist.
For more info, see https://github.com/maxogden/extract-zip/releases/tag/v2.0.0
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
This commit updates all the non-Puppeteer unit tests to run using Mocha and then deletes the custom test runner framework from this repository. The documentation has also been updated.
Rather than maintain our own test runner we should instead lean on the community and use Mocha which is very popular and also our test runner of choice in DevTools too.
Note that this commit doesn't remove the TestRunner source as it's still used for other unit tests, but they will be updated in a future PR and then we can remove the TestRunner.
The main bulk of this PR is updating the tests as the old TestRunner passed in contextual data via the `it` function callback whereas Mocha does not, so we introduce some helpers for the tests to make it easier.
Rather than use our own custom expect library, we can use expect from npm [1], which has an API almost identical to the one Puppeteer has, but with more options, better diffing, and is used by many in the community as it's the default assertions library that comes with Jest.
It's also thoroughly documented [2].
[1]: https://www.npmjs.com/package/expect
[2]: https://jestjs.io/docs/en/expect
This updates our `tsconfig.json` so it emits our JavaScript files as
well as type checking them. We compile into `./lib` which we then ship
in our npm package. The source code has moved from `./lib` into `./src`.
Because the `src/` directory is exclusively JS files, this change is a
no-op in terms of code functionality but is the first step towards being
able to replace `src/X.js` with `src/X.ts` in a way that allows us to
migrate incrementally.
The `lib` directory is gitignored, and the `src` directory is
npmignored. On `npm publish` we will now run `npm run tsc` in order to
generate the outputted code.
TypeScript seems to struggle to understand `Promise.all` when the items in the array return different types. If we were authoring in TS we could fix this with TS generics (`Promise.all<OurTypeHere>(...)`) but for now we can typecast the result. We'll fix this properly when we author in TS.
Continues the work to get up to TS 3.8 (latest release at time of writing).
This version of TS introduced built in definitions for web workers that include an `interface Worker` so TS gets confused when it sees us reference a `Worker`. I have renamed the imports to `PuppeteerWorker` as I couldn't figure out a way to tell TS to not load in the worker types; longer term we might consider renaming `Worker` to `PuppeteerWorker` (or an alternative) but that would be a breaking change that we don't need right now.
The other fix is similar; TypeScript doesn't differentiate between the built-in `WebSocket` type and the `ws` library. Renaming the import solves this too.
TS 3.5 got much stricter on writing changes to objects with varied types [1] so we have to do a bit of typecasting work to convince TS about the types of keys and values that we are setting.
Longer term we should think about a better data structure that avoids us having to jump through some hoops but for now I think this is a reasonable step to get us onto 3.5.
Same story regarding bindings on `window`: the easiest fix is to cast `window` to `any` for the code that adds to it. I'm sure we can come up with a more type-safe way of doing this in the future.
[1]: https://github.com/Microsoft/TypeScript/wiki/Breaking-Changes#fixes-to-unsound-writes-to-indexed-access-types
* (feat) Add option to fetch Firefox Nightly
Add Firefox support to BrowserFetcher and the install script.
By default, the latest Firefox Nightly is downloaded
directly from archive.mozilla.org (dmg, tar.bz2 and zip)
This also required changes that impact `puppeteer.launch()`
and `puppeteer.executablePath()`
Fixes#5151
* Update docs/api.md
Co-Authored-By: Mathias Bynens <mathias@qiwi.be>
* Clean up revision promise
* Improve error handling in revision check
* Remove matchAll
* Use explicit octal mode
* Update .gitignore
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
* chore: update relevant Node.js versions from 8 to 10
* chore: remove node6 and node8 folders from puppeteer-firefox ci
* fix: loosen definition for proc.stdio
* fix: update typescript version used in npm run test-types
