diff --git a/.prettierrc.cjs b/.prettierrc.cjs index f994e8a1..01c7bea5 100644 --- a/.prettierrc.cjs +++ b/.prettierrc.cjs @@ -1 +1,7 @@ -module.exports = require('gts/.prettierrc.json'); +/** + * @type {import('prettier').Config} + */ +module.exports = { + ...require('gts/.prettierrc.json'), + // proseWrap: 'always', // Uncomment this while working on Markdown documents. MAKE SURE TO COMMENT THIS BEFORE RUNNING CHECKS/FORMATS OR EVERYTHING WILL BE MODIFIED. +}; diff --git a/README.md b/README.md index 4fd09938..f5aa1e6d 100644 --- a/README.md +++ b/README.md @@ -1,22 +1,34 @@ # Puppeteer -[![Build status](https://github.com/puppeteer/puppeteer/workflows/CI/badge.svg)](https://github.com/puppeteer/puppeteer/actions?query=workflow%3ACI) [![npm puppeteer package](https://img.shields.io/npm/v/puppeteer.svg)](https://npmjs.org/package/puppeteer) +[![Build status](https://github.com/puppeteer/puppeteer/workflows/CI/badge.svg)](https://github.com/puppeteer/puppeteer/actions?query=workflow%3ACI) +[![npm puppeteer package](https://img.shields.io/npm/v/puppeteer.svg)](https://npmjs.org/package/puppeteer) #### [API](https://pptr.dev/api) | [FAQ](https://pptr.dev/faq) | [Contributing](https://pptr.dev/contributing) | [Troubleshooting](https://pptr.dev/troubleshooting) -> Puppeteer is a Node.js library which provides a high-level API to control Chrome/Chromium over the [DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/). Puppeteer runs in [headless](https://developers.google.com/web/updates/2017/04/headless-chrome) mode by default, but can be configured to run in full (non-headless) Chrome/Chromium. +> Puppeteer is a Node.js library which provides a high-level API to control +> Chrome/Chromium over the +> [DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/). +> Puppeteer runs in +> [headless](https://developers.google.com/web/updates/2017/04/headless-chrome) +> mode by default, but can be configured to run in full (non-headless) +> Chrome/Chromium. #### What can I do? -Most things that you can do manually in the browser can be done using Puppeteer! Here are a few examples to get you started: +Most things that you can do manually in the browser can be done using Puppeteer! +Here are a few examples to get you started: - Generate screenshots and PDFs of pages. -- Crawl a SPA (Single-Page Application) and generate pre-rendered content (i.e. "SSR" (Server-Side Rendering)). +- Crawl a SPA (Single-Page Application) and generate pre-rendered content (i.e. + "SSR" (Server-Side Rendering)). - Automate form submission, UI testing, keyboard input, etc. -- Create an automated testing environment using the latest JavaScript and browser features. -- Capture a [timeline trace](https://developers.google.com/web/tools/chrome-devtools/evaluate-performance/reference) of your site to help diagnose performance issues. +- Create an automated testing environment using the latest JavaScript and + browser features. +- Capture a + [timeline trace](https://developers.google.com/web/tools/chrome-devtools/evaluate-performance/reference) + of your site to help diagnose performance issues. - Test Chrome Extensions. ## Getting Started @@ -31,25 +43,57 @@ npm i puppeteer # or `pnpm i puppeteer` ``` -When you install Puppeteer, it automatically downloads a recent version of Chromium (~170MB macOS, ~282MB Linux, ~280MB Windows) that is [guaranteed to work](https://pptr.dev/faq#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) with Puppeteer. For a version of Puppeteer without installation, see [`puppeteer-core`](#puppeteer-core). +When you install Puppeteer, it automatically downloads a recent version of +Chromium (~170MB macOS, ~282MB Linux, ~280MB Windows) that is +[guaranteed to work](https://pptr.dev/faq#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) +with Puppeteer. For a version of Puppeteer without installation, see +[`puppeteer-core`](#puppeteer-core). #### Environment Variables -Puppeteer looks for certain [environment variables](https://en.wikipedia.org/wiki/Environment_variable) for customizing behavior. -If Puppeteer doesn't find them in the environment during the installation step, a lowercased variant of these variables will be used from the [npm config](https://docs.npmjs.com/cli/config). +Puppeteer looks for certain +[environment variables](https://en.wikipedia.org/wiki/Environment_variable) for +customizing behavior. If Puppeteer doesn't find them in the environment during +the installation step, a lowercased variant of these variables will be used from +the [npm config](https://docs.npmjs.com/cli/config). -- `HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY` - defines HTTP proxy settings that are used to download and run the browser. -- `PUPPETEER_CACHE_DIR` - defines the directory to be used by Puppeteer for caching. Defaults to [`os.homedir()/.cache/puppeteer`](https://nodejs.org/api/os.html#os_os_homedir). -- `PUPPETEER_SKIP_CHROMIUM_DOWNLOAD` - do not download bundled Chromium during installation step. -- `PUPPETEER_TMP_DIR` - defines the directory to be used by Puppeteer for creating temporary files. Defaults to [`os.tmpdir()`](https://nodejs.org/api/os.html#os_os_tmpdir). -- `PUPPETEER_DOWNLOAD_HOST` - specifies the URL prefix that is used to download Chromium. Note: this includes protocol and might even include path prefix. Defaults to `https://storage.googleapis.com`. -- `PUPPETEER_DOWNLOAD_PATH` - specifies the path for the downloads folder. Defaults to `/chromium`, where `` is Puppeteer's cache directory. -- `PUPPETEER_BROWSER_REVISION` - specifies a certain version of the browser you'd like Puppeteer to use. See [`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) on how executable path is inferred. -- `PUPPETEER_EXECUTABLE_PATH` - specifies an executable path to be used in [`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch). -- `PUPPETEER_PRODUCT` - specifies which browser you'd like Puppeteer to use. Must be either `chrome` or `firefox`. This can also be used during installation to fetch the recommended browser binary. Setting `product` programmatically in [`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) supersedes this environment variable. -- `PUPPETEER_EXPERIMENTAL_CHROMIUM_MAC_ARM` — specify Puppeteer download Chromium for Apple M1. On Apple M1 devices Puppeteer by default downloads the version for Intel's processor which runs via Rosetta. It works without any problems, however, with this option, you should get more efficient resource usage (CPU and RAM) that could lead to a faster execution time. +- `HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY` - defines HTTP proxy settings that are + used to download and run the browser. +- `PUPPETEER_CACHE_DIR` - defines the directory to be used by Puppeteer for + caching. Defaults to + [`os.homedir()/.cache/puppeteer`](https://nodejs.org/api/os.html#os_os_homedir). +- `PUPPETEER_SKIP_CHROMIUM_DOWNLOAD` - do not download bundled Chromium during + installation step. +- `PUPPETEER_TMP_DIR` - defines the directory to be used by Puppeteer for + creating temporary files. Defaults to + [`os.tmpdir()`](https://nodejs.org/api/os.html#os_os_tmpdir). +- `PUPPETEER_DOWNLOAD_HOST` - specifies the URL prefix that is used to download + Chromium. Note: this includes protocol and might even include path prefix. + Defaults to `https://storage.googleapis.com`. +- `PUPPETEER_DOWNLOAD_PATH` - specifies the path for the downloads folder. + Defaults to `/chromium`, where `` is Puppeteer's cache + directory. +- `PUPPETEER_BROWSER_REVISION` - specifies a certain version of the browser + you'd like Puppeteer to use. See + [`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) on + how executable path is inferred. +- `PUPPETEER_EXECUTABLE_PATH` - specifies an executable path to be used in + [`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch). +- `PUPPETEER_PRODUCT` - specifies which browser you'd like Puppeteer to use. + Must be either `chrome` or `firefox`. This can also be used during + installation to fetch the recommended browser binary. Setting `product` + programmatically in + [`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) + supersedes this environment variable. +- `PUPPETEER_EXPERIMENTAL_CHROMIUM_MAC_ARM` — specify Puppeteer download + Chromium for Apple M1. On Apple M1 devices Puppeteer by default downloads the + version for Intel's processor which runs via Rosetta. It works without any + problems, however, with this option, you should get more efficient resource + usage (CPU and RAM) that could lead to a faster execution time. -Environment variables except for `PUPPETEER_CACHE_DIR` are not used for [`puppeteer-core`](#puppeteer-core) since core does not automatically handle browser downloading. +Environment variables except for `PUPPETEER_CACHE_DIR` are not used for +[`puppeteer-core`](#puppeteer-core) since core does not automatically handle +browser downloading. #### `puppeteer-core` @@ -58,12 +102,23 @@ Every release since v1.7.0 we publish two packages: - [`puppeteer`](https://www.npmjs.com/package/puppeteer) - [`puppeteer-core`](https://www.npmjs.com/package/puppeteer-core) -`puppeteer` is a _product_ for browser automation. When installed, it downloads a version of -Chromium, which it then drives using `puppeteer-core`. Being an end-user product, `puppeteer` supports a bunch of convenient `PUPPETEER_*` env variables to tweak its behavior. +`puppeteer` is a _product_ for browser automation. When installed, it downloads +a version of Chromium, which it then drives using `puppeteer-core`. Being an +end-user product, `puppeteer` supports a bunch of convenient `PUPPETEER_*` env +variables to tweak its behavior. -`puppeteer-core` is a _library_ to help drive anything that supports DevTools protocol. `puppeteer-core` doesn't download Chromium when installed. Being a library, `puppeteer-core` is fully driven through its programmatic interface. +`puppeteer-core` is a _library_ to help drive anything that supports DevTools +protocol. `puppeteer-core` doesn't download Chromium when installed. Being a +library, `puppeteer-core` is fully driven through its programmatic interface. -You should only use `puppeteer-core` if you are [connecting to a remote browser](https://pptr.dev/api/puppeteer.puppeteer.connect) or [managing browsers yourself](https://pptr.dev/api/puppeteer.browserfetcher). If you are managing browsers yourself, you will need to call [`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) with an explicit [`executablePath`](https://pptr.dev/api/puppeteer.launchoptions.executablepath) or [`channel`](https://pptr.dev/api/puppeteer.launchoptions.channel). +You should only use `puppeteer-core` if you are +[connecting to a remote browser](https://pptr.dev/api/puppeteer.puppeteer.connect) +or [managing browsers yourself](https://pptr.dev/api/puppeteer.browserfetcher). +If you are managing browsers yourself, you will need to call +[`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) with +an explicit +[`executablePath`](https://pptr.dev/api/puppeteer.launchoptions.executablepath) +or [`channel`](https://pptr.dev/api/puppeteer.launchoptions.channel). When using `puppeteer-core`, remember to change the import: @@ -73,15 +128,25 @@ import puppeteer from 'puppeteer-core'; ### Usage -Puppeteer follows the latest [maintenance LTS](https://github.com/nodejs/Release#release-schedule) version of Node. +Puppeteer follows the latest +[maintenance LTS](https://github.com/nodejs/Release#release-schedule) version of +Node. -Puppeteer will be familiar to people using other browser testing frameworks. You [launch](https://pptr.dev/api/puppeteer.puppeteernode.launch)/[connect](https://pptr.dev/api/puppeteer.puppeteernode.connect) a [browser](https://pptr.dev/api/puppeteer.browser), [create](https://pptr.dev/api/puppeteer.browser.newpage) some [pages](https://pptr.dev/api/puppeteer.page), and then manipulate them with [Puppeteer's API](https://pptr.dev/api). +Puppeteer will be familiar to people using other browser testing frameworks. You +[launch](https://pptr.dev/api/puppeteer.puppeteernode.launch)/[connect](https://pptr.dev/api/puppeteer.puppeteernode.connect) +a [browser](https://pptr.dev/api/puppeteer.browser), +[create](https://pptr.dev/api/puppeteer.browser.newpage) some +[pages](https://pptr.dev/api/puppeteer.page), and then manipulate them with +[Puppeteer's API](https://pptr.dev/api). -For more in-depth usage, check our [guides](https://pptr.dev/guides) and [examples](https://github.com/puppeteer/puppeteer/tree/main/examples). +For more in-depth usage, check our [guides](https://pptr.dev/guides) and +[examples](https://github.com/puppeteer/puppeteer/tree/main/examples). #### Example -The following example searches [developers.google.com/web](https://developers.google.com/web) for articles tagged "Headless Chrome" and scrape results from the results page. +The following example searches +[developers.google.com/web](https://developers.google.com/web) for articles +tagged "Headless Chrome" and scrape results from the results page. ```ts import puppeteer from 'puppeteer'; @@ -123,7 +188,11 @@ import puppeteer from 'puppeteer'; **1. Uses Headless mode** -Puppeteer launches Chromium in [headless mode](https://developers.google.com/web/updates/2017/04/headless-chrome). To launch a full version of Chromium, set the [`headless`](https://pptr.dev/api/puppeteer.browserlaunchargumentoptions.headless) option when launching a browser: +Puppeteer launches Chromium in +[headless mode](https://developers.google.com/web/updates/2017/04/headless-chrome). +To launch a full version of Chromium, set the +[`headless`](https://pptr.dev/api/puppeteer.browserlaunchargumentoptions.headless) +option when launching a browser: ```ts const browser = await puppeteer.launch({headless: false}); // default is true @@ -131,21 +200,29 @@ const browser = await puppeteer.launch({headless: false}); // default is true **2. Runs a bundled version of Chromium** -By default, Puppeteer downloads and uses a specific version of Chromium so its API -is guaranteed to work out of the box. To use Puppeteer with a different version of Chrome or Chromium, -pass in the executable's path when creating a `Browser` instance: +By default, Puppeteer downloads and uses a specific version of Chromium so its +API is guaranteed to work out of the box. To use Puppeteer with a different +version of Chrome or Chromium, pass in the executable's path when creating a +`Browser` instance: ```ts const browser = await puppeteer.launch({executablePath: '/path/to/Chrome'}); ``` -You can also use Puppeteer with Firefox Nightly (experimental support). See [`Puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) for more information. +You can also use Puppeteer with Firefox Nightly (experimental support). See +[`Puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) for +more information. -See [`this article`](https://www.howtogeek.com/202825/what%E2%80%99s-the-difference-between-chromium-and-chrome/) for a description of the differences between Chromium and Chrome. [`This article`](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/chromium_browser_vs_google_chrome.md) describes some differences for Linux users. +See +[`this article`](https://www.howtogeek.com/202825/what%E2%80%99s-the-difference-between-chromium-and-chrome/) +for a description of the differences between Chromium and Chrome. +[`This article`](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/chromium_browser_vs_google_chrome.md) +describes some differences for Linux users. **3. Creates a fresh user profile** -Puppeteer creates its own browser user profile which it **cleans up on every run**. +Puppeteer creates its own browser user profile which it **cleans up on every +run**. #### Using Docker @@ -153,7 +230,8 @@ See our [guide on using Docker](https://pptr.dev/guides/docker). #### Using Chrome Extensions -See our [guide on using Chrome extensions](https://pptr.dev/guides/chrome-extensions). +See our +[guide on using Chrome extensions](https://pptr.dev/guides/chrome-extensions). ## Resources @@ -164,8 +242,10 @@ See our [guide on using Chrome extensions](https://pptr.dev/guides/chrome-extens ## Contributing -Check out our [contributing guide](https://pptr.dev/contributing) to get an overview of Puppeteer development. +Check out our [contributing guide](https://pptr.dev/contributing) to get an +overview of Puppeteer development. ## FAQ -Our [FAQ](https://pptr.dev/faq) has migrated to [our site](https://pptr.dev/faq). +Our [FAQ](https://pptr.dev/faq) has migrated to +[our site](https://pptr.dev/faq). diff --git a/docs/contributing.md b/docs/contributing.md index 2060ce00..1bc48d64 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -4,8 +4,8 @@ sidebar_position: 4 # Contributing -First of all, thank you for your interest in Puppeteer! -We'd love to accept your patches and contributions! +First of all, thank you for your interest in Puppeteer! We'd love to accept your +patches and contributions! ## Contributor License Agreement @@ -48,7 +48,9 @@ again. ### macOS ARM and custom executables. -- To run experimental Chromium macOS ARM tests, firstly ensure you have correct Chromium version installed locally (you only need to do this once, not on every test run) and then you can run the tests: +- To run experimental Chromium macOS ARM tests, firstly ensure you have correct + Chromium version installed locally (you only need to do this once, not on + every test run) and then you can run the tests: ```bash PUPPETEER_EXPERIMENTAL_CHROMIUM_MAC_ARM=1 npm install @@ -69,11 +71,17 @@ To build a single package, you can run npm run build --workspace # e.g. puppeteer ``` -This will build all dependent packages automatically, so specifying a single packages is sufficient. This is all possible due to [wireit](https://github.com/google/wireit) which behaves similar to [GNU Make](https://www.gnu.org/software/make/). +This will build all dependent packages automatically, so specifying a single +packages is sufficient. This is all possible due to +[wireit](https://github.com/google/wireit) which behaves similar to +[GNU Make](https://www.gnu.org/software/make/). ## Removing stale artifacts -It's possible some generated artifacts (such as `packages/puppeteer-core/src/types.ts`) can become stale since these artifacts rely on complex conditions (such as names of distinct files) that cannot be captured by the build system. To clean artifacts, you can run +It's possible some generated artifacts (such as +`packages/puppeteer-core/src/types.ts`) can become stale since these artifacts +rely on complex conditions (such as names of distinct files) that cannot be +captured by the build system. To clean artifacts, you can run ```sh npm run clean # or npm run clean --workspace @@ -81,18 +89,25 @@ npm run clean # or npm run clean --workspace ## Comprehensive testing -Outside of `npm test`, there are several other [`npm` scripts](https://docs.npmjs.com/cli/v8/using-npm/scripts) that are usually check through CI: +Outside of `npm test`, there are several other +[`npm` scripts](https://docs.npmjs.com/cli/v8/using-npm/scripts) that are +usually check through CI: -- `test-install` - Tests whether `puppeteer` and `puppeteer-core` install properly and are functional. -- `test-types` - Tests the TypeScript types in `puppeteer` using [`tsd`](https://github.com/SamVerschueren/tsd). +- `test-install` - Tests whether `puppeteer` and `puppeteer-core` install + properly and are functional. +- `test-types` - Tests the TypeScript types in `puppeteer` using + [`tsd`](https://github.com/SamVerschueren/tsd). - `test:chrome:**` - Tests `puppeteer` on Chromium. - `test:firefox:**` - Tests `puppeteer` on Firefox. -The default `npm test` runs `test:{chrome,firefox}:headless` which is generally sufficient. +The default `npm test` runs `test:{chrome,firefox}:headless` which is generally +sufficient. -Puppeteer uses a custom test runner on top of Mocha that consults -the [TestExpectations.json](https://github.com/puppeteer/puppeteer/blob/main/test/TestExpectations.json) -to see if a given test result is expected or not. See more info about the test runner in [`tools/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/tools/mochaRunner). +Puppeteer uses a custom test runner on top of Mocha that consults the +[TestExpectations.json](https://github.com/puppeteer/puppeteer/blob/main/test/TestExpectations.json) +to see if a given test result is expected or not. See more info about the test +runner in +[`tools/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/tools/mochaRunner). ## Code reviews @@ -129,7 +144,8 @@ The following is a description of the primary folders in Puppeteer: - `packages` contains all public source code. - `test` contains all test source code. -- `test-d` contains type tests using [`tsd`](https://github.com/SamVerschueren/tsd). +- `test-d` contains type tests using + [`tsd`](https://github.com/SamVerschueren/tsd). - `tools` contains miscellaneous scripts that are used in building and etc. - `tools/mochaRunner` - contains the source code for our test runner. @@ -137,17 +153,23 @@ The following is a description of the primary folders in Puppeteer: When authoring new API methods, consider the following: -- Expose as little information as needed. When in doubt, don’t expose new information. +- Expose as little information as needed. When in doubt, don’t expose new + information. - Methods are used in favor of getters/setters. - The only exception is namespaces, e.g. `page.keyboard` and `page.coverage` -- All string literals must be small case. This includes event names and option values. -- Avoid adding "sugar" API (API that is trivially implementable in user-space) unless they're **extremely** demanded. +- All string literals must be small case. This includes event names and option + values. +- Avoid adding "sugar" API (API that is trivially implementable in user-space) + unless they're **extremely** demanded. ## Commit messages -Commit messages should follow [the Conventional Commits format](https://www.conventionalcommits.org/en/v1.0.0/#summary). This is enforced via `npm run commitlint`. +Commit messages should follow +[the Conventional Commits format](https://www.conventionalcommits.org/en/v1.0.0/#summary). +This is enforced via `npm run commitlint`. -In particular, breaking changes should clearly be noted as “BREAKING CHANGE:” in the commit message footer. Example: +In particular, breaking changes should clearly be noted as “BREAKING CHANGE:” in +the commit message footer. Example: ``` fix(page): fix page.pizza method @@ -163,19 +185,28 @@ To deliver to a different location, use the "deliver" option: ## Writing documentation -Documentation is generated via `npm run docs`. It is automatically published to our documentation site on merge and gets versioned on release. +Documentation is generated via `npm run docs`. It is automatically published to +our documentation site on merge and gets versioned on release. ## Writing TSDoc comments -Each change to Puppeteer should be thoroughly documented using TSDoc comments. Refer to the [API Extractor documentation](https://api-extractor.com/pages/tsdoc/doc_comment_syntax/) for information on the exact syntax. +Each change to Puppeteer should be thoroughly documented using TSDoc comments. +Refer to the +[API Extractor documentation](https://api-extractor.com/pages/tsdoc/doc_comment_syntax/) +for information on the exact syntax. -- Every new method needs to have either `@public` or `@internal` added as a tag depending on if it is part of the public API. -- Keep each line in a comment to no more than 90 characters (ESLint will warn you if you go over this). If you're a VSCode user the [Rewrap plugin](https://marketplace.visualstudio.com/items?itemName=stkb.rewrap) is highly recommended! +- Every new method needs to have either `@public` or `@internal` added as a tag + depending on if it is part of the public API. +- Keep each line in a comment to no more than 90 characters (ESLint will warn + you if you go over this). If you're a VSCode user the + [Rewrap plugin](https://marketplace.visualstudio.com/items?itemName=stkb.rewrap) + is highly recommended! ## Running the documentation site locally 1. At root, install all dependencies with `npm i --ignore-scripts`. -2. run `npm run docs` which will generate all the `.md` files on `puppeteer/docs/api`. +2. run `npm run docs` which will generate all the `.md` files on + `puppeteer/docs/api`. 3. run `npm i` in `puppeteer/website`. 4. run `npm start` in `puppeteer/website`. @@ -183,32 +214,43 @@ Each change to Puppeteer should be thoroughly documented using TSDoc comments. R For all dependencies (both installation and development): -- **Do not add** a dependency if the desired functionality is easily implementable. +- **Do not add** a dependency if the desired functionality is easily + implementable. - If adding a dependency, it should be well-maintained and trustworthy. A barrier for introducing new installation dependencies is especially high: -- **Do not add** installation dependency unless it's critical to project success. +- **Do not add** installation dependency unless it's critical to project + success. -There are additional considerations for dependencies that are environment agonistic. See the [`third_party/README.md`](https://github.com/puppeteer/puppeteer/blob/main/third_party/README.md) for details. +There are additional considerations for dependencies that are environment +agonistic. See the +[`third_party/README.md`](https://github.com/puppeteer/puppeteer/blob/main/third_party/README.md) +for details. ## Testing tips - Every feature should be accompanied by a test. - Every public api event/method should be accompanied by a test. - Tests should not depend on external services. -- Tests should work on all three platforms: Mac, Linux and Win. This is especially important for screenshot tests. +- Tests should work on all three platforms: Mac, Linux and Win. This is + especially important for screenshot tests. -If a test is expected to fail on certain configurations or became flaky, update [TestExpectations.json](https://github.com/puppeteer/puppeteer/blob/main/test/TestExpectations.json) -to reflect that. See more info about TestExpectations.json in [`tools/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/tools/mochaRunner). +If a test is expected to fail on certain configurations or became flaky, update +[TestExpectations.json](https://github.com/puppeteer/puppeteer/blob/main/test/TestExpectations.json) +to reflect that. See more info about TestExpectations.json in +[`tools/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/tools/mochaRunner). ## API Coverage -Every public API method or event should be called at least once in tests. To ensure this, the main `test` command runs coverage during testing. +Every public API method or event should be called at least once in tests. To +ensure this, the main `test` command runs coverage during testing. ## Debugging Puppeteer -See [Debugging Tips](https://github.com/puppeteer/puppeteer/blob/main/README.md#debugging-tips) in the readme. +See +[Debugging Tips](https://github.com/puppeteer/puppeteer/blob/main/README.md#debugging-tips) +in the readme. # For Project Maintainers @@ -216,26 +258,46 @@ See [Debugging Tips](https://github.com/puppeteer/puppeteer/blob/main/README.md# The following steps are needed to update the Chromium version. -1. Find a suitable Chromium revision. Not all revisions have builds for all platforms, so we need to find one that does. The easiest way is to run `tools/check_availability.js -rd` to find the latest suitable `dev` Chromium revision (see `tools/check_availability.js -help` for more options). -2. Update `packages/puppeteer-core/src/revisions.ts` with the found revision number. -3. Update `versions.js` with the new Chromium-to-Puppeteer version mapping and update `lastMaintainedChromiumVersion` with the latest stable Chrome version. -4. Run `npm run check:protocol-revision`. If it fails, update `package.json` with the expected `devtools-protocol` version. +1. Find a suitable Chromium revision. Not all revisions have builds for all + platforms, so we need to find one that does. The easiest way is to run + `tools/check_availability.js -rd` to find the latest suitable `dev` Chromium + revision (see `tools/check_availability.js -help` for more options). +2. Update `packages/puppeteer-core/src/revisions.ts` with the found revision + number. +3. Update `versions.js` with the new Chromium-to-Puppeteer version mapping and + update `lastMaintainedChromiumVersion` with the latest stable Chrome version. +4. Run `npm run check:protocol-revision`. If it fails, update `package.json` + with the expected `devtools-protocol` version. 5. Run `npm run build` and `npm install`. -6. Run `npm test` and ensure that all tests pass. If a test fails, [bisect](#bisecting-upstream-changes) the upstream cause of the failure, and either update the test expectations accordingly (if it was an intended change) or work around the changes in Puppeteer (if it’s not desirable to change Puppeteer’s observable behavior). -7. Commit and push your changes and open a pull request. - The commit message must contain the version in `Chromium ()` format to ensure that [pptr.dev](https://pptr.dev/) can parse it correctly, e.g. `'feat(chromium): roll to Chromium 90.0.4427.0 (r856583)'`. +6. Run `npm test` and ensure that all tests pass. If a test fails, + [bisect](#bisecting-upstream-changes) the upstream cause of the failure, and + either update the test expectations accordingly (if it was an intended + change) or work around the changes in Puppeteer (if it’s not desirable to + change Puppeteer’s observable behavior). +7. Commit and push your changes and open a pull request. The commit message must + contain the version in `Chromium ()` format to ensure + that [pptr.dev](https://pptr.dev/) can parse it correctly, e.g. + `'feat(chromium): roll to Chromium 90.0.4427.0 (r856583)'`. ### Bisecting upstream changes -Sometimes, performing a Chromium roll causes tests to fail. To figure out the cause, you need to bisect Chromium revisions to figure out the earliest possible revision that changed the behavior. The `bisect` script can be helpful here. Given a pattern for one or more unit tests, it will automatically bisect the current range: +Sometimes, performing a Chromium roll causes tests to fail. To figure out the +cause, you need to bisect Chromium revisions to figure out the earliest possible +revision that changed the behavior. The `bisect` script can be helpful here. +Given a pattern for one or more unit tests, it will automatically bisect the +current range: ```sh npm run bisect -- --good 686378 --bad 706915 script.js npm run bisect -- --unit-test Response.fromCache ``` -By default, it will use the Chromium revision in `packages/puppeteer-core/src/revisions.ts` from the `main` branch and from the working tree to determine the range to bisect. +By default, it will use the Chromium revision in +`packages/puppeteer-core/src/revisions.ts` from the `main` branch and from the +working tree to determine the range to bisect. ## Releasing to npm -We use [release-please](https://github.com/googleapis/release-please) to automate releases. When a release should be done, check for the release PR in our [pull requests](https://github.com/puppeteer/puppeteer/pulls) and merge it. +We use [release-please](https://github.com/googleapis/release-please) to +automate releases. When a release should be done, check for the release PR in +our [pull requests](https://github.com/puppeteer/puppeteer/pulls) and merge it. diff --git a/docs/faq.md b/docs/faq.md index 2dab314a..8ddf79e6 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -6,62 +6,107 @@ sidebar_position: 2 ## Q: Who maintains Puppeteer? -The Chrome DevTools team maintains the library, but we'd love your help and expertise on the project! -See our [contributing guide](https://pptr.dev/contributing). +The Chrome DevTools team maintains the library, but we'd love your help and +expertise on the project! See our +[contributing guide](https://pptr.dev/contributing). ## Q: What is the status of cross-browser support? -Official Firefox support is currently experimental. The ongoing collaboration with Mozilla aims to support common end-to-end testing use cases, for which developers expect cross-browser coverage. The Puppeteer team needs input from users to stabilize Firefox support and to bring missing APIs to our attention. +Official Firefox support is currently experimental. The ongoing collaboration +with Mozilla aims to support common end-to-end testing use cases, for which +developers expect cross-browser coverage. The Puppeteer team needs input from +users to stabilize Firefox support and to bring missing APIs to our attention. -From Puppeteer v2.1.0 onwards you can specify [`puppeteer.launch({product: 'firefox'})`](https://pptr.dev/api/puppeteer.puppeteernode.launch) to run your Puppeteer scripts in Firefox Nightly, without any additional custom patches. While [an older experiment](https://www.npmjs.com/package/puppeteer-firefox) required a patched version of Firefox, [the current approach](https://wiki.mozilla.org/Remote) works with “stock” Firefox. +From Puppeteer v2.1.0 onwards you can specify +[`puppeteer.launch({product: 'firefox'})`](https://pptr.dev/api/puppeteer.puppeteernode.launch) +to run your Puppeteer scripts in Firefox Nightly, without any additional custom +patches. While +[an older experiment](https://www.npmjs.com/package/puppeteer-firefox) required +a patched version of Firefox, +[the current approach](https://wiki.mozilla.org/Remote) works with “stock” +Firefox. -We will continue to collaborate with other browser vendors to bring Puppeteer support to browsers such as Safari. -This effort includes exploration of a standard for executing cross-browser commands (instead of relying on the non-standard DevTools Protocol used by Chrome). +We will continue to collaborate with other browser vendors to bring Puppeteer +support to browsers such as Safari. This effort includes exploration of a +standard for executing cross-browser commands (instead of relying on the +non-standard DevTools Protocol used by Chrome). ## Q: What are Puppeteer’s goals and principles? The goals of the project are: -- Provide a slim, canonical library that highlights the capabilities of the [DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/). -- Provide a reference implementation for similar testing libraries. Eventually, these other frameworks could adopt Puppeteer as their foundational layer. +- Provide a slim, canonical library that highlights the capabilities of the + [DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/). +- Provide a reference implementation for similar testing libraries. Eventually, + these other frameworks could adopt Puppeteer as their foundational layer. - Grow the adoption of headless/automated browser testing. - Help dogfood new DevTools Protocol features...and catch bugs! -- Learn more about the pain points of automated browser testing and help fill those gaps. +- Learn more about the pain points of automated browser testing and help fill + those gaps. -We adapt [Chromium principles](https://www.chromium.org/developers/core-principles) to help us drive product decisions: +We adapt +[Chromium principles](https://www.chromium.org/developers/core-principles) to +help us drive product decisions: -- **Speed**: Puppeteer has almost zero performance overhead over an automated page. -- **Security**: Puppeteer operates off-process with respect to Chromium, making it safe to automate potentially malicious pages. +- **Speed**: Puppeteer has almost zero performance overhead over an automated + page. +- **Security**: Puppeteer operates off-process with respect to Chromium, making + it safe to automate potentially malicious pages. - **Stability**: Puppeteer should not be flaky and should not leak memory. -- **Simplicity**: Puppeteer provides a high-level API that’s easy to use, understand, and debug. +- **Simplicity**: Puppeteer provides a high-level API that’s easy to use, + understand, and debug. ## Q: Is Puppeteer replacing Selenium/WebDriver? **No**. Both projects are valuable for very different reasons: -- Selenium/WebDriver focuses on cross-browser automation; its value proposition is a single standard API that works across all major browsers. -- Puppeteer focuses on Chromium; its value proposition is richer functionality and higher reliability. +- Selenium/WebDriver focuses on cross-browser automation; its value proposition + is a single standard API that works across all major browsers. +- Puppeteer focuses on Chromium; its value proposition is richer functionality + and higher reliability. -That said, you **can** use Puppeteer to run tests against Chromium, e.g. using the community-driven [jest-puppeteer](https://github.com/smooth-code/jest-puppeteer). While this probably shouldn’t be your only testing solution, it does have a few good points compared to WebDriver: +That said, you **can** use Puppeteer to run tests against Chromium, e.g. using +the community-driven +[jest-puppeteer](https://github.com/smooth-code/jest-puppeteer). While this +probably shouldn’t be your only testing solution, it does have a few good points +compared to WebDriver: -- Puppeteer requires zero setup and comes bundled with the Chromium version it works best with, making it [very easy to start with](https://github.com/puppeteer/puppeteer/#getting-started). At the end of the day, it’s better to have a few tests running chromium-only, than no tests at all. -- Puppeteer has event-driven architecture, which removes a lot of potential flakiness. There’s no need for evil “sleep(1000)” calls in puppeteer scripts. -- Puppeteer runs headless by default, which makes it fast to run. Puppeteer v1.5.0 also exposes browser contexts, making it possible to efficiently parallelize test execution. -- Puppeteer shines when it comes to debugging: flip the “headless” bit to false, add “slowMo”, and you’ll see what the browser is doing. You can even open Chrome DevTools to inspect the test environment. +- Puppeteer requires zero setup and comes bundled with the Chromium version it + works best with, making it + [very easy to start with](https://github.com/puppeteer/puppeteer/#getting-started). + At the end of the day, it’s better to have a few tests running chromium-only, + than no tests at all. +- Puppeteer has event-driven architecture, which removes a lot of potential + flakiness. There’s no need for evil “sleep(1000)” calls in puppeteer scripts. +- Puppeteer runs headless by default, which makes it fast to run. Puppeteer + v1.5.0 also exposes browser contexts, making it possible to efficiently + parallelize test execution. +- Puppeteer shines when it comes to debugging: flip the “headless” bit to false, + add “slowMo”, and you’ll see what the browser is doing. You can even open + Chrome DevTools to inspect the test environment. ## Q: Why doesn’t Puppeteer v.XXX work with Chromium v.YYY? -We see Puppeteer as an **indivisible entity** with Chromium. Each version of Puppeteer bundles a specific version of Chromium – **the only** version it is guaranteed to work with. +We see Puppeteer as an **indivisible entity** with Chromium. Each version of +Puppeteer bundles a specific version of Chromium – **the only** version it is +guaranteed to work with. -This is not an artificial constraint: A lot of work on Puppeteer is actually taking place in the Chromium repository. Here’s a typical story: +This is not an artificial constraint: A lot of work on Puppeteer is actually +taking place in the Chromium repository. Here’s a typical story: -- A Puppeteer bug is reported: https://github.com/puppeteer/puppeteer/issues/2709 -- It turned out this is an issue with the DevTools protocol, so we’re fixing it in Chromium: https://chromium-review.googlesource.com/c/chromium/src/+/1102154 -- Once the upstream fix is landed, we roll updated Chromium into Puppeteer: https://github.com/puppeteer/puppeteer/pull/2769 +- A Puppeteer bug is reported: + https://github.com/puppeteer/puppeteer/issues/2709 +- It turned out this is an issue with the DevTools protocol, so we’re fixing it + in Chromium: https://chromium-review.googlesource.com/c/chromium/src/+/1102154 +- Once the upstream fix is landed, we roll updated Chromium into Puppeteer: + https://github.com/puppeteer/puppeteer/pull/2769 -However, oftentimes it is desirable to use Puppeteer with the official Google Chrome rather than Chromium. For this to work, you should install a `puppeteer-core` version that corresponds to the Chrome version. +However, oftentimes it is desirable to use Puppeteer with the official Google +Chrome rather than Chromium. For this to work, you should install a +`puppeteer-core` version that corresponds to the Chrome version. -For example, in order to drive Chrome 71 with puppeteer-core, use `chrome-71` npm tag: +For example, in order to drive Chrome 71 with puppeteer-core, use `chrome-71` +npm tag: ```bash npm install puppeteer-core@chrome-71 @@ -71,12 +116,24 @@ npm install puppeteer-core@chrome-71 Find the version using one of the following ways: -- Look for the `chromium` entry in [revisions.ts](https://github.com/puppeteer/puppeteer/blob/main/src/revisions.ts). To find the corresponding Chromium commit and version number, search for the revision prefixed by an `r` in [OmahaProxy](https://omahaproxy.appspot.com/)'s "Find Releases" section. -- Look for the `versionsPerRelease` map in [versions.js](https://github.com/puppeteer/puppeteer/blob/main/versions.js) which contains mapping between Chromium and the smallest Puppeteer version that supports it. +- Look for the `chromium` entry in + [revisions.ts](https://github.com/puppeteer/puppeteer/blob/main/src/revisions.ts). + To find the corresponding Chromium commit and version number, search for the + revision prefixed by an `r` in [OmahaProxy](https://omahaproxy.appspot.com/)'s + "Find Releases" section. +- Look for the `versionsPerRelease` map in + [versions.js](https://github.com/puppeteer/puppeteer/blob/main/versions.js) + which contains mapping between Chromium and the smallest Puppeteer version + that supports it. ## Q: Which Firefox version does Puppeteer use? -Since Firefox support is experimental, Puppeteer downloads the latest [Firefox Nightly](https://wiki.mozilla.org/Nightly) when the `PUPPETEER_PRODUCT` environment variable is set to `firefox`. That's also why the value of `firefox` in [revisions.ts](https://github.com/puppeteer/puppeteer/blob/main/src/revisions.ts) is `latest` -- Puppeteer isn't tied to a particular Firefox version. +Since Firefox support is experimental, Puppeteer downloads the latest +[Firefox Nightly](https://wiki.mozilla.org/Nightly) when the `PUPPETEER_PRODUCT` +environment variable is set to `firefox`. That's also why the value of `firefox` +in +[revisions.ts](https://github.com/puppeteer/puppeteer/blob/main/src/revisions.ts) +is `latest` -- Puppeteer isn't tied to a particular Firefox version. To fetch Firefox Nightly as part of Puppeteer installation: @@ -87,24 +144,39 @@ PUPPETEER_PRODUCT=firefox npm i puppeteer #### Q: What’s considered a “Navigation”? -From Puppeteer’s standpoint, **“navigation” is anything that changes a page’s URL**. -Aside from regular navigation where the browser hits the network to fetch a new document from the web server, this includes [anchor navigations](https://www.w3.org/TR/html5/single-page.html#scroll-to-fragid) and [History API](https://developer.mozilla.org/en-US/docs/Web/API/History_API) usage. +From Puppeteer’s standpoint, **“navigation” is anything that changes a page’s +URL**. Aside from regular navigation where the browser hits the network to fetch +a new document from the web server, this includes +[anchor navigations](https://www.w3.org/TR/html5/single-page.html#scroll-to-fragid) +and [History API](https://developer.mozilla.org/en-US/docs/Web/API/History_API) +usage. -With this definition of “navigation,” **Puppeteer works seamlessly with single-page applications.** +With this definition of “navigation,” **Puppeteer works seamlessly with +single-page applications.** #### Q: What’s the difference between a “trusted" and "untrusted" input event? -In browsers, input events could be divided into two big groups: trusted vs. untrusted. +In browsers, input events could be divided into two big groups: trusted vs. +untrusted. -- **Trusted events**: events generated by users interacting with the page, e.g. using a mouse or keyboard. -- **Untrusted event**: events generated by Web APIs, e.g. `document.createEvent` or `element.click()` methods. +- **Trusted events**: events generated by users interacting with the page, e.g. + using a mouse or keyboard. +- **Untrusted event**: events generated by Web APIs, e.g. `document.createEvent` + or `element.click()` methods. Websites can distinguish between these two groups: -- using an [`Event.isTrusted`](https://developer.mozilla.org/en-US/docs/Web/API/Event/isTrusted) event flag -- sniffing for accompanying events. For example, every trusted `'click'` event is preceded by `'mousedown'` and `'mouseup'` events. +- using an + [`Event.isTrusted`](https://developer.mozilla.org/en-US/docs/Web/API/Event/isTrusted) + event flag +- sniffing for accompanying events. For example, every trusted `'click'` event + is preceded by `'mousedown'` and `'mouseup'` events. -For automation purposes it’s important to generate trusted events. **All input events generated with Puppeteer are trusted and fire proper accompanying events.** If, for some reason, one needs an untrusted event, it’s always possible to hop into a page context with `page.evaluate` and generate a fake event: +For automation purposes it’s important to generate trusted events. **All input +events generated with Puppeteer are trusted and fire proper accompanying +events.** If, for some reason, one needs an untrusted event, it’s always +possible to hop into a page context with `page.evaluate` and generate a fake +event: ```ts await page.evaluate(() => { @@ -114,20 +186,39 @@ await page.evaluate(() => { #### Q: What features does Puppeteer not support? -You may find that Puppeteer does not behave as expected when controlling pages that incorporate audio and video. (For example, [video playback/screenshots is likely to fail](https://github.com/puppeteer/puppeteer/issues/291).) There are two reasons for this: +You may find that Puppeteer does not behave as expected when controlling pages +that incorporate audio and video. (For example, +[video playback/screenshots is likely to fail](https://github.com/puppeteer/puppeteer/issues/291).) +There are two reasons for this: -- Puppeteer is bundled with Chromium — not Chrome — and so by default, it inherits all of [Chromium's media-related limitations](https://www.chromium.org/audio-video). This means that Puppeteer does not support licensed formats such as AAC or H.264. (However, it is possible to force Puppeteer to use a separately-installed version Chrome instead of Chromium via the [`executablePath` option to `puppeteer.launch`](https://pptr.dev/api/puppeteer.launchoptions.executablepath). You should only use this configuration if you need an official release of Chrome that supports these media formats.) -- Since Puppeteer (in all configurations) controls a desktop version of Chromium/Chrome, features that are only supported by the mobile version of Chrome are not supported. This means that Puppeteer [does not support HTTP Live Streaming (HLS)](https://caniuse.com/#feat=http-live-streaming). +- Puppeteer is bundled with Chromium — not Chrome — and so by default, it + inherits all of + [Chromium's media-related limitations](https://www.chromium.org/audio-video). + This means that Puppeteer does not support licensed formats such as AAC or + H.264. (However, it is possible to force Puppeteer to use a + separately-installed version Chrome instead of Chromium via the + [`executablePath` option to `puppeteer.launch`](https://pptr.dev/api/puppeteer.launchoptions.executablepath). + You should only use this configuration if you need an official release of + Chrome that supports these media formats.) +- Since Puppeteer (in all configurations) controls a desktop version of + Chromium/Chrome, features that are only supported by the mobile version of + Chrome are not supported. This means that Puppeteer + [does not support HTTP Live Streaming (HLS)](https://caniuse.com/#feat=http-live-streaming). #### Q: I am having trouble installing / running Puppeteer in my test environment. Where should I look for help? -We have a [troubleshooting](https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md) guide for various operating systems that lists the required dependencies. +We have a +[troubleshooting](https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md) +guide for various operating systems that lists the required dependencies. #### Q: Chromium gets downloaded on every `npm ci` run. How can I cache the download? -The default download path is `node_modules/puppeteer/.local-chromium`. However, you can change that path with the `PUPPETEER_DOWNLOAD_PATH` environment variable. +The default download path is `node_modules/puppeteer/.local-chromium`. However, +you can change that path with the `PUPPETEER_DOWNLOAD_PATH` environment +variable. -Puppeteer uses that variable to resolve the Chromium executable location during launch, so you don’t need to specify `PUPPETEER_EXECUTABLE_PATH` as well. +Puppeteer uses that variable to resolve the Chromium executable location during +launch, so you don’t need to specify `PUPPETEER_EXECUTABLE_PATH` as well. For example, if you wish to keep the Chromium download in `~/.npm/chromium`: diff --git a/docs/guides/chrome-extensions.md b/docs/guides/chrome-extensions.md index 35b9bec6..0bc8d71b 100644 --- a/docs/guides/chrome-extensions.md +++ b/docs/guides/chrome-extensions.md @@ -4,11 +4,14 @@ Puppeteer can be used for testing Chrome Extensions. :::caution -Extensions in Chrome/Chromium currently only work in non-headless mode and experimental Chrome headless mode. +Extensions in Chrome/Chromium currently only work in non-headless mode and +experimental Chrome headless mode. ::: -The following is code for getting a handle to the [background page](https://developer.chrome.com/extensions/background_pages) of an extension whose source is located in `./my-extension`: +The following is code for getting a handle to the +[background page](https://developer.chrome.com/extensions/background_pages) of +an extension whose source is located in `./my-extension`: ```ts const puppeteer = require('puppeteer'); @@ -33,7 +36,8 @@ const puppeteer = require('puppeteer'); :::note -Chrome Manifest V3 extensions have a background ServiceWorker of type 'service_worker', instead of a page of type 'background_page'. +Chrome Manifest V3 extensions have a background ServiceWorker of type +'service_worker', instead of a page of type 'background_page'. ::: diff --git a/docs/guides/debugging.md b/docs/guides/debugging.md index 9378fed0..14b5940e 100644 --- a/docs/guides/debugging.md +++ b/docs/guides/debugging.md @@ -1,18 +1,36 @@ # Debugging -Debugging with Puppeteer can be an arduous task. There is no _single_ method for debugging all possible issues since Puppeteer touches many distinct components of a browser such as network requests and Web APIs. On a high note, Puppeteer provides _several_ methods for debugging which hopefully does cover all possible issues. +Debugging with Puppeteer can be an arduous task. There is no _single_ method for +debugging all possible issues since Puppeteer touches many distinct components +of a browser such as network requests and Web APIs. On a high note, Puppeteer +provides _several_ methods for debugging which hopefully does cover all possible +issues. ## Background -In general, there are two possible sources of an issue: Code running on Node.js (which we call _server code_), and [code running in the browser](<[`Page.evaluate()`](https://pptr.dev/api/puppeteer.page.evaluate)>) (which we call _client code_). There is also a third possible source being the browser itself (which we call _internal code_), but if you suspect this is the source **after attempting the methods below**, we suggest [searching existing issues](https://github.com/puppeteer/puppeteer/issues) before [filing an issue](https://github.com/puppeteer/puppeteer/issues/new/choose). +In general, there are two possible sources of an issue: Code running on Node.js +(which we call _server code_), and +[code running in the browser](<[`Page.evaluate()`](https://pptr.dev/api/puppeteer.page.evaluate)>) +(which we call _client code_). There is also a third possible source being the +browser itself (which we call _internal code_), but if you suspect this is the +source **after attempting the methods below**, we suggest +[searching existing issues](https://github.com/puppeteer/puppeteer/issues) +before +[filing an issue](https://github.com/puppeteer/puppeteer/issues/new/choose). ## Debugging methods for all situations -These methods can be used to debug any situation. These should be used as a quick sanity check before diving into more complex methods. +These methods can be used to debug any situation. These should be used as a +quick sanity check before diving into more complex methods. ### Turn off [`headless`](https://pptr.dev/api/puppeteer.browserlaunchargumentoptions.headless) -Sometimes it's useful to see what the browser is displaying. Instead of launching in [`headless`](https://pptr.dev/api/puppeteer.browserlaunchargumentoptions.headless) mode, launch a full version of the browser with [`headless`](https://pptr.dev/api/puppeteer.browserlaunchargumentoptions.headless) set to `false`: +Sometimes it's useful to see what the browser is displaying. Instead of +launching in +[`headless`](https://pptr.dev/api/puppeteer.browserlaunchargumentoptions.headless) +mode, launch a full version of the browser with +[`headless`](https://pptr.dev/api/puppeteer.browserlaunchargumentoptions.headless) +set to `false`: ```ts const browser = await puppeteer.launch({headless: false}); @@ -20,7 +38,9 @@ const browser = await puppeteer.launch({headless: false}); ### Puppeteer "slow-mo" -The [`slowMo`](https://pptr.dev/api/puppeteer.browserconnectoptions.slowmo) option slows down Puppeteer operations by a specified amount of milliseconds. It's another way to help see what's going on. +The [`slowMo`](https://pptr.dev/api/puppeteer.browserconnectoptions.slowmo) +option slows down Puppeteer operations by a specified amount of milliseconds. +It's another way to help see what's going on. ```ts const browser = await puppeteer.launch({ @@ -33,7 +53,11 @@ const browser = await puppeteer.launch({ ### Capture `console.*` output -Since client code runs in the browser, doing `console.*` in client code will not directly log to Node.js. However, you can [listen](https://pptr.dev/api/puppeteer.page.on) for the [`console`](https://pptr.dev/api/puppeteer.pageeventobject.console) event which returns a payload with the logged text. +Since client code runs in the browser, doing `console.*` in client code will not +directly log to Node.js. However, you can +[listen](https://pptr.dev/api/puppeteer.page.on) for the +[`console`](https://pptr.dev/api/puppeteer.pageeventobject.console) event which +returns a payload with the logged text. ```ts page.on('console', msg => console.log('PAGE LOG:', msg.text())); @@ -43,7 +67,9 @@ await page.evaluate(() => console.log(`url is ${location.href}`)); ### Use the debugger in the browser -1. Set [`devtools`](https://pptr.dev/api/puppeteer.browserlaunchargumentoptions.devtools) to `true` when launching Puppeteer: +1. Set + [`devtools`](https://pptr.dev/api/puppeteer.browserlaunchargumentoptions.devtools) + to `true` when launching Puppeteer: ```ts const browser = await puppeteer.launch({devtools: true}); @@ -57,18 +83,26 @@ await page.evaluate(() => console.log(`url is ${location.href}`)); }); ``` - The Browser will now stop in the location the `debugger` word is found in debug mode. + The Browser will now stop in the location the `debugger` word is found in + debug mode. ## Debugging methods for server code ### Use the debugger in Node.js (Chrome/Chromium-only) -Since server code intermingles with client code, this method of debugging is closely tied with the browser. For example, you can step over `await page.click()` in the server script and see the click happen in the browser. +Since server code intermingles with client code, this method of debugging is +closely tied with the browser. For example, you can step over +`await page.click()` in the server script and see the click happen in the +browser. -Note that you won't be able to run `await page.click()` in DevTools console due to this [Chromium bug](https://bugs.chromium.org/p/chromium/issues/detail?id=833928), so if -you want to try something out, you have to add it to your test file. +Note that you won't be able to run `await page.click()` in DevTools console due +to this +[Chromium bug](https://bugs.chromium.org/p/chromium/issues/detail?id=833928), so +if you want to try something out, you have to add it to your test file. -1. Set [`headless`](https://pptr.dev/api/puppeteer.browserlaunchargumentoptions.headless) to `false`. +1. Set + [`headless`](https://pptr.dev/api/puppeteer.browserlaunchargumentoptions.headless) + to `false`. 2. Add `debugger` to any server code you want debugged. For example, ```ts @@ -82,9 +116,11 @@ you want to try something out, you have to add it to your test file. node --inspect-brk path/to/script.js ``` -4. In the opened Chrome/Chromium browser, open `chrome://inspect/#devices` and click `inspect`. +4. In the opened Chrome/Chromium browser, open `chrome://inspect/#devices` and + click `inspect`. 5. In the newly opened test browser, press `F8` to resume test execution. -6. Now your `debugger` statement will be hit and you can debug in the test browser. +6. Now your `debugger` statement will be hit and you can debug in the test + browser. ### Use [ndb](https://github.com/GoogleChromeLabs/ndb) @@ -111,7 +147,10 @@ you want to try something out, you have to add it to your test file. ### Log DevTools protocol traffic -If all else fails, it's possible there may be an issue between Puppeteer and the DevTools protocol. You can debug this by setting the `DEBUG` environment variable before running your script. This will log internal traffic via [`debug`](https://github.com/visionmedia/debug) under the `puppeteer` namespace. +If all else fails, it's possible there may be an issue between Puppeteer and the +DevTools protocol. You can debug this by setting the `DEBUG` environment +variable before running your script. This will log internal traffic via +[`debug`](https://github.com/visionmedia/debug) under the `puppeteer` namespace. ```sh # Basic verbose logging diff --git a/docs/guides/docker.md b/docs/guides/docker.md index 446d2b94..fdce6710 100644 --- a/docs/guides/docker.md +++ b/docs/guides/docker.md @@ -1,13 +1,19 @@ # Docker -Puppeteer offers a Docker image that includes Chromium along with the required dependencies and a pre-installed Puppeteer version. The image is available via the [GitHub Container Registry](https://github.com/puppeteer/puppeteer/pkgs/container/puppeteer). The latest image is tagged as `latest` and other tags match Puppeteer versions. For example, +Puppeteer offers a Docker image that includes Chromium along with the required +dependencies and a pre-installed Puppeteer version. The image is available via +the +[GitHub Container Registry](https://github.com/puppeteer/puppeteer/pkgs/container/puppeteer). +The latest image is tagged as `latest` and other tags match Puppeteer versions. +For example, ```sh docker pull ghcr.io/puppeteer/puppeteer:latest # pulls the latest docker pull ghcr.io/puppeteer/puppeteer:16.1.0 # pulls the image that contains Puppeteer v16.1.0 ``` -The image is meant for running the browser in sandbox mode and therefore, running the image requires the `SYS_ADMIN` capability. +The image is meant for running the browser in sandbox mode and therefore, +running the image requires the `SYS_ADMIN` capability. ## Usage @@ -17,6 +23,10 @@ To use the docker image directly, run: docker run -i --init --cap-add=SYS_ADMIN --rm ghcr.io/puppeteer/puppeteer:latest node -e "$(cat path/to/script.js)" ``` -where `path/to/script.js` is the path relative to your working directory. Note the image requires the `SYS_ADMIN` capability since the browser runs in sandbox mode. +where `path/to/script.js` is the path relative to your working directory. Note +the image requires the `SYS_ADMIN` capability since the browser runs in sandbox +mode. -If you need to build an image based on a different base image, you can use our [`Dockerfile`](https://github.com/puppeteer/puppeteer/blob/main/docker/Dockerfile) as the starting point. +If you need to build an image based on a different base image, you can use our +[`Dockerfile`](https://github.com/puppeteer/puppeteer/blob/main/docker/Dockerfile) +as the starting point. diff --git a/docs/guides/index.md b/docs/guides/index.md deleted file mode 100644 index 3615e66d..00000000 --- a/docs/guides/index.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -sidebar_label: Guides -sidebar_position: 6 ---- - -- [Request interception](request-interception.md) diff --git a/docs/guides/request-interception.md b/docs/guides/request-interception.md index 1299a60d..bfdccb68 100644 --- a/docs/guides/request-interception.md +++ b/docs/guides/request-interception.md @@ -1,6 +1,7 @@ # Request interception -Once request interception is enabled, every request will stall unless it's continued, responded or aborted. +Once request interception is enabled, every request will stall unless it's +continued, responded or aborted. An example of a naïve request interceptor that aborts all image requests: @@ -27,14 +28,22 @@ const puppeteer = require('puppeteer'); ## Multiple Intercept Handlers and Asynchronous Resolutions -By default Puppeteer will raise a `Request is already handled!` exception if `request.abort`, `request.continue`, or `request.respond` are called after any of them have already been called. +By default Puppeteer will raise a `Request is already handled!` exception if +`request.abort`, `request.continue`, or `request.respond` are called after any +of them have already been called. -Always assume that an unknown handler may have already called `abort/continue/respond`. Even if your handler is the only one you registered, -3rd party packages may register their own handlers. It is therefore -important to always check the resolution status using [request.isInterceptResolutionHandled](#httprequestisinterceptresolutionhandled) +Always assume that an unknown handler may have already called +`abort/continue/respond`. Even if your handler is the only one you registered, +3rd party packages may register their own handlers. It is therefore important to +always check the resolution status using +[request.isInterceptResolutionHandled](#httprequestisinterceptresolutionhandled) before calling `abort/continue/respond`. -Importantly, the intercept resolution may get handled by another listener while your handler is awaiting an asynchronous operation. Therefore, the return value of `request.isInterceptResolutionHandled` is only safe in a synchronous code block. Always execute `request.isInterceptResolutionHandled` and `abort/continue/respond` **synchronously** together. +Importantly, the intercept resolution may get handled by another listener while +your handler is awaiting an asynchronous operation. Therefore, the return value +of `request.isInterceptResolutionHandled` is only safe in a synchronous code +block. Always execute `request.isInterceptResolutionHandled` and +`abort/continue/respond` **synchronously** together. This example demonstrates two synchronous handlers working together: @@ -94,7 +103,10 @@ page.on('request', async interceptedRequest => { }); ``` -For finer-grained introspection (see Cooperative Intercept Mode below), you may also call [request.interceptResolutionState](#httprequestinterceptresolutionstate) synchronously before using `abort/continue/respond`. +For finer-grained introspection (see Cooperative Intercept Mode below), you may +also call +[request.interceptResolutionState](#httprequestinterceptresolutionstate) +synchronously before using `abort/continue/respond`. Here is the example above rewritten using `request.interceptResolutionState` @@ -145,22 +157,40 @@ page.on('request', async interceptedRequest => { ## Cooperative Intercept Mode -`request.abort`, `request.continue`, and `request.respond` can accept an optional `priority` to work in Cooperative Intercept Mode. When all -handlers are using Cooperative Intercept Mode, Puppeteer guarantees that all intercept handlers will run and be awaited in order of registration. The interception is resolved to the highest-priority resolution. Here are the rules of Cooperative Intercept Mode: +`request.abort`, `request.continue`, and `request.respond` can accept an +optional `priority` to work in Cooperative Intercept Mode. When all handlers are +using Cooperative Intercept Mode, Puppeteer guarantees that all intercept +handlers will run and be awaited in order of registration. The interception is +resolved to the highest-priority resolution. Here are the rules of Cooperative +Intercept Mode: -- All resolutions must supply a numeric `priority` argument to `abort/continue/respond`. -- If any resolution does not supply a numeric `priority`, Legacy Mode is active and Cooperative Intercept Mode is inactive. +- All resolutions must supply a numeric `priority` argument to + `abort/continue/respond`. +- If any resolution does not supply a numeric `priority`, Legacy Mode is active + and Cooperative Intercept Mode is inactive. - Async handlers finish before intercept resolution is finalized. -- The highest priority interception resolution "wins", i.e. the interception is ultimately aborted/responded/continued according to which resolution was given the highest priority. +- The highest priority interception resolution "wins", i.e. the interception is + ultimately aborted/responded/continued according to which resolution was given + the highest priority. - In the event of a tie, `abort` > `respond` > `continue`. -For standardization, when specifying a Cooperative Intercept Mode priority use `0` or `DEFAULT_INTERCEPT_RESOLUTION_PRIORITY` (exported from `HTTPRequest`) unless you have a clear reason to use a higher priority. This gracefully prefers `respond` over `continue` and `abort` over `respond` and allows other handlers to work cooperatively. If you do intentionally want to use a different priority, higher priorities win over lower priorities. Negative priorities are allowed. For example, `continue({}, 4)` would win over `continue({}, -2)`. +For standardization, when specifying a Cooperative Intercept Mode priority use +`0` or `DEFAULT_INTERCEPT_RESOLUTION_PRIORITY` (exported from `HTTPRequest`) +unless you have a clear reason to use a higher priority. This gracefully prefers +`respond` over `continue` and `abort` over `respond` and allows other handlers +to work cooperatively. If you do intentionally want to use a different priority, +higher priorities win over lower priorities. Negative priorities are allowed. +For example, `continue({}, 4)` would win over `continue({}, -2)`. -To preserve backward compatibility, any handler resolving the intercept without specifying `priority` (Legacy Mode) causes immediate resolution. For Cooperative Intercept Mode to work, all resolutions must use a `priority`. In practice, this means you must still test for -`request.isInterceptResolutionHandled` because a handler beyond your control may have called `abort/continue/respond` without a +To preserve backward compatibility, any handler resolving the intercept without +specifying `priority` (Legacy Mode) causes immediate resolution. For Cooperative +Intercept Mode to work, all resolutions must use a `priority`. In practice, this +means you must still test for `request.isInterceptResolutionHandled` because a +handler beyond your control may have called `abort/continue/respond` without a priority (Legacy Mode). -In this example, Legacy Mode prevails and the request is aborted immediately because at least one handler omits `priority` when resolving the intercept: +In this example, Legacy Mode prevails and the request is aborted immediately +because at least one handler omits `priority` when resolving the intercept: ```ts // Final outcome: immediate abort() @@ -180,7 +210,8 @@ page.on('request', request => { }); ``` -In this example, Legacy Mode prevails and the request is continued because at least one handler does not specify a `priority`: +In this example, Legacy Mode prevails and the request is continued because at +least one handler does not specify a `priority`: ```ts // Final outcome: immediate continue() @@ -208,7 +239,9 @@ page.on('request', request => { }); ``` -In this example, Cooperative Intercept Mode is active because all handlers specify a `priority`. `continue()` wins because it has a higher priority than `abort()`. +In this example, Cooperative Intercept Mode is active because all handlers +specify a `priority`. `continue()` wins because it has a higher priority than +`abort()`. ```ts // Final outcome: cooperative continue() @ 5 @@ -231,7 +264,9 @@ page.on('request', request => { }); ``` -In this example, Cooperative Intercept Mode is active because all handlers specify `priority`. `respond()` wins because its priority ties with `continue()`, but `respond()` beats `continue()`. +In this example, Cooperative Intercept Mode is active because all handlers +specify `priority`. `respond()` wins because its priority ties with +`continue()`, but `respond()` beats `continue()`. ```ts // Final outcome: cooperative respond() @ 15 @@ -268,23 +303,41 @@ page.on('request', request => { ## Cooperative Request Continuation -Puppeteer requires `request.continue()` to be called explicitly or the request will hang. Even if -your handler means to take no special action, or 'opt out', `request.continue()` must still be called. +Puppeteer requires `request.continue()` to be called explicitly or the request +will hang. Even if your handler means to take no special action, or 'opt out', +`request.continue()` must still be called. -With the introduction of Cooperative Intercept Mode, two use cases arise for cooperative request continuations: -Unopinionated and Opinionated. +With the introduction of Cooperative Intercept Mode, two use cases arise for +cooperative request continuations: Unopinionated and Opinionated. -The first case (common) is that your handler means to opt out of doing anything special the request. It has no opinion on further action and simply intends to continue by default and/or defer to other handlers that might have an opinion. But in case there are no other handlers, we must call `request.continue()` to ensure that the request doesn't hang. +The first case (common) is that your handler means to opt out of doing anything +special the request. It has no opinion on further action and simply intends to +continue by default and/or defer to other handlers that might have an opinion. +But in case there are no other handlers, we must call `request.continue()` to +ensure that the request doesn't hang. -We call this an **Unopinionated continuation** because the intent is to continue the request if nobody else has a better idea. Use `request.continue({...}, DEFAULT_INTERCEPT_RESOLUTION_PRIORITY)` (or `0`) for this type of continuation. +We call this an **Unopinionated continuation** because the intent is to continue +the request if nobody else has a better idea. Use +`request.continue({...}, DEFAULT_INTERCEPT_RESOLUTION_PRIORITY)` (or `0`) for +this type of continuation. -The second case (uncommon) is that your handler actually does have an opinion and means to force continuation by overriding a lower-priority `abort()` or `respond()` issued elsewhere. We call this an **Opinionated continuation**. In these rare cases where you mean to specify an overriding continuation priority, use a custom priority. +The second case (uncommon) is that your handler actually does have an opinion +and means to force continuation by overriding a lower-priority `abort()` or +`respond()` issued elsewhere. We call this an **Opinionated continuation**. In +these rare cases where you mean to specify an overriding continuation priority, +use a custom priority. -To summarize, reason through whether your use of `request.continue` is just meant to be default/bypass behavior vs falling within the intended use case of your handler. Consider using a custom priority for in-scope use cases, and a default priority otherwise. Be aware that your handler may have both Opinionated and Unopinionated cases. +To summarize, reason through whether your use of `request.continue` is just +meant to be default/bypass behavior vs falling within the intended use case of +your handler. Consider using a custom priority for in-scope use cases, and a +default priority otherwise. Be aware that your handler may have both Opinionated +and Unopinionated cases. ## Upgrading to Cooperative Intercept Mode for package maintainers -If you are package maintainer and your package uses intercept handlers, you can update your intercept handlers to use Cooperative Intercept Mode. Suppose you have the following existing handler: +If you are package maintainer and your package uses intercept handlers, you can +update your intercept handlers to use Cooperative Intercept Mode. Suppose you +have the following existing handler: ```ts page.on('request', interceptedRequest => { @@ -316,14 +369,30 @@ page.on('request', interceptedRequest => { }); ``` -With those simple upgrades, your handler now uses Cooperative Intercept Mode instead. +With those simple upgrades, your handler now uses Cooperative Intercept Mode +instead. -However, we recommend a slightly more robust solution because the above introduces several subtle issues: +However, we recommend a slightly more robust solution because the above +introduces several subtle issues: -1. **Backward compatibility.** If any handler still uses a Legacy Mode resolution (ie, does not specify a priority), that handler will resolve the interception immediately even if your handler runs first. This could cause disconcerting behavior for your users because suddenly your handler is not resolving the interception and a different handler is taking priority when all the user did was upgrade your package. -2. **Hard-coded priority.** Your package user has no ability to specify the default resolution priority for your handlers. This can become important when the user wishes to manipulate the priorities based on use case. For example, one user might want your package to take a high priority while another user might want it to take a low priority. +1. **Backward compatibility.** If any handler still uses a Legacy Mode + resolution (ie, does not specify a priority), that handler will resolve the + interception immediately even if your handler runs first. This could cause + disconcerting behavior for your users because suddenly your handler is not + resolving the interception and a different handler is taking priority when + all the user did was upgrade your package. +2. **Hard-coded priority.** Your package user has no ability to specify the + default resolution priority for your handlers. This can become important when + the user wishes to manipulate the priorities based on use case. For example, + one user might want your package to take a high priority while another user + might want it to take a low priority. -To resolve both of these issues, our recommended approach is to export a `setInterceptResolutionConfig()` from your package. The user can then call `setInterceptResolutionConfig()` to explicitly activate Cooperative Intercept Mode in your package so they aren't surprised by changes in how the interception is resolved. They can also optionally specify a custom priority using `setInterceptResolutionConfig(priority)` that works for their use case: +To resolve both of these issues, our recommended approach is to export a +`setInterceptResolutionConfig()` from your package. The user can then call +`setInterceptResolutionConfig()` to explicitly activate Cooperative Intercept +Mode in your package so they aren't surprised by changes in how the interception +is resolved. They can also optionally specify a custom priority using +`setInterceptResolutionConfig(priority)` that works for their use case: ```ts // Defaults to undefined which preserves Legacy Mode behavior @@ -352,7 +421,8 @@ page.on('request', interceptedRequest => { }); ``` -If your package calls for more fine-grained control over resolution priorities, use a config pattern like this: +If your package calls for more fine-grained control over resolution priorities, +use a config pattern like this: ```ts interface InterceptResolutionConfig { @@ -395,4 +465,9 @@ page.on('request', interceptedRequest => { }); ``` -The above solutions ensure backward compatibility while also allowing the user to adjust the importance of your package in the resolution chain when Cooperative Intercept Mode is being used. Your package continues to work as expected until the user has fully upgraded their code and all third party packages to use Cooperative Intercept Mode. If any handler or package still uses Legacy Mode, your package can still operate in Legacy Mode too. +The above solutions ensure backward compatibility while also allowing the user +to adjust the importance of your package in the resolution chain when +Cooperative Intercept Mode is being used. Your package continues to work as +expected until the user has fully upgraded their code and all third party +packages to use Cooperative Intercept Mode. If any handler or package still uses +Legacy Mode, your package can still operate in Legacy Mode too. diff --git a/docs/index.md b/docs/index.md index ba221d72..3f1d4a00 100644 --- a/docs/index.md +++ b/docs/index.md @@ -4,23 +4,35 @@ sidebar_position: 1 # Puppeteer -[![Build status](https://github.com/puppeteer/puppeteer/workflows/CI/badge.svg)](https://github.com/puppeteer/puppeteer/actions?query=workflow%3ACI) [![npm puppeteer package](https://img.shields.io/npm/v/puppeteer.svg)](https://npmjs.org/package/puppeteer) +[![Build status](https://github.com/puppeteer/puppeteer/workflows/CI/badge.svg)](https://github.com/puppeteer/puppeteer/actions?query=workflow%3ACI) +[![npm puppeteer package](https://img.shields.io/npm/v/puppeteer.svg)](https://npmjs.org/package/puppeteer) #### [API](https://pptr.dev/api) | [FAQ](https://pptr.dev/faq) | [Contributing](https://pptr.dev/contributing) | [Troubleshooting](https://pptr.dev/troubleshooting) -> Puppeteer is a Node.js library which provides a high-level API to control Chrome/Chromium over the [DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/). Puppeteer runs in [headless](https://developers.google.com/web/updates/2017/04/headless-chrome) mode by default, but can be configured to run in full (non-headless) Chrome/Chromium. +> Puppeteer is a Node.js library which provides a high-level API to control +> Chrome/Chromium over the +> [DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/). +> Puppeteer runs in +> [headless](https://developers.google.com/web/updates/2017/04/headless-chrome) +> mode by default, but can be configured to run in full (non-headless) +> Chrome/Chromium. #### What can I do? -Most things that you can do manually in the browser can be done using Puppeteer! Here are a few examples to get you started: +Most things that you can do manually in the browser can be done using Puppeteer! +Here are a few examples to get you started: - Generate screenshots and PDFs of pages. -- Crawl a SPA (Single-Page Application) and generate pre-rendered content (i.e. "SSR" (Server-Side Rendering)). +- Crawl a SPA (Single-Page Application) and generate pre-rendered content (i.e. + "SSR" (Server-Side Rendering)). - Automate form submission, UI testing, keyboard input, etc. -- Create an automated testing environment using the latest JavaScript and browser features. -- Capture a [timeline trace](https://developers.google.com/web/tools/chrome-devtools/evaluate-performance/reference) of your site to help diagnose performance issues. +- Create an automated testing environment using the latest JavaScript and + browser features. +- Capture a + [timeline trace](https://developers.google.com/web/tools/chrome-devtools/evaluate-performance/reference) + of your site to help diagnose performance issues. - Test Chrome Extensions. ## Getting Started @@ -35,25 +47,57 @@ npm i puppeteer # or `pnpm i puppeteer` ``` -When you install Puppeteer, it automatically downloads a recent version of Chromium (~170MB macOS, ~282MB Linux, ~280MB Windows) that is [guaranteed to work](https://pptr.dev/faq#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) with Puppeteer. For a version of Puppeteer without installation, see [`puppeteer-core`](#puppeteer-core). +When you install Puppeteer, it automatically downloads a recent version of +Chromium (~170MB macOS, ~282MB Linux, ~280MB Windows) that is +[guaranteed to work](https://pptr.dev/faq#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) +with Puppeteer. For a version of Puppeteer without installation, see +[`puppeteer-core`](#puppeteer-core). #### Environment Variables -Puppeteer looks for certain [environment variables](https://en.wikipedia.org/wiki/Environment_variable) for customizing behavior. -If Puppeteer doesn't find them in the environment during the installation step, a lowercased variant of these variables will be used from the [npm config](https://docs.npmjs.com/cli/config). +Puppeteer looks for certain +[environment variables](https://en.wikipedia.org/wiki/Environment_variable) for +customizing behavior. If Puppeteer doesn't find them in the environment during +the installation step, a lowercased variant of these variables will be used from +the [npm config](https://docs.npmjs.com/cli/config). -- `HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY` - defines HTTP proxy settings that are used to download and run the browser. -- `PUPPETEER_CACHE_DIR` - defines the directory to be used by Puppeteer for caching. Defaults to [`os.homedir()/.cache/puppeteer`](https://nodejs.org/api/os.html#os_os_homedir). -- `PUPPETEER_SKIP_CHROMIUM_DOWNLOAD` - do not download bundled Chromium during installation step. -- `PUPPETEER_TMP_DIR` - defines the directory to be used by Puppeteer for creating temporary files. Defaults to [`os.tmpdir()`](https://nodejs.org/api/os.html#os_os_tmpdir). -- `PUPPETEER_DOWNLOAD_HOST` - specifies the URL prefix that is used to download Chromium. Note: this includes protocol and might even include path prefix. Defaults to `https://storage.googleapis.com`. -- `PUPPETEER_DOWNLOAD_PATH` - specifies the path for the downloads folder. Defaults to `/chromium`, where `` is Puppeteer's cache directory. -- `PUPPETEER_BROWSER_REVISION` - specifies a certain version of the browser you'd like Puppeteer to use. See [`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) on how executable path is inferred. -- `PUPPETEER_EXECUTABLE_PATH` - specifies an executable path to be used in [`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch). -- `PUPPETEER_PRODUCT` - specifies which browser you'd like Puppeteer to use. Must be either `chrome` or `firefox`. This can also be used during installation to fetch the recommended browser binary. Setting `product` programmatically in [`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) supersedes this environment variable. -- `PUPPETEER_EXPERIMENTAL_CHROMIUM_MAC_ARM` — specify Puppeteer download Chromium for Apple M1. On Apple M1 devices Puppeteer by default downloads the version for Intel's processor which runs via Rosetta. It works without any problems, however, with this option, you should get more efficient resource usage (CPU and RAM) that could lead to a faster execution time. +- `HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY` - defines HTTP proxy settings that are + used to download and run the browser. +- `PUPPETEER_CACHE_DIR` - defines the directory to be used by Puppeteer for + caching. Defaults to + [`os.homedir()/.cache/puppeteer`](https://nodejs.org/api/os.html#os_os_homedir). +- `PUPPETEER_SKIP_CHROMIUM_DOWNLOAD` - do not download bundled Chromium during + installation step. +- `PUPPETEER_TMP_DIR` - defines the directory to be used by Puppeteer for + creating temporary files. Defaults to + [`os.tmpdir()`](https://nodejs.org/api/os.html#os_os_tmpdir). +- `PUPPETEER_DOWNLOAD_HOST` - specifies the URL prefix that is used to download + Chromium. Note: this includes protocol and might even include path prefix. + Defaults to `https://storage.googleapis.com`. +- `PUPPETEER_DOWNLOAD_PATH` - specifies the path for the downloads folder. + Defaults to `/chromium`, where `` is Puppeteer's cache + directory. +- `PUPPETEER_BROWSER_REVISION` - specifies a certain version of the browser + you'd like Puppeteer to use. See + [`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) on + how executable path is inferred. +- `PUPPETEER_EXECUTABLE_PATH` - specifies an executable path to be used in + [`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch). +- `PUPPETEER_PRODUCT` - specifies which browser you'd like Puppeteer to use. + Must be either `chrome` or `firefox`. This can also be used during + installation to fetch the recommended browser binary. Setting `product` + programmatically in + [`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) + supersedes this environment variable. +- `PUPPETEER_EXPERIMENTAL_CHROMIUM_MAC_ARM` — specify Puppeteer download + Chromium for Apple M1. On Apple M1 devices Puppeteer by default downloads the + version for Intel's processor which runs via Rosetta. It works without any + problems, however, with this option, you should get more efficient resource + usage (CPU and RAM) that could lead to a faster execution time. -Environment variables except for `PUPPETEER_CACHE_DIR` are not used for [`puppeteer-core`](#puppeteer-core) since core does not automatically handle browser downloading. +Environment variables except for `PUPPETEER_CACHE_DIR` are not used for +[`puppeteer-core`](#puppeteer-core) since core does not automatically handle +browser downloading. #### `puppeteer-core` @@ -62,12 +106,23 @@ Every release since v1.7.0 we publish two packages: - [`puppeteer`](https://www.npmjs.com/package/puppeteer) - [`puppeteer-core`](https://www.npmjs.com/package/puppeteer-core) -`puppeteer` is a _product_ for browser automation. When installed, it downloads a version of -Chromium, which it then drives using `puppeteer-core`. Being an end-user product, `puppeteer` supports a bunch of convenient `PUPPETEER_*` env variables to tweak its behavior. +`puppeteer` is a _product_ for browser automation. When installed, it downloads +a version of Chromium, which it then drives using `puppeteer-core`. Being an +end-user product, `puppeteer` supports a bunch of convenient `PUPPETEER_*` env +variables to tweak its behavior. -`puppeteer-core` is a _library_ to help drive anything that supports DevTools protocol. `puppeteer-core` doesn't download Chromium when installed. Being a library, `puppeteer-core` is fully driven through its programmatic interface. +`puppeteer-core` is a _library_ to help drive anything that supports DevTools +protocol. `puppeteer-core` doesn't download Chromium when installed. Being a +library, `puppeteer-core` is fully driven through its programmatic interface. -You should only use `puppeteer-core` if you are [connecting to a remote browser](https://pptr.dev/api/puppeteer.puppeteer.connect) or [managing browsers yourself](https://pptr.dev/api/puppeteer.browserfetcher). If you are managing browsers yourself, you will need to call [`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) with an explicit [`executablePath`](https://pptr.dev/api/puppeteer.launchoptions.executablepath) or [`channel`](https://pptr.dev/api/puppeteer.launchoptions.channel). +You should only use `puppeteer-core` if you are +[connecting to a remote browser](https://pptr.dev/api/puppeteer.puppeteer.connect) +or [managing browsers yourself](https://pptr.dev/api/puppeteer.browserfetcher). +If you are managing browsers yourself, you will need to call +[`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) with +an explicit +[`executablePath`](https://pptr.dev/api/puppeteer.launchoptions.executablepath) +or [`channel`](https://pptr.dev/api/puppeteer.launchoptions.channel). When using `puppeteer-core`, remember to change the import: @@ -77,15 +132,25 @@ import puppeteer from 'puppeteer-core'; ### Usage -Puppeteer follows the latest [maintenance LTS](https://github.com/nodejs/Release#release-schedule) version of Node. +Puppeteer follows the latest +[maintenance LTS](https://github.com/nodejs/Release#release-schedule) version of +Node. -Puppeteer will be familiar to people using other browser testing frameworks. You [launch](https://pptr.dev/api/puppeteer.puppeteernode.launch)/[connect](https://pptr.dev/api/puppeteer.puppeteernode.connect) a [browser](https://pptr.dev/api/puppeteer.browser), [create](https://pptr.dev/api/puppeteer.browser.newpage) some [pages](https://pptr.dev/api/puppeteer.page), and then manipulate them with [Puppeteer's API](https://pptr.dev/api). +Puppeteer will be familiar to people using other browser testing frameworks. You +[launch](https://pptr.dev/api/puppeteer.puppeteernode.launch)/[connect](https://pptr.dev/api/puppeteer.puppeteernode.connect) +a [browser](https://pptr.dev/api/puppeteer.browser), +[create](https://pptr.dev/api/puppeteer.browser.newpage) some +[pages](https://pptr.dev/api/puppeteer.page), and then manipulate them with +[Puppeteer's API](https://pptr.dev/api). -For more in-depth usage, check our [guides](https://pptr.dev/guides) and [examples](https://github.com/puppeteer/puppeteer/tree/main/examples). +For more in-depth usage, check our [guides](https://pptr.dev/guides) and +[examples](https://github.com/puppeteer/puppeteer/tree/main/examples). #### Example -The following example searches [developers.google.com/web](https://developers.google.com/web) for articles tagged "Headless Chrome" and scrape results from the results page. +The following example searches +[developers.google.com/web](https://developers.google.com/web) for articles +tagged "Headless Chrome" and scrape results from the results page. ```ts import puppeteer from 'puppeteer'; @@ -127,7 +192,11 @@ import puppeteer from 'puppeteer'; **1. Uses Headless mode** -Puppeteer launches Chromium in [headless mode](https://developers.google.com/web/updates/2017/04/headless-chrome). To launch a full version of Chromium, set the [`headless`](https://pptr.dev/api/puppeteer.browserlaunchargumentoptions.headless) option when launching a browser: +Puppeteer launches Chromium in +[headless mode](https://developers.google.com/web/updates/2017/04/headless-chrome). +To launch a full version of Chromium, set the +[`headless`](https://pptr.dev/api/puppeteer.browserlaunchargumentoptions.headless) +option when launching a browser: ```ts const browser = await puppeteer.launch({headless: false}); // default is true @@ -135,21 +204,29 @@ const browser = await puppeteer.launch({headless: false}); // default is true **2. Runs a bundled version of Chromium** -By default, Puppeteer downloads and uses a specific version of Chromium so its API -is guaranteed to work out of the box. To use Puppeteer with a different version of Chrome or Chromium, -pass in the executable's path when creating a `Browser` instance: +By default, Puppeteer downloads and uses a specific version of Chromium so its +API is guaranteed to work out of the box. To use Puppeteer with a different +version of Chrome or Chromium, pass in the executable's path when creating a +`Browser` instance: ```ts const browser = await puppeteer.launch({executablePath: '/path/to/Chrome'}); ``` -You can also use Puppeteer with Firefox Nightly (experimental support). See [`Puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) for more information. +You can also use Puppeteer with Firefox Nightly (experimental support). See +[`Puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) for +more information. -See [`this article`](https://www.howtogeek.com/202825/what%E2%80%99s-the-difference-between-chromium-and-chrome/) for a description of the differences between Chromium and Chrome. [`This article`](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/chromium_browser_vs_google_chrome.md) describes some differences for Linux users. +See +[`this article`](https://www.howtogeek.com/202825/what%E2%80%99s-the-difference-between-chromium-and-chrome/) +for a description of the differences between Chromium and Chrome. +[`This article`](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/chromium_browser_vs_google_chrome.md) +describes some differences for Linux users. **3. Creates a fresh user profile** -Puppeteer creates its own browser user profile which it **cleans up on every run**. +Puppeteer creates its own browser user profile which it **cleans up on every +run**. #### Using Docker @@ -157,7 +234,8 @@ See our [guide on using Docker](https://pptr.dev/guides/docker). #### Using Chrome Extensions -See our [guide on using Chrome extensions](https://pptr.dev/guides/chrome-extensions). +See our +[guide on using Chrome extensions](https://pptr.dev/guides/chrome-extensions). ## Resources @@ -168,8 +246,10 @@ See our [guide on using Chrome extensions](https://pptr.dev/guides/chrome-extens ## Contributing -Check out our [contributing guide](https://pptr.dev/contributing) to get an overview of Puppeteer development. +Check out our [contributing guide](https://pptr.dev/contributing) to get an +overview of Puppeteer development. ## FAQ -Our [FAQ](https://pptr.dev/faq) has migrated to [our site](https://pptr.dev/faq). +Our [FAQ](https://pptr.dev/faq) has migrated to +[our site](https://pptr.dev/faq). diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 94106d3a..98029522 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -4,12 +4,39 @@ sidebar_position: 3 # Troubleshooting +## `Cannot find module 'puppeteer-core/internal/...'` + +This can occur is your Node.js version is lower than 14 or you are using a +custom resolver (such as +[`jest-resolve`](https://www.npmjs.com/package/jest-resolve)). For the former, +we do not support deprecated versions of Node.js. For the latter, usually +upgrading the resolver (or its parent module such as `jest`) will work (e.g. +https://github.com/puppeteer/puppeteer/issues/9121) + +## `Could not find expected browser locally` + +Starting from v19.0.0, Puppeteer will download browsers into +`~/.cache/puppeteer` using +[`os.homedir`](https://nodejs.org/api/os.html#oshomedir) for better caching +between Puppeteer upgrades. Generally the home directory is well-defined (even +on Windows), but occasionally the home directory may not be available. In this +case, we provide the `PUPPETEER_CACHE_DIR` variable which allows you to change +the installation directory. + +For example, + +```sh +PUPPETEER_CACHE_DIR=$(pwd) npm install puppeteer +PUPPETEER_CACHE_DIR=$(pwd) node +``` + ## Chrome headless doesn't launch on Windows -Some [chrome policies](https://support.google.com/chrome/a/answer/7532015) might enforce running Chrome/Chromium -with certain extensions. +Some [chrome policies](https://support.google.com/chrome/a/answer/7532015) might +enforce running Chrome/Chromium with certain extensions. -Puppeteer passes `--disable-extensions` flag by default and will fail to launch when such policies are active. +Puppeteer passes `--disable-extensions` flag by default and will fail to launch +when such policies are active. To work around this, try running without the flag: @@ -19,12 +46,14 @@ const browser = await puppeteer.launch({ }); ``` -> Context: [issue 3681](https://github.com/puppeteer/puppeteer/issues/3681#issuecomment-447865342). +> Context: +> [issue 3681](https://github.com/puppeteer/puppeteer/issues/3681#issuecomment-447865342). ## Chrome headless doesn't launch on UNIX -Make sure all the necessary dependencies are installed. You can run `ldd chrome | grep not` on a Linux -machine to check which dependencies are missing. The common ones are provided below. +Make sure all the necessary dependencies are installed. You can run +`ldd chrome | grep not` on a Linux machine to check which dependencies are +missing. The common ones are provided below.
Debian (e.g. Ubuntu) Dependencies @@ -97,7 +126,8 @@ xorg-x11-fonts-Type1 xorg-x11-utils ``` -After installing dependencies you need to update nss library using this command +After installing dependencies you need to update `nss` library using this +command ``` yum update nss -y @@ -108,15 +138,19 @@ yum update nss -y
Check out discussions -- [#290](https://github.com/puppeteer/puppeteer/issues/290) - Debian troubleshooting
-- [#391](https://github.com/puppeteer/puppeteer/issues/391) - CentOS troubleshooting
-- [#379](https://github.com/puppeteer/puppeteer/issues/379) - Alpine troubleshooting
+- [#290](https://github.com/puppeteer/puppeteer/issues/290) - Debian + troubleshooting
+- [#391](https://github.com/puppeteer/puppeteer/issues/391) - CentOS + troubleshooting
+- [#379](https://github.com/puppeteer/puppeteer/issues/379) - Alpine + troubleshooting
## Chrome headless disables GPU compositing -Chrome/Chromium requires `--use-gl=egl` to [enable GPU acceleration in headless mode](https://github.com/chromium/chromium/commit/19671359ae25aa1e30bde90f8ff92453eeaac2ba). +Chrome/Chromium requires `--use-gl=egl` to +[enable GPU acceleration in headless mode](https://github.com/chromium/chromium/commit/19671359ae25aa1e30bde90f8ff92453eeaac2ba). ```ts const browser = await puppeteer.launch({ @@ -134,16 +168,22 @@ If you get an error that looks like this when trying to launch Chromium: spawn /Users/.../node_modules/puppeteer/.local-chromium/mac-756035/chrome-mac/Chromium.app/Contents/MacOS/Chromium ENOENT ``` -This means that the browser was downloaded but failed to be extracted correctly. The most common cause is a bug in Node.js v14.0.0 which broke `extract-zip`, the module Puppeteer uses to extract browser downloads into the right place. The bug was fixed in Node.js v14.1.0, so please make sure you're running that version or higher. +This means that the browser was downloaded but failed to be extracted correctly. +The most common cause is a bug in Node.js v14.0.0 which broke `extract-zip`, the +module Puppeteer uses to extract browser downloads into the right place. The bug +was fixed in Node.js v14.1.0, so please make sure you're running that version or +higher. ## Setting Up Chrome Linux Sandbox -In order to protect the host environment from untrusted web content, Chrome uses [multiple layers of sandboxing](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/linux/sandboxing.md). For this to work properly, -the host should be configured first. If there's no good sandbox for Chrome to use, it will crash -with the error `No usable sandbox!`. +In order to protect the host environment from untrusted web content, Chrome uses +[multiple layers of sandboxing](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/linux/sandboxing.md). +For this to work properly, the host should be configured first. If there's no +good sandbox for Chrome to use, it will crash with the error +`No usable sandbox!`. -If you **absolutely trust** the content you open in Chrome, you can launch Chrome -with the `--no-sandbox` argument: +If you **absolutely trust** the content you open in Chrome, you can launch +Chrome with the `--no-sandbox` argument: ```ts const browser = await puppeteer.launch({ @@ -153,7 +193,8 @@ const browser = await puppeteer.launch({ :::caution -Running without a sandbox is **strongly discouraged**. Consider configuring a sandbox instead. +Running without a sandbox is **strongly discouraged**. Consider configuring a +sandbox instead. ::: @@ -161,9 +202,10 @@ There are 2 ways to configure a sandbox in Chromium. ### [recommended] Enable [user namespace cloning](http://man7.org/linux/man-pages/man7/user_namespaces.7.html) -User namespace cloning is only supported by modern kernels. Unprivileged user namespaces are generally fine to enable, -but in some cases they open up more kernel attack surface for (unsandboxed) non-root processes to elevate to -kernel privileges. +User namespace cloning is only supported by modern kernels. Unprivileged user +namespaces are generally fine to enable, but in some cases they open up more +kernel attack surface for (unsandboxed) non-root processes to elevate to kernel +privileges. ```bash sudo sysctl -w kernel.unprivileged_userns_clone=1 @@ -171,9 +213,10 @@ sudo sysctl -w kernel.unprivileged_userns_clone=1 ### [alternative] Setup [setuid sandbox](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/linux/suid_sandbox_development.md) -The setuid sandbox comes as a standalone executable and is located next to the Chromium that Puppeteer downloads. It is -fine to re-use the same sandbox executable for different Chromium versions, so the following could be -done only once per host environment: +The setuid sandbox comes as a standalone executable and is located next to the +Chromium that Puppeteer downloads. It is fine to re-use the same sandbox +executable for different Chromium versions, so the following could be done only +once per host environment: ```bash # cd to the downloaded instance @@ -186,8 +229,8 @@ sudo cp -p chrome_sandbox /usr/local/sbin/chrome-devel-sandbox export CHROME_DEVEL_SANDBOX=/usr/local/sbin/chrome-devel-sandbox ``` -You might want to export the `CHROME_DEVEL_SANDBOX` env variable by default. In this case, add the following to the `~/.bashrc` -or `.zshenv`: +You might want to export the `CHROME_DEVEL_SANDBOX` env variable by default. In +this case, add the following to the `~/.bashrc` or `.zshenv`: ```bash export CHROME_DEVEL_SANDBOX=/usr/local/sbin/chrome-devel-sandbox @@ -195,11 +238,15 @@ export CHROME_DEVEL_SANDBOX=/usr/local/sbin/chrome-devel-sandbox ## Running Puppeteer on Travis CI -> 👋 We ran our tests for Puppeteer on Travis CI until v6.0.0 (when we've migrated to GitHub Actions) - see our historical [`.travis.yml` (v5.5.0)](https://github.com/puppeteer/puppeteer/blob/v5.5.0/.travis.yml) for reference. +> 👋 We ran our tests for Puppeteer on Travis CI until v6.0.0 (when we've +> migrated to GitHub Actions) - see our historical +> [`.travis.yml` (v5.5.0)](https://github.com/puppeteer/puppeteer/blob/v5.5.0/.travis.yml) +> for reference. Tips-n-tricks: -- [xvfb](https://en.wikipedia.org/wiki/Xvfb) service should be launched in order to run Chromium in non-headless mode +- [xvfb](https://en.wikipedia.org/wiki/Xvfb) service should be launched in order + to run Chromium in non-headless mode - Runs on Xenial Linux on Travis by default - Runs `npm install` by default - `node_modules` is cached by default @@ -218,17 +265,17 @@ script: Running Puppeteer smoothly on CircleCI requires the following steps: -1. Start with a [NodeJS - image](https://circleci.com/docs/2.0/circleci-images/#nodejs) in your config - like so: +1. Start with a + [NodeJS image](https://circleci.com/docs/2.0/circleci-images/#nodejs) in your + config like so: ```yaml docker: - image: circleci/node:14 # Use your desired version environment: NODE_ENV: development # Only needed if puppeteer is in `devDependencies` ``` -1. Dependencies like `libXtst6` probably need to be installed via `apt-get`, - so use the +1. Dependencies like `libXtst6` probably need to be installed via `apt-get`, so + use the [threetreeslight/puppeteer](https://circleci.com/orbs/registry/orb/threetreeslight/puppeteer) orb ([instructions](https://circleci.com/orbs/registry/orb/threetreeslight/puppeteer#quick-start)), @@ -243,20 +290,27 @@ Running Puppeteer smoothly on CircleCI requires the following steps: at ChildProcess.spawn (internal/child_process.js:394:11) ``` This is likely caused by Jest autodetecting the number of processes on the - entire machine (`36`) rather than the number allowed to your container - (`2`). To fix this, set `jest --maxWorkers=2` in your test command. + entire machine (`36`) rather than the number allowed to your container (`2`). + To fix this, set `jest --maxWorkers=2` in your test command. ## Running Puppeteer in Docker -> 👋 We used [Cirrus Ci](https://cirrus-ci.org/) to run our tests for Puppeteer in a Docker container until v3.0.x - see our historical [`Dockerfile.linux` (v3.0.1)](https://github.com/puppeteer/puppeteer/blob/v3.0.1/.ci/node12/Dockerfile.linux) for reference. -> Starting from v16.0.0 we are shipping a Docker image via the GitHub registry. The Dockerfile is located [here](https://github.com/puppeteer/puppeteer/blob/main/docker/Dockerfile) and the usage instructions are in the [README.md](https://github.com/puppeteer/puppeteer#running-in-docker). The instructions below might be still helpful if you are building your own image. +> 👋 We used [Cirrus Ci](https://cirrus-ci.org/) to run our tests for Puppeteer +> in a Docker container until v3.0.x - see our historical +> [`Dockerfile.linux` (v3.0.1)](https://github.com/puppeteer/puppeteer/blob/v3.0.1/.ci/node12/Dockerfile.linux) +> for reference. Starting from v16.0.0 we are shipping a Docker image via the +> GitHub registry. The Dockerfile is located +> [here](https://github.com/puppeteer/puppeteer/blob/main/docker/Dockerfile) and +> the usage instructions are in the +> [README.md](https://github.com/puppeteer/puppeteer#running-in-docker). The +> instructions below might be still helpful if you are building your own image. -Getting headless Chrome up and running in Docker can be tricky. -The bundled Chromium that Puppeteer installs is missing the necessary -shared library dependencies. +Getting headless Chrome up and running in Docker can be tricky. The bundled +Chromium that Puppeteer installs is missing the necessary shared library +dependencies. -To fix, you'll need to install the missing dependencies and the -latest Chromium package in your Dockerfile: +To fix, you'll need to install the missing dependencies and the latest Chromium +package in your Dockerfile: ```Dockerfile FROM node:14-slim @@ -308,7 +362,8 @@ Build the container: docker build -t puppeteer-chrome-linux . ``` -Run the container by passing `node -e ""` as the command: +Run the container by passing `node -e ""` as +the command: ```bash docker run -i --init --rm --cap-add=SYS_ADMIN \ @@ -316,12 +371,15 @@ Run the container by passing `node -e ""` as node -e "`cat yourscript.js`" ``` -There's a full example at https://github.com/ebidel/try-puppeteer that shows -how to run this Dockerfile from a webserver running on App Engine Flex (Node). +There's a full example at https://github.com/ebidel/try-puppeteer that shows how +to run this Dockerfile from a webserver running on App Engine Flex (Node). ### Running on Alpine -The [newest Chromium package](https://pkgs.alpinelinux.org/package/edge/community/x86_64/chromium) supported on Alpine is 100, which corresponds to [Puppeteer v13.5.0](https://github.com/puppeteer/puppeteer/releases/tag/v13.5.0). +The +[newest Chromium package](https://pkgs.alpinelinux.org/package/edge/community/x86_64/chromium) +supported on Alpine is 100, which corresponds to +[Puppeteer v13.5.0](https://github.com/puppeteer/puppeteer/releases/tag/v13.5.0). Example Dockerfile: @@ -362,7 +420,8 @@ USER pptruser ## Running Puppeteer on GitlabCI -This is very similar to some of the instructions above, but require a bit different configuration to finally achieve success. +This is very similar to some of the instructions above, but require a bit +different configuration to finally achieve success. Usually the issue looks like this: @@ -375,23 +434,34 @@ You need to patch two places: 1. Your `gitlab-ci.yml` config 2. Arguments' list when launching pupepeteer -In `gitlab-ci.yml` we need to install some packages to make it possible to launch headless Chrome in your docker env: +In `gitlab-ci.yml` we need to install some packages to make it possible to +launch headless Chrome in your docker env: ```yml before_script: - apt-get update - - apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libnss3 lsb-release xdg-utils wget + - apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6 libcairo2 + libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 + libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 + libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 + libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 + libxss1 libxtst6 ca-certificates fonts-liberation libnss3 lsb-release + xdg-utils wget ``` -Next, you have to use `'--no-sandbox'` mode and also `'--disable-setuid-sandbox'` when launching Puppeteer. This can be done by passing them as an arguments to your `.launch()` call: `puppeteer.launch({ args: ['--no-sandbox', '--disable-setuid-sandbox'] });`. +Next, you have to use `'--no-sandbox'` mode and also +`'--disable-setuid-sandbox'` when launching Puppeteer. This can be done by +passing them as an arguments to your `.launch()` call: +`puppeteer.launch({ args: ['--no-sandbox', '--disable-setuid-sandbox'] });`. #### Tips By default, Docker runs a container with a `/dev/shm` shared memory space 64MB. -This is [typically too small](https://github.com/c0b/chrome-in-docker/issues/1) for Chrome -and will cause Chrome to crash when rendering large pages. To fix, run the container with -`docker run --shm-size=1gb` to increase the size of `/dev/shm`. Since Chrome 65, this is no -longer necessary. Instead, launch the browser with the `--disable-dev-shm-usage` flag: +This is [typically too small](https://github.com/c0b/chrome-in-docker/issues/1) +for Chrome and will cause Chrome to crash when rendering large pages. To fix, +run the container with `docker run --shm-size=1gb` to increase the size of +`/dev/shm`. Since Chrome 65, this is no longer necessary. Instead, launch the +browser with the `--disable-dev-shm-usage` flag: ```ts const browser = await puppeteer.launch({ @@ -399,11 +469,14 @@ const browser = await puppeteer.launch({ }); ``` -This will write shared memory files into `/tmp` instead of `/dev/shm`. See [crbug.com/736452](https://bugs.chromium.org/p/chromium/issues/detail?id=736452) for more details. +This will write shared memory files into `/tmp` instead of `/dev/shm`. See +[crbug.com/736452](https://bugs.chromium.org/p/chromium/issues/detail?id=736452) +for more details. -Seeing other weird errors when launching Chrome? Try running your container -with `docker run --cap-add=SYS_ADMIN` when developing locally. Since the Dockerfile -adds a `pptr` user as a non-privileged user, it may not have all the necessary privileges. +Seeing other weird errors when launching Chrome? Try running your container with +`docker run --cap-add=SYS_ADMIN` when developing locally. Since the Dockerfile +adds a `pptr` user as a non-privileged user, it may not have all the necessary +privileges. [dumb-init](https://github.com/Yelp/dumb-init) is worth checking out if you're experiencing a lot of zombies Chrome processes sticking around. There's special @@ -414,46 +487,80 @@ properly in some cases (e.g. in Docker). ### Running Puppeteer on Google App Engine -The Node.js runtime of the [App Engine standard environment](https://cloud.google.com/appengine/docs/standard/nodejs/) comes with all system packages needed to run Headless Chrome. +The Node.js runtime of the +[App Engine standard environment](https://cloud.google.com/appengine/docs/standard/nodejs/) +comes with all system packages needed to run Headless Chrome. -To use `puppeteer`, simply list the module as a dependency in your `package.json` and deploy to Google App Engine. Read more about using `puppeteer` on App Engine by following [the official tutorial](https://cloud.google.com/appengine/docs/standard/nodejs/using-headless-chrome-with-puppeteer). +To use `puppeteer`, simply list the module as a dependency in your +`package.json` and deploy to Google App Engine. Read more about using +`puppeteer` on App Engine by following +[the official tutorial](https://cloud.google.com/appengine/docs/standard/nodejs/using-headless-chrome-with-puppeteer). ### Running Puppeteer on Google Cloud Functions -The Node.js 10 runtime of [Google Cloud Functions](https://cloud.google.com/functions/docs/) comes with all system packages needed to run Headless Chrome. +The Node.js 10 runtime of +[Google Cloud Functions](https://cloud.google.com/functions/docs/) comes with +all system packages needed to run Headless Chrome. -To use `puppeteer`, simply list the module as a dependency in your `package.json` and deploy your function to Google Cloud Functions using the `nodejs10` runtime. +To use `puppeteer`, simply list the module as a dependency in your +`package.json` and deploy your function to Google Cloud Functions using the +`nodejs10` runtime. ### Running Puppeteer on Google Cloud Run -The default Node.js runtime of [Google Cloud Run](https://cloud.google.com/run/docs/) does not come with the system packages needed to run Headless Chrome. You will need to set up your own `Dockerfile` and [include the missing dependencies](#chrome-headless-doesnt-launch-on-unix). +The default Node.js runtime of +[Google Cloud Run](https://cloud.google.com/run/docs/) does not come with the +system packages needed to run Headless Chrome. You will need to set up your own +`Dockerfile` and +[include the missing dependencies](#chrome-headless-doesnt-launch-on-unix). ### Running Puppeteer on Heroku -Running Puppeteer on Heroku requires some additional dependencies that aren't included on the Linux box that Heroku spins up for you. To add the dependencies on deploy, add the Puppeteer Heroku buildpack to the list of buildpacks for your app under Settings > Buildpacks. +Running Puppeteer on Heroku requires some additional dependencies that aren't +included on the Linux box that Heroku spins up for you. To add the dependencies +on deploy, add the Puppeteer Heroku buildpack to the list of buildpacks for your +app under Settings > Buildpacks. -The url for the buildpack is https://github.com/jontewks/puppeteer-heroku-buildpack +The url for the buildpack is +https://github.com/jontewks/puppeteer-heroku-buildpack -Ensure that you're using `'--no-sandbox'` mode when launching Puppeteer. This can be done by passing it as an argument to your `.launch()` call: `puppeteer.launch({ args: ['--no-sandbox'] });`. +Ensure that you're using `'--no-sandbox'` mode when launching Puppeteer. This +can be done by passing it as an argument to your `.launch()` call: +`puppeteer.launch({ args: ['--no-sandbox'] });`. -When you click add buildpack, simply paste that url into the input, and click save. On the next deploy, your app will also install the dependencies that Puppeteer needs to run. +When you click add buildpack, simply paste that url into the input, and click +save. On the next deploy, your app will also install the dependencies that +Puppeteer needs to run. -If you need to render Chinese, Japanese, or Korean characters you may need to use a buildpack with additional font files like https://github.com/CoffeeAndCode/puppeteer-heroku-buildpack +If you need to render Chinese, Japanese, or Korean characters you may need to +use a buildpack with additional font files like +https://github.com/CoffeeAndCode/puppeteer-heroku-buildpack -There's also another [simple guide](https://timleland.com/headless-chrome-on-heroku/) from @timleland that includes a sample project: https://timleland.com/headless-chrome-on-heroku/. +There's also another +[simple guide](https://timleland.com/headless-chrome-on-heroku/) from @timleland +that includes a sample project: +https://timleland.com/headless-chrome-on-heroku/. ### Running Puppeteer on AWS Lambda -AWS Lambda [limits](https://docs.aws.amazon.com/lambda/latest/dg/limits.html) deployment package sizes to ~50MB. This presents challenges for running headless Chrome (and therefore Puppeteer) on Lambda. The community has put together a few resources that work around the issues: +AWS Lambda [limits](https://docs.aws.amazon.com/lambda/latest/dg/limits.html) +deployment package sizes to ~50MB. This presents challenges for running headless +Chrome (and therefore Puppeteer) on Lambda. The community has put together a few +resources that work around the issues: -- https://github.com/alixaxel/chrome-aws-lambda (kept updated with the latest stable release of puppeteer) -- https://github.com/adieuadieu/serverless-chrome/blob/HEAD/docs/chrome.md (serverless plugin - outdated) +- https://github.com/alixaxel/chrome-aws-lambda (kept updated with the latest + stable release of puppeteer) +- https://github.com/adieuadieu/serverless-chrome/blob/HEAD/docs/chrome.md + (serverless plugin - outdated) ### Running Puppeteer on AWS EC2 instance running Amazon-Linux -If you are using an EC2 instance running amazon-linux in your CI/CD pipeline, and if you want to run Puppeteer tests in amazon-linux, follow these steps. +If you are using an EC2 instance running amazon-linux in your CI/CD pipeline, +and if you want to run Puppeteer tests in amazon-linux, follow these steps. -1. To install Chromium, you have to first enable `amazon-linux-extras` which comes as part of [EPEL (Extra Packages for Enterprise Linux)](https://aws.amazon.com/premiumsupport/knowledge-center/ec2-enable-epel/): +1. To install Chromium, you have to first enable `amazon-linux-extras` which + comes as part of + [EPEL (Extra Packages for Enterprise Linux)](https://aws.amazon.com/premiumsupport/knowledge-center/ec2-enable-epel/): ```sh sudo amazon-linux-extras install epel -y @@ -465,13 +572,23 @@ If you are using an EC2 instance running amazon-linux in your CI/CD pipeline, an sudo yum install -y chromium ``` -Now Puppeteer can launch Chromium to run your tests. If you do not enable EPEL and if you continue installing chromium as part of `npm install`, Puppeteer cannot launch Chromium due to unavailablity of `libatk-1.0.so.0` and many more packages. +Now Puppeteer can launch Chromium to run your tests. If you do not enable EPEL +and if you continue installing chromium as part of `npm install`, Puppeteer +cannot launch Chromium due to unavailablity of `libatk-1.0.so.0` and many more +packages. ## Code Transpilation Issues -If you are using a JavaScript transpiler like babel or TypeScript, calling `evaluate()` with an async function might not work. This is because while `puppeteer` uses `Function.prototype.toString()` to serialize functions while transpilers could be changing the output code in such a way it's incompatible with `puppeteer`. +If you are using a JavaScript transpiler like babel or TypeScript, calling +`evaluate()` with an async function might not work. This is because while +`puppeteer` uses `Function.prototype.toString()` to serialize functions while +transpilers could be changing the output code in such a way it's incompatible +with `puppeteer`. -Some workarounds to this problem would be to instruct the transpiler not to mess up with the code, for example, configure TypeScript to use latest ecma version (`"target": "es2018"`). Another workaround could be using string templates instead of functions: +Some workarounds to this problem would be to instruct the transpiler not to mess +up with the code, for example, configure TypeScript to use latest ecma version +(`"target": "es2018"`). Another workaround could be using string templates +instead of functions: ```ts await page.evaluate(`(async() => {