2022-08-06 14:49:20 +00:00
|
|
|
|
---
|
2022-10-24 14:23:24 +00:00
|
|
|
|
sidebar_position: 3
|
2022-08-06 14:49:20 +00:00
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
# Troubleshooting
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
## `Cannot find module 'puppeteer-core/internal/...'`
|
|
|
|
|
|
|
|
|
|
This can occur is your Node.js version is lower than 14 or you are using a
|
|
|
|
|
custom resolver (such as
|
|
|
|
|
[`jest-resolve`](https://www.npmjs.com/package/jest-resolve)). For the former,
|
|
|
|
|
we do not support deprecated versions of Node.js. For the latter, usually
|
|
|
|
|
upgrading the resolver (or its parent module such as `jest`) will work (e.g.
|
|
|
|
|
https://github.com/puppeteer/puppeteer/issues/9121)
|
|
|
|
|
|
|
|
|
|
## `Could not find expected browser locally`
|
|
|
|
|
|
|
|
|
|
Starting from v19.0.0, Puppeteer will download browsers into
|
|
|
|
|
`~/.cache/puppeteer` using
|
|
|
|
|
[`os.homedir`](https://nodejs.org/api/os.html#oshomedir) for better caching
|
|
|
|
|
between Puppeteer upgrades. Generally the home directory is well-defined (even
|
|
|
|
|
on Windows), but occasionally the home directory may not be available. In this
|
|
|
|
|
case, we provide the `PUPPETEER_CACHE_DIR` variable which allows you to change
|
|
|
|
|
the installation directory.
|
|
|
|
|
|
|
|
|
|
For example,
|
|
|
|
|
|
|
|
|
|
```sh
|
|
|
|
|
PUPPETEER_CACHE_DIR=$(pwd) npm install puppeteer
|
|
|
|
|
PUPPETEER_CACHE_DIR=$(pwd) node <script-path>
|
|
|
|
|
```
|
|
|
|
|
|
2022-08-06 14:49:20 +00:00
|
|
|
|
## Chrome headless doesn't launch on Windows
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
Some [chrome policies](https://support.google.com/chrome/a/answer/7532015) might
|
|
|
|
|
enforce running Chrome/Chromium with certain extensions.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
Puppeteer passes `--disable-extensions` flag by default and will fail to launch
|
|
|
|
|
when such policies are active.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
To work around this, try running without the flag:
|
|
|
|
|
|
|
|
|
|
```ts
|
|
|
|
|
const browser = await puppeteer.launch({
|
|
|
|
|
ignoreDefaultArgs: ['--disable-extensions'],
|
|
|
|
|
});
|
|
|
|
|
```
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
> Context:
|
|
|
|
|
> [issue 3681](https://github.com/puppeteer/puppeteer/issues/3681#issuecomment-447865342).
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
## Chrome headless doesn't launch on UNIX
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
Make sure all the necessary dependencies are installed. You can run
|
|
|
|
|
`ldd chrome | grep not` on a Linux machine to check which dependencies are
|
|
|
|
|
missing. The common ones are provided below.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
<details>
|
|
|
|
|
<summary>Debian (e.g. Ubuntu) Dependencies</summary>
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
ca-certificates
|
|
|
|
|
fonts-liberation
|
|
|
|
|
libappindicator3-1
|
|
|
|
|
libasound2
|
|
|
|
|
libatk-bridge2.0-0
|
|
|
|
|
libatk1.0-0
|
|
|
|
|
libc6
|
|
|
|
|
libcairo2
|
|
|
|
|
libcups2
|
|
|
|
|
libdbus-1-3
|
|
|
|
|
libexpat1
|
|
|
|
|
libfontconfig1
|
|
|
|
|
libgbm1
|
|
|
|
|
libgcc1
|
|
|
|
|
libglib2.0-0
|
|
|
|
|
libgtk-3-0
|
|
|
|
|
libnspr4
|
|
|
|
|
libnss3
|
|
|
|
|
libpango-1.0-0
|
|
|
|
|
libpangocairo-1.0-0
|
|
|
|
|
libstdc++6
|
|
|
|
|
libx11-6
|
|
|
|
|
libx11-xcb1
|
|
|
|
|
libxcb1
|
|
|
|
|
libxcomposite1
|
|
|
|
|
libxcursor1
|
|
|
|
|
libxdamage1
|
|
|
|
|
libxext6
|
|
|
|
|
libxfixes3
|
|
|
|
|
libxi6
|
|
|
|
|
libxrandr2
|
|
|
|
|
libxrender1
|
|
|
|
|
libxss1
|
|
|
|
|
libxtst6
|
|
|
|
|
lsb-release
|
|
|
|
|
wget
|
|
|
|
|
xdg-utils
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
</details>
|
|
|
|
|
|
|
|
|
|
<details>
|
|
|
|
|
<summary>CentOS Dependencies</summary>
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
alsa-lib.x86_64
|
|
|
|
|
atk.x86_64
|
|
|
|
|
cups-libs.x86_64
|
|
|
|
|
gtk3.x86_64
|
|
|
|
|
ipa-gothic-fonts
|
|
|
|
|
libXcomposite.x86_64
|
|
|
|
|
libXcursor.x86_64
|
|
|
|
|
libXdamage.x86_64
|
|
|
|
|
libXext.x86_64
|
|
|
|
|
libXi.x86_64
|
|
|
|
|
libXrandr.x86_64
|
|
|
|
|
libXScrnSaver.x86_64
|
|
|
|
|
libXtst.x86_64
|
|
|
|
|
pango.x86_64
|
|
|
|
|
xorg-x11-fonts-100dpi
|
|
|
|
|
xorg-x11-fonts-75dpi
|
|
|
|
|
xorg-x11-fonts-cyrillic
|
|
|
|
|
xorg-x11-fonts-misc
|
|
|
|
|
xorg-x11-fonts-Type1
|
|
|
|
|
xorg-x11-utils
|
|
|
|
|
```
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
After installing dependencies you need to update `nss` library using this
|
|
|
|
|
command
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
yum update nss -y
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
</details>
|
|
|
|
|
|
|
|
|
|
<details>
|
|
|
|
|
<summary>Check out discussions</summary>
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
- [#290](https://github.com/puppeteer/puppeteer/issues/290) - Debian
|
|
|
|
|
troubleshooting <br/>
|
|
|
|
|
- [#391](https://github.com/puppeteer/puppeteer/issues/391) - CentOS
|
|
|
|
|
troubleshooting <br/>
|
|
|
|
|
- [#379](https://github.com/puppeteer/puppeteer/issues/379) - Alpine
|
|
|
|
|
troubleshooting <br/>
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
</details>
|
|
|
|
|
|
|
|
|
|
## Chrome headless disables GPU compositing
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
Chrome/Chromium requires `--use-gl=egl` to
|
|
|
|
|
[enable GPU acceleration in headless mode](https://github.com/chromium/chromium/commit/19671359ae25aa1e30bde90f8ff92453eeaac2ba).
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
```ts
|
|
|
|
|
const browser = await puppeteer.launch({
|
|
|
|
|
headless: true,
|
|
|
|
|
args: ['--use-gl=egl'],
|
|
|
|
|
});
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Chrome is downloaded but fails to launch on Node.js 14
|
|
|
|
|
|
|
|
|
|
If you get an error that looks like this when trying to launch Chromium:
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
(node:15505) UnhandledPromiseRejectionWarning: Error: Failed to launch the browser process!
|
|
|
|
|
spawn /Users/.../node_modules/puppeteer/.local-chromium/mac-756035/chrome-mac/Chromium.app/Contents/MacOS/Chromium ENOENT
|
|
|
|
|
```
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
This means that the browser was downloaded but failed to be extracted correctly.
|
|
|
|
|
The most common cause is a bug in Node.js v14.0.0 which broke `extract-zip`, the
|
|
|
|
|
module Puppeteer uses to extract browser downloads into the right place. The bug
|
|
|
|
|
was fixed in Node.js v14.1.0, so please make sure you're running that version or
|
|
|
|
|
higher.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
## Setting Up Chrome Linux Sandbox
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
In order to protect the host environment from untrusted web content, Chrome uses
|
|
|
|
|
[multiple layers of sandboxing](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/linux/sandboxing.md).
|
|
|
|
|
For this to work properly, the host should be configured first. If there's no
|
|
|
|
|
good sandbox for Chrome to use, it will crash with the error
|
|
|
|
|
`No usable sandbox!`.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
If you **absolutely trust** the content you open in Chrome, you can launch
|
|
|
|
|
Chrome with the `--no-sandbox` argument:
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
```ts
|
|
|
|
|
const browser = await puppeteer.launch({
|
|
|
|
|
args: ['--no-sandbox', '--disable-setuid-sandbox'],
|
|
|
|
|
});
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
:::caution
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
Running without a sandbox is **strongly discouraged**. Consider configuring a
|
|
|
|
|
sandbox instead.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
:::
|
|
|
|
|
|
|
|
|
|
There are 2 ways to configure a sandbox in Chromium.
|
|
|
|
|
|
|
|
|
|
### [recommended] Enable [user namespace cloning](http://man7.org/linux/man-pages/man7/user_namespaces.7.html)
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
User namespace cloning is only supported by modern kernels. Unprivileged user
|
|
|
|
|
namespaces are generally fine to enable, but in some cases they open up more
|
|
|
|
|
kernel attack surface for (unsandboxed) non-root processes to elevate to kernel
|
|
|
|
|
privileges.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
sudo sysctl -w kernel.unprivileged_userns_clone=1
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### [alternative] Setup [setuid sandbox](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/linux/suid_sandbox_development.md)
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
The setuid sandbox comes as a standalone executable and is located next to the
|
|
|
|
|
Chromium that Puppeteer downloads. It is fine to re-use the same sandbox
|
|
|
|
|
executable for different Chromium versions, so the following could be done only
|
|
|
|
|
once per host environment:
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
# cd to the downloaded instance
|
|
|
|
|
cd <project-dir-path>/node_modules/puppeteer/.local-chromium/linux-<revision>/chrome-linux/
|
|
|
|
|
sudo chown root:root chrome_sandbox
|
|
|
|
|
sudo chmod 4755 chrome_sandbox
|
|
|
|
|
# copy sandbox executable to a shared location
|
|
|
|
|
sudo cp -p chrome_sandbox /usr/local/sbin/chrome-devel-sandbox
|
|
|
|
|
# export CHROME_DEVEL_SANDBOX env variable
|
|
|
|
|
export CHROME_DEVEL_SANDBOX=/usr/local/sbin/chrome-devel-sandbox
|
|
|
|
|
```
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
You might want to export the `CHROME_DEVEL_SANDBOX` env variable by default. In
|
|
|
|
|
this case, add the following to the `~/.bashrc` or `.zshenv`:
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
export CHROME_DEVEL_SANDBOX=/usr/local/sbin/chrome-devel-sandbox
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Running Puppeteer on Travis CI
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
> 👋 We ran our tests for Puppeteer on Travis CI until v6.0.0 (when we've
|
|
|
|
|
> migrated to GitHub Actions) - see our historical
|
|
|
|
|
> [`.travis.yml` (v5.5.0)](https://github.com/puppeteer/puppeteer/blob/v5.5.0/.travis.yml)
|
|
|
|
|
> for reference.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
Tips-n-tricks:
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
- [xvfb](https://en.wikipedia.org/wiki/Xvfb) service should be launched in order
|
|
|
|
|
to run Chromium in non-headless mode
|
2022-08-06 14:49:20 +00:00
|
|
|
|
- Runs on Xenial Linux on Travis by default
|
|
|
|
|
- Runs `npm install` by default
|
|
|
|
|
- `node_modules` is cached by default
|
|
|
|
|
|
|
|
|
|
`.travis.yml` might look like this:
|
|
|
|
|
|
|
|
|
|
```yml
|
|
|
|
|
language: node_js
|
|
|
|
|
node_js: node
|
|
|
|
|
services: xvfb
|
|
|
|
|
script:
|
|
|
|
|
- npm test
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Running Puppeteer on CircleCI
|
|
|
|
|
|
|
|
|
|
Running Puppeteer smoothly on CircleCI requires the following steps:
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
1. Start with a
|
|
|
|
|
[NodeJS image](https://circleci.com/docs/2.0/circleci-images/#nodejs) in your
|
|
|
|
|
config like so:
|
2022-08-06 14:49:20 +00:00
|
|
|
|
```yaml
|
|
|
|
|
docker:
|
|
|
|
|
- image: circleci/node:14 # Use your desired version
|
|
|
|
|
environment:
|
|
|
|
|
NODE_ENV: development # Only needed if puppeteer is in `devDependencies`
|
|
|
|
|
```
|
2022-10-21 13:52:43 +00:00
|
|
|
|
1. Dependencies like `libXtst6` probably need to be installed via `apt-get`, so
|
|
|
|
|
use the
|
2022-08-06 14:49:20 +00:00
|
|
|
|
[threetreeslight/puppeteer](https://circleci.com/orbs/registry/orb/threetreeslight/puppeteer)
|
|
|
|
|
orb
|
|
|
|
|
([instructions](https://circleci.com/orbs/registry/orb/threetreeslight/puppeteer#quick-start)),
|
|
|
|
|
or paste parts of its
|
|
|
|
|
[source](https://circleci.com/orbs/registry/orb/threetreeslight/puppeteer#orb-source)
|
|
|
|
|
into your own config.
|
|
|
|
|
1. Lastly, if you’re using Puppeteer through Jest, then you may encounter an
|
|
|
|
|
error spawning child processes:
|
|
|
|
|
```
|
|
|
|
|
[00:00.0] jest args: --e2e --spec --max-workers=36
|
|
|
|
|
Error: spawn ENOMEM
|
|
|
|
|
at ChildProcess.spawn (internal/child_process.js:394:11)
|
|
|
|
|
```
|
|
|
|
|
This is likely caused by Jest autodetecting the number of processes on the
|
2022-10-21 13:52:43 +00:00
|
|
|
|
entire machine (`36`) rather than the number allowed to your container (`2`).
|
|
|
|
|
To fix this, set `jest --maxWorkers=2` in your test command.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
## Running Puppeteer in Docker
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
> 👋 We used [Cirrus Ci](https://cirrus-ci.org/) to run our tests for Puppeteer
|
|
|
|
|
> in a Docker container until v3.0.x - see our historical
|
|
|
|
|
> [`Dockerfile.linux` (v3.0.1)](https://github.com/puppeteer/puppeteer/blob/v3.0.1/.ci/node12/Dockerfile.linux)
|
|
|
|
|
> for reference. Starting from v16.0.0 we are shipping a Docker image via the
|
|
|
|
|
> GitHub registry. The Dockerfile is located
|
|
|
|
|
> [here](https://github.com/puppeteer/puppeteer/blob/main/docker/Dockerfile) and
|
|
|
|
|
> the usage instructions are in the
|
|
|
|
|
> [README.md](https://github.com/puppeteer/puppeteer#running-in-docker). The
|
|
|
|
|
> instructions below might be still helpful if you are building your own image.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
Getting headless Chrome up and running in Docker can be tricky. The bundled
|
|
|
|
|
Chromium that Puppeteer installs is missing the necessary shared library
|
|
|
|
|
dependencies.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
To fix, you'll need to install the missing dependencies and the latest Chromium
|
|
|
|
|
package in your Dockerfile:
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
```Dockerfile
|
|
|
|
|
FROM node:14-slim
|
|
|
|
|
|
|
|
|
|
# Install latest chrome dev package and fonts to support major charsets (Chinese, Japanese, Arabic, Hebrew, Thai and a few others)
|
|
|
|
|
# Note: this installs the necessary libs to make the bundled version of Chromium that Puppeteer
|
|
|
|
|
# installs, work.
|
|
|
|
|
RUN apt-get update \
|
|
|
|
|
&& apt-get install -y wget gnupg \
|
|
|
|
|
&& wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
|
|
|
|
|
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
|
|
|
|
|
&& apt-get update \
|
|
|
|
|
&& apt-get install -y google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf libxss1 \
|
|
|
|
|
--no-install-recommends \
|
|
|
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
|
|
|
|
|
|
# If running Docker >= 1.13.0 use docker run's --init arg to reap zombie processes, otherwise
|
|
|
|
|
# uncomment the following lines to have `dumb-init` as PID 1
|
|
|
|
|
# ADD https://github.com/Yelp/dumb-init/releases/download/v1.2.2/dumb-init_1.2.2_x86_64 /usr/local/bin/dumb-init
|
|
|
|
|
# RUN chmod +x /usr/local/bin/dumb-init
|
|
|
|
|
# ENTRYPOINT ["dumb-init", "--"]
|
|
|
|
|
|
|
|
|
|
# Uncomment to skip the chromium download when installing puppeteer. If you do,
|
|
|
|
|
# you'll need to launch puppeteer with:
|
|
|
|
|
# browser.launch({executablePath: 'google-chrome-stable'})
|
|
|
|
|
# ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true
|
|
|
|
|
|
|
|
|
|
# Install puppeteer so it's available in the container.
|
|
|
|
|
RUN npm init -y && \
|
|
|
|
|
npm i puppeteer \
|
|
|
|
|
# Add user so we don't need --no-sandbox.
|
|
|
|
|
# same layer as npm install to keep re-chowned files from using up several hundred MBs more space
|
|
|
|
|
&& groupadd -r pptruser && useradd -r -g pptruser -G audio,video pptruser \
|
|
|
|
|
&& mkdir -p /home/pptruser/Downloads \
|
|
|
|
|
&& chown -R pptruser:pptruser /home/pptruser \
|
|
|
|
|
&& chown -R pptruser:pptruser /node_modules \
|
|
|
|
|
&& chown -R pptruser:pptruser /package.json \
|
|
|
|
|
&& chown -R pptruser:pptruser /package-lock.json
|
|
|
|
|
|
|
|
|
|
# Run everything after as non-privileged user.
|
|
|
|
|
USER pptruser
|
|
|
|
|
|
|
|
|
|
CMD ["google-chrome-stable"]
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Build the container:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
docker build -t puppeteer-chrome-linux .
|
|
|
|
|
```
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
Run the container by passing `node -e "<yourscript.js content as a string>"` as
|
|
|
|
|
the command:
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
docker run -i --init --rm --cap-add=SYS_ADMIN \
|
|
|
|
|
--name puppeteer-chrome puppeteer-chrome-linux \
|
|
|
|
|
node -e "`cat yourscript.js`"
|
|
|
|
|
```
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
There's a full example at https://github.com/ebidel/try-puppeteer that shows how
|
|
|
|
|
to run this Dockerfile from a webserver running on App Engine Flex (Node).
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
### Running on Alpine
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
The
|
|
|
|
|
[newest Chromium package](https://pkgs.alpinelinux.org/package/edge/community/x86_64/chromium)
|
|
|
|
|
supported on Alpine is 100, which corresponds to
|
|
|
|
|
[Puppeteer v13.5.0](https://github.com/puppeteer/puppeteer/releases/tag/v13.5.0).
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
Example Dockerfile:
|
|
|
|
|
|
|
|
|
|
```Dockerfile
|
|
|
|
|
FROM alpine
|
|
|
|
|
|
|
|
|
|
# Installs latest Chromium (100) package.
|
|
|
|
|
RUN apk add --no-cache \
|
|
|
|
|
chromium \
|
|
|
|
|
nss \
|
|
|
|
|
freetype \
|
|
|
|
|
harfbuzz \
|
|
|
|
|
ca-certificates \
|
|
|
|
|
ttf-freefont \
|
|
|
|
|
nodejs \
|
|
|
|
|
yarn
|
|
|
|
|
|
|
|
|
|
...
|
|
|
|
|
|
|
|
|
|
# Tell Puppeteer to skip installing Chrome. We'll be using the installed package.
|
|
|
|
|
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true \
|
|
|
|
|
PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browser
|
|
|
|
|
|
|
|
|
|
# Puppeteer v13.5.0 works with Chromium 100.
|
|
|
|
|
RUN yarn add puppeteer@13.5.0
|
|
|
|
|
|
|
|
|
|
# Add user so we don't need --no-sandbox.
|
|
|
|
|
RUN addgroup -S pptruser && adduser -S -G pptruser pptruser \
|
|
|
|
|
&& mkdir -p /home/pptruser/Downloads /app \
|
|
|
|
|
&& chown -R pptruser:pptruser /home/pptruser \
|
|
|
|
|
&& chown -R pptruser:pptruser /app
|
|
|
|
|
|
|
|
|
|
# Run everything after as non-privileged user.
|
|
|
|
|
USER pptruser
|
|
|
|
|
|
|
|
|
|
...
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Running Puppeteer on GitlabCI
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
This is very similar to some of the instructions above, but require a bit
|
|
|
|
|
different configuration to finally achieve success.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
Usually the issue looks like this:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
Error: Failed to launch chrome! spawn /usr/bin/chromium-browser ENOENT
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
You need to patch two places:
|
|
|
|
|
|
|
|
|
|
1. Your `gitlab-ci.yml` config
|
|
|
|
|
2. Arguments' list when launching pupepeteer
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
In `gitlab-ci.yml` we need to install some packages to make it possible to
|
|
|
|
|
launch headless Chrome in your docker env:
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
```yml
|
|
|
|
|
before_script:
|
|
|
|
|
- apt-get update
|
2022-10-21 13:52:43 +00:00
|
|
|
|
- apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6 libcairo2
|
|
|
|
|
libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4
|
|
|
|
|
libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0
|
|
|
|
|
libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1
|
|
|
|
|
libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1
|
|
|
|
|
libxss1 libxtst6 ca-certificates fonts-liberation libnss3 lsb-release
|
|
|
|
|
xdg-utils wget
|
2022-08-06 14:49:20 +00:00
|
|
|
|
```
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
Next, you have to use `'--no-sandbox'` mode and also
|
|
|
|
|
`'--disable-setuid-sandbox'` when launching Puppeteer. This can be done by
|
|
|
|
|
passing them as an arguments to your `.launch()` call:
|
|
|
|
|
`puppeteer.launch({ args: ['--no-sandbox', '--disable-setuid-sandbox'] });`.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
#### Tips
|
|
|
|
|
|
|
|
|
|
By default, Docker runs a container with a `/dev/shm` shared memory space 64MB.
|
2022-10-21 13:52:43 +00:00
|
|
|
|
This is [typically too small](https://github.com/c0b/chrome-in-docker/issues/1)
|
|
|
|
|
for Chrome and will cause Chrome to crash when rendering large pages. To fix,
|
|
|
|
|
run the container with `docker run --shm-size=1gb` to increase the size of
|
|
|
|
|
`/dev/shm`. Since Chrome 65, this is no longer necessary. Instead, launch the
|
|
|
|
|
browser with the `--disable-dev-shm-usage` flag:
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
```ts
|
|
|
|
|
const browser = await puppeteer.launch({
|
|
|
|
|
args: ['--disable-dev-shm-usage'],
|
|
|
|
|
});
|
|
|
|
|
```
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
This will write shared memory files into `/tmp` instead of `/dev/shm`. See
|
|
|
|
|
[crbug.com/736452](https://bugs.chromium.org/p/chromium/issues/detail?id=736452)
|
|
|
|
|
for more details.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
Seeing other weird errors when launching Chrome? Try running your container with
|
|
|
|
|
`docker run --cap-add=SYS_ADMIN` when developing locally. Since the Dockerfile
|
|
|
|
|
adds a `pptr` user as a non-privileged user, it may not have all the necessary
|
|
|
|
|
privileges.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
[dumb-init](https://github.com/Yelp/dumb-init) is worth checking out if you're
|
|
|
|
|
experiencing a lot of zombies Chrome processes sticking around. There's special
|
|
|
|
|
treatment for processes with PID=1, which makes it hard to terminate Chrome
|
|
|
|
|
properly in some cases (e.g. in Docker).
|
|
|
|
|
|
|
|
|
|
## Running Puppeteer in the cloud
|
|
|
|
|
|
|
|
|
|
### Running Puppeteer on Google App Engine
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
The Node.js runtime of the
|
|
|
|
|
[App Engine standard environment](https://cloud.google.com/appengine/docs/standard/nodejs/)
|
|
|
|
|
comes with all system packages needed to run Headless Chrome.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
To use `puppeteer`, simply list the module as a dependency in your
|
|
|
|
|
`package.json` and deploy to Google App Engine. Read more about using
|
|
|
|
|
`puppeteer` on App Engine by following
|
|
|
|
|
[the official tutorial](https://cloud.google.com/appengine/docs/standard/nodejs/using-headless-chrome-with-puppeteer).
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
### Running Puppeteer on Google Cloud Functions
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
The Node.js 10 runtime of
|
|
|
|
|
[Google Cloud Functions](https://cloud.google.com/functions/docs/) comes with
|
|
|
|
|
all system packages needed to run Headless Chrome.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
To use `puppeteer`, simply list the module as a dependency in your
|
|
|
|
|
`package.json` and deploy your function to Google Cloud Functions using the
|
|
|
|
|
`nodejs10` runtime.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
### Running Puppeteer on Google Cloud Run
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
The default Node.js runtime of
|
|
|
|
|
[Google Cloud Run](https://cloud.google.com/run/docs/) does not come with the
|
|
|
|
|
system packages needed to run Headless Chrome. You will need to set up your own
|
|
|
|
|
`Dockerfile` and
|
|
|
|
|
[include the missing dependencies](#chrome-headless-doesnt-launch-on-unix).
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
### Running Puppeteer on Heroku
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
Running Puppeteer on Heroku requires some additional dependencies that aren't
|
|
|
|
|
included on the Linux box that Heroku spins up for you. To add the dependencies
|
|
|
|
|
on deploy, add the Puppeteer Heroku buildpack to the list of buildpacks for your
|
|
|
|
|
app under Settings > Buildpacks.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
The url for the buildpack is
|
|
|
|
|
https://github.com/jontewks/puppeteer-heroku-buildpack
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
Ensure that you're using `'--no-sandbox'` mode when launching Puppeteer. This
|
|
|
|
|
can be done by passing it as an argument to your `.launch()` call:
|
|
|
|
|
`puppeteer.launch({ args: ['--no-sandbox'] });`.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
When you click add buildpack, simply paste that url into the input, and click
|
|
|
|
|
save. On the next deploy, your app will also install the dependencies that
|
|
|
|
|
Puppeteer needs to run.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
If you need to render Chinese, Japanese, or Korean characters you may need to
|
|
|
|
|
use a buildpack with additional font files like
|
|
|
|
|
https://github.com/CoffeeAndCode/puppeteer-heroku-buildpack
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
There's also another
|
|
|
|
|
[simple guide](https://timleland.com/headless-chrome-on-heroku/) from @timleland
|
|
|
|
|
that includes a sample project:
|
|
|
|
|
https://timleland.com/headless-chrome-on-heroku/.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
### Running Puppeteer on AWS Lambda
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
AWS Lambda [limits](https://docs.aws.amazon.com/lambda/latest/dg/limits.html)
|
|
|
|
|
deployment package sizes to ~50MB. This presents challenges for running headless
|
|
|
|
|
Chrome (and therefore Puppeteer) on Lambda. The community has put together a few
|
|
|
|
|
resources that work around the issues:
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
- https://github.com/alixaxel/chrome-aws-lambda (kept updated with the latest
|
|
|
|
|
stable release of puppeteer)
|
|
|
|
|
- https://github.com/adieuadieu/serverless-chrome/blob/HEAD/docs/chrome.md
|
|
|
|
|
(serverless plugin - outdated)
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
### Running Puppeteer on AWS EC2 instance running Amazon-Linux
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
If you are using an EC2 instance running amazon-linux in your CI/CD pipeline,
|
|
|
|
|
and if you want to run Puppeteer tests in amazon-linux, follow these steps.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
1. To install Chromium, you have to first enable `amazon-linux-extras` which
|
|
|
|
|
comes as part of
|
|
|
|
|
[EPEL (Extra Packages for Enterprise Linux)](https://aws.amazon.com/premiumsupport/knowledge-center/ec2-enable-epel/):
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
```sh
|
|
|
|
|
sudo amazon-linux-extras install epel -y
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
1. Next, install Chromium:
|
|
|
|
|
|
|
|
|
|
```sh
|
|
|
|
|
sudo yum install -y chromium
|
|
|
|
|
```
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
Now Puppeteer can launch Chromium to run your tests. If you do not enable EPEL
|
|
|
|
|
and if you continue installing chromium as part of `npm install`, Puppeteer
|
|
|
|
|
cannot launch Chromium due to unavailablity of `libatk-1.0.so.0` and many more
|
|
|
|
|
packages.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
## Code Transpilation Issues
|
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
If you are using a JavaScript transpiler like babel or TypeScript, calling
|
|
|
|
|
`evaluate()` with an async function might not work. This is because while
|
|
|
|
|
`puppeteer` uses `Function.prototype.toString()` to serialize functions while
|
|
|
|
|
transpilers could be changing the output code in such a way it's incompatible
|
|
|
|
|
with `puppeteer`.
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
2022-10-21 13:52:43 +00:00
|
|
|
|
Some workarounds to this problem would be to instruct the transpiler not to mess
|
|
|
|
|
up with the code, for example, configure TypeScript to use latest ecma version
|
|
|
|
|
(`"target": "es2018"`). Another workaround could be using string templates
|
|
|
|
|
instead of functions:
|
2022-08-06 14:49:20 +00:00
|
|
|
|
|
|
|
|
|
```ts
|
|
|
|
|
await page.evaluate(`(async() => {
|
|
|
|
|
console.log('1');
|
|
|
|
|
})()`);
|
|
|
|
|
```
|