thunderstrike/puppeteer
thunderstrike/puppeteer
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat(launcher): add option to run Puppeteer with different browsers (#5137)

Browse Source 
* feat: Set which browser to launch via PUPPETEER_PRODUCT

This change introduces a PUPPETEER_PRODUCT environment
variable as a first step toward using Puppeteer with
many different browsers. Setting PUPPETEER_PRODUCT=firefox, for
example, enables Firefox-specific Launcher settings.

The state is also exposed as `puppeteer.product` in the API
to support adding other product-specific behaviour as needed.

The bulk of the change is a refactoring in Launcher
to decouple generic browser start-up from product-specific
configuration.

Respecting the puppeteer-core restriction for PUPPETEER_
environment variables, lazily instantiate the Launcher
based on a `product` Puppeteer.launch option, if available.

* test: Distinguish Juggler unit tests from Firefox

The funit script is renamed to fjunit (j for Juggler, which is
used only by the experimental puppeteer-firefox package.

In contrast, the funit script now refers to running Puppeteer
unit tests against the main puppeteer package with Firefox.
To do so with Firefox Nightly, run:

`BINARY=path/to/firefox npm run funit`

A number of changes in this patch make it easier to run
Puppeteer unit tests in Mozilla's CI.
This commit is contained in:
Maja Frydrychowicz 2019-11-26 04:23:19 -05:00 committed by Mathias Bynens
parent d17708ba1f
commit c5a72e9887
16 changed files with 559 additions and 214 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.cirrus.yml
View File

@ -24,12 +24,12 @@ task:
task:
matrix:
- name: Firefox (node8 + linux)
- name: Firefox Juggler (node8 + linux)
container:
dockerfile: .ci/node8/Dockerfile.linux
xvfb_start_background_script: Xvfb :99 -ac -screen 0 1024x768x24
install_script: npm install --unsafe-perm && cd experimental/puppeteer-firefox && npm install --unsafe-perm
test_script: npm run funit
test_script: npm run fjunit
task:
osx_instance:

2
.travis.yml
View File

@ -19,7 +19,7 @@ script:
- 'if [ "$NODE8" = "true" ]; then npm run lint; fi'
- 'if [ "$NODE8" = "true" ]; then npm run coverage; fi'
- 'if [ "$FIREFOX" = "true" ]; then cd experimental/puppeteer-firefox && npm i && cd ../..; fi'
- 'if [ "$FIREFOX" = "true" ]; then npm run funit; fi'
- 'if [ "$FIREFOX" = "true" ]; then npm run fjunit; fi'
- 'if [ "$NODE8" = "true" ]; then npm run test-doclint; fi'
- 'if [ "$NODE8" = "true" ]; then npm run test-types; fi'
- 'if [ "$NODE8" = "true" ]; then npm run bundle; fi'

11
CONTRIBUTING.md
View File

@ -199,10 +199,10 @@ npm run unit -- --break-on-failure
HEADLESS=false npm run unit
```
- To run tests with custom Chromium executable:
- To run tests with custom browser executable:
```bash
CHROME=<path-to-executable> npm run unit
BINARY=<path-to-executable> npm run unit
```
- To run tests in slow-mode:
@ -211,6 +211,13 @@ CHROME=<path-to-executable> npm run unit
HEADLESS=false SLOW_MO=500 npm run unit
```
- To run tests with additional Launcher options:
```bash
EXTRA_LAUNCH_OPTIONS='{"args": ["--user-data-dir=some/path"], "handleSIGINT": true}' npm run unit
```
- To debug a test, "focus" a test first and then run:
```bash

14
docs/api.md
View File

@ -31,6 +31,7 @@
* [puppeteer.errors](#puppeteererrors)
* [puppeteer.executablePath()](#puppeteerexecutablepath)
* [puppeteer.launch([options])](#puppeteerlaunchoptions)
* [puppeteer.product](#puppeteerproduct)
- [class: BrowserFetcher](#class-browserfetcher)
* [browserFetcher.canDownload(revision)](#browserfetchercandownloadrevision)
* [browserFetcher.download(revision[, progressCallback])](#browserfetcherdownloadrevision-progresscallback)
@ -388,6 +389,7 @@ If Puppeteer doesn't find them in the environment during the installation step,
- `PUPPETEER_DOWNLOAD_HOST` - overwrite URL prefix that is used to download Chromium. Note: this includes protocol and might even include path prefix. Defaults to `https://storage.googleapis.com`.
- `PUPPETEER_CHROMIUM_REVISION` - specify a certain version of Chromium you'd like Puppeteer to use. See [puppeteer.launch([options])](#puppeteerlaunchoptions) on how executable path is inferred. **BEWARE**: Puppeteer is only [guaranteed to work](https://github.com/GoogleChrome/puppeteer/#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) with the bundled Chromium, use at your own risk.
- `PUPPETEER_EXECUTABLE_PATH` - specify an executable path to be used in `puppeteer.launch`. See [puppeteer.launch([options])](#puppeteerlaunchoptions) on how the executable path is inferred. **BEWARE**: Puppeteer is only [guaranteed to work](https://github.com/GoogleChrome/puppeteer/#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) with the bundled Chromium, use at your own risk.
- `PUPPETEER_PRODUCT` - specify which browser you'd like Puppeteer to use. Must be one of `chrome` or `firefox`. Setting `product` programmatically in [puppeteer.launch([options])](#puppeteerlaunchoptions) supercedes this environment variable. The product is exposed in [`puppeteer.product`](#puppeteerproduct)
> **NOTE** PUPPETEER_* env variables are not accounted for in the [`puppeteer-core`](https://www.npmjs.com/package/puppeteer-core) package.
@ -525,9 +527,10 @@ try {
#### puppeteer.launch([options])
- `options` <[Object]> Set of configurable options to set on the browser. Can have the following fields:
- `product` <[string]> Which browser to launch. At this time, this is either `chrome` or `firefox`. See also `PUPPETEER_PRODUCT`.
- `ignoreHTTPSErrors` <[boolean]> Whether to ignore HTTPS errors during navigation. Defaults to `false`.
- `headless` <[boolean]> Whether to run browser in [headless mode](https://developers.google.com/web/updates/2017/04/headless-chrome). Defaults to `true` unless the `devtools` option is `true`.
- `executablePath` <[string]> Path to a Chromium or Chrome executable to run instead of the bundled Chromium. If `executablePath` is a relative path, then it is resolved relative to [current working directory](https://nodejs.org/api/process.html#process_process_cwd). **BEWARE**: Puppeteer is only [guaranteed to work](https://github.com/GoogleChrome/puppeteer/#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) with the bundled Chromium, use at your own risk.
- `executablePath` <[string]> Path to a browser executable to run instead of the bundled Chromium. If `executablePath` is a relative path, then it is resolved relative to [current working directory](https://nodejs.org/api/process.html#process_process_cwd). **BEWARE**: Puppeteer is only [guaranteed to work](https://github.com/GoogleChrome/puppeteer/#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) with the bundled Chromium, use at your own risk.
- `slowMo` <[number]> Slows down Puppeteer operations by the specified amount of milliseconds. Useful so that you can see what is going on.
- `defaultViewport` <?[Object]> Sets a consistent viewport for each page. Defaults to an 800x600 viewport. `null` disables the default viewport.
- `width` <[number]> page width in pixels.
@ -536,7 +539,7 @@ try {
- `isMobile` <[boolean]> Whether the `meta viewport` tag is taken into account. Defaults to `false`.
- `hasTouch`<[boolean]> Specifies if viewport supports touch events. Defaults to `false`
- `isLandscape` <[boolean]> Specifies if viewport is in landscape mode. Defaults to `false`.
- `args` <[Array]<[string]>> Additional arguments to pass to the browser instance. The list of Chromium flags can be found [here](http://peter.sh/experiments/chromium-command-line-switches/).
- `args` <[Array]<[string]>> Additional arguments to pass to the browser instance. The list of Chromium flags can be found [here](http://peter.sh/experiments/chromium-command-line-switches/), and here is the list of [Firefox flags](https://developer.mozilla.org/en-US/docs/Mozilla/Command_Line_Options).
- `ignoreDefaultArgs` <[boolean]|[Array]<[string]>> If `true`, then do not use [`puppeteer.defaultArgs()`](#puppeteerdefaultargsoptions). If an array is given, then filter out the given default arguments. Dangerous option; use with care. Defaults to `false`.
- `handleSIGINT` <[boolean]> Close the browser process on Ctrl-C. Defaults to `true`.
- `handleSIGTERM` <[boolean]> Close the browser process on SIGTERM. Defaults to `true`.
@ -547,6 +550,7 @@ try {
- `env` <[Object]> Specify environment variables that will be visible to the browser. Defaults to `process.env`.
- `devtools` <[boolean]> Whether to auto-open a DevTools panel for each tab. If this option is `true`, the `headless` option will be set `false`.
- `pipe` <[boolean]> Connects to the browser over a pipe instead of a WebSocket. Defaults to `false`.
- `extraPrefsFirefox` <[Object]> Additional [preferences](https://developer.mozilla.org/en-US/docs/Mozilla/Preferences/Preference_reference) that can be passed to Firefox (see `PUPPETEER_PRODUCT`)
- returns: <[Promise]<[Browser]>> Promise which resolves to browser instance.
@ -565,6 +569,12 @@ const browser = await puppeteer.launch({
>
> See [`this article`](https://www.howtogeek.com/202825/what%E2%80%99s-the-difference-between-chromium-and-chrome/) for a description of the differences between Chromium and Chrome. [`This article`](https://chromium.googlesource.com/chromium/src/+/lkgr/docs/chromium_browser_vs_google_chrome.md) describes some differences for Linux users.
#### puppeteer.product
- returns: <[string]> returns the name of the browser that is under automation ("chrome" or "firefox")
The product is set by the `PUPPETEER_PRODUCT` environment variable or the `product` option in [puppeteer.launch([options])](#puppeteerlaunchoptions) and defaults to `chrome`. Firefox support is experimental.
### class: BrowserFetcher
BrowserFetcher can download and manage different versions of Chromium.

6
experimental/puppeteer-firefox/.cirrus.yml
View File

@ -7,7 +7,7 @@ task:
dockerfile: .ci/node8/Dockerfile.linux
xvfb_start_background_script: Xvfb :99 -ac -screen 0 1024x768x24
install_script: npm install
test_script: npm run funit
test_script: npm run fjunit
task:
name: node8 (macOS)
@ -19,7 +19,7 @@ task:
- brew install node@8
- brew link --force node@8
install_script: npm install
test_script: npm run funit
test_script: npm run fjunit
# task:
# allow_failures: true
@ -28,4 +28,4 @@ task:
# os_version: 2016
# name: node8 (windows)
# install_script: npm install --unsafe-perm
# test_script: npm run funit
# test_script: npm run fjunit

9
index.js
View File

@ -28,4 +28,11 @@ const packageJson = require('./package.json');
const preferredRevision = packageJson.puppeteer.chromium_revision;
const isPuppeteerCore = packageJson.name === 'puppeteer-core';
module.exports = new Puppeteer(__dirname, preferredRevision, isPuppeteerCore);
const puppeteer = new Puppeteer(__dirname, preferredRevision, isPuppeteerCore);
// The introspection in `Helper.installAsyncStackHooks` references `Puppeteer._launcher`
// before the Puppeteer ctor is called, such that an invalid Launcher is selected at import,
// so we reset it.
puppeteer._lazyLauncher = undefined;
module.exports = puppeteer;

580
lib/Launcher.js
View File

@ -26,43 +26,159 @@ const {Browser} = require('./Browser');
const readline = require('readline');
const fs = require('fs');
const {helper, assert, debugError} = require('./helper');
const debugLauncher = require('debug')(`puppeteer:launcher`);
const {TimeoutError} = require('./Errors');
const WebSocketTransport = require('./WebSocketTransport');
const PipeTransport = require('./PipeTransport');
const mkdtempAsync = helper.promisify(fs.mkdtemp);
const removeFolderAsync = helper.promisify(removeFolder);
const writeFileAsync = helper.promisify(fs.writeFile);
const CHROME_PROFILE_PATH = path.join(os.tmpdir(), 'puppeteer_dev_profile-');
class BrowserRunner {
const DEFAULT_ARGS = [
'--disable-background-networking',
'--enable-features=NetworkService,NetworkServiceInProcess',
'--disable-background-timer-throttling',
'--disable-backgrounding-occluded-windows',
'--disable-breakpad',
'--disable-client-side-phishing-detection',
'--disable-component-extensions-with-background-pages',
'--disable-default-apps',
'--disable-dev-shm-usage',
'--disable-extensions',
// BlinkGenPropertyTrees disabled due to crbug.com/937609
'--disable-features=TranslateUI,BlinkGenPropertyTrees',
'--disable-hang-monitor',
'--disable-ipc-flooding-protection',
'--disable-popup-blocking',
'--disable-prompt-on-repost',
'--disable-renderer-backgrounding',
'--disable-sync',
'--force-color-profile=srgb',
'--metrics-recording-only',
'--no-first-run',
'--enable-automation',
'--password-store=basic',
'--use-mock-keychain',
];
/**
* @param {string} executablePath
* @param {!Array<string>} processArguments
* @param {string=} tempDirectory
*/
constructor(executablePath, processArguments, tempDirectory) {
this._executablePath = executablePath;
this._processArguments = processArguments;
this._tempDirectory = tempDirectory;
this.proc = null;
this.connection = null;
this._closed = true;
this._listeners = [];
}
class Launcher {
/**
* @param {!(Launcher.LaunchOptions)=} options
*/
start(options = {}) {
const {
handleSIGINT,
handleSIGTERM,
handleSIGHUP,
dumpio,
env,
pipe
} = options;
/** @type {!Array<"ignore"|"pipe">} */
let stdio = ['pipe', 'pipe', 'pipe'];
if (pipe) {
if (dumpio)
stdio = ['ignore', 'pipe', 'pipe', 'pipe', 'pipe'];
else
stdio = ['ignore', 'ignore', 'ignore', 'pipe', 'pipe'];
}
assert(!this.proc, 'This process has previously been started.');
debugLauncher(`Calling ${this._executablePath} ${this._processArguments.join(' ')}`);
this.proc = childProcess.spawn(
this._executablePath,
this._processArguments,
{
// On non-windows platforms, `detached: true` makes child process a leader of a new
// process group, making it possible to kill child process tree with `.kill(-pid)` command.
// @see https://nodejs.org/api/child_process.html#child_process_options_detached
detached: process.platform !== 'win32',
env,
stdio
}
);
if (dumpio) {
this.proc.stderr.pipe(process.stderr);
this.proc.stdout.pipe(process.stdout);
}
this._closed = false;
this._processClosing = new Promise((fulfill, reject) => {
this.proc.once('exit', () => {
this._closed = true;
// Cleanup as processes exit.
if (this._tempDirectory) {
removeFolderAsync(this._tempDirectory)
.then(() => fulfill())
.catch(err => console.error(err));
} else {
fulfill();
}
});
});
this._listeners = [ helper.addEventListener(process, 'exit', this.kill.bind(this)) ];
if (handleSIGINT)
this._listeners.push(helper.addEventListener(process, 'SIGINT', () => { this.kill(); process.exit(130); }));
if (handleSIGTERM)
this._listeners.push(helper.addEventListener(process, 'SIGTERM', this.close.bind(this)));
if (handleSIGHUP)
this._listeners.push(helper.addEventListener(process, 'SIGHUP', this.close.bind(this)));
}
/**
* @return {Promise}
*/
close() {
if (this._closed)
return Promise.resolve();
helper.removeEventListeners(this._listeners);
if (this._tempDirectory) {
this.kill();
} else if (this.connection) {
// Attempt to close the browser gracefully
this.connection.send('Browser.close').catch(error => {
debugError(error);
this.kill();
});
}
return this._processClosing;
}
// This function has to be sync to be used as 'exit' event handler.
kill() {
helper.removeEventListeners(this._listeners);
if (this.proc && this.proc.pid && !this.proc.killed && !this._closed) {
try {
if (process.platform === 'win32')
childProcess.execSync(`taskkill /pid ${this.proc.pid} /T /F`);
else
process.kill(-this.proc.pid, 'SIGKILL');
} catch (error) {
// the process might have already stopped
}
}
// Attempt to remove temporary profile directory to avoid littering.
try {
removeFolder.sync(this._tempDirectory);
} catch (error) { }
}
/**
* @param {!({usePipe?: boolean, timeout: number, slowMo: number, preferredRevision: string})} options
*
* @return {!Promise<!Connection>}
*/
async setupConnection(options) {
const {
usePipe,
timeout,
slowMo,
preferredRevision
} = options;
if (!usePipe) {
const browserWSEndpoint = await waitForWSEndpoint(this.proc, timeout, preferredRevision);
const transport = await WebSocketTransport.create(browserWSEndpoint);
this.connection = new Connection(browserWSEndpoint, transport, slowMo);
} else {
const transport = new PipeTransport(/** @type {!NodeJS.WritableStream} */(this.proc.stdio[3]), /** @type {!NodeJS.ReadableStream} */ (this.proc.stdio[4]));
this.connection = new Connection('', transport, slowMo);
}
return this.connection;
}
}
/**
* @implements {!Puppeteer.ProductLauncher}
*/
class ChromeLauncher {
/**
* @param {string} projectRoot
* @param {string} preferredRevision
@ -95,11 +211,12 @@ class Launcher {
timeout = 30000
} = options;
const profilePath = path.join(os.tmpdir(), 'puppeteer_dev_chrome_profile-');
const chromeArguments = [];
if (!ignoreDefaultArgs)
chromeArguments.push(...this.defaultArgs(options));
else if (Array.isArray(ignoreDefaultArgs))
chromeArguments.push(...this.defaultArgs(options).filter(arg => ignoreDefaultArgs.indexOf(arg) === -1));
chromeArguments.push(...this.defaultArgs(options).filter(arg => !ignoreDefaultArgs.includes(arg)));
else
chromeArguments.push(...args);
@ -108,121 +225,30 @@ class Launcher {
if (!chromeArguments.some(argument => argument.startsWith('--remote-debugging-')))
chromeArguments.push(pipe ? '--remote-debugging-pipe' : '--remote-debugging-port=0');
if (!chromeArguments.some(arg => arg.startsWith('--user-data-dir'))) {
temporaryUserDataDir = await mkdtempAsync(CHROME_PROFILE_PATH);
temporaryUserDataDir = await mkdtempAsync(profilePath);
chromeArguments.push(`--user-data-dir=${temporaryUserDataDir}`);
}
let chromeExecutable = executablePath;
if (!executablePath) {
const {missingText, executablePath} = this._resolveExecutablePath();
const {missingText, executablePath} = resolveExecutablePath(this);
if (missingText)
throw new Error(missingText);
chromeExecutable = executablePath;
}
const usePipe = chromeArguments.includes('--remote-debugging-pipe');
/** @type {!Array<"ignore"|"pipe">} */
let stdio = ['pipe', 'pipe', 'pipe'];
if (usePipe) {
if (dumpio)
stdio = ['ignore', 'pipe', 'pipe', 'pipe', 'pipe'];
else
stdio = ['ignore', 'ignore', 'ignore', 'pipe', 'pipe'];
}
const chromeProcess = childProcess.spawn(
chromeExecutable,
chromeArguments,
{
// On non-windows platforms, `detached: true` makes child process a leader of a new
// process group, making it possible to kill child process tree with `.kill(-pid)` command.
// @see https://nodejs.org/api/child_process.html#child_process_options_detached
detached: process.platform !== 'win32',
env,
stdio
}
);
const runner = new BrowserRunner(chromeExecutable, chromeArguments, temporaryUserDataDir);
runner.start({handleSIGHUP, handleSIGTERM, handleSIGINT, dumpio, env, pipe: usePipe});
if (dumpio) {
chromeProcess.stderr.pipe(process.stderr);
chromeProcess.stdout.pipe(process.stdout);
}
let chromeClosed = false;
const waitForChromeToClose = new Promise((fulfill, reject) => {
chromeProcess.once('exit', () => {
chromeClosed = true;
// Cleanup as processes exit.
if (temporaryUserDataDir) {
removeFolderAsync(temporaryUserDataDir)
.then(() => fulfill())
.catch(err => console.error(err));
} else {
fulfill();
}
});
});
const listeners = [ helper.addEventListener(process, 'exit', killChrome) ];
if (handleSIGINT)
listeners.push(helper.addEventListener(process, 'SIGINT', () => { killChrome(); process.exit(130); }));
if (handleSIGTERM)
listeners.push(helper.addEventListener(process, 'SIGTERM', gracefullyCloseChrome));
if (handleSIGHUP)
listeners.push(helper.addEventListener(process, 'SIGHUP', gracefullyCloseChrome));
/** @type {?Connection} */
let connection = null;
try {
if (!usePipe) {
const browserWSEndpoint = await waitForWSEndpoint(chromeProcess, timeout, this._preferredRevision);
const transport = await WebSocketTransport.create(browserWSEndpoint);
connection = new Connection(browserWSEndpoint, transport, slowMo);
} else {
const transport = new PipeTransport(/** @type {!NodeJS.WritableStream} */(chromeProcess.stdio[3]), /** @type {!NodeJS.ReadableStream} */ (chromeProcess.stdio[4]));
connection = new Connection('', transport, slowMo);
}
const browser = await Browser.create(connection, [], ignoreHTTPSErrors, defaultViewport, chromeProcess, gracefullyCloseChrome);
const connection = await runner.setupConnection({usePipe, timeout, slowMo, preferredRevision: this._preferredRevision});
const browser = await Browser.create(connection, [], ignoreHTTPSErrors, defaultViewport, runner.proc, runner.close.bind(runner));
await browser.waitForTarget(t => t.type() === 'page');
return browser;
} catch (e) {
killChrome();
throw e;
}
/**
* @return {Promise}
*/
function gracefullyCloseChrome() {
helper.removeEventListeners(listeners);
if (temporaryUserDataDir) {
killChrome();
} else if (connection) {
// Attempt to close chrome gracefully
connection.send('Browser.close').catch(error => {
debugError(error);
killChrome();
});
}
return waitForChromeToClose;
}
// This method has to be sync to be used as 'exit' event handler.
function killChrome() {
helper.removeEventListeners(listeners);
if (chromeProcess.pid && !chromeProcess.killed && !chromeClosed) {
// Force kill chrome.
try {
if (process.platform === 'win32')
childProcess.execSync(`taskkill /pid ${chromeProcess.pid} /T /F`);
else
process.kill(-chromeProcess.pid, 'SIGKILL');
} catch (e) {
// the process might have already stopped
}
}
// Attempt to remove temporary profile directory to avoid littering.
try {
removeFolder.sync(temporaryUserDataDir);
} catch (e) { }
} catch (error) {
runner.kill();
throw error;
}
}
@ -231,13 +257,38 @@ class Launcher {
* @return {!Array<string>}
*/
defaultArgs(options = {}) {
const chromeArguments = [
'--disable-background-networking',
'--enable-features=NetworkService,NetworkServiceInProcess',
'--disable-background-timer-throttling',
'--disable-backgrounding-occluded-windows',
'--disable-breakpad',
'--disable-client-side-phishing-detection',
'--disable-component-extensions-with-background-pages',
'--disable-default-apps',
'--disable-dev-shm-usage',
'--disable-extensions',
// BlinkGenPropertyTrees disabled due to crbug.com/937609
'--disable-features=TranslateUI,BlinkGenPropertyTrees',
'--disable-hang-monitor',
'--disable-ipc-flooding-protection',
'--disable-popup-blocking',
'--disable-prompt-on-repost',
'--disable-renderer-backgrounding',
'--disable-sync',
'--force-color-profile=srgb',
'--metrics-recording-only',
'--no-first-run',
'--enable-automation',
'--password-store=basic',
'--use-mock-keychain',
];
const {
devtools = false,
headless = !devtools,
args = [],
userDataDir = null
} = options;
const chromeArguments = [...DEFAULT_ARGS];
if (userDataDir)
chromeArguments.push(`--user-data-dir=${userDataDir}`);
if (devtools)
@ -259,7 +310,123 @@ class Launcher {
* @return {string}
*/
executablePath() {
return this._resolveExecutablePath().executablePath;
return resolveExecutablePath(this).executablePath;
}
/**
* @return {string}
*/
get product() {
return 'chrome';
}
/**
* @param {!(Launcher.BrowserOptions & {browserWSEndpoint?: string, browserURL?: string, transport?: !Puppeteer.ConnectionTransport})} options
* @return {!Promise<!Browser>}
*/
async connect(options) {
const {
browserWSEndpoint,
browserURL,
ignoreHTTPSErrors = false,
defaultViewport = {width: 800, height: 600},
transport,
slowMo = 0,
} = options;
assert(Number(!!browserWSEndpoint) + Number(!!browserURL) + Number(!!transport) === 1, 'Exactly one of browserWSEndpoint, browserURL or transport must be passed to puppeteer.connect');
let connection = null;
if (transport) {
connection = new Connection('', transport, slowMo);
} else if (browserWSEndpoint) {
const connectionTransport = await WebSocketTransport.create(browserWSEndpoint);
connection = new Connection(browserWSEndpoint, connectionTransport, slowMo);
} else if (browserURL) {
const connectionURL = await getWSEndpoint(browserURL);
const connectionTransport = await WebSocketTransport.create(connectionURL);
connection = new Connection(connectionURL, connectionTransport, slowMo);
}
const {browserContextIds} = await connection.send('Target.getBrowserContexts');
return Browser.create(connection, browserContextIds, ignoreHTTPSErrors, defaultViewport, null, () => connection.send('Browser.close').catch(debugError));
}
}
/**
* @implements {!Puppeteer.ProductLauncher}
*/
class FirefoxLauncher {
/**
* @param {string} projectRoot
* @param {string} preferredRevision
* @param {boolean} isPuppeteerCore
*/
constructor(projectRoot, preferredRevision, isPuppeteerCore) {
this._projectRoot = projectRoot;
this._preferredRevision = preferredRevision;
this._isPuppeteerCore = isPuppeteerCore;
}
/**
* @param {!(Launcher.LaunchOptions & Launcher.ChromeArgOptions & Launcher.BrowserOptions & {extraPrefsFirefox?: !object})=} options
* @return {!Promise<!Browser>}
*/
async launch(options = {}) {
const {
ignoreDefaultArgs = false,
args = [],
dumpio = false,
executablePath = null,
pipe = false,
env = process.env,
handleSIGINT = true,
handleSIGTERM = true,
handleSIGHUP = true,
ignoreHTTPSErrors = false,
defaultViewport = {width: 800, height: 600},
slowMo = 0,
timeout = 30000,
extraPrefsFirefox = {}
} = options;
const firefoxArguments = [];
if (!ignoreDefaultArgs)
firefoxArguments.push(...this.defaultArgs(options));
else if (Array.isArray(ignoreDefaultArgs))
firefoxArguments.push(...this.defaultArgs(options).filter(arg => !ignoreDefaultArgs.includes(arg)));
else
firefoxArguments.push(...args);
let temporaryUserDataDir = null;
if (!firefoxArguments.includes('-profile') && !firefoxArguments.includes('--profile')) {
temporaryUserDataDir = await this._createProfile(extraPrefsFirefox);
firefoxArguments.push('--profile');
firefoxArguments.push(temporaryUserDataDir);
}
let executable = executablePath;
if (!executablePath) {
const {missingText, executablePath} = resolveExecutablePath(this);
if (missingText)
throw new Error(missingText);
executable = executablePath;
}
const runner = new BrowserRunner(executable, firefoxArguments, temporaryUserDataDir);
runner.start({handleSIGHUP, handleSIGTERM, handleSIGINT, dumpio, env, pipe});
try {
const connection = await runner.setupConnection({usePipe: pipe, timeout, slowMo, preferredRevision: this._preferredRevision});
const browser = await Browser.create(connection, [], ignoreHTTPSErrors, defaultViewport, runner.proc, runner.close.bind(runner));
await browser.waitForTarget(t => t.type() === 'page');
return browser;
} catch (error) {
runner.kill();
throw error;
}
}
/**
@ -295,48 +462,92 @@ class Launcher {
}
/**
* @return {{executablePath: string, missingText: ?string}}
* @return {string}
*/
_resolveExecutablePath() {
// puppeteer-core doesn't take into account PUPPETEER_* env variables.
if (!this._isPuppeteerCore) {
const executablePath = process.env.PUPPETEER_EXECUTABLE_PATH || process.env.npm_config_puppeteer_executable_path || process.env.npm_package_config_puppeteer_executable_path;
if (executablePath) {
const missingText = !fs.existsSync(executablePath) ? 'Tried to use PUPPETEER_EXECUTABLE_PATH env variable to launch browser but did not find any executable at: ' + executablePath : null;
return { executablePath, missingText };
}
}
const browserFetcher = new BrowserFetcher(this._projectRoot);
if (!this._isPuppeteerCore) {
const revision = process.env['PUPPETEER_CHROMIUM_REVISION'];
if (revision) {
const revisionInfo = browserFetcher.revisionInfo(revision);
const missingText = !revisionInfo.local ? 'Tried to use PUPPETEER_CHROMIUM_REVISION env variable to launch browser but did not find executable at: ' + revisionInfo.executablePath : null;
return {executablePath: revisionInfo.executablePath, missingText};
}
}
const revisionInfo = browserFetcher.revisionInfo(this._preferredRevision);
const missingText = !revisionInfo.local ? `Chromium revision is not downloaded. Run "npm install" or "yarn install"` : null;
return {executablePath: revisionInfo.executablePath, missingText};
executablePath() {
const executablePath = process.env.PUPPETEER_EXECUTABLE_PATH || process.env.npm_config_puppeteer_executable_path || process.env.npm_package_config_puppeteer_executable_path;
// TODO get resolveExecutablePath working for Firefox
if (!executablePath)
throw new Error('Please set PUPPETEER_EXECUTABLE_PATH to a Firefox binary.');
return executablePath;
}
/**
* @return {string}
*/
get product() {
return 'firefox';
}
/**
* @param {!Launcher.ChromeArgOptions=} options
* @return {!Array<string>}
*/
defaultArgs(options = {}) {
const firefoxArguments = [
'--remote-debugging-port=0',
'--no-remote',
'--foreground',
];
const {
devtools = false,
headless = !devtools,
args = [],
userDataDir = null
} = options;
if (userDataDir) {
firefoxArguments.push('--profile');
firefoxArguments.push(userDataDir);
}
if (headless)
firefoxArguments.push('--headless');
if (devtools)
firefoxArguments.push('--devtools');
if (args.every(arg => arg.startsWith('-')))
firefoxArguments.push('about:blank');
firefoxArguments.push(...args);
return firefoxArguments;
}
/**
* @param {!Object=} extraPrefs
* @return {!Promise<string>}
*/
async _createProfile(extraPrefs) {
const profilePath = await mkdtempAsync(path.join(os.tmpdir(), 'puppeteer_dev_firefox_profile-'));
const prefsJs = [];
const userJs = [];
const preferences = {
// https://bugzilla.mozilla.org/show_bug.cgi?id=1544393
'remote.enabled': true,
// https://bugzilla.mozilla.org/show_bug.cgi?id=1543115
'browser.dom.window.dump.enabled': true
};
Object.assign(preferences, extraPrefs);
for (const [key, value] of Object.entries(preferences))
userJs.push(`user_pref(${JSON.stringify(key)}, ${JSON.stringify(value)});`);
await writeFileAsync(path.join(profilePath, 'user.js'), userJs.join('\n'));
await writeFileAsync(path.join(profilePath, 'prefs.js'), prefsJs.join('\n'));
return profilePath;
}
}
/**
* @param {!Puppeteer.ChildProcess} chromeProcess
* @param {!Puppeteer.ChildProcess} browserProcess
* @param {number} timeout
* @param {string} preferredRevision
* @return {!Promise<string>}
*/
function waitForWSEndpoint(chromeProcess, timeout, preferredRevision) {
function waitForWSEndpoint(browserProcess, timeout, preferredRevision) {
return new Promise((resolve, reject) => {
const rl = readline.createInterface({ input: chromeProcess.stderr });
const rl = readline.createInterface({ input: browserProcess.stderr });
let stderr = '';
const listeners = [
helper.addEventListener(rl, 'line', onLine),
helper.addEventListener(rl, 'close', () => onClose()),
helper.addEventListener(chromeProcess, 'exit', () => onClose()),
helper.addEventListener(chromeProcess, 'error', error => onClose(error))
helper.addEventListener(browserProcess, 'exit', () => onClose()),
helper.addEventListener(browserProcess, 'error', error => onClose(error))
];
const timeoutId = timeout ? setTimeout(onTimeout, timeout) : 0;
@ -346,7 +557,7 @@ function waitForWSEndpoint(chromeProcess, timeout, preferredRevision) {
function onClose(error) {
cleanup();
reject(new Error([
'Failed to launch chrome!' + (error ? ' ' + error.message : ''),
'Failed to launch the browser process!' + (error ? ' ' + error.message : ''),
stderr,
'',
'TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md',
@ -356,7 +567,7 @@ function waitForWSEndpoint(chromeProcess, timeout, preferredRevision) {
function onTimeout() {
cleanup();
reject(new TimeoutError(`Timed out after ${timeout} ms while trying to connect to Chrome! The only Chrome revision guaranteed to work is r${preferredRevision}`));
reject(new TimeoutError(`Timed out after ${timeout} ms while trying to connect to the browser! Only Chrome at revision r${preferredRevision} is guaranteed to work.`));
}
/**
@ -412,6 +623,55 @@ function getWSEndpoint(browserURL) {
});
}
/**
* @param {ChromeLauncher|FirefoxLauncher} launcher
*
* @return {{executablePath: string, missingText: ?string}}
*/
function resolveExecutablePath(launcher) {
// puppeteer-core doesn't take into account PUPPETEER_* env variables.
if (!launcher._isPuppeteerCore) {
const executablePath = process.env.PUPPETEER_EXECUTABLE_PATH || process.env.npm_config_puppeteer_executable_path || process.env.npm_package_config_puppeteer_executable_path;
if (executablePath) {
const missingText = !fs.existsSync(executablePath) ? 'Tried to use PUPPETEER_EXECUTABLE_PATH env variable to launch browser but did not find any executable at: ' + executablePath : null;
return { executablePath, missingText };
}
}
const browserFetcher = new BrowserFetcher(launcher._projectRoot);
if (!launcher._isPuppeteerCore) {
const revision = process.env['PUPPETEER_CHROMIUM_REVISION'];
if (revision) {
const revisionInfo = browserFetcher.revisionInfo(revision);
const missingText = !revisionInfo.local ? 'Tried to use PUPPETEER_CHROMIUM_REVISION env variable to launch browser but did not find executable at: ' + revisionInfo.executablePath : null;
return {executablePath: revisionInfo.executablePath, missingText};
}
}
const revisionInfo = browserFetcher.revisionInfo(launcher._preferredRevision);
const missingText = !revisionInfo.local ? `Browser is not downloaded. Run "npm install" or "yarn install"` : null;
return {executablePath: revisionInfo.executablePath, missingText};
}
/**
* @param {string} projectRoot
* @param {string} preferredRevision
* @param {boolean} isPuppeteerCore
* @param {string=} product
* @return {!Puppeteer.ProductLauncher}
*/
function Launcher(projectRoot, preferredRevision, isPuppeteerCore, product) {
// puppeteer-core doesn't take into account PUPPETEER_* env variables.
if (!product && !isPuppeteerCore)
product = process.env.PUPPETEER_PRODUCT || process.env.npm_config_puppeteer_product || process.env.npm_package_config_puppeteer_product;
switch (product) {
case 'firefox':
return new FirefoxLauncher(projectRoot, preferredRevision, isPuppeteerCore);
case 'chrome':
default:
return new ChromeLauncher(projectRoot, preferredRevision, isPuppeteerCore);
}
}
/**
* @typedef {Object} Launcher.ChromeArgOptions
* @property {boolean=} headless

24
lib/Puppeteer.js
View File

@ -26,14 +26,17 @@ module.exports = class {
*/
constructor(projectRoot, preferredRevision, isPuppeteerCore) {
this._projectRoot = projectRoot;
this._launcher = new Launcher(projectRoot, preferredRevision, isPuppeteerCore);
this._preferredRevision = preferredRevision;
this._isPuppeteerCore = isPuppeteerCore;
}
/**
* @param {!(Launcher.LaunchOptions & Launcher.ChromeArgOptions & Launcher.BrowserOptions)=} options
* @param {!(Launcher.LaunchOptions & Launcher.ChromeArgOptions & Launcher.BrowserOptions & {product?: string, extraPrefsFirefox?: !object})=} options
* @return {!Promise<!Puppeteer.Browser>}
*/
launch(options) {
if (!this._productName && options)
this._productName = options.product;
return this._launcher.launch(options);
}
@ -52,6 +55,23 @@ module.exports = class {
return this._launcher.executablePath();
}
/**
* @return {!Puppeteer.ProductLauncher}
*/
get _launcher() {
if (!this._lazyLauncher)
this._lazyLauncher = Launcher(this._projectRoot, this._preferredRevision, this._isPuppeteerCore, this._productName);
return this._lazyLauncher;
}
/**
* @return {string}
*/
get product() {
return this._launcher.product;
}
/**
* @return {Object}
*/

8
lib/externs.d.ts vendored
View File

@ -44,6 +44,14 @@ declare global {
onclose?: () => void,
}
export interface ProductLauncher {
launch(object)
connect(object)
executablePath: () => string,
defaultArgs(object)
product:string,
}
export interface ChildProcess extends child_process.ChildProcess { }
export type Viewport = {

2
lib/helper.js
View File

@ -137,7 +137,7 @@ class Helper {
static removeEventListeners(listeners) {
for (const listener of listeners)
listener.emitter.removeListener(listener.eventName, listener.handler);
listeners.splice(0, listeners.length);
listeners.length = 0;
}
/**

3
package.json
View File

@ -12,7 +12,8 @@
},
"scripts": {
"unit": "node test/test.js",
"funit": "BROWSER=firefox node test/test.js",
"fjunit": "PUPPETEER_PRODUCT=juggler node test/test.js",
"funit": "PUPPETEER_PRODUCT=firefox node test/test.js",
"debug-unit": "node --inspect-brk test/test.js",
"test-doclint": "node utils/doclint/check_public_api/test/test.js && node utils/doclint/preprocessor/test.js",
"test": "npm run lint --silent && npm run coverage && npm run test-doclint && npm run test-types && node utils/testrunner/test/test.js",

14
test/fixtures.spec.js
View File

@ -16,7 +16,7 @@
const path = require('path');
module.exports.addTests = function({testRunner, expect, defaultBrowserOptions, puppeteer, puppeteerPath, CHROME}) {
module.exports.addTests = function({testRunner, expect, defaultBrowserOptions, puppeteer, puppeteerPath, JUGGLER}) {
const {describe, xdescribe, fdescribe, describe_fails_ffox} = testRunner;
const {it, fit, xit, it_fails_ffox} = testRunner;
const {beforeAll, beforeEach, afterAll, afterEach} = testRunner;
@ -38,16 +38,16 @@ module.exports.addTests = function({testRunner, expect, defaultBrowserOptions, p
const options = Object.assign({}, defaultBrowserOptions, {dumpio: true});
const res = spawn('node',
[path.join(__dirname, 'fixtures', 'dumpio.js'), puppeteerPath, JSON.stringify(options)]);
if (CHROME)
res.stderr.on('data', data => dumpioData += data.toString('utf8'));
else
if (JUGGLER)
res.stdout.on('data', data => dumpioData += data.toString('utf8'));
else
res.stderr.on('data', data => dumpioData += data.toString('utf8'));
await new Promise(resolve => res.on('close', resolve));
if (CHROME)
expect(dumpioData).toContain('DevTools listening on ws://');
else
if (JUGGLER)
expect(dumpioData).toContain('Juggler listening on ws://');
else
expect(dumpioData).toContain('DevTools listening on ws://');
});
it('should close the browser when the node process closes', async({ server }) => {
const {spawn, execSync} = require('child_process');

8
test/launcher.spec.js
View File

@ -24,7 +24,7 @@ const statAsync = helper.promisify(fs.stat);
const TMP_FOLDER = path.join(os.tmpdir(), 'pptr_tmp_folder-');
const utils = require('./utils');
module.exports.addTests = function({testRunner, expect, defaultBrowserOptions, puppeteer, CHROME, puppeteerPath}) {
module.exports.addTests = function({testRunner, expect, defaultBrowserOptions, puppeteer, CHROME, FFOX, JUGGLER, puppeteerPath}) {
const {describe, xdescribe, fdescribe, describe_fails_ffox} = testRunner;
const {it, fit, xit, it_fails_ffox} = testRunner;
const {beforeAll, beforeEach, afterAll, afterEach} = testRunner;
@ -199,6 +199,12 @@ module.exports.addTests = function({testRunner, expect, defaultBrowserOptions, p
expect(puppeteer.defaultArgs({userDataDir: 'foo'})).toContain('foo');
}
});
it('should report the correct product', async() => {
if (CHROME)
expect(puppeteer.product).toBe('chrome');
else if (FFOX && !JUGGLER)
expect(puppeteer.product).toBe('firefox');
});
it('should work with no default arguments', async() => {
const options = Object.assign({}, defaultBrowserOptions);
options.ignoreDefaultArgs = true;

30
test/puppeteer.spec.js
View File

@ -28,31 +28,40 @@ module.exports.addTests = ({testRunner, product, puppeteerPath}) => {
const {beforeAll, beforeEach, afterAll, afterEach} = testRunner;
const CHROME = product === 'Chromium';
const FFOX = product === 'Firefox';
const FFOX = (product === 'Firefox' || product === 'Juggler');
const JUGGLER = product === 'Juggler';
const puppeteer = require(puppeteerPath);
const headless = (process.env.HEADLESS || 'true').trim().toLowerCase() === 'true';
const slowMo = parseInt((process.env.SLOW_MO || '0').trim(), 10);
let extraLaunchOptions = {};
try {
extraLaunchOptions = JSON.parse(process.env.EXTRA_LAUNCH_OPTIONS || '{}');
} catch (error) {
console.warn(`${YELLOW_COLOR}Error parsing EXTRA_LAUNCH_OPTIONS: ${error.message}. Skipping.${RESET_COLOR}`);
}
const defaultBrowserOptions = {
const defaultBrowserOptions = Object.assign({
handleSIGINT: false,
executablePath: CHROME ? process.env.CHROME : process.env.FFOX,
executablePath: process.env.BINARY,
slowMo,
headless,
dumpio: !!process.env.DUMPIO,
};
}, extraLaunchOptions);
if (defaultBrowserOptions.executablePath) {
console.warn(`${YELLOW_COLOR}WARN: running ${product} tests with ${defaultBrowserOptions.executablePath}${RESET_COLOR}`);
} else {
// Make sure the `npm install` was run after the chromium roll.
if (!fs.existsSync(puppeteer.executablePath()))
throw new Error(`Browser is not downloaded. Run 'npm install' and try to re-run tests`);
const executablePath = puppeteer.executablePath();
if (!fs.existsSync(executablePath))
throw new Error(`Browser is not downloaded at ${executablePath}. Run 'npm install' and try to re-run tests`);
}
const GOLDEN_DIR = path.join(__dirname, 'golden-' + product.toLowerCase());
const OUTPUT_DIR = path.join(__dirname, 'output-' + product.toLowerCase());
const suffix = JUGGLER ? 'firefox' : product.toLowerCase();
const GOLDEN_DIR = path.join(__dirname, 'golden-' + suffix);
const OUTPUT_DIR = path.join(__dirname, 'output-' + suffix);
if (fs.existsSync(OUTPUT_DIR))
rm(OUTPUT_DIR);
const {expect} = new Matchers({
@ -64,6 +73,7 @@ module.exports.addTests = ({testRunner, product, puppeteerPath}) => {
product,
FFOX,
CHROME,
JUGGLER,
puppeteer,
expect,
defaultBrowserOptions,
@ -72,7 +82,7 @@ module.exports.addTests = ({testRunner, product, puppeteerPath}) => {
};
beforeAll(async() => {
if (FFOX && defaultBrowserOptions.executablePath)
if (JUGGLER && defaultBrowserOptions.executablePath)
await require('../experimental/puppeteer-firefox/misc/install-preferences')(defaultBrowserOptions.executablePath);
});

56
test/test.js
View File

@ -76,28 +76,44 @@ const CHROMIUM_NO_COVERAGE = new Set([
'page.emulateMedia', // Legacy alias for `page.emulateMediaType`.
]);
if (process.env.BROWSER === 'firefox') {
testRunner.addTestDSL('it_fails_ffox', 'skip');
testRunner.addSuiteDSL('describe_fails_ffox', 'skip');
describe('Firefox', () => {
require('./puppeteer.spec.js').addTests({
product: 'Firefox',
puppeteerPath: path.resolve(__dirname, '../experimental/puppeteer-firefox/'),
testRunner,
switch (process.env.PUPPETEER_PRODUCT) {
case 'firefox':
testRunner.addTestDSL('it_fails_ffox', 'skip');
testRunner.addSuiteDSL('describe_fails_ffox', 'skip');
describe('Firefox', () => {
require('./puppeteer.spec.js').addTests({
product: 'Firefox',
puppeteerPath: utils.projectRoot(),
testRunner,
});
if (process.env.COVERAGE)
utils.recordAPICoverage(testRunner, require('../lib/api'), require('../lib/Events').Events, CHROMIUM_NO_COVERAGE);
});
});
} else {
testRunner.addTestDSL('it_fails_ffox', 'run');
testRunner.addSuiteDSL('describe_fails_ffox', 'run');
describe('Chromium', () => {
require('./puppeteer.spec.js').addTests({
product: 'Chromium',
puppeteerPath: utils.projectRoot(),
testRunner,
break;
case 'juggler':
testRunner.addTestDSL('it_fails_ffox', 'skip');
testRunner.addSuiteDSL('describe_fails_ffox', 'skip');
describe('Firefox (Juggler)', () => {
require('./puppeteer.spec.js').addTests({
product: 'Juggler',
puppeteerPath: path.resolve(__dirname, '../experimental/puppeteer-firefox/'),
testRunner,
});
});
break;
case 'chrome':
default:
testRunner.addTestDSL('it_fails_ffox', 'run');
testRunner.addSuiteDSL('describe_fails_ffox', 'run');
describe('Chromium', () => {
require('./puppeteer.spec.js').addTests({
product: 'Chromium',
puppeteerPath: utils.projectRoot(),
testRunner,
});
if (process.env.COVERAGE)
utils.recordAPICoverage(testRunner, require('../lib/api'), require('../lib/Events').Events, CHROMIUM_NO_COVERAGE);
});
if (process.env.COVERAGE)
utils.recordAPICoverage(testRunner, require('../lib/api'), require('../lib/Events').Events, CHROMIUM_NO_COVERAGE);
});
}
if (process.env.CI && testRunner.hasFocusedTestsOrSuites()) {

2
utils/protocol-types-generator/index.js
View File

@ -3,7 +3,7 @@ const path = require('path');
const puppeteer = require('../..');
module.exports = puppeteer.launch({
pipe: false,
executablePath: process.env.CHROME,
executablePath: process.env.BINARY,
}).then(async browser => {
const origin = browser.wsEndpoint().match(/ws:\/\/([0-9A-Za-z:\.]*)\//)[1];
const page = await browser.newPage();