feat(launcher): add option to run Puppeteer with different browsers (#5137)

* feat: Set which browser to launch via PUPPETEER_PRODUCT

This change introduces a PUPPETEER_PRODUCT environment
variable as a first step toward using Puppeteer with
many different browsers. Setting PUPPETEER_PRODUCT=firefox, for
example, enables Firefox-specific Launcher settings.

The state is also exposed as `puppeteer.product` in the API
to support adding other product-specific behaviour as needed.

The bulk of the change is a refactoring in Launcher
to decouple generic browser start-up from product-specific
configuration.

Respecting the puppeteer-core restriction for PUPPETEER_
environment variables, lazily instantiate the Launcher
based on a `product` Puppeteer.launch option, if available.

* test: Distinguish Juggler unit tests from Firefox

The funit script is renamed to fjunit (j for Juggler, which is
used only by the experimental puppeteer-firefox package.

In contrast, the funit script now refers to running Puppeteer
unit tests against the main puppeteer package with Firefox.
To do so with Firefox Nightly, run:

`BINARY=path/to/firefox npm run funit`

A number of changes in this patch make it easier to run
Puppeteer unit tests in Mozilla's CI.
This commit is contained in:
Maja Frydrychowicz 2019-11-26 04:23:19 -05:00 committed by Mathias Bynens
parent d17708ba1f
commit c5a72e9887
16 changed files with 559 additions and 214 deletions

View File

@ -24,12 +24,12 @@ task:
task:
matrix:
- name: Firefox (node8 + linux)
- name: Firefox Juggler (node8 + linux)
container:
dockerfile: .ci/node8/Dockerfile.linux
xvfb_start_background_script: Xvfb :99 -ac -screen 0 1024x768x24
install_script: npm install --unsafe-perm && cd experimental/puppeteer-firefox && npm install --unsafe-perm
test_script: npm run funit
test_script: npm run fjunit
task:
osx_instance:

View File

@ -19,7 +19,7 @@ script:
- 'if [ "$NODE8" = "true" ]; then npm run lint; fi'
- 'if [ "$NODE8" = "true" ]; then npm run coverage; fi'
- 'if [ "$FIREFOX" = "true" ]; then cd experimental/puppeteer-firefox && npm i && cd ../..; fi'
- 'if [ "$FIREFOX" = "true" ]; then npm run funit; fi'
- 'if [ "$FIREFOX" = "true" ]; then npm run fjunit; fi'
- 'if [ "$NODE8" = "true" ]; then npm run test-doclint; fi'
- 'if [ "$NODE8" = "true" ]; then npm run test-types; fi'
- 'if [ "$NODE8" = "true" ]; then npm run bundle; fi'

View File

@ -199,10 +199,10 @@ npm run unit -- --break-on-failure
HEADLESS=false npm run unit
```
- To run tests with custom Chromium executable:
- To run tests with custom browser executable:
```bash
CHROME=<path-to-executable> npm run unit
BINARY=<path-to-executable> npm run unit
```
- To run tests in slow-mode:
@ -211,6 +211,13 @@ CHROME=<path-to-executable> npm run unit
HEADLESS=false SLOW_MO=500 npm run unit
```
- To run tests with additional Launcher options:
```bash
EXTRA_LAUNCH_OPTIONS='{"args": ["--user-data-dir=some/path"], "handleSIGINT": true}' npm run unit
```
- To debug a test, "focus" a test first and then run:
```bash

View File

@ -31,6 +31,7 @@
* [puppeteer.errors](#puppeteererrors)
* [puppeteer.executablePath()](#puppeteerexecutablepath)
* [puppeteer.launch([options])](#puppeteerlaunchoptions)
* [puppeteer.product](#puppeteerproduct)
- [class: BrowserFetcher](#class-browserfetcher)
* [browserFetcher.canDownload(revision)](#browserfetchercandownloadrevision)
* [browserFetcher.download(revision[, progressCallback])](#browserfetcherdownloadrevision-progresscallback)
@ -388,6 +389,7 @@ If Puppeteer doesn't find them in the environment during the installation step,
- `PUPPETEER_DOWNLOAD_HOST` - overwrite URL prefix that is used to download Chromium. Note: this includes protocol and might even include path prefix. Defaults to `https://storage.googleapis.com`.
- `PUPPETEER_CHROMIUM_REVISION` - specify a certain version of Chromium you'd like Puppeteer to use. See [puppeteer.launch([options])](#puppeteerlaunchoptions) on how executable path is inferred. **BEWARE**: Puppeteer is only [guaranteed to work](https://github.com/GoogleChrome/puppeteer/#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) with the bundled Chromium, use at your own risk.
- `PUPPETEER_EXECUTABLE_PATH` - specify an executable path to be used in `puppeteer.launch`. See [puppeteer.launch([options])](#puppeteerlaunchoptions) on how the executable path is inferred. **BEWARE**: Puppeteer is only [guaranteed to work](https://github.com/GoogleChrome/puppeteer/#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) with the bundled Chromium, use at your own risk.
- `PUPPETEER_PRODUCT` - specify which browser you'd like Puppeteer to use. Must be one of `chrome` or `firefox`. Setting `product` programmatically in [puppeteer.launch([options])](#puppeteerlaunchoptions) supercedes this environment variable. The product is exposed in [`puppeteer.product`](#puppeteerproduct)
> **NOTE** PUPPETEER_* env variables are not accounted for in the [`puppeteer-core`](https://www.npmjs.com/package/puppeteer-core) package.
@ -525,9 +527,10 @@ try {
#### puppeteer.launch([options])
- `options` <[Object]> Set of configurable options to set on the browser. Can have the following fields:
- `product` <[string]> Which browser to launch. At this time, this is either `chrome` or `firefox`. See also `PUPPETEER_PRODUCT`.
- `ignoreHTTPSErrors` <[boolean]> Whether to ignore HTTPS errors during navigation. Defaults to `false`.
- `headless` <[boolean]> Whether to run browser in [headless mode](https://developers.google.com/web/updates/2017/04/headless-chrome). Defaults to `true` unless the `devtools` option is `true`.
- `executablePath` <[string]> Path to a Chromium or Chrome executable to run instead of the bundled Chromium. If `executablePath` is a relative path, then it is resolved relative to [current working directory](https://nodejs.org/api/process.html#process_process_cwd). **BEWARE**: Puppeteer is only [guaranteed to work](https://github.com/GoogleChrome/puppeteer/#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) with the bundled Chromium, use at your own risk.
- `executablePath` <[string]> Path to a browser executable to run instead of the bundled Chromium. If `executablePath` is a relative path, then it is resolved relative to [current working directory](https://nodejs.org/api/process.html#process_process_cwd). **BEWARE**: Puppeteer is only [guaranteed to work](https://github.com/GoogleChrome/puppeteer/#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) with the bundled Chromium, use at your own risk.
- `slowMo` <[number]> Slows down Puppeteer operations by the specified amount of milliseconds. Useful so that you can see what is going on.
- `defaultViewport` <?[Object]> Sets a consistent viewport for each page. Defaults to an 800x600 viewport. `null` disables the default viewport.
- `width` <[number]> page width in pixels.
@ -536,7 +539,7 @@ try {
- `isMobile` <[boolean]> Whether the `meta viewport` tag is taken into account. Defaults to `false`.
- `hasTouch`<[boolean]> Specifies if viewport supports touch events. Defaults to `false`
- `isLandscape` <[boolean]> Specifies if viewport is in landscape mode. Defaults to `false`.
- `args` <[Array]<[string]>> Additional arguments to pass to the browser instance. The list of Chromium flags can be found [here](http://peter.sh/experiments/chromium-command-line-switches/).
- `args` <[Array]<[string]>> Additional arguments to pass to the browser instance. The list of Chromium flags can be found [here](http://peter.sh/experiments/chromium-command-line-switches/), and here is the list of [Firefox flags](https://developer.mozilla.org/en-US/docs/Mozilla/Command_Line_Options).
- `ignoreDefaultArgs` <[boolean]|[Array]<[string]>> If `true`, then do not use [`puppeteer.defaultArgs()`](#puppeteerdefaultargsoptions). If an array is given, then filter out the given default arguments. Dangerous option; use with care. Defaults to `false`.
- `handleSIGINT` <[boolean]> Close the browser process on Ctrl-C. Defaults to `true`.
- `handleSIGTERM` <[boolean]> Close the browser process on SIGTERM. Defaults to `true`.
@ -547,6 +550,7 @@ try {
- `env` <[Object]> Specify environment variables that will be visible to the browser. Defaults to `process.env`.
- `devtools` <[boolean]> Whether to auto-open a DevTools panel for each tab. If this option is `true`, the `headless` option will be set `false`.
- `pipe` <[boolean]> Connects to the browser over a pipe instead of a WebSocket. Defaults to `false`.
- `extraPrefsFirefox` <[Object]> Additional [preferences](https://developer.mozilla.org/en-US/docs/Mozilla/Preferences/Preference_reference) that can be passed to Firefox (see `PUPPETEER_PRODUCT`)
- returns: <[Promise]<[Browser]>> Promise which resolves to browser instance.
@ -565,6 +569,12 @@ const browser = await puppeteer.launch({
>
> See [`this article`](https://www.howtogeek.com/202825/what%E2%80%99s-the-difference-between-chromium-and-chrome/) for a description of the differences between Chromium and Chrome. [`This article`](https://chromium.googlesource.com/chromium/src/+/lkgr/docs/chromium_browser_vs_google_chrome.md) describes some differences for Linux users.
#### puppeteer.product
- returns: <[string]> returns the name of the browser that is under automation ("chrome" or "firefox")
The product is set by the `PUPPETEER_PRODUCT` environment variable or the `product` option in [puppeteer.launch([options])](#puppeteerlaunchoptions) and defaults to `chrome`. Firefox support is experimental.
### class: BrowserFetcher
BrowserFetcher can download and manage different versions of Chromium.

View File

@ -7,7 +7,7 @@ task:
dockerfile: .ci/node8/Dockerfile.linux
xvfb_start_background_script: Xvfb :99 -ac -screen 0 1024x768x24
install_script: npm install
test_script: npm run funit
test_script: npm run fjunit
task:
name: node8 (macOS)
@ -19,7 +19,7 @@ task:
- brew install node@8
- brew link --force node@8
install_script: npm install
test_script: npm run funit
test_script: npm run fjunit
# task:
# allow_failures: true
@ -28,4 +28,4 @@ task:
# os_version: 2016
# name: node8 (windows)
# install_script: npm install --unsafe-perm
# test_script: npm run funit
# test_script: npm run fjunit

View File

@ -28,4 +28,11 @@ const packageJson = require('./package.json');
const preferredRevision = packageJson.puppeteer.chromium_revision;
const isPuppeteerCore = packageJson.name === 'puppeteer-core';
module.exports = new Puppeteer(__dirname, preferredRevision, isPuppeteerCore);
const puppeteer = new Puppeteer(__dirname, preferredRevision, isPuppeteerCore);
// The introspection in `Helper.installAsyncStackHooks` references `Puppeteer._launcher`
// before the Puppeteer ctor is called, such that an invalid Launcher is selected at import,
// so we reset it.
puppeteer._lazyLauncher = undefined;
module.exports = puppeteer;

View File

@ -26,43 +26,159 @@ const {Browser} = require('./Browser');
const readline = require('readline');
const fs = require('fs');
const {helper, assert, debugError} = require('./helper');
const debugLauncher = require('debug')(`puppeteer:launcher`);
const {TimeoutError} = require('./Errors');
const WebSocketTransport = require('./WebSocketTransport');
const PipeTransport = require('./PipeTransport');
const mkdtempAsync = helper.promisify(fs.mkdtemp);
const removeFolderAsync = helper.promisify(removeFolder);
const writeFileAsync = helper.promisify(fs.writeFile);
const CHROME_PROFILE_PATH = path.join(os.tmpdir(), 'puppeteer_dev_profile-');
class BrowserRunner {
const DEFAULT_ARGS = [
'--disable-background-networking',
'--enable-features=NetworkService,NetworkServiceInProcess',
'--disable-background-timer-throttling',
'--disable-backgrounding-occluded-windows',
'--disable-breakpad',
'--disable-client-side-phishing-detection',
'--disable-component-extensions-with-background-pages',
'--disable-default-apps',
'--disable-dev-shm-usage',
'--disable-extensions',
// BlinkGenPropertyTrees disabled due to crbug.com/937609
'--disable-features=TranslateUI,BlinkGenPropertyTrees',
'--disable-hang-monitor',
'--disable-ipc-flooding-protection',
'--disable-popup-blocking',
'--disable-prompt-on-repost',
'--disable-renderer-backgrounding',
'--disable-sync',
'--force-color-profile=srgb',
'--metrics-recording-only',
'--no-first-run',
'--enable-automation',
'--password-store=basic',
'--use-mock-keychain',
];
/**
* @param {string} executablePath
* @param {!Array<string>} processArguments
* @param {string=} tempDirectory
*/
constructor(executablePath, processArguments, tempDirectory) {
this._executablePath = executablePath;
this._processArguments = processArguments;
this._tempDirectory = tempDirectory;
this.proc = null;
this.connection = null;
this._closed = true;
this._listeners = [];
}
class Launcher {
/**
* @param {!(Launcher.LaunchOptions)=} options
*/
start(options = {}) {
const {
handleSIGINT,
handleSIGTERM,
handleSIGHUP,
dumpio,
env,
pipe
} = options;
/** @type {!Array<"ignore"|"pipe">} */
let stdio = ['pipe', 'pipe', 'pipe'];
if (pipe) {
if (dumpio)
stdio = ['ignore', 'pipe', 'pipe', 'pipe', 'pipe'];
else
stdio = ['ignore', 'ignore', 'ignore', 'pipe', 'pipe'];
}
assert(!this.proc, 'This process has previously been started.');
debugLauncher(`Calling ${this._executablePath} ${this._processArguments.join(' ')}`);
this.proc = childProcess.spawn(
this._executablePath,
this._processArguments,
{
// On non-windows platforms, `detached: true` makes child process a leader of a new
// process group, making it possible to kill child process tree with `.kill(-pid)` command.
// @see https://nodejs.org/api/child_process.html#child_process_options_detached
detached: process.platform !== 'win32',
env,
stdio
}
);
if (dumpio) {
this.proc.stderr.pipe(process.stderr);
this.proc.stdout.pipe(process.stdout);
}
this._closed = false;
this._processClosing = new Promise((fulfill, reject) => {
this.proc.once('exit', () => {
this._closed = true;
// Cleanup as processes exit.
if (this._tempDirectory) {
removeFolderAsync(this._tempDirectory)
.then(() => fulfill())
.catch(err => console.error(err));
} else {
fulfill();
}
});
});
this._listeners = [ helper.addEventListener(process, 'exit', this.kill.bind(this)) ];
if (handleSIGINT)
this._listeners.push(helper.addEventListener(process, 'SIGINT', () => { this.kill(); process.exit(130); }));
if (handleSIGTERM)
this._listeners.push(helper.addEventListener(process, 'SIGTERM', this.close.bind(this)));
if (handleSIGHUP)
this._listeners.push(helper.addEventListener(process, 'SIGHUP', this.close.bind(this)));
}
/**
* @return {Promise}
*/
close() {
if (this._closed)
return Promise.resolve();
helper.removeEventListeners(this._listeners);
if (this._tempDirectory) {
this.kill();
} else if (this.connection) {
// Attempt to close the browser gracefully
this.connection.send('Browser.close').catch(error => {
debugError(error);
this.kill();
});
}
return this._processClosing;
}
// This function has to be sync to be used as 'exit' event handler.
kill() {
helper.removeEventListeners(this._listeners);
if (this.proc && this.proc.pid && !this.proc.killed && !this._closed) {
try {
if (process.platform === 'win32')
childProcess.execSync(`taskkill /pid ${this.proc.pid} /T /F`);
else
process.kill(-this.proc.pid, 'SIGKILL');
} catch (error) {
// the process might have already stopped
}
}
// Attempt to remove temporary profile directory to avoid littering.
try {
removeFolder.sync(this._tempDirectory);
} catch (error) { }
}
/**
* @param {!({usePipe?: boolean, timeout: number, slowMo: number, preferredRevision: string})} options
*
* @return {!Promise<!Connection>}
*/
async setupConnection(options) {
const {
usePipe,
timeout,
slowMo,
preferredRevision
} = options;
if (!usePipe) {
const browserWSEndpoint = await waitForWSEndpoint(this.proc, timeout, preferredRevision);
const transport = await WebSocketTransport.create(browserWSEndpoint);
this.connection = new Connection(browserWSEndpoint, transport, slowMo);
} else {
const transport = new PipeTransport(/** @type {!NodeJS.WritableStream} */(this.proc.stdio[3]), /** @type {!NodeJS.ReadableStream} */ (this.proc.stdio[4]));
this.connection = new Connection('', transport, slowMo);
}
return this.connection;
}
}
/**
* @implements {!Puppeteer.ProductLauncher}
*/
class ChromeLauncher {
/**
* @param {string} projectRoot
* @param {string} preferredRevision
@ -95,11 +211,12 @@ class Launcher {
timeout = 30000
} = options;
const profilePath = path.join(os.tmpdir(), 'puppeteer_dev_chrome_profile-');
const chromeArguments = [];
if (!ignoreDefaultArgs)
chromeArguments.push(...this.defaultArgs(options));
else if (Array.isArray(ignoreDefaultArgs))
chromeArguments.push(...this.defaultArgs(options).filter(arg => ignoreDefaultArgs.indexOf(arg) === -1));
chromeArguments.push(...this.defaultArgs(options).filter(arg => !ignoreDefaultArgs.includes(arg)));
else
chromeArguments.push(...args);
@ -108,121 +225,30 @@ class Launcher {
if (!chromeArguments.some(argument => argument.startsWith('--remote-debugging-')))
chromeArguments.push(pipe ? '--remote-debugging-pipe' : '--remote-debugging-port=0');
if (!chromeArguments.some(arg => arg.startsWith('--user-data-dir'))) {
temporaryUserDataDir = await mkdtempAsync(CHROME_PROFILE_PATH);
temporaryUserDataDir = await mkdtempAsync(profilePath);
chromeArguments.push(`--user-data-dir=${temporaryUserDataDir}`);
}
let chromeExecutable = executablePath;
if (!executablePath) {
const {missingText, executablePath} = this._resolveExecutablePath();
const {missingText, executablePath} = resolveExecutablePath(this);
if (missingText)
throw new Error(missingText);
chromeExecutable = executablePath;
}
const usePipe = chromeArguments.includes('--remote-debugging-pipe');
/** @type {!Array<"ignore"|"pipe">} */
let stdio = ['pipe', 'pipe', 'pipe'];
if (usePipe) {
if (dumpio)
stdio = ['ignore', 'pipe', 'pipe', 'pipe', 'pipe'];
else
stdio = ['ignore', 'ignore', 'ignore', 'pipe', 'pipe'];
}
const chromeProcess = childProcess.spawn(
chromeExecutable,
chromeArguments,
{
// On non-windows platforms, `detached: true` makes child process a leader of a new
// process group, making it possible to kill child process tree with `.kill(-pid)` command.
// @see https://nodejs.org/api/child_process.html#child_process_options_detached
detached: process.platform !== 'win32',
env,
stdio
}
);
const runner = new BrowserRunner(chromeExecutable, chromeArguments, temporaryUserDataDir);
runner.start({handleSIGHUP, handleSIGTERM, handleSIGINT, dumpio, env, pipe: usePipe});
if (dumpio) {
chromeProcess.stderr.pipe(process.stderr);
chromeProcess.stdout.pipe(process.stdout);
}
let chromeClosed = false;
const waitForChromeToClose = new Promise((fulfill, reject) => {
chromeProcess.once('exit', () => {
chromeClosed = true;
// Cleanup as processes exit.
if (temporaryUserDataDir) {
removeFolderAsync(temporaryUserDataDir)
.then(() => fulfill())
.catch(err => console.error(err));
} else {
fulfill();
}
});
});
const listeners = [ helper.addEventListener(process, 'exit', killChrome) ];
if (handleSIGINT)
listeners.push(helper.addEventListener(process, 'SIGINT', () => { killChrome(); process.exit(130); }));
if (handleSIGTERM)
listeners.push(helper.addEventListener(process, 'SIGTERM', gracefullyCloseChrome));
if (handleSIGHUP)
listeners.push(helper.addEventListener(process, 'SIGHUP', gracefullyCloseChrome));
/** @type {?Connection} */
let connection = null;
try {
if (!usePipe) {
const browserWSEndpoint = await waitForWSEndpoint(chromeProcess, timeout, this._preferredRevision);
const transport = await WebSocketTransport.create(browserWSEndpoint);
connection = new Connection(browserWSEndpoint, transport, slowMo);
} else {
const transport = new PipeTransport(/** @type {!NodeJS.WritableStream} */(chromeProcess.stdio[3]), /** @type {!NodeJS.ReadableStream} */ (chromeProcess.stdio[4]));
connection = new Connection('', transport, slowMo);
}
const browser = await Browser.create(connection, [], ignoreHTTPSErrors, defaultViewport, chromeProcess, gracefullyCloseChrome);
const connection = await runner.setupConnection({usePipe, timeout, slowMo, preferredRevision: this._preferredRevision});
const browser = await Browser.create(connection, [], ignoreHTTPSErrors, defaultViewport, runner.proc, runner.close.bind(runner));
await browser.waitForTarget(t => t.type() === 'page');
return browser;
} catch (e) {
killChrome();
throw e;
}
/**
* @return {Promise}
*/
function gracefullyCloseChrome() {
helper.removeEventListeners(listeners);
if (temporaryUserDataDir) {
killChrome();
} else if (connection) {
// Attempt to close chrome gracefully
connection.send('Browser.close').catch(error => {
debugError(error);
killChrome();
});
}
return waitForChromeToClose;
}
// This method has to be sync to be used as 'exit' event handler.
function killChrome() {
helper.removeEventListeners(listeners);
if (chromeProcess.pid && !chromeProcess.killed && !chromeClosed) {
// Force kill chrome.
try {
if (process.platform === 'win32')
childProcess.execSync(`taskkill /pid ${chromeProcess.pid} /T /F`);
else
process.kill(-chromeProcess.pid, 'SIGKILL');
} catch (e) {
// the process might have already stopped
}
}
// Attempt to remove temporary profile directory to avoid littering.
try {
removeFolder.sync(temporaryUserDataDir);
} catch (e) { }
} catch (error) {
runner.kill();
throw error;
}
}
@ -231,13 +257,38 @@ class Launcher {
* @return {!Array<string>}
*/
defaultArgs(options = {}) {
const chromeArguments = [
'--disable-background-networking',
'--enable-features=NetworkService,NetworkServiceInProcess',
'--disable-background-timer-throttling',
'--disable-backgrounding-occluded-windows',
'--disable-breakpad',
'--disable-client-side-phishing-detection',
'--disable-component-extensions-with-background-pages',
'--disable-default-apps',
'--disable-dev-shm-usage',
'--disable-extensions',
// BlinkGenPropertyTrees disabled due to crbug.com/937609
'--disable-features=TranslateUI,BlinkGenPropertyTrees',
'--disable-hang-monitor',
'--disable-ipc-flooding-protection',
'--disable-popup-blocking',
'--disable-prompt-on-repost',
'--disable-renderer-backgrounding',
'--disable-sync',
'--force-color-profile=srgb',
'--metrics-recording-only',
'--no-first-run',
'--enable-automation',
'--password-store=basic',
'--use-mock-keychain',
];
const {
devtools = false,
headless = !devtools,
args = [],
userDataDir = null
} = options;
const chromeArguments = [...DEFAULT_ARGS];
if (userDataDir)
chromeArguments.push(`--user-data-dir=${userDataDir}`);
if (devtools)
@ -259,7 +310,123 @@ class Launcher {
* @return {string}
*/
executablePath() {
return this._resolveExecutablePath().executablePath;
return resolveExecutablePath(this).executablePath;
}
/**
* @return {string}
*/
get product() {
return 'chrome';
}
/**
* @param {!(Launcher.BrowserOptions & {browserWSEndpoint?: string, browserURL?: string, transport?: !Puppeteer.ConnectionTransport})} options
* @return {!Promise<!Browser>}
*/
async connect(options) {
const {
browserWSEndpoint,
browserURL,
ignoreHTTPSErrors = false,
defaultViewport = {width: 800, height: 600},
transport,
slowMo = 0,
} = options;
assert(Number(!!browserWSEndpoint) + Number(!!browserURL) + Number(!!transport) === 1, 'Exactly one of browserWSEndpoint, browserURL or transport must be passed to puppeteer.connect');
let connection = null;
if (transport) {
connection = new Connection('', transport, slowMo);
} else if (browserWSEndpoint) {
const connectionTransport = await WebSocketTransport.create(browserWSEndpoint);
connection = new Connection(browserWSEndpoint, connectionTransport, slowMo);
} else if (browserURL) {
const connectionURL = await getWSEndpoint(browserURL);
const connectionTransport = await WebSocketTransport.create(connectionURL);
connection = new Connection(connectionURL, connectionTransport, slowMo);
}
const {browserContextIds} = await connection.send('Target.getBrowserContexts');
return Browser.create(connection, browserContextIds, ignoreHTTPSErrors, defaultViewport, null, () => connection.send('Browser.close').catch(debugError));
}
}
/**
* @implements {!Puppeteer.ProductLauncher}
*/
class FirefoxLauncher {
/**
* @param {string} projectRoot
* @param {string} preferredRevision
* @param {boolean} isPuppeteerCore
*/
constructor(projectRoot, preferredRevision, isPuppeteerCore) {
this._projectRoot = projectRoot;
this._preferredRevision = preferredRevision;
this._isPuppeteerCore = isPuppeteerCore;
}
/**
* @param {!(Launcher.LaunchOptions & Launcher.ChromeArgOptions & Launcher.BrowserOptions & {extraPrefsFirefox?: !object})=} options
* @return {!Promise<!Browser>}
*/
async launch(options = {}) {
const {
ignoreDefaultArgs = false,
args = [],
dumpio = false,
executablePath = null,
pipe = false,
env = process.env,
handleSIGINT = true,
handleSIGTERM = true,
handleSIGHUP = true,
ignoreHTTPSErrors = false,
defaultViewport = {width: 800, height: 600},
slowMo = 0,
timeout = 30000,
extraPrefsFirefox = {}
} = options;
const firefoxArguments = [];
if (!ignoreDefaultArgs)
firefoxArguments.push(...this.defaultArgs(options));
else if (Array.isArray(ignoreDefaultArgs))
firefoxArguments.push(...this.defaultArgs(options).filter(arg => !ignoreDefaultArgs.includes(arg)));
else
firefoxArguments.push(...args);
let temporaryUserDataDir = null;
if (!firefoxArguments.includes('-profile') && !firefoxArguments.includes('--profile')) {
temporaryUserDataDir = await this._createProfile(extraPrefsFirefox);
firefoxArguments.push('--profile');
firefoxArguments.push(temporaryUserDataDir);
}
let executable = executablePath;
if (!executablePath) {
const {missingText, executablePath} = resolveExecutablePath(this);
if (missingText)
throw new Error(missingText);
executable = executablePath;
}
const runner = new BrowserRunner(executable, firefoxArguments, temporaryUserDataDir);
runner.start({handleSIGHUP, handleSIGTERM, handleSIGINT, dumpio, env, pipe});
try {
const connection = await runner.setupConnection({usePipe: pipe, timeout, slowMo, preferredRevision: this._preferredRevision});
const browser = await Browser.create(connection, [], ignoreHTTPSErrors, defaultViewport, runner.proc, runner.close.bind(runner));
await browser.waitForTarget(t => t.type() === 'page');
return browser;
} catch (error) {
runner.kill();
throw error;
}
}
/**
@ -295,48 +462,92 @@ class Launcher {
}
/**
* @return {{executablePath: string, missingText: ?string}}
* @return {string}
*/
_resolveExecutablePath() {
// puppeteer-core doesn't take into account PUPPETEER_* env variables.
if (!this._isPuppeteerCore) {
executablePath() {
const executablePath = process.env.PUPPETEER_EXECUTABLE_PATH || process.env.npm_config_puppeteer_executable_path || process.env.npm_package_config_puppeteer_executable_path;
if (executablePath) {
const missingText = !fs.existsSync(executablePath) ? 'Tried to use PUPPETEER_EXECUTABLE_PATH env variable to launch browser but did not find any executable at: ' + executablePath : null;
return { executablePath, missingText };
}
}
const browserFetcher = new BrowserFetcher(this._projectRoot);
if (!this._isPuppeteerCore) {
const revision = process.env['PUPPETEER_CHROMIUM_REVISION'];
if (revision) {
const revisionInfo = browserFetcher.revisionInfo(revision);
const missingText = !revisionInfo.local ? 'Tried to use PUPPETEER_CHROMIUM_REVISION env variable to launch browser but did not find executable at: ' + revisionInfo.executablePath : null;
return {executablePath: revisionInfo.executablePath, missingText};
}
}
const revisionInfo = browserFetcher.revisionInfo(this._preferredRevision);
const missingText = !revisionInfo.local ? `Chromium revision is not downloaded. Run "npm install" or "yarn install"` : null;
return {executablePath: revisionInfo.executablePath, missingText};
}
// TODO get resolveExecutablePath working for Firefox
if (!executablePath)
throw new Error('Please set PUPPETEER_EXECUTABLE_PATH to a Firefox binary.');
return executablePath;
}
/**
* @param {!Puppeteer.ChildProcess} chromeProcess
* @return {string}
*/
get product() {
return 'firefox';
}
/**
* @param {!Launcher.ChromeArgOptions=} options
* @return {!Array<string>}
*/
defaultArgs(options = {}) {
const firefoxArguments = [
'--remote-debugging-port=0',
'--no-remote',
'--foreground',
];
const {
devtools = false,
headless = !devtools,
args = [],
userDataDir = null
} = options;
if (userDataDir) {
firefoxArguments.push('--profile');
firefoxArguments.push(userDataDir);
}
if (headless)
firefoxArguments.push('--headless');
if (devtools)
firefoxArguments.push('--devtools');
if (args.every(arg => arg.startsWith('-')))
firefoxArguments.push('about:blank');
firefoxArguments.push(...args);
return firefoxArguments;
}
/**
* @param {!Object=} extraPrefs
* @return {!Promise<string>}
*/
async _createProfile(extraPrefs) {
const profilePath = await mkdtempAsync(path.join(os.tmpdir(), 'puppeteer_dev_firefox_profile-'));
const prefsJs = [];
const userJs = [];
const preferences = {
// https://bugzilla.mozilla.org/show_bug.cgi?id=1544393
'remote.enabled': true,
// https://bugzilla.mozilla.org/show_bug.cgi?id=1543115
'browser.dom.window.dump.enabled': true
};
Object.assign(preferences, extraPrefs);
for (const [key, value] of Object.entries(preferences))
userJs.push(`user_pref(${JSON.stringify(key)}, ${JSON.stringify(value)});`);
await writeFileAsync(path.join(profilePath, 'user.js'), userJs.join('\n'));
await writeFileAsync(path.join(profilePath, 'prefs.js'), prefsJs.join('\n'));
return profilePath;
}
}
/**
* @param {!Puppeteer.ChildProcess} browserProcess
* @param {number} timeout
* @param {string} preferredRevision
* @return {!Promise<string>}
*/
function waitForWSEndpoint(chromeProcess, timeout, preferredRevision) {
function waitForWSEndpoint(browserProcess, timeout, preferredRevision) {
return new Promise((resolve, reject) => {
const rl = readline.createInterface({ input: chromeProcess.stderr });
const rl = readline.createInterface({ input: browserProcess.stderr });
let stderr = '';
const listeners = [
helper.addEventListener(rl, 'line', onLine),
helper.addEventListener(rl, 'close', () => onClose()),
helper.addEventListener(chromeProcess, 'exit', () => onClose()),
helper.addEventListener(chromeProcess, 'error', error => onClose(error))
helper.addEventListener(browserProcess, 'exit', () => onClose()),
helper.addEventListener(browserProcess, 'error', error => onClose(error))
];
const timeoutId = timeout ? setTimeout(onTimeout, timeout) : 0;
@ -346,7 +557,7 @@ function waitForWSEndpoint(chromeProcess, timeout, preferredRevision) {
function onClose(error) {
cleanup();
reject(new Error([
'Failed to launch chrome!' + (error ? ' ' + error.message : ''),
'Failed to launch the browser process!' + (error ? ' ' + error.message : ''),
stderr,
'',
'TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md',
@ -356,7 +567,7 @@ function waitForWSEndpoint(chromeProcess, timeout, preferredRevision) {
function onTimeout() {
cleanup();
reject(new TimeoutError(`Timed out after ${timeout} ms while trying to connect to Chrome! The only Chrome revision guaranteed to work is r${preferredRevision}`));
reject(new TimeoutError(`Timed out after ${timeout} ms while trying to connect to the browser! Only Chrome at revision r${preferredRevision} is guaranteed to work.`));
}
/**
@ -412,6 +623,55 @@ function getWSEndpoint(browserURL) {
});
}
/**
* @param {ChromeLauncher|FirefoxLauncher} launcher
*
* @return {{executablePath: string, missingText: ?string}}
*/
function resolveExecutablePath(launcher) {
// puppeteer-core doesn't take into account PUPPETEER_* env variables.
if (!launcher._isPuppeteerCore) {
const executablePath = process.env.PUPPETEER_EXECUTABLE_PATH || process.env.npm_config_puppeteer_executable_path || process.env.npm_package_config_puppeteer_executable_path;
if (executablePath) {
const missingText = !fs.existsSync(executablePath) ? 'Tried to use PUPPETEER_EXECUTABLE_PATH env variable to launch browser but did not find any executable at: ' + executablePath : null;
return { executablePath, missingText };
}
}
const browserFetcher = new BrowserFetcher(launcher._projectRoot);
if (!launcher._isPuppeteerCore) {
const revision = process.env['PUPPETEER_CHROMIUM_REVISION'];
if (revision) {
const revisionInfo = browserFetcher.revisionInfo(revision);
const missingText = !revisionInfo.local ? 'Tried to use PUPPETEER_CHROMIUM_REVISION env variable to launch browser but did not find executable at: ' + revisionInfo.executablePath : null;
return {executablePath: revisionInfo.executablePath, missingText};
}
}
const revisionInfo = browserFetcher.revisionInfo(launcher._preferredRevision);
const missingText = !revisionInfo.local ? `Browser is not downloaded. Run "npm install" or "yarn install"` : null;
return {executablePath: revisionInfo.executablePath, missingText};
}
/**
* @param {string} projectRoot
* @param {string} preferredRevision
* @param {boolean} isPuppeteerCore
* @param {string=} product
* @return {!Puppeteer.ProductLauncher}
*/
function Launcher(projectRoot, preferredRevision, isPuppeteerCore, product) {
// puppeteer-core doesn't take into account PUPPETEER_* env variables.
if (!product && !isPuppeteerCore)
product = process.env.PUPPETEER_PRODUCT || process.env.npm_config_puppeteer_product || process.env.npm_package_config_puppeteer_product;
switch (product) {
case 'firefox':
return new FirefoxLauncher(projectRoot, preferredRevision, isPuppeteerCore);
case 'chrome':
default:
return new ChromeLauncher(projectRoot, preferredRevision, isPuppeteerCore);
}
}
/**
* @typedef {Object} Launcher.ChromeArgOptions
* @property {boolean=} headless

View File

@ -26,14 +26,17 @@ module.exports = class {
*/
constructor(projectRoot, preferredRevision, isPuppeteerCore) {
this._projectRoot = projectRoot;
this._launcher = new Launcher(projectRoot, preferredRevision, isPuppeteerCore);
this._preferredRevision = preferredRevision;
this._isPuppeteerCore = isPuppeteerCore;
}
/**
* @param {!(Launcher.LaunchOptions & Launcher.ChromeArgOptions & Launcher.BrowserOptions)=} options
* @param {!(Launcher.LaunchOptions & Launcher.ChromeArgOptions & Launcher.BrowserOptions & {product?: string, extraPrefsFirefox?: !object})=} options
* @return {!Promise<!Puppeteer.Browser>}
*/
launch(options) {
if (!this._productName && options)
this._productName = options.product;
return this._launcher.launch(options);
}
@ -52,6 +55,23 @@ module.exports = class {
return this._launcher.executablePath();
}
/**
* @return {!Puppeteer.ProductLauncher}
*/
get _launcher() {
if (!this._lazyLauncher)
this._lazyLauncher = Launcher(this._projectRoot, this._preferredRevision, this._isPuppeteerCore, this._productName);
return this._lazyLauncher;
}
/**
* @return {string}
*/
get product() {
return this._launcher.product;
}
/**
* @return {Object}
*/

8
lib/externs.d.ts vendored
View File

@ -44,6 +44,14 @@ declare global {
onclose?: () => void,
}
export interface ProductLauncher {
launch(object)
connect(object)
executablePath: () => string,
defaultArgs(object)
product:string,
}
export interface ChildProcess extends child_process.ChildProcess { }
export type Viewport = {

View File

@ -137,7 +137,7 @@ class Helper {
static removeEventListeners(listeners) {
for (const listener of listeners)
listener.emitter.removeListener(listener.eventName, listener.handler);
listeners.splice(0, listeners.length);
listeners.length = 0;
}
/**

View File

@ -12,7 +12,8 @@
},
"scripts": {
"unit": "node test/test.js",
"funit": "BROWSER=firefox node test/test.js",
"fjunit": "PUPPETEER_PRODUCT=juggler node test/test.js",
"funit": "PUPPETEER_PRODUCT=firefox node test/test.js",
"debug-unit": "node --inspect-brk test/test.js",
"test-doclint": "node utils/doclint/check_public_api/test/test.js && node utils/doclint/preprocessor/test.js",
"test": "npm run lint --silent && npm run coverage && npm run test-doclint && npm run test-types && node utils/testrunner/test/test.js",

View File

@ -16,7 +16,7 @@
const path = require('path');
module.exports.addTests = function({testRunner, expect, defaultBrowserOptions, puppeteer, puppeteerPath, CHROME}) {
module.exports.addTests = function({testRunner, expect, defaultBrowserOptions, puppeteer, puppeteerPath, JUGGLER}) {
const {describe, xdescribe, fdescribe, describe_fails_ffox} = testRunner;
const {it, fit, xit, it_fails_ffox} = testRunner;
const {beforeAll, beforeEach, afterAll, afterEach} = testRunner;
@ -38,16 +38,16 @@ module.exports.addTests = function({testRunner, expect, defaultBrowserOptions, p
const options = Object.assign({}, defaultBrowserOptions, {dumpio: true});
const res = spawn('node',
[path.join(__dirname, 'fixtures', 'dumpio.js'), puppeteerPath, JSON.stringify(options)]);
if (CHROME)
res.stderr.on('data', data => dumpioData += data.toString('utf8'));
else
if (JUGGLER)
res.stdout.on('data', data => dumpioData += data.toString('utf8'));
else
res.stderr.on('data', data => dumpioData += data.toString('utf8'));
await new Promise(resolve => res.on('close', resolve));
if (CHROME)
expect(dumpioData).toContain('DevTools listening on ws://');
else
if (JUGGLER)
expect(dumpioData).toContain('Juggler listening on ws://');
else
expect(dumpioData).toContain('DevTools listening on ws://');
});
it('should close the browser when the node process closes', async({ server }) => {
const {spawn, execSync} = require('child_process');

View File

@ -24,7 +24,7 @@ const statAsync = helper.promisify(fs.stat);
const TMP_FOLDER = path.join(os.tmpdir(), 'pptr_tmp_folder-');
const utils = require('./utils');
module.exports.addTests = function({testRunner, expect, defaultBrowserOptions, puppeteer, CHROME, puppeteerPath}) {
module.exports.addTests = function({testRunner, expect, defaultBrowserOptions, puppeteer, CHROME, FFOX, JUGGLER, puppeteerPath}) {
const {describe, xdescribe, fdescribe, describe_fails_ffox} = testRunner;
const {it, fit, xit, it_fails_ffox} = testRunner;
const {beforeAll, beforeEach, afterAll, afterEach} = testRunner;
@ -199,6 +199,12 @@ module.exports.addTests = function({testRunner, expect, defaultBrowserOptions, p
expect(puppeteer.defaultArgs({userDataDir: 'foo'})).toContain('foo');
}
});
it('should report the correct product', async() => {
if (CHROME)
expect(puppeteer.product).toBe('chrome');
else if (FFOX && !JUGGLER)
expect(puppeteer.product).toBe('firefox');
});
it('should work with no default arguments', async() => {
const options = Object.assign({}, defaultBrowserOptions);
options.ignoreDefaultArgs = true;

View File

@ -28,31 +28,40 @@ module.exports.addTests = ({testRunner, product, puppeteerPath}) => {
const {beforeAll, beforeEach, afterAll, afterEach} = testRunner;
const CHROME = product === 'Chromium';
const FFOX = product === 'Firefox';
const FFOX = (product === 'Firefox' || product === 'Juggler');
const JUGGLER = product === 'Juggler';
const puppeteer = require(puppeteerPath);
const headless = (process.env.HEADLESS || 'true').trim().toLowerCase() === 'true';
const slowMo = parseInt((process.env.SLOW_MO || '0').trim(), 10);
let extraLaunchOptions = {};
try {
extraLaunchOptions = JSON.parse(process.env.EXTRA_LAUNCH_OPTIONS || '{}');
} catch (error) {
console.warn(`${YELLOW_COLOR}Error parsing EXTRA_LAUNCH_OPTIONS: ${error.message}. Skipping.${RESET_COLOR}`);
}
const defaultBrowserOptions = {
const defaultBrowserOptions = Object.assign({
handleSIGINT: false,
executablePath: CHROME ? process.env.CHROME : process.env.FFOX,
executablePath: process.env.BINARY,
slowMo,
headless,
dumpio: !!process.env.DUMPIO,
};
}, extraLaunchOptions);
if (defaultBrowserOptions.executablePath) {
console.warn(`${YELLOW_COLOR}WARN: running ${product} tests with ${defaultBrowserOptions.executablePath}${RESET_COLOR}`);
} else {
// Make sure the `npm install` was run after the chromium roll.
if (!fs.existsSync(puppeteer.executablePath()))
throw new Error(`Browser is not downloaded. Run 'npm install' and try to re-run tests`);
const executablePath = puppeteer.executablePath();
if (!fs.existsSync(executablePath))
throw new Error(`Browser is not downloaded at ${executablePath}. Run 'npm install' and try to re-run tests`);
}
const GOLDEN_DIR = path.join(__dirname, 'golden-' + product.toLowerCase());
const OUTPUT_DIR = path.join(__dirname, 'output-' + product.toLowerCase());
const suffix = JUGGLER ? 'firefox' : product.toLowerCase();
const GOLDEN_DIR = path.join(__dirname, 'golden-' + suffix);
const OUTPUT_DIR = path.join(__dirname, 'output-' + suffix);
if (fs.existsSync(OUTPUT_DIR))
rm(OUTPUT_DIR);
const {expect} = new Matchers({
@ -64,6 +73,7 @@ module.exports.addTests = ({testRunner, product, puppeteerPath}) => {
product,
FFOX,
CHROME,
JUGGLER,
puppeteer,
expect,
defaultBrowserOptions,
@ -72,7 +82,7 @@ module.exports.addTests = ({testRunner, product, puppeteerPath}) => {
};
beforeAll(async() => {
if (FFOX && defaultBrowserOptions.executablePath)
if (JUGGLER && defaultBrowserOptions.executablePath)
await require('../experimental/puppeteer-firefox/misc/install-preferences')(defaultBrowserOptions.executablePath);
});

View File

@ -76,17 +76,33 @@ const CHROMIUM_NO_COVERAGE = new Set([
'page.emulateMedia', // Legacy alias for `page.emulateMediaType`.
]);
if (process.env.BROWSER === 'firefox') {
switch (process.env.PUPPETEER_PRODUCT) {
case 'firefox':
testRunner.addTestDSL('it_fails_ffox', 'skip');
testRunner.addSuiteDSL('describe_fails_ffox', 'skip');
describe('Firefox', () => {
require('./puppeteer.spec.js').addTests({
product: 'Firefox',
puppeteerPath: utils.projectRoot(),
testRunner,
});
if (process.env.COVERAGE)
utils.recordAPICoverage(testRunner, require('../lib/api'), require('../lib/Events').Events, CHROMIUM_NO_COVERAGE);
});
break;
case 'juggler':
testRunner.addTestDSL('it_fails_ffox', 'skip');
testRunner.addSuiteDSL('describe_fails_ffox', 'skip');
describe('Firefox (Juggler)', () => {
require('./puppeteer.spec.js').addTests({
product: 'Juggler',
puppeteerPath: path.resolve(__dirname, '../experimental/puppeteer-firefox/'),
testRunner,
});
});
} else {
break;
case 'chrome':
default:
testRunner.addTestDSL('it_fails_ffox', 'run');
testRunner.addSuiteDSL('describe_fails_ffox', 'run');
describe('Chromium', () => {

View File

@ -3,7 +3,7 @@ const path = require('path');
const puppeteer = require('../..');
module.exports = puppeteer.launch({
pipe: false,
executablePath: process.env.CHROME,
executablePath: process.env.BINARY,
}).then(async browser => {
const origin = browser.wsEndpoint().match(/ws:\/\/([0-9A-Za-z:\.]*)\//)[1];
const page = await browser.newPage();