diff --git a/README.md b/README.md index 2f87f664314..39ea7cb4441 100644 --- a/README.md +++ b/README.md @@ -46,10 +46,10 @@ pnpm i puppeteer ``` When you install Puppeteer, it automatically downloads a recent version of -[Chrome for Testing](https://developer.chrome.com/blog/chrome-for-testing/) (~170MB macOS, ~282MB Linux, ~280MB Windows) that is [guaranteed to +[Chrome for Testing](https://developer.chrome.com/blog/chrome-for-testing/) (~170MB macOS, ~282MB Linux, ~280MB Windows) and a `chrome-headless-shell` binary (starting with Puppeteer v21.6.0) that is [guaranteed to work](https://pptr.dev/faq#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) with Puppeteer. The browser is downloaded to the `$HOME/.cache/puppeteer` folder -by default (starting with Puppeteer v19.0.0). +by default (starting with Puppeteer v19.0.0). See [configuration](https://pptr.dev/api/puppeteer.configuration) for configuration options and environmental variables to control the download behavor. If you deploy a project using Puppeteer to a hosting provider, such as Render or Heroku, you might need to reconfigure the location of the cache to be within diff --git a/docs/api/puppeteer.configuration.md b/docs/api/puppeteer.configuration.md index c8a49caacc7..8f80db88516 100644 --- a/docs/api/puppeteer.configuration.md +++ b/docs/api/puppeteer.configuration.md @@ -16,15 +16,17 @@ export interface Configuration ## Properties -| Property | Modifiers | Type | Description | Default | -| ------------------ | --------------------- | ------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| browserRevision | optional | string |

Specifies a certain version of the browser you'd like Puppeteer to use.

Can be overridden by PUPPETEER_BROWSER_REVISION.

See [puppeteer.launch](./puppeteer.puppeteernode.launch.md) on how executable path is inferred.

| A compatible-revision of the browser. | -| cacheDirectory | optional | string |

Defines the directory to be used by Puppeteer for caching.

Can be overridden by PUPPETEER_CACHE_DIR.

| path.join(os.homedir(), '.cache', 'puppeteer') | -| defaultProduct | optional | [Product](./puppeteer.product.md) |

Specifies which browser you'd like Puppeteer to use.

Can be overridden by PUPPETEER_PRODUCT.

| chrome | -| downloadBaseUrl | optional | string |

Specifies the URL prefix that is used to download the browser.

Can be overridden by PUPPETEER_DOWNLOAD_BASE_URL.

| Either https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing or https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central, depending on the product. | -| downloadPath | optional | string |

Specifies the path for the downloads folder.

Can be overridden by PUPPETEER_DOWNLOAD_PATH.

| <cacheDirectory> | -| executablePath | optional | string |

Specifies an executable path to be used in [puppeteer.launch](./puppeteer.puppeteernode.launch.md).

Can be overridden by PUPPETEER_EXECUTABLE_PATH.

| **Auto-computed.** | -| experiments | optional | [ExperimentsConfiguration](./puppeteer.experimentsconfiguration.md) | Defines experimental options for Puppeteer. | | -| logLevel | optional | 'silent' \| 'error' \| 'warn' | Tells Puppeteer to log at the given level. | warn | -| skipDownload | optional | boolean |

Tells Puppeteer to not download during installation.

Can be overridden by PUPPETEER_SKIP_DOWNLOAD.

| | -| temporaryDirectory | optional | string |

Defines the directory to be used by Puppeteer for creating temporary files.

Can be overridden by PUPPETEER_TMP_DIR.

| os.tmpdir() | +| Property | Modifiers | Type | Description | Default | +| ------------------------------- | --------------------- | ------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| browserRevision | optional | string |

Specifies a certain version of the browser you'd like Puppeteer to use.

Can be overridden by PUPPETEER_BROWSER_REVISION.

See [puppeteer.launch](./puppeteer.puppeteernode.launch.md) on how executable path is inferred.

| A compatible-revision of the browser. | +| cacheDirectory | optional | string |

Defines the directory to be used by Puppeteer for caching.

Can be overridden by PUPPETEER_CACHE_DIR.

| path.join(os.homedir(), '.cache', 'puppeteer') | +| defaultProduct | optional | [Product](./puppeteer.product.md) |

Specifies which browser you'd like Puppeteer to use.

Can be overridden by PUPPETEER_PRODUCT.

| chrome | +| downloadBaseUrl | optional | string |

Specifies the URL prefix that is used to download the browser.

Can be overridden by PUPPETEER_DOWNLOAD_BASE_URL.

| Either https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing or https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central, depending on the product. | +| downloadPath | optional | string |

Specifies the path for the downloads folder.

Can be overridden by PUPPETEER_DOWNLOAD_PATH.

| <cacheDirectory> | +| executablePath | optional | string |

Specifies an executable path to be used in [puppeteer.launch](./puppeteer.puppeteernode.launch.md).

Can be overridden by PUPPETEER_EXECUTABLE_PATH.

| **Auto-computed.** | +| experiments | optional | [ExperimentsConfiguration](./puppeteer.experimentsconfiguration.md) | Defines experimental options for Puppeteer. | | +| logLevel | optional | 'silent' \| 'error' \| 'warn' | Tells Puppeteer to log at the given level. | warn | +| skipChromeDownload | optional | boolean |

Tells Puppeteer to not Chrome download during installation.

Can be overridden by PUPPETEER_SKIP_CHROME_DOWNLOAD.

| | +| skipChromeHeadlessShellDownload | optional | boolean |

Tells Puppeteer to not chrome-headless-shell download during installation.

Can be overridden by PUPPETEER_SKIP_CHROME_HEADLESSS_HELL_DOWNLOAD.

| | +| skipDownload | optional | boolean |

Tells Puppeteer to not download during installation.

Can be overridden by PUPPETEER_SKIP_DOWNLOAD.

| | +| temporaryDirectory | optional | string |

Defines the directory to be used by Puppeteer for creating temporary files.

Can be overridden by PUPPETEER_TMP_DIR.

| os.tmpdir() | diff --git a/docs/index.md b/docs/index.md index 2f87f664314..39ea7cb4441 100644 --- a/docs/index.md +++ b/docs/index.md @@ -46,10 +46,10 @@ pnpm i puppeteer ``` When you install Puppeteer, it automatically downloads a recent version of -[Chrome for Testing](https://developer.chrome.com/blog/chrome-for-testing/) (~170MB macOS, ~282MB Linux, ~280MB Windows) that is [guaranteed to +[Chrome for Testing](https://developer.chrome.com/blog/chrome-for-testing/) (~170MB macOS, ~282MB Linux, ~280MB Windows) and a `chrome-headless-shell` binary (starting with Puppeteer v21.6.0) that is [guaranteed to work](https://pptr.dev/faq#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) with Puppeteer. The browser is downloaded to the `$HOME/.cache/puppeteer` folder -by default (starting with Puppeteer v19.0.0). +by default (starting with Puppeteer v19.0.0). See [configuration](https://pptr.dev/api/puppeteer.configuration) for configuration options and environmental variables to control the download behavor. If you deploy a project using Puppeteer to a hosting provider, such as Render or Heroku, you might need to reconfigure the location of the cache to be within diff --git a/packages/puppeteer-core/src/common/Configuration.ts b/packages/puppeteer-core/src/common/Configuration.ts index a6f9619f0d2..b7cf81d352e 100644 --- a/packages/puppeteer-core/src/common/Configuration.ts +++ b/packages/puppeteer-core/src/common/Configuration.ts @@ -105,6 +105,18 @@ export interface Configuration { * Can be overridden by `PUPPETEER_SKIP_DOWNLOAD`. */ skipDownload?: boolean; + /** + * Tells Puppeteer to not Chrome download during installation. + * + * Can be overridden by `PUPPETEER_SKIP_CHROME_DOWNLOAD`. + */ + skipChromeDownload?: boolean; + /** + * Tells Puppeteer to not chrome-headless-shell download during installation. + * + * Can be overridden by `PUPPETEER_SKIP_CHROME_HEADLESSS_HELL_DOWNLOAD`. + */ + skipChromeHeadlessShellDownload?: boolean; /** * Tells Puppeteer to log at the given level. * diff --git a/packages/puppeteer-core/src/node/ChromeLauncher.ts b/packages/puppeteer-core/src/node/ChromeLauncher.ts index f35bad0824d..06332fd5148 100644 --- a/packages/puppeteer-core/src/node/ChromeLauncher.ts +++ b/packages/puppeteer-core/src/node/ChromeLauncher.ts @@ -156,7 +156,7 @@ export class ChromeLauncher extends ProductLauncher { channel || !this.puppeteer._isPuppeteerCore, `An \`executablePath\` or \`channel\` must be specified for \`puppeteer-core\`` ); - chromeExecutable = this.executablePath(channel); + chromeExecutable = this.executablePath(channel, options.headless ?? true); } return { @@ -279,14 +279,17 @@ export class ChromeLauncher extends ProductLauncher { return chromeArguments; } - override executablePath(channel?: ChromeReleaseChannel): string { + override executablePath( + channel?: ChromeReleaseChannel, + headless?: boolean | 'new' + ): string { if (channel) { return computeSystemExecutablePath({ browser: SupportedBrowsers.CHROME, channel: convertPuppeteerChannelToBrowsersChannel(channel), }); } else { - return this.resolveExecutablePath(); + return this.resolveExecutablePath(headless); } } } diff --git a/packages/puppeteer-core/src/node/ProductLauncher.ts b/packages/puppeteer-core/src/node/ProductLauncher.ts index c6a238323d8..81c5e478e29 100644 --- a/packages/puppeteer-core/src/node/ProductLauncher.ts +++ b/packages/puppeteer-core/src/node/ProductLauncher.ts @@ -386,7 +386,7 @@ export abstract class ProductLauncher { /** * @internal */ - protected resolveExecutablePath(): string { + protected resolveExecutablePath(headless?: boolean | 'new'): string { let executablePath = this.puppeteer.configuration.executablePath; if (executablePath) { if (!existsSync(executablePath)) { @@ -397,9 +397,12 @@ export abstract class ProductLauncher { return executablePath; } - function productToBrowser(product?: Product) { + function productToBrowser(product?: Product, headless?: boolean | 'new') { switch (product) { case 'chrome': + if (headless === true) { + return InstalledBrowser.CHROMEHEADLESSSHELL; + } return InstalledBrowser.CHROME; case 'firefox': return InstalledBrowser.FIREFOX; @@ -409,7 +412,7 @@ export abstract class ProductLauncher { executablePath = computeExecutablePath({ cacheDir: this.puppeteer.defaultDownloadPath!, - browser: productToBrowser(this.product), + browser: productToBrowser(this.product, headless), buildId: this.puppeteer.browserRevision, }); diff --git a/packages/puppeteer-core/src/revisions.ts b/packages/puppeteer-core/src/revisions.ts index f8a06199170..d68961c745f 100644 --- a/packages/puppeteer-core/src/revisions.ts +++ b/packages/puppeteer-core/src/revisions.ts @@ -19,5 +19,6 @@ */ export const PUPPETEER_REVISIONS = Object.freeze({ chrome: '120.0.6099.71', + 'chrome-headless-shell': '120.0.6099.71', firefox: 'latest', }); diff --git a/packages/puppeteer/src/getConfiguration.ts b/packages/puppeteer/src/getConfiguration.ts index 94b7cffce96..f6cfd6a40c6 100644 --- a/packages/puppeteer/src/getConfiguration.ts +++ b/packages/puppeteer/src/getConfiguration.ts @@ -72,6 +72,24 @@ export const getConfiguration = (): Configuration => { configuration.skipDownload ); + // Set skipChromeDownload explicitly or from default + configuration.skipChromeDownload = Boolean( + process.env['PUPPETEER_SKIP_CHROME_DOWNLOAD'] ?? + process.env['npm_config_puppeteer_skip_chrome_download'] ?? + process.env['npm_package_config_puppeteer_skip_chrome_download'] ?? + configuration.skipChromeDownload + ); + + // Set skipChromeDownload explicitly or from default + configuration.skipChromeHeadlessShellDownload = Boolean( + process.env['PUPPETEER_SKIP_CHROME_HEADLESS_SHELL_DOWNLOAD'] ?? + process.env['npm_config_puppeteer_skip_chrome_headless_shell_download'] ?? + process.env[ + 'npm_package_config_puppeteer_skip_chrome_headless_shell_download' + ] ?? + configuration.skipChromeHeadlessShellDownload + ); + // Prepare variables used in browser downloading if (!configuration.skipDownload) { configuration.browserRevision = diff --git a/packages/puppeteer/src/node/cli.ts b/packages/puppeteer/src/node/cli.ts index d0520d7f8f2..7498d9a4817 100644 --- a/packages/puppeteer/src/node/cli.ts +++ b/packages/puppeteer/src/node/cli.ts @@ -37,5 +37,6 @@ void new CLI({ pinnedBrowsers: { [Browser.CHROME]: PUPPETEER_REVISIONS.chrome, [Browser.FIREFOX]: PUPPETEER_REVISIONS.firefox, + [Browser.CHROMEHEADLESSSHELL]: PUPPETEER_REVISIONS['chrome-headless-shell'], }, }).run(process.argv); diff --git a/packages/puppeteer/src/node/install.ts b/packages/puppeteer/src/node/install.ts index 5a1a410d825..b3bb6858049 100644 --- a/packages/puppeteer/src/node/install.ts +++ b/packages/puppeteer/src/node/install.ts @@ -58,24 +58,64 @@ export async function downloadBrowser(): Promise { const unresolvedBuildId = configuration.browserRevision || PUPPETEER_REVISIONS[product] || 'latest'; + const unresolvedShellBuildId = + configuration.browserRevision || + PUPPETEER_REVISIONS['chrome-headless-shell'] || + 'latest'; + + const shellBuildId = + browser === Browser.CHROME + ? await resolveBuildId(browser, platform, unresolvedShellBuildId) + : ''; const buildId = await resolveBuildId(browser, platform, unresolvedBuildId); // TODO: deprecate downloadPath in favour of cacheDirectory. const cacheDir = configuration.downloadPath ?? configuration.cacheDirectory!; try { - const result = await install({ - browser, - cacheDir, - platform, - buildId, - downloadProgressCallback: makeProgressCallback(browser, buildId), - baseUrl: downloadBaseUrl, - }); + const installationJobs = []; - logPolitely( - `${supportedProducts[product]} (${result.buildId}) downloaded to ${result.path}` - ); + if (configuration.skipChromeDownload) { + logPolitely('**INFO** Skipping Chrome download as instructed.'); + } else { + installationJobs.push( + install({ + browser, + cacheDir, + platform, + buildId, + downloadProgressCallback: makeProgressCallback(browser, buildId), + baseUrl: downloadBaseUrl, + }).then(result => { + logPolitely( + `${supportedProducts[product]} (${result.buildId}) downloaded to ${result.path}` + ); + }) + ); + } + + if (browser === Browser.CHROME) { + if (configuration.skipChromeHeadlessShellDownload) { + logPolitely('**INFO** Skipping Chrome download as instructed.'); + } else { + installationJobs.push( + install({ + browser: Browser.CHROMEHEADLESSSHELL, + cacheDir, + platform, + buildId: shellBuildId, + downloadProgressCallback: makeProgressCallback(browser, buildId), + baseUrl: downloadBaseUrl, + }).then(result => { + logPolitely( + `${Browser.CHROMEHEADLESSSHELL} (${result.buildId}) downloaded to ${result.path}` + ); + }) + ); + } + } + + await Promise.all(installationJobs); } catch (error) { console.error( `ERROR: Failed to set up ${supportedProducts[product]} r${buildId}! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.` diff --git a/test/installation/src/puppeteer-configuration.spec.ts b/test/installation/src/puppeteer-configuration.spec.ts index acabdd0c4a3..edd2e3c188b 100644 --- a/test/installation/src/puppeteer-configuration.spec.ts +++ b/test/installation/src/puppeteer-configuration.spec.ts @@ -40,8 +40,9 @@ describe('`puppeteer` with configuration', () => { it('evaluates', async function () { const files = await readdir(join(this.sandbox, '.cache', 'puppeteer')); - assert.equal(files.length, 1); - assert.equal(files[0], 'chrome'); + assert.equal(files.length, 2); + assert(files.includes('chrome')); + assert(files.includes('chrome-headless-shell')); const script = await readAsset('puppeteer', 'basic.js'); await this.runScript(script, 'mjs'); @@ -71,8 +72,9 @@ describe('`puppeteer` with configuration', () => { it('evaluates', async function () { const files = await readdir(join(this.sandbox, '.cache', 'puppeteer')); - assert.equal(files.length, 1); - assert.equal(files[0], 'chrome'); + assert.equal(files.length, 2); + assert(files.includes('chrome')); + assert(files.includes('chrome-headless-shell')); const script = await readAsset('puppeteer', 'basic.js'); await this.runScript(script, 'mjs'); diff --git a/test/installation/src/puppeteer.spec.ts b/test/installation/src/puppeteer.spec.ts index 64109251c8a..777199f65a2 100644 --- a/test/installation/src/puppeteer.spec.ts +++ b/test/installation/src/puppeteer.spec.ts @@ -35,8 +35,10 @@ describe('`puppeteer`', () => { it('evaluates CommonJS', async function () { const files = await readdir(join(this.sandbox, '.cache', 'puppeteer')); - assert.equal(files.length, 1); - assert.equal(files[0], 'chrome'); + assert.equal(files.length, 2); + assert(files.includes('chrome')); + assert(files.includes('chrome-headless-shell')); + const script = await readAsset('puppeteer-core', 'requires.cjs'); await this.runScript(script, 'cjs'); }); @@ -62,8 +64,9 @@ describe('`puppeteer`', () => { it('evaluates', async function () { const files = await readdir(join(this.sandbox, '.cache', 'puppeteer')); - assert.equal(files.length, 1); - assert.equal(files[0], 'chrome'); + assert.equal(files.length, 2); + assert(files.includes('chrome')); + assert(files.includes('chrome-headless-shell')); const script = await readAsset('puppeteer', 'basic.js'); await this.runScript(script, 'mjs'); diff --git a/tools/update_chrome_revision.mjs b/tools/update_chrome_revision.mjs index 0d0429742f1..32cccdec606 100644 --- a/tools/update_chrome_revision.mjs +++ b/tools/update_chrome_revision.mjs @@ -72,7 +72,7 @@ async function formatUpdateFiles() { async function replaceInFile(filePath, search, replace) { const buffer = await readFile(filePath); - const update = buffer.toString().replace(search, replace); + const update = buffer.toString().replaceAll(search, replace); await writeFile(filePath, update);