github/puppeteer
0
0
Fork 0
mirror of https://github.com/puppeteer/puppeteer synced 2024-06-14 14:02:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: download chrome-headless-shell by default and use it for the old headless mode (#11754)

Browse Source
This commit is contained in:
Alex Rudenko 2024-01-25 21:39:07 +01:00 committed by GitHub
parent f67b205776
commit ce894a2ffc
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
13 changed files with 154 additions and 46 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -46,10 +46,10 @@ pnpm i puppeteer
``` ```
When you install Puppeteer, it automatically downloads a recent version of When you install Puppeteer, it automatically downloads a recent version of
[Chrome for Testing](https://developer.chrome.com/blog/chrome-for-testing/) (~170MB macOS, ~282MB Linux, ~280MB Windows) that is [guaranteed to [Chrome for Testing](https://developer.chrome.com/blog/chrome-for-testing/) (~170MB macOS, ~282MB Linux, ~280MB Windows) and a `chrome-headless-shell` binary (starting with Puppeteer v21.6.0) that is [guaranteed to
work](https://pptr.dev/faq#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) work](https://pptr.dev/faq#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy)
with Puppeteer. The browser is downloaded to the `$HOME/.cache/puppeteer` folder with Puppeteer. The browser is downloaded to the `$HOME/.cache/puppeteer` folder
by default (starting with Puppeteer v19.0.0). by default (starting with Puppeteer v19.0.0). See [configuration](https://pptr.dev/api/puppeteer.configuration) for configuration options and environmental variables to control the download behavor.
If you deploy a project using Puppeteer to a hosting provider, such as Render or If you deploy a project using Puppeteer to a hosting provider, such as Render or
Heroku, you might need to reconfigure the location of the cache to be within Heroku, you might need to reconfigure the location of the cache to be within

26
docs/api/puppeteer.configuration.md
View File

@ -16,15 +16,17 @@ export interface Configuration
## Properties ## Properties
| Property | Modifiers | Type | Description | Default | | Property | Modifiers | Type | Description | Default |
| ------------------ | --------------------- | ------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ------------------------------- | --------------------- | ------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| browserRevision | <code>optional</code> | string | <p>Specifies a certain version of the browser you'd like Puppeteer to use.</p><p>Can be overridden by <code>PUPPETEER_BROWSER_REVISION</code>.</p><p>See [puppeteer.launch](./puppeteer.puppeteernode.launch.md) on how executable path is inferred.</p> | A compatible-revision of the browser. | | browserRevision | <code>optional</code> | string | <p>Specifies a certain version of the browser you'd like Puppeteer to use.</p><p>Can be overridden by <code>PUPPETEER_BROWSER_REVISION</code>.</p><p>See [puppeteer.launch](./puppeteer.puppeteernode.launch.md) on how executable path is inferred.</p> | A compatible-revision of the browser. |
| cacheDirectory | <code>optional</code> | string | <p>Defines the directory to be used by Puppeteer for caching.</p><p>Can be overridden by <code>PUPPETEER_CACHE_DIR</code>.</p> | <code>path.join(os.homedir(), '.cache', 'puppeteer')</code> | | cacheDirectory | <code>optional</code> | string | <p>Defines the directory to be used by Puppeteer for caching.</p><p>Can be overridden by <code>PUPPETEER_CACHE_DIR</code>.</p> | <code>path.join(os.homedir(), '.cache', 'puppeteer')</code> |
| defaultProduct | <code>optional</code> | [Product](./puppeteer.product.md) | <p>Specifies which browser you'd like Puppeteer to use.</p><p>Can be overridden by <code>PUPPETEER_PRODUCT</code>.</p> | <code>chrome</code> | | defaultProduct | <code>optional</code> | [Product](./puppeteer.product.md) | <p>Specifies which browser you'd like Puppeteer to use.</p><p>Can be overridden by <code>PUPPETEER_PRODUCT</code>.</p> | <code>chrome</code> |
| downloadBaseUrl | <code>optional</code> | string | <p>Specifies the URL prefix that is used to download the browser.</p><p>Can be overridden by <code>PUPPETEER_DOWNLOAD_BASE_URL</code>.</p> | Either https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing or https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central, depending on the product. | | downloadBaseUrl | <code>optional</code> | string | <p>Specifies the URL prefix that is used to download the browser.</p><p>Can be overridden by <code>PUPPETEER_DOWNLOAD_BASE_URL</code>.</p> | Either https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing or https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central, depending on the product. |
| downloadPath | <code>optional</code> | string | <p>Specifies the path for the downloads folder.</p><p>Can be overridden by <code>PUPPETEER_DOWNLOAD_PATH</code>.</p> | <code>&lt;cacheDirectory&gt;</code> | | downloadPath | <code>optional</code> | string | <p>Specifies the path for the downloads folder.</p><p>Can be overridden by <code>PUPPETEER_DOWNLOAD_PATH</code>.</p> | <code>&lt;cacheDirectory&gt;</code> |
| executablePath | <code>optional</code> | string | <p>Specifies an executable path to be used in [puppeteer.launch](./puppeteer.puppeteernode.launch.md).</p><p>Can be overridden by <code>PUPPETEER_EXECUTABLE_PATH</code>.</p> | **Auto-computed.** | | executablePath | <code>optional</code> | string | <p>Specifies an executable path to be used in [puppeteer.launch](./puppeteer.puppeteernode.launch.md).</p><p>Can be overridden by <code>PUPPETEER_EXECUTABLE_PATH</code>.</p> | **Auto-computed.** |
| experiments | <code>optional</code> | [ExperimentsConfiguration](./puppeteer.experimentsconfiguration.md) | Defines experimental options for Puppeteer. | | | experiments | <code>optional</code> | [ExperimentsConfiguration](./puppeteer.experimentsconfiguration.md) | Defines experimental options for Puppeteer. | |
| logLevel | <code>optional</code> | 'silent' \| 'error' \| 'warn' | Tells Puppeteer to log at the given level. | <code>warn</code> | | logLevel | <code>optional</code> | 'silent' \| 'error' \| 'warn' | Tells Puppeteer to log at the given level. | <code>warn</code> |
| skipDownload | <code>optional</code> | boolean | <p>Tells Puppeteer to not download during installation.</p><p>Can be overridden by <code>PUPPETEER_SKIP_DOWNLOAD</code>.</p> | | | skipChromeDownload | <code>optional</code> | boolean | <p>Tells Puppeteer to not Chrome download during installation.</p><p>Can be overridden by <code>PUPPETEER_SKIP_CHROME_DOWNLOAD</code>.</p> | |
| temporaryDirectory | <code>optional</code> | string | <p>Defines the directory to be used by Puppeteer for creating temporary files.</p><p>Can be overridden by <code>PUPPETEER_TMP_DIR</code>.</p> | <code>os.tmpdir()</code> | | skipChromeHeadlessShellDownload | <code>optional</code> | boolean | <p>Tells Puppeteer to not chrome-headless-shell download during installation.</p><p>Can be overridden by <code>PUPPETEER_SKIP_CHROME_HEADLESSS_HELL_DOWNLOAD</code>.</p> | |
| skipDownload | <code>optional</code> | boolean | <p>Tells Puppeteer to not download during installation.</p><p>Can be overridden by <code>PUPPETEER_SKIP_DOWNLOAD</code>.</p> | |
| temporaryDirectory | <code>optional</code> | string | <p>Defines the directory to be used by Puppeteer for creating temporary files.</p><p>Can be overridden by <code>PUPPETEER_TMP_DIR</code>.</p> | <code>os.tmpdir()</code> |

4
docs/index.md
View File

@ -46,10 +46,10 @@ pnpm i puppeteer
``` ```
When you install Puppeteer, it automatically downloads a recent version of When you install Puppeteer, it automatically downloads a recent version of
[Chrome for Testing](https://developer.chrome.com/blog/chrome-for-testing/) (~170MB macOS, ~282MB Linux, ~280MB Windows) that is [guaranteed to [Chrome for Testing](https://developer.chrome.com/blog/chrome-for-testing/) (~170MB macOS, ~282MB Linux, ~280MB Windows) and a `chrome-headless-shell` binary (starting with Puppeteer v21.6.0) that is [guaranteed to
work](https://pptr.dev/faq#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy) work](https://pptr.dev/faq#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy)
with Puppeteer. The browser is downloaded to the `$HOME/.cache/puppeteer` folder with Puppeteer. The browser is downloaded to the `$HOME/.cache/puppeteer` folder
by default (starting with Puppeteer v19.0.0). by default (starting with Puppeteer v19.0.0). See [configuration](https://pptr.dev/api/puppeteer.configuration) for configuration options and environmental variables to control the download behavor.
If you deploy a project using Puppeteer to a hosting provider, such as Render or If you deploy a project using Puppeteer to a hosting provider, such as Render or
Heroku, you might need to reconfigure the location of the cache to be within Heroku, you might need to reconfigure the location of the cache to be within

12
packages/puppeteer-core/src/common/Configuration.ts
View File

@ -95,6 +95,18 @@ export interface Configuration {
* Can be overridden by `PUPPETEER_SKIP_DOWNLOAD`. * Can be overridden by `PUPPETEER_SKIP_DOWNLOAD`.
*/ */
skipDownload?: boolean; skipDownload?: boolean;
/**
* Tells Puppeteer to not Chrome download during installation.
*
* Can be overridden by `PUPPETEER_SKIP_CHROME_DOWNLOAD`.
*/
skipChromeDownload?: boolean;
/**
* Tells Puppeteer to not chrome-headless-shell download during installation.
*
* Can be overridden by `PUPPETEER_SKIP_CHROME_HEADLESSS_HELL_DOWNLOAD`.
*/
skipChromeHeadlessShellDownload?: boolean;
/** /**
* Tells Puppeteer to log at the given level. * Tells Puppeteer to log at the given level.
* *

9
packages/puppeteer-core/src/node/ChromeLauncher.ts
View File

@ -146,7 +146,7 @@ export class ChromeLauncher extends ProductLauncher {
channel || !this.puppeteer._isPuppeteerCore, channel || !this.puppeteer._isPuppeteerCore,
`An \`executablePath\` or \`channel\` must be specified for \`puppeteer-core\`` `An \`executablePath\` or \`channel\` must be specified for \`puppeteer-core\``
); );
chromeExecutable = this.executablePath(channel); chromeExecutable = this.executablePath(channel, options.headless ?? true);
} }
return { return {
@ -269,14 +269,17 @@ export class ChromeLauncher extends ProductLauncher {
return chromeArguments; return chromeArguments;
} }
override executablePath(channel?: ChromeReleaseChannel): string { override executablePath(
channel?: ChromeReleaseChannel,
headless?: boolean | 'new'
): string {
if (channel) { if (channel) {
return computeSystemExecutablePath({ return computeSystemExecutablePath({
browser: SupportedBrowsers.CHROME, browser: SupportedBrowsers.CHROME,
channel: convertPuppeteerChannelToBrowsersChannel(channel), channel: convertPuppeteerChannelToBrowsersChannel(channel),
}); });
} else { } else {
return this.resolveExecutablePath(); return this.resolveExecutablePath(headless);
} }
} }
} }

9
packages/puppeteer-core/src/node/ProductLauncher.ts
View File

@ -393,7 +393,7 @@ export abstract class ProductLauncher {
/** /**
* @internal * @internal
*/ */
protected resolveExecutablePath(): string { protected resolveExecutablePath(headless?: boolean | 'new'): string {
let executablePath = this.puppeteer.configuration.executablePath; let executablePath = this.puppeteer.configuration.executablePath;
if (executablePath) { if (executablePath) {
if (!existsSync(executablePath)) { if (!existsSync(executablePath)) {
@ -404,9 +404,12 @@ export abstract class ProductLauncher {
return executablePath; return executablePath;
} }
function productToBrowser(product?: Product) { function productToBrowser(product?: Product, headless?: boolean | 'new') {
switch (product) { switch (product) {
case 'chrome': case 'chrome':
if (headless === true) {
return InstalledBrowser.CHROMEHEADLESSSHELL;
}
return InstalledBrowser.CHROME; return InstalledBrowser.CHROME;
case 'firefox': case 'firefox':
return InstalledBrowser.FIREFOX; return InstalledBrowser.FIREFOX;
@ -416,7 +419,7 @@ export abstract class ProductLauncher {
executablePath = computeExecutablePath({ executablePath = computeExecutablePath({
cacheDir: this.puppeteer.defaultDownloadPath!, cacheDir: this.puppeteer.defaultDownloadPath!,
browser: productToBrowser(this.product), browser: productToBrowser(this.product, headless),
buildId: this.puppeteer.browserRevision, buildId: this.puppeteer.browserRevision,
}); });

1
packages/puppeteer-core/src/revisions.ts
View File

@ -9,5 +9,6 @@
*/ */
export const PUPPETEER_REVISIONS = Object.freeze({ export const PUPPETEER_REVISIONS = Object.freeze({
chrome: '121.0.6167.85', chrome: '121.0.6167.85',
'chrome-headless-shell': '121.0.6167.85',
firefox: 'latest', firefox: 'latest',
}); });

18
packages/puppeteer/src/getConfiguration.ts
View File

@ -64,6 +64,24 @@ export const getConfiguration = (): Configuration => {
configuration.skipDownload configuration.skipDownload
); );
// Set skipChromeDownload explicitly or from default
configuration.skipChromeDownload = Boolean(
process.env['PUPPETEER_SKIP_CHROME_DOWNLOAD'] ??
process.env['npm_config_puppeteer_skip_chrome_download'] ??
process.env['npm_package_config_puppeteer_skip_chrome_download'] ??
configuration.skipChromeDownload
);
// Set skipChromeDownload explicitly or from default
configuration.skipChromeHeadlessShellDownload = Boolean(
process.env['PUPPETEER_SKIP_CHROME_HEADLESS_SHELL_DOWNLOAD'] ??
process.env['npm_config_puppeteer_skip_chrome_headless_shell_download'] ??
process.env[
'npm_package_config_puppeteer_skip_chrome_headless_shell_download'
] ??
configuration.skipChromeHeadlessShellDownload
);
// Prepare variables used in browser downloading // Prepare variables used in browser downloading
if (!configuration.skipDownload) { if (!configuration.skipDownload) {
configuration.browserRevision = configuration.browserRevision =

1
packages/puppeteer/src/node/cli.ts
View File

@ -27,5 +27,6 @@ void new CLI({
pinnedBrowsers: { pinnedBrowsers: {
[Browser.CHROME]: PUPPETEER_REVISIONS.chrome, [Browser.CHROME]: PUPPETEER_REVISIONS.chrome,
[Browser.FIREFOX]: PUPPETEER_REVISIONS.firefox, [Browser.FIREFOX]: PUPPETEER_REVISIONS.firefox,
[Browser.CHROMEHEADLESSSHELL]: PUPPETEER_REVISIONS['chrome-headless-shell'],
}, },
}).run(process.argv); }).run(process.argv);

93
packages/puppeteer/src/node/install.ts
View File

@ -48,28 +48,91 @@ export async function downloadBrowser(): Promise<void> {
const unresolvedBuildId = const unresolvedBuildId =
configuration.browserRevision || PUPPETEER_REVISIONS[product] || 'latest'; configuration.browserRevision || PUPPETEER_REVISIONS[product] || 'latest';
const unresolvedShellBuildId =
configuration.browserRevision ||
PUPPETEER_REVISIONS['chrome-headless-shell'] ||
'latest';
const buildId = await resolveBuildId(browser, platform, unresolvedBuildId);
// TODO: deprecate downloadPath in favour of cacheDirectory. // TODO: deprecate downloadPath in favour of cacheDirectory.
const cacheDir = configuration.downloadPath ?? configuration.cacheDirectory!; const cacheDir = configuration.downloadPath ?? configuration.cacheDirectory!;
try { try {
const result = await install({ const installationJobs = [];
browser,
cacheDir,
platform,
buildId,
downloadProgressCallback: makeProgressCallback(browser, buildId),
baseUrl: downloadBaseUrl,
});
logPolitely( if (configuration.skipChromeDownload) {
`${supportedProducts[product]} (${result.buildId}) downloaded to ${result.path}` logPolitely('**INFO** Skipping Chrome download as instructed.');
); } else {
const buildId = await resolveBuildId(
browser,
platform,
unresolvedBuildId
);
installationJobs.push(
install({
browser,
cacheDir,
platform,
buildId,
downloadProgressCallback: makeProgressCallback(browser, buildId),
baseUrl: downloadBaseUrl,
})
.then(result => {
logPolitely(
`${supportedProducts[product]} (${result.buildId}) downloaded to ${result.path}`
);
})
.catch(error => {
throw new Error(
`ERROR: Failed to set up ${supportedProducts[product]} v${buildId}! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.`,
{
cause: error,
}
);
})
);
}
if (browser === Browser.CHROME) {
if (configuration.skipChromeHeadlessShellDownload) {
logPolitely('**INFO** Skipping Chrome download as instructed.');
} else {
const shellBuildId = await resolveBuildId(
browser,
platform,
unresolvedShellBuildId
);
installationJobs.push(
install({
browser: Browser.CHROMEHEADLESSSHELL,
cacheDir,
platform,
buildId: shellBuildId,
downloadProgressCallback: makeProgressCallback(
browser,
shellBuildId
),
baseUrl: downloadBaseUrl,
})
.then(result => {
logPolitely(
`${Browser.CHROMEHEADLESSSHELL} (${result.buildId}) downloaded to ${result.path}`
);
})
.catch(error => {
throw new Error(
`ERROR: Failed to set up ${Browser.CHROMEHEADLESSSHELL} v${shellBuildId}! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.`,
{
cause: error,
}
);
})
);
}
}
await Promise.all(installationJobs);
} catch (error) { } catch (error) {
console.error(
`ERROR: Failed to set up ${supportedProducts[product]} r${buildId}! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.`
);
console.error(error); console.error(error);
process.exit(1); process.exit(1);
} }

10
test/installation/src/puppeteer-configuration.spec.ts
View File

@ -30,8 +30,9 @@ describe('`puppeteer` with configuration', () => {
it('evaluates', async function () { it('evaluates', async function () {
const files = await readdir(join(this.sandbox, '.cache', 'puppeteer')); const files = await readdir(join(this.sandbox, '.cache', 'puppeteer'));
assert.equal(files.length, 1); assert.equal(files.length, 2);
assert.equal(files[0], 'chrome'); assert(files.includes('chrome'));
assert(files.includes('chrome-headless-shell'));
const script = await readAsset('puppeteer', 'basic.js'); const script = await readAsset('puppeteer', 'basic.js');
await this.runScript(script, 'mjs'); await this.runScript(script, 'mjs');
@ -61,8 +62,9 @@ describe('`puppeteer` with configuration', () => {
it('evaluates', async function () { it('evaluates', async function () {
const files = await readdir(join(this.sandbox, '.cache', 'puppeteer')); const files = await readdir(join(this.sandbox, '.cache', 'puppeteer'));
assert.equal(files.length, 1); assert.equal(files.length, 2);
assert.equal(files[0], 'chrome'); assert(files.includes('chrome'));
assert(files.includes('chrome-headless-shell'));
const script = await readAsset('puppeteer', 'basic.js'); const script = await readAsset('puppeteer', 'basic.js');
await this.runScript(script, 'mjs'); await this.runScript(script, 'mjs');

11
test/installation/src/puppeteer.spec.ts
View File

@ -25,8 +25,10 @@ describe('`puppeteer`', () => {
it('evaluates CommonJS', async function () { it('evaluates CommonJS', async function () {
const files = await readdir(join(this.sandbox, '.cache', 'puppeteer')); const files = await readdir(join(this.sandbox, '.cache', 'puppeteer'));
assert.equal(files.length, 1); assert.equal(files.length, 2);
assert.equal(files[0], 'chrome'); assert(files.includes('chrome'));
assert(files.includes('chrome-headless-shell'));
const script = await readAsset('puppeteer-core', 'requires.cjs'); const script = await readAsset('puppeteer-core', 'requires.cjs');
await this.runScript(script, 'cjs'); await this.runScript(script, 'cjs');
}); });
@ -52,8 +54,9 @@ describe('`puppeteer`', () => {
it('evaluates', async function () { it('evaluates', async function () {
const files = await readdir(join(this.sandbox, '.cache', 'puppeteer')); const files = await readdir(join(this.sandbox, '.cache', 'puppeteer'));
assert.equal(files.length, 1); assert.equal(files.length, 2);
assert.equal(files[0], 'chrome'); assert(files.includes('chrome'));
assert(files.includes('chrome-headless-shell'));
const script = await readAsset('puppeteer', 'basic.js'); const script = await readAsset('puppeteer', 'basic.js');
await this.runScript(script, 'mjs'); await this.runScript(script, 'mjs');

2
tools/update_chrome_revision.mjs
View File

@ -62,7 +62,7 @@ async function formatUpdateFiles() {
async function replaceInFile(filePath, search, replace) { async function replaceInFile(filePath, search, replace) {
const buffer = await readFile(filePath); const buffer = await readFile(filePath);
const update = buffer.toString().replace(search, replace); const update = buffer.toString().replaceAll(search, replace);
await writeFile(filePath, update); await writeFile(filePath, update);