chore: refactor utils (#9053)

This PR

- renames the `utils` folder to `tools` (follows internal practice),
- migrates the contents of `scripts` into `tools` and removes scripts.

.github/workflows/pre-release.yml

@@ -17,7 +17,7 @@ jobs:
         with:
           ssh-key: ${{ secrets.SSH_PRIVATE_KEY }}
       - name: Pin dependencies
-        run: utils/pin_dependencies.py
+        run: tools/pin_dependencies.py
       - name: Install dependencies
         run: npm install
       - name: Build

.github/workflows/tot-ci.yml

@@ -39,7 +39,7 @@ jobs:
           PUPPETEER_PRODUCT=firefox npm install
           npm install
           ls .local-chromium .local-firefox
-          REV=$(node utils/check_availability.js -p linux)
+          REV=$(node tools/check_availability.js -p linux)
           echo "Installing revision $REV"
           cat src/revisions.ts | sed "s/[0-9]\{6,\}/$REV/" > src/revisions.ts.replaced
           mv src/revisions.ts.replaced src/revisions.ts
@@ -77,7 +77,7 @@ jobs:
           PUPPETEER_PRODUCT=firefox npm install
           npm install
           ls .local-chromium .local-firefox
-          REV=$(node utils/check_availability.js -p linux)
+          REV=$(node tools/check_availability.js -p linux)
           echo "Installing revision $REV"
           cat src/revisions.ts | sed "s/[0-9]\{6,\}/$REV/" > src/revisions.ts.replaced
           mv src/revisions.ts.replaced src/revisions.ts
@@ -115,7 +115,7 @@ jobs:
           PUPPETEER_PRODUCT=firefox npm install
           npm install
           ls .local-chromium .local-firefox
-          REV=$(node utils/check_availability.js -p linux)
+          REV=$(node tools/check_availability.js -p linux)
           echo "Installing revision $REV"
           cat src/revisions.ts | sed "s/[0-9]\{6,\}/$REV/" > src/revisions.ts.replaced
           mv src/revisions.ts.replaced src/revisions.ts

docs/contributing.md

@@ -59,7 +59,7 @@ We also have other tests such as `test:types` that tests types and
 
 Puppeteer is using a custom test runner on top of Mocha that consults
 the [TestExpectations.json](https://github.com/puppeteer/puppeteer/blob/main/test/TestExpectations.json)
-to see if a given test result is expected or not. See more info about the test runner in [`utils/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/utils/mochaRunner).
+to see if a given test result is expected or not. See more info about the test runner in [`tools/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/tools/mochaRunner).
 
 ## Code reviews
 
@@ -97,8 +97,8 @@ The following is a description of the primary folders in Puppeteer:
 - `src` - contains the source code for Puppeteer.
 - `test/src` - contains the source code for Puppeteer tests.
 - `utils`/`scripts` - contains various scripts.
-- `utils/testserver` - contains the source code for our test servers in testing.
+- `tools/testserver` - contains the source code for our test servers in testing.
-- `utils/mochaRunner` - contains the source code for our test runner.
+- `tools/mochaRunner` - contains the source code for our test runner.
 - `compat` - contains code separated by module import type. See [`compat/README.md`](https://github.com/puppeteer/puppeteer/blob/main/compat/README.md) for details.
 - `test-d` contains type tests using [`tsd`](https://github.com/SamVerschueren/tsd).
 - `third_party` contains all dependencies that we vendor into the final build. See the [`third_party/README.md`](https://github.com/puppeteer/puppeteer/blob/main/third_party/README.md) for details.
@@ -251,7 +251,7 @@ BINARY=<path-to-executable> npm run test:chrome:headless # Or npm run test:firef
 ```
 
 If a test is expected to fail on certain configurations or became flaky, update [TestExpectations.json](https://github.com/puppeteer/puppeteer/blob/main/test/TestExpectations.json)
-to reflect that. See more info about TestExpectations.json in [`utils/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/utils/mochaRunner).
+to reflect that. See more info about TestExpectations.json in [`tools/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/tools/mochaRunner).
 
 ## API Coverage
 
@@ -269,7 +269,7 @@ The following steps are needed to update the Chromium version.
 
 1. Find a suitable Chromium revision
    Not all revisions have builds for all platforms, so we need to find one that does.
-   To do so, run `utils/check_availability.js -rd` to find the latest suitable `dev` Chromium revision (see `utils/check_availability.js -help` for more options).
+   To do so, run `tools/check_availability.js -rd` to find the latest suitable `dev` Chromium revision (see `tools/check_availability.js -help` for more options).
 1. Update `src/revisions.ts` with the found revision number.
 1. Update `versions.js` with the new Chromium-to-Puppeteer version mapping and update `lastMaintainedChromiumVersion` with the latest stable Chrome version.
 1. Run `npm run check:protocol-revision`.
@@ -281,11 +281,11 @@ The following steps are needed to update the Chromium version.
 
 ### Bisecting upstream changes
 
-Sometimes, performing a Chromium roll causes tests to fail. To figure out the cause, you need to bisect Chromium revisions to figure out the earliest possible revision that changed the behavior. The script in `utils/bisect.js` can be helpful here. Given a pattern for one or more unit tests, it will automatically bisect the current range:
+Sometimes, performing a Chromium roll causes tests to fail. To figure out the cause, you need to bisect Chromium revisions to figure out the earliest possible revision that changed the behavior. The script in `tools/bisect.js` can be helpful here. Given a pattern for one or more unit tests, it will automatically bisect the current range:
 
 ```sh
-node utils/bisect.js --good 686378 --bad 706915 script.js
+node tools/bisect.js --good 686378 --bad 706915 script.js
-node utils/bisect.js --unit-test Response.fromCache
+node tools/bisect.js --unit-test Response.fromCache
 ```
 
 By default, it will use the Chromium revision in `src/revisions.ts` from the `main` branch and from the working tree to determine the range to bisect.

package.json

@@ -8,7 +8,7 @@
   "scripts": {
     "build:test": "wireit",
     "build": "npm run build --workspaces --if-present",
-    "check:pinned-deps": "tsx scripts/ensure-pinned-deps",
+    "check:pinned-deps": "tsx tools/ensure-pinned-deps",
     "check": "npm run check --workspaces --if-present && run-p check:*",
     "clean": "npm run clean --workspaces --if-present && rimraf **/.wireit",
     "commitlint": "commitlint --from=HEAD~1",
@@ -17,7 +17,7 @@
     "format:eslint": "eslint --ext js --ext ts --fix .",
     "format:prettier": "prettier --write .",
     "format": "run-s format:*",
-    "generate:markdown": "tsx utils/generate_docs.ts",
+    "generate:markdown": "tsx tools/generate_docs.ts",
     "lint:eslint": "([ \"$CI\" = true ] && eslint --ext js --ext ts --quiet -f codeframe . || eslint --ext js --ext ts .)",
     "lint:prettier": "prettier --check .",
     "lint": "run-s lint:prettier lint:eslint",
@@ -32,7 +32,7 @@
     "test:firefox": "run-s test:firefox:*",
     "test:install": "test/test-install.sh",
     "test:types": "tsd -t packages/puppeteer",
-    "test": "cross-env PUPPETEER_DEFERRED_PROMISE_DEBUG_TIMEOUT=20000 node utils/mochaRunner/lib/main.js"
+    "test": "cross-env PUPPETEER_DEFERRED_PROMISE_DEBUG_TIMEOUT=20000 node tools/mochaRunner/lib/main.js"
   },
   "wireit": {
     "build:test": {

packages/puppeteer-core/src/injected/README.md

@@ -2,4 +2,4 @@
 
 This folder contains code that is injected into every Puppeteer execution context. Each file is transpiled using esbuild into a script in `src/generated` which is then imported into server code.
 
-See `utils/generate_injected.ts` for more information.
+See `tools/generate_injected.ts` for more information.

packages/puppeteer-core/tools/generate_sources.ts

@@ -4,7 +4,7 @@ import {mkdir, mkdtemp, readFile, rm, writeFile} from 'fs/promises';
 import {sync as glob} from 'glob';
 import path, {join, resolve} from 'path';
 import {chdir} from 'process';
-import {job} from '../../../utils/internal/job.js';
+import {job} from '../../../tools/internal/job.js';
 
 const packageRoot = resolve(join(__dirname, '..'));
 chdir(packageRoot);

test/tsconfig.json

@@ -7,5 +7,5 @@
     "rootDir": "src"
   },
   "include": ["src"],
-  "references": [{"path": "../utils/mochaRunner/tsconfig.json"}]
+  "references": [{"path": "../tools/mochaRunner/tsconfig.json"}]
 }

tools/bisect.js

@@ -42,9 +42,9 @@ Parameters:
   --script     path to a script that returns non-zero code for BAD revisions and 0 for good
 
 Example:
-  node utils/bisect.js --unit-test test
+  node tools/bisect.js --unit-test test
-  node utils/bisect.js --good 577361 --bad 599821 --script simple.js
+  node tools/bisect.js --good 577361 --bad 599821 --script simple.js
-  node utils/bisect.js --good 577361 --bad 599821 --unit-test test
+  node tools/bisect.js --good 577361 --bad 599821 --unit-test test
 `;
 
 if (argv.h || argv.help) {

tools/mochaRunner/README.md

@@ -5,7 +5,7 @@ Mocha Runner is a test runner on top of mocha. It uses `/test/TestSuites.json` a
 ## Running tests for Mocha Runner itself.
 
 ```
-npm run build:test && npx c8 node utils/mochaRunner/lib/test.js
+npm run build:test && npx c8 node tools/mochaRunner/lib/test.js
 ```
 
 ## Running tests using Mocha Runner

utils/tsconfig.json

@@ -1,12 +0,0 @@
-/**
- * This configuration only exists for the API Extractor tool and for VSCode to use. It is NOT the tsconfig used for compilation.
- * For CJS builds, `tsconfig.cjs.json` is used, and for ESM, it's `tsconfig.esm.json`.
- * See the details in CONTRIBUTING.md that describes our TypeScript setup.
- */
-{
-  "extends": "../tsconfig.base.json",
-  "compilerOptions": {
-    "noEmit": true,
-    "module": "CommonJS"
-  }
-}

website/versioned_docs/version-18.2.0/contributing.md

@@ -59,7 +59,7 @@ We also have other tests such as `test:types` that tests types and
 
 Puppeteer is using a custom test runner on top of Mocha that consults
 the [TestExpectations.json](https://github.com/puppeteer/puppeteer/blob/main/test/TestExpectations.json)
-to see if a given test result is expected or not. See more info about the test runner in [`utils/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/utils/mochaRunner).
+to see if a given test result is expected or not. See more info about the test runner in [`tools/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/tools/mochaRunner).
 
 ## Code reviews
 
@@ -97,8 +97,8 @@ The following is a description of the primary folders in Puppeteer:
 - `src` - contains the source code for Puppeteer.
 - `test/src` - contains the source code for Puppeteer tests.
 - `utils`/`scripts` - contains various scripts.
-- `utils/testserver` - contains the source code for our test servers in testing.
+- `tools/testserver` - contains the source code for our test servers in testing.
-- `utils/mochaRunner` - contains the source code for our test runner.
+- `tools/mochaRunner` - contains the source code for our test runner.
 - `compat` - contains code separated by module import type. See [`compat/README.md`](https://github.com/puppeteer/puppeteer/blob/main/compat/README.md) for details.
 - `test-d` contains type tests using [`tsd`](https://github.com/SamVerschueren/tsd).
 - `third_party` contains all dependencies that we vendor into the final build. See the [`third_party/README.md`](https://github.com/puppeteer/puppeteer/blob/main/third_party/README.md) for details.
@@ -251,7 +251,7 @@ BINARY=<path-to-executable> npm run test:chrome:headless # Or npm run test:firef
 ```
 
 If a test is expected to fail on certain configurations or became flaky, update [TestExpectations.json](https://github.com/puppeteer/puppeteer/blob/main/test/TestExpectations.json)
-to reflect that. See more info about TestExpectations.json in [`utils/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/utils/mochaRunner).
+to reflect that. See more info about TestExpectations.json in [`tools/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/tools/mochaRunner).
 
 ## API Coverage
 
@@ -269,7 +269,7 @@ The following steps are needed to update the Chromium version.
 
 1. Find a suitable Chromium revision
    Not all revisions have builds for all platforms, so we need to find one that does.
-   To do so, run `utils/check_availability.js -rd` to find the latest suitable `dev` Chromium revision (see `utils/check_availability.js -help` for more options).
+   To do so, run `tools/check_availability.js -rd` to find the latest suitable `dev` Chromium revision (see `tools/check_availability.js -help` for more options).
 1. Update `src/revisions.ts` with the found revision number.
 1. Update `versions.js` with the new Chromium-to-Puppeteer version mapping and update `lastMaintainedChromiumVersion` with the latest stable Chrome version.
 1. Run `npm run check:protocol-revision`.
@@ -281,11 +281,11 @@ The following steps are needed to update the Chromium version.
 
 ### Bisecting upstream changes
 
-Sometimes, performing a Chromium roll causes tests to fail. To figure out the cause, you need to bisect Chromium revisions to figure out the earliest possible revision that changed the behavior. The script in `utils/bisect.js` can be helpful here. Given a pattern for one or more unit tests, it will automatically bisect the current range:
+Sometimes, performing a Chromium roll causes tests to fail. To figure out the cause, you need to bisect Chromium revisions to figure out the earliest possible revision that changed the behavior. The script in `tools/bisect.js` can be helpful here. Given a pattern for one or more unit tests, it will automatically bisect the current range:
 
 ```sh
-node utils/bisect.js --good 686378 --bad 706915 script.js
+node tools/bisect.js --good 686378 --bad 706915 script.js
-node utils/bisect.js --unit-test Response.fromCache
+node tools/bisect.js --unit-test Response.fromCache
 ```
 
 By default, it will use the Chromium revision in `src/revisions.ts` from the `main` branch and from the working tree to determine the range to bisect.