puppeteer/test
Jack Franklin 5518bac291
chore: update how we track coverage during unit tests (#5779)
* chore: update how we track coverage during unit tests

The old method of tracking coverage was causing issues. If a test failed
on CI, that test's failure would be lost because the test failing would
in turn cause the coverage to fail, but the `process.exit(1)` in the
coverage code caused Mocha to not output anything useful.

Instead the coverage checker now:

* tracks the coverage in memory in a Map (this hasn't changed)
* after all tests, writes that to disk in test/coverage.json (which is
gitignored)
* we then run a single Mocha test that asserts every method was called.

This means if the test run fails, the build will fail and give the error
about that test run, and that output won't be lost when the coverage
then fails too.

Co-authored-by: Mathias Bynens <mathias@qiwi.be>
2020-04-30 13:00:41 +01:00
..
assets (feat) Add option to fetch Firefox Nightly (#5467) 2020-03-10 21:59:03 +01:00
fixtures fix(launcher): make dumpio and pipe options work together (#4727) 2019-07-18 16:42:11 -07:00
golden-chromium test: add page.screenshot viewport clipping test (#5079) 2019-10-24 14:05:13 +02:00
golden-firefox test: add page.screenshot viewport clipping test (#5079) 2019-10-24 14:05:13 +02:00
accessibility.spec.js chore: enforce consistent spacing around object curlys (#5700) 2020-04-21 10:40:04 +01:00
assert-coverage-test.js chore: update how we track coverage during unit tests (#5779) 2020-04-30 13:00:41 +01:00
browser.spec.js chore: enforce consistent spacing around object curlys (#5700) 2020-04-21 10:40:04 +01:00
browsercontext.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
CDPSession.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
chromiumonly.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
click.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
cookies.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
coverage-utils.js chore: update how we track coverage during unit tests (#5779) 2020-04-30 13:00:41 +01:00
coverage.spec.js chore: enforce consistent spacing around object curlys (#5700) 2020-04-21 10:40:04 +01:00
defaultbrowsercontext.spec.js chore: enforce consistent spacing around object curlys (#5700) 2020-04-21 10:40:04 +01:00
dialog.spec.js chore: enforce consistent spacing around object curlys (#5700) 2020-04-21 10:40:04 +01:00
diffstyle.css Implement FrameManager 2017-06-21 14:11:52 -07:00
elementhandle.spec.js feat: add support for string-based custom queries (#5753) 2020-04-30 12:45:52 +01:00
emulation.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
evaluation.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
fixtures.spec.js chore: enforce consistent spacing around object curlys (#5700) 2020-04-21 10:40:04 +01:00
frame.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
golden-utils.js chore: enforce consistent spacing around object curlys (#5700) 2020-04-21 10:40:04 +01:00
headful.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
ignorehttpserrors.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
input.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
jshandle.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
keyboard.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
launcher.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
mocha-utils.js chore: update how we track coverage during unit tests (#5779) 2020-04-30 13:00:41 +01:00
mouse.spec.js chore: enforce consistent spacing around object curlys (#5700) 2020-04-21 10:40:04 +01:00
navigation.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
network.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
oopif.spec.js chore: migrate src/Browser to TS (#5761) 2020-04-28 14:26:37 +02:00
page.spec.js chore: disable flaky setUserAgent test in Firefox (#5780) 2020-04-30 12:10:28 +02:00
queryselector.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
README.md chore: migrate remaining tests to Mocha (#5616) 2020-04-09 20:12:32 +02:00
requestinterception.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
run_static_server.js chore(testserver): prepare test server (#3294) 2018-09-24 12:46:39 -07:00
screenshot.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
target.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
touchscreen.spec.js chore: migrate unit tests to Mocha (#5600) 2020-04-09 07:56:25 +02:00
tracing.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
utils.js chore: migrate src/FrameManager to TypeScript (#5773) 2020-04-29 13:28:16 +02:00
waittask.spec.js chore: enforce naming of errors in catch blocks (#5763) 2020-04-28 15:16:28 +02:00
worker.spec.js chore: enforce consistent spacing around object curlys (#5700) 2020-04-21 10:40:04 +01:00

Puppeteer unit tests

Unit tests in Puppeteer are written using Mocha as the test runner and Expect as the assertions library.

Test state

We have some common setup that runs before each test and is defined in mocha-utils.js.

You can use the getTestState function to read state. It exposes the following that you can use in your tests. These will be reset/tidied between tests automatically for you:

  • puppeteer: an instance of the Puppeteer library. This is exactly what you'd get if you ran require('puppeteer').
  • puppeteerPath: the path to the root source file for Puppeteer.
  • defaultBrowserOptions: the default options the Puppeteer browser is launched from in test mode, so tests can use them and override if required.
  • server: a dummy test server instance (see utils/testserver for more).
  • httpsServer: a dummy test server HTTPS instance (see utils/testserver for more).
  • isFirefox: true if running in Firefox.
  • isChrome: true if running Chromium.
  • isHeadless: true if the test is in headless mode.

If your test needs a browser instance, you can use the setupTestBrowserHooks() function which will automatically configure a browser that will be cleaned between each test suite run. You access this via getTestState().

If your test needs a Puppeteer page and context, you can use the setupTestPageAndContextHooks() function which will configure these. You can access page and context from getTestState() once you have done this.

The best place to look is an existing test to see how they use the helpers.

Skipping tests for Firefox

Tests that are not expected to pass in Firefox can be skipped. You can skip an individual test by using itFailsFirefox rather than it. Similarly you can skip a describe block with describeFailsFirefox.

There is also describeChromeOnly which will only execute the test if running in Chromium. Note that this is different from describeFailsFirefox: the goal is to get any FailsFirefox calls passing in Firefox, whereas describeChromeOnly should be used to test behaviour that will only ever apply in Chromium.

Running tests

Despite being named 'unit', these are integration tests, making sure public API methods and events work as expected.

  • To run all tests:
npm run unit
  • To run a specific test, substitute the it with it.only:
  ...
  it.only('should work', async function() {
    const {server, page} = getTestState();
    const response = await page.goto(server.EMPTY_PAGE);
    expect(response.ok).toBe(true);
  });
  • To disable a specific test, substitute the it with xit (mnemonic rule: 'cross it'):
  ...
  // Using "xit" to skip specific test
  xit('should work', async function({server, page}) {
    const {server, page} = getTestState();
    const response = await page.goto(server.EMPTY_PAGE);
    expect(response.ok).toBe(true);
  });
  • To run tests in non-headless mode:
HEADLESS=false npm run unit
  • To run tests with custom browser executable:
BINARY=<path-to-executable> npm run unit