puppeteer/test
Andrey Lushnikov f753ec6b04
chore(testrunner): distinguish between TERMINATED and CRASHED (#4821)
`testRunner.run()` might have 4 different outcomes:
- `ok` - all non-skipped tests passed
- `failed` - some tests failed or timed out
- `terminated` - process received SIGHUP/SIGINT while testrunner was running tests. This happens on CI's under certain circumstances, e.g. when
  VM is getting re-scheduled.
- `crashed` - testrunner terminated test execution due to either `UnhandledPromiseRejection` or
  some of the hooks (`beforeEach/afterEach/beforeAll/afterAll`) failures.

As an implication, there are 2 new test results: `terminated` and `crashed`.
All possible test results are:
- `ok` - test worked just fine
- `skipped` - test was skipped with `xit`
- `timedout` - test timed out
- `failed` - test threw an exception while running
- `terminated` - testrunner got terminated while running this test
- `crashed` - some `beforeEach` / `afterEach` hook corresponding to this
test timed out of threw an exception.

This patch changes a few parts of the testrunner API:
- `testRunner.run()` now returns an object `{result: string,
terminationError?: Error, terminationMessage?: string}`
- the same object is dispatched via `testRunner.on('finished')` event
- `testRunner.on('terminated')` got removed
- tests now might have `crashed` and `terminated` results
- `testRunner.on('teststarted')` dispatched before running all related
`beforeEach` hooks, and `testRunner.on('testfinished')` dispatched after
running all related `afterEach` hooks.
2019-08-08 15:15:09 -07:00
..
assets fix(launcher): disable component extensions (#4704) 2019-07-15 17:54:42 -07:00
fixtures fix(launcher): make dumpio and pipe options work together (#4727) 2019-07-18 16:42:11 -07:00
golden-chromium chore: further unify Puppeteer-Firefox tests with Puppeteer (#3931) 2019-02-06 13:49:14 -08:00
golden-firefox fix(firefox): properly round clip when doing element screenshots (#4001) 2019-02-13 19:47:14 -08:00
accessibility.spec.js feat: root option in page.accessibility.snapshot() (#4318) 2019-05-09 23:39:42 -07:00
browser.spec.js fix(test): browser.disconnect should not be awaited (#4446) 2019-05-21 17:26:04 +03:00
browsercontext.spec.js fix(test): browser.disconnect should not be awaited (#4446) 2019-05-21 17:26:04 +03:00
CDPSession.spec.js test: cleanup tests (#4078) 2019-02-26 20:36:55 -08:00
chromiumonly.spec.js feat(page): introduce file chooser interception (#4653) 2019-07-22 21:30:49 -07:00
click.spec.js fix(firefox): fix missing awaits in mouse.click (#4561) 2019-06-10 16:53:38 -07:00
cookies.spec.js test: add cookie tests (#4365) 2019-04-30 00:35:41 -07:00
coverage.spec.js chore(test): label coverage+debuger test with upstream bug (#4807) 2019-08-05 15:22:01 -07:00
dialog.spec.js test: split out dialog tests (#3586) 2018-11-20 20:18:57 -08:00
diffstyle.css Implement FrameManager 2017-06-21 14:11:52 -07:00
elementhandle.spec.js feat(firefox): support Page.emualteMedia (#4056) 2019-02-22 14:04:40 -08:00
emulation.spec.js feat(puppeteer): introduce puppeteer.errors and puppeteer.devices (#4312) 2019-04-19 15:33:06 -07:00
evaluation.spec.js fix(tests): make eval test resilient to error format (#4793) 2019-08-01 14:18:35 -07:00
fixtures.spec.js fix(launcher): make dumpio and pipe options work together (#4727) 2019-07-18 16:42:11 -07:00
frame.spec.js test: make sure frames are reported from-inside shadow DOM (#4167) 2019-03-15 21:01:36 -07:00
golden-utils.js fix: set JPG background to white when omitBackground option is used (#3240) 2018-09-14 11:03:33 +01:00
headful.spec.js test: add page.bringToFront test (#4778) 2019-08-01 17:08:29 -07:00
ignorehttpserrors.spec.js fix(tests): Get TLS version from Node instead of hardcoding it (#4700) 2019-07-14 18:33:31 -07:00
input.spec.js feat(page): introduce file chooser interception (#4653) 2019-07-22 21:30:49 -07:00
jshandle.spec.js fix(firefox): enable more firefox tests (#4007) 2019-02-13 23:59:38 -08:00
keyboard.spec.js fix(firefox): keyboard tests (#4082) 2019-02-27 15:13:17 -08:00
launcher.spec.js fix(tests): Get TLS version from Node instead of hardcoding it (#4700) 2019-07-14 18:33:31 -07:00
mouse.spec.js fix(test): enable cross process viewport test (#4806) 2019-08-05 15:22:16 -07:00
navigation.spec.js feat(chromium): roll Chromium to r654752 (#4358) 2019-04-28 20:19:01 -07:00
network.spec.js test: split out request interception tests (#4278) 2019-04-11 21:33:01 -04:00
oopif.spec.js feat(chromium): roll Chromium to r665405 (#4516) 2019-06-04 11:03:15 -07:00
page.spec.js fix(page): do evaluations with one roundtrip (#4539) 2019-06-07 13:46:43 -07:00
puppeteer.spec.js test: add oopif tests (#4495) 2019-05-29 16:49:44 -07:00
queryselector.spec.js test: split queryselector tests into separate file (#3579) 2018-11-20 18:45:37 -08:00
requestinterception.spec.js fix(network): be able to remove headers using request.continue (#4797) 2019-08-05 15:26:17 -07:00
run_static_server.js chore(testserver): prepare test server (#3294) 2018-09-24 12:46:39 -07:00
screenshot.spec.js fix(firefox): properly round clip when doing element screenshots (#4001) 2019-02-13 19:47:14 -08:00
target.spec.js feat(workers): create workers from service workers and shared workers (#4397) 2019-05-09 17:29:18 -07:00
test.js chore(flakiness): update flakiness format (#4808) 2019-08-06 15:32:55 -07:00
touchscreen.spec.js feat(puppeteer): introduce puppeteer.errors and puppeteer.devices (#4312) 2019-04-19 15:33:06 -07:00
tracing.spec.js fix(tracing): start without options (#4388) 2019-05-09 17:15:33 -07:00
utils.js chore(testrunner): distinguish between TERMINATED and CRASHED (#4821) 2019-08-08 15:15:09 -07:00
waittask.spec.js feat(puppeteer): introduce puppeteer.errors and puppeteer.devices (#4312) 2019-04-19 15:33:06 -07:00
worker.spec.js chore(firefox): mark all Puppeteer tests that are failing on FF (#3924) 2019-02-05 22:32:41 -08:00