Commit Graph

27 Commits

Author SHA1 Message Date
Andrey Lushnikov
addd7f4c6a
chore(testrunner): introduce verbose mode (#3946) 2019-02-07 13:17:29 -08:00
Andrey Lushnikov
6887ad881a
chore: further unify Puppeteer-Firefox tests with Puppeteer (#3931)
This patch:
* unifies assets between tests
* enables a few puppeteer tests on Puppeteer-Firefox

Drive-by: beautify failing output of `expect.toEqual` matcher.

References #3889
2019-02-06 13:49:14 -08:00
Andrey Lushnikov
14fb3e38db
chore(firefox): mark all Puppeteer tests that are failing on FF (#3924)
This patch:
- introduces new testRunner methods `addTestDSL` and `addSuiteDSL`
  to add annotated test / suite.
- introduces new test/suite declaration methods: `it_fails_ffox` and
  `describe_fails_ffox`. These are equal to `it`/`describe` for chromium
  tests and to `xit`/`xdescribe` for firefox.
- marks all unsupported tests with `it_fails_ffox`
- adds a new command-line flag `'--firefox-status'` to `//test/test.js`.
  This flag dumps current amount of tests that are intentionally skipped
  for Firefox.

End goal: get rid of all `it_fails_ffox` and `describe_fails_ffox`
tests.

Drive-By: remove cookie tests  "afterEach" hook that was removing
cookies - it's not needed any more since every test is run in a
designated browser context.

References #3889
2019-02-05 22:32:41 -08:00
Andrey Lushnikov
bc71e9236d
chore(testrunner): nicer colors for test reporter (#3921) 2019-02-05 19:02:29 -08:00
Andrey Lushnikov
a210dd7ee2
chore(testrunner): show 5 slowest tests on CI (#3919)
Show top slowest tests on CI.

Drive-by: some facelifting for the test reporter.
2019-02-05 15:28:35 -08:00
Andrey Lushnikov
bea26a7f58
chore(testrunner): sort workers numerically when terminating (#3782) 2019-01-15 16:40:58 -08:00
Andrey Lushnikov
9c89090f73
chore(testrunner): fix readme description (#3293) 2018-09-21 20:44:43 -07:00
Andrey Lushnikov
12e317ce17
chore: add .npmignore for testrunner (#3290) 2018-09-21 17:27:10 -07:00
Andrey Lushnikov
5b3ddf554a
chore(testrunner): bump version to v0.5.0-post (#3291) 2018-09-21 15:25:17 -07:00
Andrey Lushnikov
907d9bed6d
chore: prepare testrunner to be published to npm (#3289)
testrunner will be published as @pptr/testrunner.
2018-09-21 14:51:22 -07:00
Andrey Lushnikov
a1a211d9e7
chore: nicer stack highlight (#3259)
Highlight part of the stack that points to where the actual
test failure happened.
2018-09-17 23:22:53 +01:00
Andrey Lushnikov
204c7ec8c4
feat: introduce puppeteer/Errors (#3056)
This patch adds a new require, `puppeteer/Errors`, that
holds all the Puppeteer-specific error classes.

Currently, the only custom error class we use is `TimeoutError`. We'll
expand in future with `CrashError` and some others.

Fixes #1694.
2018-08-09 16:51:12 -07:00
Andrey Lushnikov
862ad96bde
test: dump workers state after the test runner is terminated (#2991)
This will help us to debug the flakes on the CI.
2018-07-30 18:57:48 -07:00
Andrey Lushnikov
0b94fa70eb
chore: stop using console.assert everywhere (#2646)
Since Node 10, `console.assert` no longer throws an AssertionError.
(This is generally good since it aligns Node.js with Browsers.)

This patch migrates all usages of `console.assert` in our codebase.
- All the `lib/` and testing code is migrated onto a handmade `assert`
function. This is to make Puppeteer transpilation / bundling easier.
- All the tooling is switched to use Node's `assert` module.

Fixes #2547.
2018-05-31 16:53:51 -07:00
Joel Einbinder
1bbd094624 feat(test): enable dumpio in tests #2610
This patch allows logging the output of the Chromium process to be enabled in tests by passing in the environment variable `DUMPIO=true`.

Additionally, the `stderr` of the Chromium process will always be logged in the the "Output" section of failing page tests.
2018-05-29 15:45:03 -07:00
Joel Einbinder
8e12d197a2 fix(tests): normalize scrollbars in headful tests (#2622)
Scrollbars look different on different platforms, so must be made invisible in tests. As a drive-by, xdescribe was broken with the new test runner.

References #2524
2018-05-29 15:30:08 -07:00
Andrey Lushnikov
8a62b10fd0
chore: testrunner's ".not" should print appropriate message (#2459)
When failing, the ".not" matchers should print their name
with ".not" prefix.
2018-04-26 11:13:22 -07:00
Andrey Lushnikov
beea6f9c82
chore(testrunner): fix typo in readme 2018-04-10 12:25:14 -07:00
Andrey Lushnikov
afcc74e7c7
test: fix Matcher.toEqual to not depend on key insertion order (#2091)
Objects `{foo: 1, bar: 2}` and `{bar: 2, foo: 1}` should be considered
equal.
2018-03-15 11:49:53 -07:00
Alix Axel
bc6902623a fix: allow timeouts of be 0 (#1964)
This patch fixes timeouts for `puppeteer.launch` and `page.waitForFunction`
to be `0`.

Fixes #1960 .
2018-02-05 17:42:54 -05:00
Andrey Lushnikov
bae44a82d8
test: teach test reporter to highlight test stackframe on failures. (#1930)
This patch teaches reporter to highlight test stack frame on
test failures.
2018-01-30 15:23:47 -08:00
Andrey Lushnikov
508a78d99b
test: prohibit focused tests/suites on bots (#1686) 2017-12-28 15:41:57 -08:00
Andrey Lushnikov
be438c59c1
chore(testrunner): support first-class test debugging (#1606)
This patch teaches testrunner to override both timeout and
parallel execution option if there's attached inspector.
2017-12-15 14:25:06 -08:00
Andrey Lushnikov
a3a3774926
test: nicer matchers output (#1605)
This patch improves matchers to provide more information once
some of them fail.
2017-12-15 14:14:27 -08:00
Andrey Lushnikov
a5db6d412c
test: migrate test.js to support concurrent test execution (#1531)
This patch migrates tests so that they can be run concurrently.

- By default, tests still run in one thread.
- To run tests in 4 parallel threads, run node test/test.js -j 4 or npm run unit -- -j 4
- Environment variable PPTR_PARALLEL_TESTS could be set to override default parallelization

Every test gets passed in a state. State is set up in the beforeAll and beforeEach callbacks,
and should be teared down in the afterAll and afterEach callbacks.

By default, state has a parallelIndex variable initialized that defines the thread index that runs the execution.
2017-12-12 13:34:21 -08:00
Andrey Lushnikov
fc1f15e251
test(testrunner): fix the 'timeout: 0' test runner setting (#1561) 2017-12-08 16:44:24 -08:00
Andrey Lushnikov
e6725e15af
tests: drop jasmine test runner (#1519)
This patch introduces a tiny test runner to run puppeteer tests.
The test runner is self-container and allows parallel (wrt IO) test execution.
It will also allow us to split tests into multiple files if necessary.

Comparing to the jasmine, the testrunner supports parallel execution, properly
handles "unhandled promise rejection" event and signals.

Comparing to ava/jest, the testrunner doesn't run multiple node processes,
which makes it simpler but sufficient for our goals.
2017-12-07 16:37:22 -08:00