This patch:
- updates Flakiness Dashboard format to define version per-build
and to pass COMMIT information
- drops the README.md generation - we'll move on to a designated flakiness
dashboard viewer
- fix `FLAKINESS_DASHBOARD_BUILD_URL` to point to a task instead of a build
- do not pretty-print `dashboard.json` when serializing flakiness results
- filter out 'COVERAGE' test(s) so that they don't add up to `dashboard.json` payload. These are useless
- validate certain important options of flakiness dashboard
- more logging to STDOUT to actually say which repo and what branch is getting used
- enhance commit message with a build URL
- use a more compact format for JSON. For 100 runs of 700 tests it yields 21MB json instead of 23MB.
- bump default builds number to 100
This patch introduces a dashboard that records test results and
uploads them to https://github.com/aslushnikov/puppeteer-flakiness-dashboard
Since many bots might push results in parallel, each bot pushes
results to its own git branch.
FlakinessDashboard also generates a simple README.md with a flakiness
summary. If this proves to be not enough, we can build a website that
fetches flakiness data and renders it nicely.
Cirrus CI recently started supporting Mac OS builds. This PR adds a CI task to test Puppeteer on MacOS.
It uses Node 8 since only [Node 6 and Node 8 are LTS](https://github.com/nodejs/Release#release-schedule) so `brew` only has receipts for those two TLS versions besides the latest one.
Cirrus CI got some optimizations for containers based of `microsoft/windowsservercore:latest`.
Now startup time for windows builds is around 1:30 seconds instead of around 4 minutes.
to: @aslushnikov
This patch adds experimental support for Cirrus CI builds.
Cirrus CI has both Windows and Linux, uses google cloud and runs builds
in Docker containers.
This seems to be a promising combination for our needs.
