When using a custom Firefox profile for Puppeteer the modified
preferences as present in prefs.js need to be reset once the
profile is no longer needed by Puppeteer. If not done this could
cause side-effects when the profile is used next time outside
of Puppeteer.
As ride-along fix the "--foreground" argument for Firefox will
only be used on MacOS because that's the only supported platform.
This updates the regular expression used to parse aria attribute
selectors so that single quotes may be used as an alternative to double
quotes, e.g. `aria/Single button[role='button']`.
Issues: #7721
Co-authored-by: Andy Earnshaw <andy.earnshaw@gmail.com>
This pull request to adds better support for OOP iframes (see #2548)
The current problem with OOP iframes is that they are moved to a different target. Because of this, the previous versions of Puppeteer pretty much ignored them.
This change extends the FrameManager to already take OOP iframes into account and hides the fact that those frames are actually in different targets.
Further work needs to be done to also make the NetworkManager aware of these and to make sure that settings like emulations etc. are also properly passed down to the new targets.
In some situations, Puppeteer is left in an invalid state because protocol errors that could have been handled by the user where just hidden from them. This patch removes some of these cases and also makes sure that unhandled promise rejections lead to a test failure in mocha.
Enable developers to handle 'Invalid header' errors instead of hiding them to make sure they can address them properly.
Co-authored-by: Jan Scheffler <janscheffler@chromium.org>
Sometimes an element has not been layed out yet and, in this case,
clickablePoint fails because backend cannot compute content quads.
Co-authored-by: Jan Scheffler <janscheffler@chromium.org>
Puppeteer already allows creating a new CDP session
via target.createCDPSession but there is no way
to get access to any existing session to send
some additional commands.
Until now, the click would be always sent to the middle
point of the target element. With this change, one can define
offsets relative to the border box of the elements and click
different areas of an element.
Up to now, only strings starting with '//' are considered as to XPath selectors. Unfortunately, this is too restricting. This fix allows valid XPath selectors starting with: '/', './', and even '(//*[1])'
This change adds a new `channel` parameter to `puppeteer.launch`. When specified, Puppeteer will search for the locally installed release channel of Google Chrome and use it to launch. Available values are `chrome`, `chrome-beta`, `chrome-canary`, `chrome-dev`. This parameter is mutually exclusive with `executablePath`.
With this change,`request.respond`, `request.abort`, and `request.continue` can accept an optional `priority` to activate Cooperative Intercept Mode. In Cooperative Mode, all intercept handlers are guaranteed to run and all async handlers are awaited. The interception is resolved to the highest-priority resolution. See _Cooperative Intercept Mode and Legacy Intercept Mode_ in `docs/api.md` for details.
This commit adds drag-and-drop support, leveraging new additions to the CDP Input domain (Input.setInterceptDrags, Input.dispatchDragEvent, and Input.dragIntercepted).
Just some code reorder. We had a describe between it calls. I'm moving that describe to the end
Co-authored-by: Jack Franklin <jacktfranklin@chromium.org>
* chore: enforce pinned dependencies
Because we don't check our `package-lock.json` in, we can end up with
different versions installed locally vs CI, or even two devs having
different versions. Let's pin and enforce we pin every version to
avoid this.
This PR updates some code to remove constant ESLint warnings. It also
upgrades those warnings to errors - so that they have to be resolved
as part of the PR, rather than landing as a warning and causing noise.
Fixes#7229.
We're seeing odd failures with Prettier on some CI branches; my hunch is that they are installing different versions of the package and therefore getting formatting conflicts. This PR updates them all and pins them to specific versions - something we should probably consider generally, or remove our `package-lock.json` from the gitignore.
The existing behavior is expected to be unchanged as the value defaults to true.
Adding such option would allow user to skip the initial wait.
Issue: #3630
* fix: make `$` and `$$` selectors generic
This means, much like TS's in built `querySelector` type, you can now do:
```ts
const listItems = page.$$<HTMLLIElement>('ul li');
```
And/or:
```ts
const h2 = page.$<HTMLHeadingElement>('h2');
```
And the return value will be of type `ElementHandle<T>|null`, where `T`
is the type you provided. By default `T` is an `Element`, so you don't
have to provide this if you don't care as a consumer about the exact
type you get back.
* chore: fix test assertions
* feat(chromium): roll Chromium to r856583
This corresponds to Chromium 90.0.4427.0
This roll includes:
- Add sourceScheme, sourcePort, and sameParty to DevTools backend (https://crbug.com/1170548, https://crbug.com/1142606)
We were blocked on doing this because API Extractor didn't support it,
but now it does, so we can bump TS and the API tooling in one go. None
of the breaking changes in TS4 cause us any issues.
During the migration to TS we changed `jsonValue` so it returned
`<Record<string, unknown>>`. This is only true if all the JSON values it
returns are objects; but it could return an array, a string, a number,
etc. Therefore we make the type generic, setting the default to
`unknown`, so the user has control over the type.
This corresponds to Chromium 90.0.4403.0
This roll includes:
- Cut screenshot by ViewPort size, not position (crrev.com/c/2643792)
BREAKING CHANGE:
- `page.screenshot` cuts screenshot content by the ViewPort size, not ViewPort position.
This corresponds to Chromium 89.0.4389.0.
This roll includes:
- Add `SameParty` attribute to cookies
https://crrev.com/c/2598846
- Anchor `target=_blank` implies `rel=noopener`
https://crrev.com/c/1630010
- Don’t expect ignored elements in the AXTree
https://crrev.com/c/2505362
BREAKING CHANGE: The built-in `aria/` selector query handler doesn’t return ignored elements anymore.
Issue: #6758
This patch enables more tests for Firefox. These tests are enabled in Mozilla's CI for Firefox. The extra error handling here prevents hangs in the test harness in that environment.
The `Puppeteer` class had two concerns:
* connect to an existing browser
* launch a new browser
The first of those concerns is needed in all environments, but the
second is only needed in Node.
https://github.com/puppeteer/puppeteer/pull/6484 landing enabled us to
pull the `Puppeteer` class apart into two:
1. `Puppeteer` which hosts the behaviour for connecting to existing
browsers.
2. `PuppeteerNode`, which extends `Puppeteer` and also adds the ability
to launch a new browser.
This is a non-breaking change, because Node users will still get an
instance of a class with all the methods they expect, but it'll be a
`PuppeteerNode` rather than `Puppeteer`. I don't expect this to cause
people any issues.
We also now have new files that are effectively the entry points for
Puppeteer:
* `node.ts`: the main entry point for Puppeteer on Node.
* `web.ts`: the main entry point for Puppeteer on the web.
* `node-puppeteer-core.ts`: for those using puppeteer-core (which only
exists in Node, not on the web).
Launching headless with a relative `userDataDir` hangs on Windows. Fix by calling `path.resolve` (idempotent) to add an absolute path instead in `defaultArgs`.
Issues: #3453
The `Launcher` class was serving two purposes:
1. Launch browsers
2. Connect to browsers
Number 1) only needs to be done in Node land, but 2) is agnostic; in a
browser version of Puppeteer we'll need the ability to connect over a
websocket to send commands back and forth.
As part of the agnostification work we needed to split the `Launcher` up
so that the connection part can be made agnostic. Additionally, I
removed dependencies on `https`, `http` and `URL` from Node, instead
leaning on fetch (via `node-fetch` if in Node land) and the browser
`URL` API (which was added to Node in Node 10).
This commit adds a new built-in handler for querying by accessible name and role (#6307).
Support for waitForSelector will be added in a follow-up commit.
In `src/common` we now use `fs.promises.X` which we can dynamically
`import`. In a browser environment this code will never run because it's
gated on `isNode` (in a future PR we will add tree-shaking to the bundle
step such that this code is eliminated). By using `import`, we ensure
TypeScript still can track types and give good type information.
In `src/node` we continue to use `util.promisify` but that's not a
concern as that code explicitly is never run in the browser.
This commit changes the internal representation of query handlers to contain Puppeteer-level code instead of page functions.
The interface `CustomQueryHandler` is introduced for user-defined query handlers. When a `CustomQueryHandler` is registered using `registerCustomQueryHandler` a corresponding Puppeteer-level handler is created through `makeQueryHandler` by wrapping the page functions as appropriate.
The internal query handlers (defined by the interface `QueryHandler`) contain two new functions: `waitFor` and `queryAllArray`.
- `waitFor` allows page-based handlers to make use of the `WaitTask`-backed implementation in `DOMWorld`, whereas purely Puppeteer-based handlers can define an alternative approach instead.
- `queryAllArray` is similar to `queryAll` but with a slightly different interface; it returns a `JSHandle` to an array with the results as opposed to an array of `ElementHandle`. It is used by `$$eval`.
After this change, we can introduce built-in query handlers that are not executed in the page context (#6307).
The logic for waitForXPath and waitForSelector is currently very tightly coupled. This commit tries to untangle that relationship. This is the first step towards introducing built-in query handlers that are not executed in the page context (#6307).
This corresponds to Chromium 85.0.4182.0.
This roll includes:
- Enable SameSiteByDefaultCookies and CookiesWithoutSameSiteMustBeSecure
https://crrev.com/c/2231445
- [FlexNG] Enable FlexNG by default
https://crrev.com/c/2216595Closes#6151.
* chore: vendor Mitt into src/common/third-party
As discussed in #6203 we need to vendor our common dependencies in so
that when we ship an ESM build all imports point to file paths and do
not rely on Node resolution (e.g. a browser does not understand `import
mitt from 'mitt'`).
* chore: enforce file extensions on imports
To make our output agnostic it should include file extensions in the
output, as per the ESM spec. It's a bit odd for Node packages but makes
it easier to publish a browser build.
* feat(chromium): roll Chromium to r768783
* fix: update unit test for crrev:2135046
* chore: update devtools-protocol revision
Co-authored-by: Changhao Han <changhaohan@chromium.org>
Now the async hooks helper is gone api.ts was only used by the coverage
tools and by doclint.
DocLint is nearing the end of its lifespan with the TSDoc work, so I
focused on how best to define a list of modules for the coverage
tooling. They define an object of classes, and the path to that module.
They need the full path because we also check if the module exports any
events that need to be emitted - the coverage tool asserts that the
emitting of those events is also tested.
It's not _great_ that DocLint relies on a constant defined in the
coverage utils, but it should only be this way for a short period of
time and no one is actively working on DocLint (bar the effort to remove
it) so I don't think this is worth worrying about.
This change also broke the DocLint tests; based on the fact that DocLint is on its way out it doesn't feel worth fixing the tests, so this commit also removes them.
* chore: remove `installAsyncStackHooks` helper
This code was written when browsers/Node didn't support errors in async
functions very well. They now do a much better job of this, so we can
lose the additonal complexity from our codebase and leave it to the host
environment :)
* lazy launcher is private
* remove async stack test
This pulls in the types (based on the DefinitelyTyped repo) for
`page.$eval` (and the `$eval` method on other classes). The `$eval`
method is quite hard to type due to the way we wrap and unwrap
ElementHandles that are passed to / returned from the `pageFunction`
that users provide.
Longer term we can improve the types by providing type overloads as
DefinitelyTyped does but I've deferred that for now (see the `TODO` in
the code for more details).
This change started as a small change to pull types from DefinitelyTyped over to
Puppeteer for the `evaluateHandle` function but instead ended up also fixing
what looks to be a long standing issue with our existing documentation.
`evaluateHandle` can in fact return an `ElementHandle` rather than a `JSHandle`.
Note that `ElementHandle` extends `JSHandle` so whilst the docs are technically
correct (all ElementHandles are JSHandles) it's confusing because JSHandles
don't have methods like `click` on them, but ElementHandles do.
if you return something that is an HTML element:
```
const button = page.evaluateHandle(() => document.querySelector('button'));
// this is an ElementHandle, not a JSHandle
```
Therefore I've updated the original docs and added a large explanation to the
TSDoc for `page.evaluateHandle`.
In TypeScript land we'll assume the function will return a `JSHandle` but you
can tell TS otherwise via the generic argument, which can only be `JSHandle`
(the default) or `ElementHandle`:
```
const button = page.evaluateHandle<ElementHandle>(() => document.querySelector('button'));
```
The headful one I'm permanently skipping as I don't know what the issue is and I can't debug without getting my hands on a Windows machine. If anyone has one or is able to help, that'd be great!
The other I'm deferring another month and will ping the FF folks :)
This CL migrates all the tests to TypeScript. The main benefits of this is that we start consuming our TypeScript definitions and therefore find errors in them. The act of migrating found some bugs in our definitions and now we can be sure to avoid them going forwards.
You'll notice the addition of some `TODO`s in the code; I didn't want this CL to get any bigger than it already is but I intend to follow those up once this lands. It's mostly figuring out how to extend the `expect` types with our `toBeGolden` helpers and some other slight confusions with types that the tests exposed.
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
As far as I can tell these became irrelevant as of v1.15 which added
`puppeteer.errors` and `puppeteer.devices [1]. This is a breaking change
but one that's easily mitigated. We've said that we don't consider
changes to our folder/file structure a breaking change, but we can't
really do that if we have these two top level files that we've
documented.
[1]: e3abb0aa32 (diff-522b24108d7446af4c59873472a90444)
These files will be used by both the web and node versions of Puppeteer.
Another name for this might be "core" but I don't want to cause
confusion with the puppeteer-core package that we publish at the moment.
Fix child process killing when the parent process SIGINTs.
If you `ctrl + c` the Puppeteer parent process, we would sometimes not properly handle killing of the child processes. This would then leave child processes behind, with running Chromium instances. This in turn could block Puppeteer from launching again and results in
cryptic errors.
Instead of using the generic `process.kill` with the process id (which for some reason is negative the pid, which I don't get), we can kill the child process directly by calling `proc.kill`.
Fixes#5729.
Fixes#4796.
Fixes#4963.
Fixes#4333.
Fixes#1825.
* chore: remove "Extracting..." log message
Fixes#5741.
* test: support extra Launcher options and skips
The extra Launcher options and skipping conditions enable
unit tests to be run more easily by third-parties, e.g.
browser vendors that are interested in Puppeteer support.
Extra Launcher options were previously removed as part of
switching away from the custom test harness.
* test: enable more tests for Firefox
We deferred this initially because our Windows CI built wasn't stable
and so debugging this was hard. It's now much more stable so let's push
this back a month but at the same time I'll reach out to the Moz folks
as it should be easier to debug reliably now CI is stable on Windows.
* Don't use expect within Promises (#5466)
If a call to expect fails within a Promise it will not
be resolved, and causing the test to crash.
The patch aligns the code similar to what is used by all
the other tests.
* chore: fix invalid SSL assertion on Catalina
The error Chrome gives with an invalid cert changes between older Mac
versions and Catalina as detailed here:
https://support.google.com/chrome/thread/18125056?hl=en.
This PR changes Travis to run Catalina (and we think most devs run up to
date OS versions) so this fix ensures the test behaviour is consistent
locally and on Travis.
For those on older Mac versions I've left a comment by the tests to
hopefully save them debugging!
Co-authored-by: Mathias Bynens <mathias@qiwi.be>
The coverage utils depend on `src/api.ts` being up to date and pointing to the right modules. If they aren't, you would get a cryptic error on CI:
```
1) "before all" hook in "{root}":
TypeError: Cannot read property 'prototype' of undefined
at traceAPICoverage (test/coverage-utils.js:40:54)
at Context.before (test/coverage-utils.js:103:7)
2) "after all" hook in "{root}":
TypeError: Cannot read property 'stop' of undefined
at Context.after (test/mocha-utils.js:168:22)
```
This change logs a clearer error that highlights the missing class and exits, so it's much easier to realise what's gone wrong.
Ideally the coverage wouldn't need a hardcoded list of sources, but until then this will help spot this error in the future.
