Using RxJS greatly simplifies the control flow for locators and comes with automatic cleanup on failure. It greatly simplifies the `signal` logic and the retry logic.
This PR fixes the following edge case:
- `const oldPromise = world.puppeteerUtil`.
- setContext occurs but context is immediately destroyed, i.e.
`world.#puppeteerUtil === oldPromise` is not resolved.
- clearContext occurs due to destruction, i.e. `world.#puppeteerUtil` is
replaced (`world.#puppeteerUtil !== oldPromise`).
- `oldPromise` never resolves.
<!-- Thanks for submitting a pull request! Please provide enough
information so that others can review your pull request. -->
**What kind of change does this PR introduce?**
Test expectation file update
**Did you add tests for your changes?**
no
**Summary**
Unskippes tests for Firefox and linux, which were missed in the previous
update
**Does this PR introduce a breaking change?**
no
Issue: #9394
**What kind of change does this PR introduce?**
Feature - Added support of the `referrerPolicy` parameter (from CDP's
`Page.navigate`) to Puppeteer's `page.goto`.
**Did you add tests for your changes?**
No, currently it has no meaning that isn't browser implementation based,
which might get broken in the future. If there are suggestions to tests,
please let me know so I'll add them.
**If relevant, did you update the documentation?**
Yes, the documentation of the `goto` method has been updated.
**Summary**
I wanted to contribute to this project, which I used for testing on our
environment and saw issue #9394, so I decided to resolve it :)
**Does this PR introduce a breaking change?**
No. I added the `referrerPolicy` as an optional parameter, which will
use the HTTP header `Referrer-Policy` if not provided (much like the
`referer` parameter) so it will not interfere.
Co-authored-by: Ophir Back <ophir.back@broadcom.com>
What kind of change does this PR introduce?
Test expectation file update
Summary
Closes https://github.com/puppeteer/puppeteer/issues/9118.
This PR updates the test expectation file with more specific status for
firefox (like fail, fail-pass), removes the duplications.
Does this PR introduce a breaking change?
no
Bumps [glob](https://github.com/isaacs/node-glob) from 8.0.3 to 8.1.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/isaacs/node-glob/blob/main/changelog.md">glob's
changelog</a>.</em></p>
<blockquote>
<h2>8.1</h2>
<ul>
<li>Add <code>windowsPathsNoEscape</code> option</li>
</ul>
<h2>8.0</h2>
<ul>
<li>Only support node v12 and higher</li>
<li><code>\</code> is now <strong>only</strong> used as an escape
character, and never as a
path separator in glob patterns, so that Windows users have a
way to match against filenames containing literal glob pattern
characters.</li>
<li>Glob pattern paths <strong>must</strong> use forward-slashes as path
separators, since <code>\</code> is an escape character to match literal
glob pattern characters.</li>
<li>(8.0.2) <code>cwd</code> and <code>root</code> will always be
automatically coerced
to use <code>/</code> as path separators on Windows, as they cannot
contain glob patterns anyway, and are often supplied by
<code>path.resolve()</code> and other methods that will use
<code>\</code> path
separators by default.</li>
</ul>
<h2>7.2</h2>
<ul>
<li>Add fs option to allow passing virtual filesystem</li>
</ul>
<h2>7.1</h2>
<ul>
<li>Ignore stat errors that are not <code>ENOENT</code> to work around
Windows issues.</li>
<li>Support using root and absolute options together</li>
<li>Bring back lumpy space princess</li>
<li>force 'en' locale in string sorting</li>
</ul>
<h2>7.0</h2>
<ul>
<li>Raise error if <code>options.cwd</code> is specified, and not a
directory</li>
</ul>
<h2>6.0</h2>
<ul>
<li>Remove comment and negation pattern support</li>
<li>Ignore patterns are always in <code>dot:true</code> mode</li>
</ul>
<h2>5.0</h2>
<ul>
<li>Deprecate comment and negation patterns</li>
<li>Fix regression in <code>mark</code> and <code>nodir</code> options
from making all cache
keys absolute path.</li>
<li>Abort if <code>fs.readdir</code> returns an error that's
unexpected</li>
<li>Don't emit <code>match</code> events for ignored items</li>
<li>Treat ENOTSUP like ENOTDIR in readdir</li>
</ul>
<h2>4.5</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="1b6bf20239"><code>1b6bf20</code></a>
8.1.0</li>
<li><a
href="1756fccfe7"><code>1756fcc</code></a>
add windowsPathsNoEscape option</li>
<li><a
href="af57da21c7"><code>af57da2</code></a>
update tap, libtap</li>
<li><a
href="e19db65d23"><code>e19db65</code></a>
Remove dependency on path-is-absolute</li>
<li>See full diff in <a
href="https://github.com/isaacs/node-glob/compare/v8.0.3...v8.1.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
<!-- Thanks for submitting a pull request! Please provide enough
information so that others can review your pull request. -->
**What kind of change does this PR introduce?**
types fix
**Did you add tests for your changes?**
No
**If relevant, did you update the documentation?**
Not relevant
**Summary**
I am updating `import`s of `devtools-protocol` to always use named
import instead of the default.
For some reason, typescript doesn't understand when it is imported as
default.
```
node_modules/puppeteer-core/lib/esm/puppeteer/common/TargetManager.d.ts:23:59 - error TS2694: Namespace '"/Users/kyrylo/pacakge/node_modules/puppeteer-core/node_modules/devtools-protocol/types/protocol"' has no exported member 'Target'.
23 export declare type TargetFactory = (targetInfo: Protocol.Target.TargetInfo, session?: CDPSession) => Target;
~~~~~~
Found 1 error in node_modules/puppeteer-core/lib/esm/puppeteer/common/TargetManager.d.ts:23
```
**Does this PR introduce a breaking change?**
No
<!-- If this PR introduces a breaking change, please describe the impact
and a migration path for existing applications. -->
**Other information**
I am updating `import`s of `devtools-protocol` to always use named
import instead of the default.
<!-- Thanks for submitting a pull request! Please provide enough
information so that others can review your pull request. -->
**What kind of change does this PR introduce?**
This is a change to a custom mocha runner to look for expectation of the
test case by the whole test name instead of by the part of the name.
**Summary**
Working on integration of the puppeteer expectation file in mozilla repo
and unskipping a lot of tests, I've noticed that some tests get wrong
statuses. For example, a test case with the name `navigation Page.goto
should fail when navigating to bad SSL` got the status of `navigation
Page.goto should fail when navigating to bad SSL after redirects` or
`ElementHandle specs ElementHandle.boundingBox should work` get the
status of `ElementHandle specs ElementHandle.boundingBox should work
with SVG nodes`. So it seems like checking for the whole name of the
test should be safer, but let me know if I'm missing something here.
**Does this PR introduce a breaking change?**
no
Issue: #1313
<!-- Thanks for submitting a pull request! Please provide enough
information so that others can review your pull request. -->
**What kind of change does this PR introduce?**
Feature.
**Did you add tests for your changes?**
Yes.
**If relevant, did you update the documentation?**
Yes.
**Summary**
Supports keyboard shotcuts on MacOS. See Chrome Devtools Protocol
document:
https://chromedevtools.github.io/devtools-protocol/tot/Input/#method-dispatchKeyEvent.
<!-- Thanks for submitting a pull request! Please provide enough
information so that others can review your pull request. -->
**What kind of change does this PR introduce?**
- adds missing golden asset for firefox
**Summary**
- This change will let us see in mozilla ci, where we don't skip this
test, a better error message. Right now it fails with missing asset
error.
**Does this PR introduce a breaking change?**
no
This PR adds configurations files to `puppeteer`'s methods for
configuration. Under the hood, `puppeteer` relies on
https://www.npmjs.com/package/cosmiconfig which resolves several formats
of configuration:
- a `puppeteer` property in package.json
- a `.puppeteerrc` file in JSON or YAML format
- a `.puppeteerrc.json`, `.puppeteerrc.yaml`, `.puppeteerrc.yml`,
`.puppeteerrc.js`, or `.puppeteerrc.cjs` file
- a `puppeteer.config.js` or `puppeteer.config.cjs` CommonJS module
exporting an object
Documentation will be added later.
Fixed: #9128
This PR removes the deprecated query selector API from the `puppeteer`
instance. Direct imports of the query selector API have also been
deprecated and users are now expected to use the static methods defined
on `Puppeteer`.
This PR removes the deprecated `puppeteer.devices` in favor of a new
exported object `KnownDevices`. `devices` can also be exported, but has
been deprecated.
This PR separates the installations tests from normal tests and changes
the matrices as follows:
- Installations tests on all platforms and the latest 3 node versions.
- Chrome tests are done on all platforms on the latest node version.
- Firefox tests are done only on linux on the latest node version.
This PR moves the puppeteer source code into separate mono-repo packages:
- `puppeteer` and `puppeteer-core` are now separated into their own
packages.
- `puppeteer-core` has a new exports called `puppeteer-core/internal`
for internal usage.
Tests and various tools have been updated to accommodate the migration.
This PR starts the monorepo migrations as per
https://github.com/puppeteer/puppeteer/issues/8922. To scope migrations,
we are only moving the `testserver` into a separate package. Further
migrations will come later.
This PR adds a basic support for WebDriver BiDi that currently includes only the ability to establish a connection and shutdown the browser. Therefore, the implementation is marked as internal and won't show up in the changelog as it's barely useful at the moment.
The API classes are kept as classes instead of interfaces so that clients relying on instanceof checks still work.
Instead of checking skipped tests in mocha-utils this PR
implements a custom mocha interface for better flexibility
when skipping tests. That should allow skipping tests without
running before and after hooks.
* chore: implement a test runner on top of mocha
This PR implements a test runner on top of mocha
that performs multiple mocha runs as defined in
TestSuites.json and compares the outcome of the runs
against TestExpectations.json. This allows us to
remove most of helpers from mocha-utils and be more
flexible when defining the test configurations.
Two main sources of flakiness addressed:
1) we should dispose the lifecycle watcher after we waited for the navigation response (bad API? we need to refactor but I think it'd be valuable to stabilize tests first without too many changes).
2) we should wait for the navigation request's response if there is a navigation request in the watcher.
Closes#8644
Previously, if timeout is falsy, the targets would only
be checked if a browser-level event fires which lead to
a race: if the events arrived before waiting for a target,
the promise would never resolve.
Fixes#8763
This PR implements automatic detection of the Firefox product when the `.connect()` method is used. This partially undoes the breaking change in https://github.com/puppeteer/puppeteer/pull/8520 but it's also a breaking change on its own since we don't accept an explicit product name anymore (it does not look like it was used anyway).
When we attach to a frame, we send a call to get
the page frame tree from CDP. Based on the tree data
we look up the parent frame if parentId is provided.
The problem is that the call to get the page frame
tree could take arbitrary time and the calls for the
parent and child frames might happen at the same time.
So the situation where the frame tree for the child frame
is resolved before the parent frame is known is fairly
common.
This PR addresses the issue by awaiting for the parent
frame id before attempting to register a child frame.
* feat: use CDP's auto-attach mechanism
In this PR, we refactor Puppeteer to make use of the CDP's auto-attach mechanism. This allows the backend to pause
new targets and give Puppeteer a chance to configure them properly. This fixes the flakiness related to dealing with
OOPIFs and should fix some other issues related to the network interception and navigations. If those are not fixed completely by this PR, the PR serves a solid base for fixing them.
Closes https://github.com/puppeteer/puppeteer/issues/8507, https://github.com/puppeteer/puppeteer/issues/7990
Unlocks https://github.com/puppeteer/puppeteer/issues/3667
BREAKING CHANGE: With Chromium, Puppeteer will now attach to page/iframe targets immediately to allow reliable configuration of targets.
This patch fixes page.#scrollIntoViewIfNeeded, so that it works with devtools protocol.
Now it blocks the main thread and waits until the scrolling action finishes in Chrome.
Fallbacks to the old implementation if `DOM.scrollIntoViewIfNeeded` is not supported for Firefox.
Issues: #8627, #1805
This PR greatly improves the types within Puppeteer:
- **Almost everything** is auto-deduced.
- Parameters don't need to be specified in the function. They are deduced from the spread.
- Return types don't need to be specified. They are deduced from the function. (More on this below)
- Selections based on tag names correctly deduce element type, similar to TypeScript's mechanism for `getElementByTagName`.
- [**BREAKING CHANGE**] We've removed the ability to declare return types in type arguments for the following reasons:
1. Setting them will indubitably break auto-deduction.
2. You can just use `as ...` in TypeScript to coerce the correct type (given it makes sense).
- [**BREAKING CHANGE**] `waitFor` is officially gone.
To migrate to these changes, there are only four things you may need to change:
- If you set a return type using the `ReturnType` type parameter, remove it and use `as ...` and `HandleFor` (if necessary).
⛔ `evaluate<ReturnType>(a: number, b: number) => {...}, a, b)`
✅ `(await evaluate(a, b) => {...}, a, b)) as ReturnType`
⛔ `evaluateHandle<ReturnType>(a: number, b: number) => {...}, a, b)`
✅ `(await evaluateHandle(a, b) => {...}, a, b)) as HandleFor<ReturnType>`
- If you set any type parameters in the *parameters* of an evaluation function, remove them.
⛔ `evaluate(a: number, b: number) => {...}, a, b)`
✅ `evaluate(a, b) => {...}, a, b)`
- If you set any type parameters in the method's declaration, remove them.
⛔ `evaluate<(a: number, b: number) => void>((a, b) => {...}, a, b)`
✅ `evaluate(a, b) => {...}, a, b)`
* The testing tsconfig.json inherits from the base TS config.
* A lot of type assertions have been inserted...a lot.
* All testing utilities have migrated to TS.
* text-diff is being replaced with diff for TS compatibility.
* ProtocolError has been added to PuppeteerErrors and PuppeteerErrors is no longer a record (it's been frozen).
* Fixes a small bug where null was an allowable media type in emulation (should be undefined).
* fix: If currentNode and root are the same, do not include them in the result
* fix: Tests that only child element is included in the result
Co-authored-by: jrandolf <101637635+jrandolf@users.noreply.github.com>
* fix: test failing in headful
* fix: install Firefox for headful tests
* fix: skip favicon.ico requests in test
* fix: auth test in headful
* fix: disable NetworkTimeServiceQuerying
* fix: filter more favicon requests
* fix: network test with favicon
* fix: improve fixes
Without this patch, two tests ignore the BINARY envvar, and fail when not using
the embedded chromium, unless the chromium executable path is defined via
PUPPETEER_EXECUTABLE_PATH, which should not be considered according to the
docs.
* fix: ensure dom binding is not called after detatch
Fixes#7814
* refactor: detach listeners instead
* refactor: safer approach
* fix: test in test/page.spec.ts
Co-authored-by: Alex Rudenko <OrKoN@users.noreply.github.com>
Some recent changes to allow arm64 environments (including M1 macs) to
launch a chromium installation successfully before arm-compatible builds
were downloadable prevented the usage of PUPPETEER_EXECUTABLE_PATH in
some environments. Currently, when the platform is not darwin and the
arch is arm64, an executable cannot be specified using the environment
variable.
Generally speaking, environment variables have highest precedence for
options such as this since they depend on system configuration.
These change:
1. allow the ENV variable to always be used when defined and not
specified in LaunchOptions (and when not puppeteer-core)
2. Retain the existing behavior of assuming /usr/bin/chromium-browser on
platforms like Ubuntu (exact if-conditions preserved to avoid any
breaking changes)
3. Add some tests for this particular portion of the code.
The doc for boundingBox says that it should return the boundingBox
relative to the main frame, therefore, this fix would make the
actual implementation correspond to the documentation. boxModel
documentation does not have this note but I think it'd make sense
to have it match the behaviour of the boundingBox API.
So it appears that all bindings are added to the secondary world and all
evaluations are also running there. ElementHandle.evaluate is returning
handles from the main world though. Therefore, we need to be careful
and adopt handles to the right context before doing waitForSelector
So it appears that all bindings are added to the secondary world and all
evaluations are also running there. ElementHandle.evaluate is returning
handles from the main world though. Therefore, we need to be careful
and adopt handles to the right context before doing waitForSelector.
When using a custom Firefox profile for Puppeteer the modified
preferences as present in prefs.js need to be reset once the
profile is no longer needed by Puppeteer. If not done this could
cause side-effects when the profile is used next time outside
of Puppeteer.
As ride-along fix the "--foreground" argument for Firefox will
only be used on MacOS because that's the only supported platform.
This updates the regular expression used to parse aria attribute
selectors so that single quotes may be used as an alternative to double
quotes, e.g. `aria/Single button[role='button']`.
Issues: #7721
Co-authored-by: Andy Earnshaw <andy.earnshaw@gmail.com>
This pull request to adds better support for OOP iframes (see #2548)
The current problem with OOP iframes is that they are moved to a different target. Because of this, the previous versions of Puppeteer pretty much ignored them.
This change extends the FrameManager to already take OOP iframes into account and hides the fact that those frames are actually in different targets.
Further work needs to be done to also make the NetworkManager aware of these and to make sure that settings like emulations etc. are also properly passed down to the new targets.
In some situations, Puppeteer is left in an invalid state because protocol errors that could have been handled by the user where just hidden from them. This patch removes some of these cases and also makes sure that unhandled promise rejections lead to a test failure in mocha.
Enable developers to handle 'Invalid header' errors instead of hiding them to make sure they can address them properly.
Co-authored-by: Jan Scheffler <janscheffler@chromium.org>
Sometimes an element has not been layed out yet and, in this case,
clickablePoint fails because backend cannot compute content quads.
Co-authored-by: Jan Scheffler <janscheffler@chromium.org>
