With #8520 Puppeteer is now aware of all targets it connects
to. In order to have a not flaky init, Puppeteer waits for
all existing targets to be configured during the connection process.
This does not work well in case of concurrent connections because
while one connection might initializing a target the other one
might be closed it. In general, that is expected because we
can only be eventually consistent about the target state but we
also should not crash the init if some targets have been closed.
This PR implements checks to see if the errors are caused by the
target or session closures and suppresses them if it's the case.
Issue #8832 makes a good point that we should not be making
implicit assumptions about the client's performance
when waiting for internal events. At the same time,
we want to be able to get the debug info if some promises
never resolve because of missing backend events.
This PR adds a variable to turn on timeouts for deferred
promises created using `createDebuggableDeferredPromise`.
We can use it in our tests to catch never-resolving
promises or when reproducing bug reports where Puppeteer
hangs indefinitely.
Closes#8832
Two main sources of flakiness addressed:
1) we should dispose the lifecycle watcher after we waited for the navigation response (bad API? we need to refactor but I think it'd be valuable to stabilize tests first without too many changes).
2) we should wait for the navigation request's response if there is a navigation request in the watcher.
Closes#8644
When we attach to a frame, we send a call to get
the page frame tree from CDP. Based on the tree data
we look up the parent frame if parentId is provided.
The problem is that the call to get the page frame
tree could take arbitrary time and the calls for the
parent and child frames might happen at the same time.
So the situation where the frame tree for the child frame
is resolved before the parent frame is known is fairly
common.
This PR addresses the issue by awaiting for the parent
frame id before attempting to register a child frame.
* feat: use CDP's auto-attach mechanism
In this PR, we refactor Puppeteer to make use of the CDP's auto-attach mechanism. This allows the backend to pause
new targets and give Puppeteer a chance to configure them properly. This fixes the flakiness related to dealing with
OOPIFs and should fix some other issues related to the network interception and navigations. If those are not fixed completely by this PR, the PR serves a solid base for fixing them.
Closes https://github.com/puppeteer/puppeteer/issues/8507, https://github.com/puppeteer/puppeteer/issues/7990
Unlocks https://github.com/puppeteer/puppeteer/issues/3667
BREAKING CHANGE: With Chromium, Puppeteer will now attach to page/iframe targets immediately to allow reliable configuration of targets.
This PR greatly improves the types within Puppeteer:
- **Almost everything** is auto-deduced.
- Parameters don't need to be specified in the function. They are deduced from the spread.
- Return types don't need to be specified. They are deduced from the function. (More on this below)
- Selections based on tag names correctly deduce element type, similar to TypeScript's mechanism for `getElementByTagName`.
- [**BREAKING CHANGE**] We've removed the ability to declare return types in type arguments for the following reasons:
1. Setting them will indubitably break auto-deduction.
2. You can just use `as ...` in TypeScript to coerce the correct type (given it makes sense).
- [**BREAKING CHANGE**] `waitFor` is officially gone.
To migrate to these changes, there are only four things you may need to change:
- If you set a return type using the `ReturnType` type parameter, remove it and use `as ...` and `HandleFor` (if necessary).
⛔ `evaluate<ReturnType>(a: number, b: number) => {...}, a, b)`
✅ `(await evaluate(a, b) => {...}, a, b)) as ReturnType`
⛔ `evaluateHandle<ReturnType>(a: number, b: number) => {...}, a, b)`
✅ `(await evaluateHandle(a, b) => {...}, a, b)) as HandleFor<ReturnType>`
- If you set any type parameters in the *parameters* of an evaluation function, remove them.
⛔ `evaluate(a: number, b: number) => {...}, a, b)`
✅ `evaluate(a, b) => {...}, a, b)`
- If you set any type parameters in the method's declaration, remove them.
⛔ `evaluate<(a: number, b: number) => void>((a, b) => {...}, a, b)`
✅ `evaluate(a, b) => {...}, a, b)`
This PR works around the upstream bug crbug.com/1325782. Previously Puppeteer relied on the presence of the loaderId to determine the kind of navigation and expected events. It does not look like there is a reason to do so: instead, we could see what events we get and proceed accordingly.
This pull request to adds better support for OOP iframes (see #2548)
The current problem with OOP iframes is that they are moved to a different target. Because of this, the previous versions of Puppeteer pretty much ignored them.
This change extends the FrameManager to already take OOP iframes into account and hides the fact that those frames are actually in different targets.
Further work needs to be done to also make the NetworkManager aware of these and to make sure that settings like emulations etc. are also properly passed down to the new targets.
Up to now, only strings starting with '//' are considered as to XPath selectors. Unfortunately, this is too restricting. This fix allows valid XPath selectors starting with: '/', './', and even '(//*[1])'
* fix: make `$` and `$$` selectors generic
This means, much like TS's in built `querySelector` type, you can now do:
```ts
const listItems = page.$$<HTMLLIElement>('ul li');
```
And/or:
```ts
const h2 = page.$<HTMLHeadingElement>('h2');
```
And the return value will be of type `ElementHandle<T>|null`, where `T`
is the type you provided. By default `T` is an `Element`, so you don't
have to provide this if you don't care as a consumer about the exact
type you get back.
* chore: fix test assertions
Debug message clarifying the problem like `Issue 1060080: Event Network.loadingFinished is not sent when a frame is loaded from another domain`: https://bugs.chromium.org/p/chromium/issues/detail?id=1060080).
It can help users to identify problem with OOPIF easier without digging into the CDP protocol implementation like in the bug `1060080` mentioned above.
To reproduce:
1. Run `DEBUG="puppeteer:frame" NODE_PATH=../ node examples/oopif.js`.
2. Verify the output contains the debug message:
`
puppeteer:frame The frame '...' moved to another session. Out-of-proccess
iframes (OOPIF) are not supported by Puppeteer yet.
https://github.com/puppeteer/puppeteer/issues/2548
`
