Commit Graph

84 Commits

Author SHA1 Message Date
Jake Causon
90a1032300 fix(network): stringify response headers for intercepted requests (#4436)
Stringifying the headers was the behaviour before v1.15

References #4379
2019-05-20 11:05:32 +03:00
Andrey Lushnikov
2265974ce5
refactor: migrate onto Fetch domain (#4265)
Request interception in Network domain is deprecated.
Move on onto Fetch domain - the new bright future.
2019-04-11 15:02:06 -04:00
Andrey Lushnikov
19606a3b79
fix: cache disabling should stick when toggling request interception (#4260)
This patch:
- refactors `NetworkManager`/`FrameManager` so that they enable all the
relevant domains themselves. This is a preparation for OOPIF support and
migration onto fetch domain.
- moves `networkManager` ownership into `FrameManager`. This way it's clear who owns what.
- stops enabling Security domain: it saves quite some traffic over
websocket since it no longer sends annoying "SecurityStateChanged" events.
Instead, use `Security.setIgnoreCertificateErrors` method.
- consolidates network cache state in network manager. This even fixes a
bug with caching and request interception interop.
2019-04-10 00:42:42 -04:00
Joseph Arhar
fd0f58e9c2 fix: Use new requestId for interception request matching (#4248)
Fix #3471
2019-04-08 14:17:57 -07:00
Andrey Lushnikov
62da2366c6
chore: introduce //lib/api.js (#3835)
Introduce `//lib/api.js` that declares a list of publicly exposed
classes.

The `//lib/api.js` list superceedes dynamic `helper.tracePublicAPI()` calls
and is used in the following places:
- [ASYNC STACKS]: generate "async stacks" for publicy exposed API in `//index.js`
- [COVERAGE]: move coverage support from `//lib/helper` to `//test/utils`
- [DOCLINT]: get rid of 'exluded classes' hardcoded list

This will help us to re-use our coverage and doclint infrastructure
for Puppeteer-Firefox.

Drive-By: it turns out we didn't run coverage for `SecurityDetails`
class, so we lack coverage for a few methods there. These are excluded
for now, sanity tests will be added in a follow-up.
2019-01-25 23:21:14 -05:00
Andrey Lushnikov
9fd4b67d0c
fix(requestinterception): filter out "intervention" header (#3814)
Fixes #3798
2019-01-20 00:02:13 -05:00
Andrey Lushnikov
04fbbd7cf0
fix(network): relax request matching heuristic (#3775)
Drop requirement for matching "origin" and "content-type" headers
in requests and request interceptions. This way javascript redirects
that use form submission start working.

Fix #3684.
2019-01-15 11:37:53 -08:00
Andrey Lushnikov
4e9e3bc614
refactor: consolidate all events in Events.js (#3772)
This will allow us to break all dependency cycles that were forcing
us to put many things in a single file (e.g. ExecutionContext and
ElementHandle).
2019-01-14 19:57:05 -08:00
Andrey Lushnikov
1899e7931c
feat: enable Network Service by default (#3738)
References #3471.
2019-01-09 15:47:08 -08:00
Joel Einbinder
568734386d fix(types): fix parameters where they disagree with the api (#3525) 2018-11-12 12:59:21 -08:00
Joel Einbinder
cd54ce388e fix(types): upgrade node types to 8.10.34 (#3341) 2018-10-03 16:59:49 -07:00
Andrey Lushnikov
5acf953104
feat(frame): introduce Frame.goto and Frame.waitForNavigation (#3276)
This patch introduces API to manage frame navigations.
As a drive-by, the `response.frame()` method is added as a shortcut
for `response.request().frame()`.

Fixes #2918.
2018-09-20 11:31:19 -07:00
Andrey Lushnikov
9223bca964
refactor: move navigation management to FrameManager (#3266)
This patch:
- moves implementation of page.goto and page.waitForNavigation
  into FrameManager. The defaultNavigationTimeout gets moved to
  FrameManager as well.
- moves NavigatorWatcher into FrameManager to avoid circular dependency

References #2918
2018-09-19 13:12:28 -07:00
Andrey Lushnikov
7ec0801729
fix: expect Network.responseReceived event is never dispatched (#3233)
Fixes #2888.
2018-09-12 22:08:32 +01:00
Andrey Lushnikov
7f9e276733
feat(network): introduce Response.statusText() (#3193)
Fixes #317.
2018-09-05 21:03:24 +01:00
Andrey Lushnikov
84c2621dd4
refactor(network): Create Request class right away from payload (#3194)
Similarly to Responses, we now always create Request classes from
requestWillBeSent events.
2018-09-05 21:02:28 +01:00
Andrey Lushnikov
1ba2b8540d
feat(network): introduce Response.remoteAddress() (#3192)
Closes #2171.
2018-09-04 20:39:59 +01:00
Andrey Lushnikov
52cf16c73c
refactor(network): Create Response class right away from payload (#3191)
Before v1.7.0 we were creating Response objects either from protocol's
Network.Response struct, or from the data available in the
requestIntercepted.

With the recent chagens to the request interception logic, we can
always create response from Newtork.Response struct; this allows
us to simply create Response objects from protocol payload.
2018-09-04 20:20:20 +01:00
Joel Einbinder
3d7ae2a259 fix: fix null-type bugs (#3137)
I ran TypeScript against our code with `strictNullChecks` on. Most of the errors generated are noise, because TypeScript doesn't understand how our `assert` method works. But some were legitimate bugs. They are fixed in this patch.
2018-08-24 15:17:36 -07:00
Joel Einbinder
2e0007669d chore: upgrade to TypeScript 3.0.1 (#3024) 2018-08-06 11:31:33 -07:00
Andrey Lushnikov
1931cb479e
feat(Chromium): roll Chromium to r579032 (#2989)
This patch rolls Chromium to r579032. The patch includes:
- https://crrev.com/577366 - DevTools: report redirect responses only if response interception is enabled
- https://crrev.com/577212 - DevTools: intercept requests resulting from redirects
- https://crrev.com/578934 - DevTools: Add a protocol method to insertText

Interception Logic in DevTools protocol has changed regarding redirects;
this patch migrates interceptions to dispatch "request" events based on
requestWillBeSent event.
2018-07-30 19:09:10 -07:00
Donovan Allen
0b12119843 fix(NetworkManager): Remove header 'cookie' from request hash (#2954)
This fixes an issue in which a response may not be recorded or trigger an event because of differing cookie headers from the request.

References #2970
2018-07-30 17:06:53 -07:00
Mickael van der Beek
1064aa4476 feat(request): add new error reasons for request.abort() (#2771)
As the title indicated, adds two allowed error codes that are specified in the Chrome Developer Tools Protocol specifications but missing from the Puppeteer enum:

https://chromedevtools.github.io/devtools-protocol/tot/Network#type-ErrorReason
2018-06-19 10:46:17 -07:00
Andrey Lushnikov
724fa512cf
feat(network): introduce Request.isNavigationRequest() method (#2633)
This patch introduces `Request.isNavigationRequest()` method.

Fixes #2627, #1132.
2018-05-31 17:38:30 -07:00
Andrey Lushnikov
0b94fa70eb
chore: stop using console.assert everywhere (#2646)
Since Node 10, `console.assert` no longer throws an AssertionError.
(This is generally good since it aligns Node.js with Browsers.)

This patch migrates all usages of `console.assert` in our codebase.
- All the `lib/` and testing code is migrated onto a handmade `assert`
function. This is to make Puppeteer transpilation / bundling easier.
- All the tooling is switched to use Node's `assert` module.

Fixes #2547.
2018-05-31 16:53:51 -07:00
Andrey Lushnikov
f8cba455ba
fix(Network): explicitly throw on content request for redirect response (#2352)
DevTools protocol doesn't support returning body of redirect responses.
We should explicitly throw in this case.

References #1896.
2018-04-10 20:22:18 -07:00
JoelEinbinder
294f33b75c chore(types): type check the rest of the protocol (#2328)
This adds events and commands into generated `protocol.d.ts`.
2018-04-07 17:58:52 -07:00
JoelEinbinder
8c54f41552 chore(types): generate a d.ts file for protocol types (#2325)
This uses the `/json/protocol` endpoint to generate type definitions for the protocol.

Currently it is lacking protocol events and commands, but I will add those later.
2018-04-06 18:20:48 -07:00
Andrey Lushnikov
ec3025fe84
feat(Network): implement request.redirectChain() api (#2194)
The request.redirectChain() returns all the requests issued to fetch
the resource.

Fixes #1816.
2018-03-15 17:17:38 -07:00
JoelEinbinder
826fe42c48 fix(Network): response.ok should be true for file:// urls (#2152) 2018-03-12 11:34:33 -07:00
Meir017
cad71d9bd9 chore: removed duplicated jsdoc annotation in NetworkManager.js (#2132) 2018-03-07 12:35:15 +03:00
Andrey Lushnikov
43c0feb2f2
fix(Network): fulfill security details for response redirects (#2025)
This patch:
- starts fulfilling security details for redirect responses
- changes `response.securityDetails()` to return null if the response
  is served over non-secure connection
2018-02-14 16:08:20 -08:00
Tomáš Trnka
856c431914 feat(Network): introduce response.securityDetails() method (#1880)
This patch:
- introduces `SecurityDetails` class that exposes a set of fields that describe properties of secure connection
- introduces method `response.securityDetails()` that returns an instance of `SecurityDetails` object.
2018-02-13 11:26:18 -08:00
Andrey Lushnikov
ecc3adc279
feat(Response): add Response.fromCache / Response.fromServiceWorker (#1971)
This patch:
- introduces `test/assets/cached` folder and teaches server to cache
  all the assets from the folder
- introduces `test/assets/serviceworkers` folder that stores all the
  service workers and makes them register with unique URL prefix
- introduces `Response.fromCache()` and `Response.fromServiceWorker()`
  methods

Fixes #1551.
2018-02-05 17:59:07 -05:00
Andrey Lushnikov
5368051610
feat: expose raw devtools protocol connection (#1770)
feat: expose raw devtools protocol connection

This patch introduces `target.createCDPSession` method that
allows directly communicating with the target over the
Chrome DevTools Protocol.

Fixes #31.
2018-01-10 19:33:22 -08:00
Andrey Lushnikov
9634d13be2
fix: requestInterception should play nicely with canceled redirects (#1758)
Since interception events and `loadingFailed` events come from
different processes and are not serialized, we might get `loadingFailed` event and a subsequent outdated `requestIntercepted`.

Short-term, this patch stops assuming that interception events are
aligned with `loadingFailed`.

Long-term, this will be resolved as @caseq completes network
servicification effort in chromium.

Fixes #880.
2018-01-09 22:59:21 -08:00
yujiosaka
c29822d663 feat: Attribute network requests to frames (#1646)
This patch introduces `request.frame()` method that returns the frame that
initiated request.

Fixes #1579
2018-01-09 18:47:21 -08:00
yujiosaka
5843f6f775 fix: setRequestInterception and setExtraHTTPHeaders not working together (#1734)
This patch starts lowering header keys while generating request hashes.

Fixes #1729.
2018-01-09 16:52:28 -08:00
Andrey Lushnikov
b73737302a
fix: convert all getters to methods (#1621)
The patch converts all the getters in the codebase into the methods.
For example, the `request.url` getter becomes the `request.url()`
method.

This is done in order to unify the API and make it more predictable.
The general rule for all further changes would be:
- there are no getters/fields exposed in the api
- the only exceptions are "namespaces", e.g. `page.keyboard`

Fixes #280.

BREAKING CHANGE:
This patch ditches getters and replaces them with methods throughout
the API. The following methods were added instead of the fields:
- dialog.type()
- consoleMessage.args()
- consoleMessage.text()
- consoleMessage.type()
- request.headers()
- request.method()
- request.postData()
- request.resourceType()
- request.url()
- response.headers()
- response.ok()
- response.status()
- response.url()
2017-12-18 17:05:57 -08:00
Andrey Lushnikov
2becd156d0
feat(Chromium): roll Chromium to r518818 (#1455)
This roll includes:
- crrev.com/518362 DevTools: override referrer policy for the request if
  referer header is overriden

Fixes #1373.
2017-11-23 07:42:48 +03:00
Andrey Lushnikov
cafd040bf2
fix(Network): be prepared to miss requestWillBeSent events (#1420)
With the addition of `browser.targets()` api, we now can connect to
in-flight targets.

For Puppeteer, it means that it can "miss" certain events happenning
while it wasn't attached to the target.

This patch:
- fixes this problem with NetworkManager, preparing it for the
  missed `requestWillBeSent` event.
- adds a new test to ensure that not a single unhandled promise
  rejection has happened during test execution.

Fixes #1363.
2017-11-20 15:59:07 -08:00
Andrey Lushnikov
f5bb333cd0
fix: support PlzNavigate in puppeteer. (#1239)
This patch migrates puppeteer to support PlzNavigate chromium
project.

As a consequence of this patch, we no longer wait for both
requestWillBeSent and requestIntercepted events to happen. This should
resolve a ton of request interception bugs that "hanged" the loading.

Fixes #877.
2017-11-01 14:04:10 -07:00
Andrey Lushnikov
b9ab6fe4bb fix(interception): Disable newtork caching when intercepting (#1154)
Request interception might not work properly if caching is enabled.
2017-10-24 15:14:25 -07:00
Andrey Lushnikov
ce005d480c feat(Chromium): Roll chromium to r511134 (#1153)
This roll includes:
- crrev.com/510651 that changes request interception methods in protocol
- s/Page.setRequestInterceptionEnabled/Page.setRequestInterception

BREAKING CHANGE

Page.setRequestInterceptionEnabled is renamed into
Page.setRequestInterception.
2017-10-24 14:45:03 -07:00
Andrey Lushnikov
bcc969ccc4 feat(interception): Implement request.mockResponse method (#1064)
feat(interception): Implement request.respond method

This patch implements a new Request.respond method. This
allows users to fulfill the intercepted request with a hand-crafted
response if they wish so.

References #1020.
2017-10-20 16:55:15 -07:00
Andrey Lushnikov
4f64dfd993 feat(Request): allow aborting intercepted requests with custom reasons (#1080)
This patch adds optional parameter to the `request.abort()` method
that specifies abort reason.

References #1020.
2017-10-18 00:26:48 -07:00
Andrey Lushnikov
273c733237 feat(Request): introduce Request.failure() method (#1065)
Similarly to the `request.response()` method, this patch adds
`request.failure()` method that returns error details for the failed
requests.

Fixes #901.
2017-10-17 17:48:04 -07:00
Andrey Lushnikov
a02347e3ef feat(Page): implement page.setOfflineMode (#1032)
This patch implements page.setOfflineMode method.

Fixes #63.
2017-10-13 14:41:39 -07:00
Andrey Lushnikov
c3fb367148 fix(Request): convert resourceType to all small-caps (#990)
This patch moves resourceType to be all small-caps. This aligns
with our convention that all string constants should be smallcaps.

BREAKING CHANGE: this patch changes the constants of the
request.resourceType to be all small-caps.
2017-10-10 10:53:37 -07:00
JoelEinbinder
e59172de83 chore: Use Typescript to lint JSDoc annotations (#986)
This patch starts using typescript to lint JSDoc annotations.

Note: this uses typescript's bleeding edge. We should migrate to stable once
it has all the necessary bugfixes.

References #65.
2017-10-09 22:31:40 -07:00