mirror of https://github.com/puppeteer/puppeteer synced 2024-06-14 14:02:48 +00:00

Andrey Lushnikov a63a0198de Implement waitFor which survives navigation (#99 )

This patch implements page.waitFor method which survives navigation.

References #89.

2017-07-19 19:04:51 -07:00

39 KiB

Raw Blame History

Puppeteer API

class: Browser
class: Page
class: Keyboard
class: Dialog
class: Frame
class: Request
class: Response
class: InterceptedRequest
class: Headers
class: Body

class: Browser

Browser manages a browser instance, creating it with a predefined settings, opening and closing pages. Instantiating Browser class does not necessarily result in launching browser; the instance will be launched when the need will arise.

A typical scenario of using Browser is opening a new page and navigating it to a desired URL:

const {Browser} = require('puppeteer');
const browser = new Browser();
browser.newPage().then(async page => {
  await page.navigate('https://example.com');
  browser.close();
})

new Browser([options])

options <Object> Set of configurable options to set on the browser. Can have the following fields:
- headless <boolean> Wether to run chromium in headless mode. Defaults to true.
- executablePath <string> Path to a chromium executable to run instead of bundled chromium.
- args <Array<string>> Additional arguments to pass to the chromium instance. List of chromium flags could be found here.

browser.close()

Closes browser with all the pages (if any were opened). The browser object itself is considered to be disposed and could not be used anymore.

browser.closePage(page)

page <Page> A page to be closed.
returns: <Promise> Promise which resolves when the page is closed.

This is an alias for the page.close() method.

browser.newPage()

returns: <Promise<Page>> Promise which resolves to a new Page object.

browser.stderr

<stream.Readable>

A Readable Stream that represents the browser process's stderr. For example, stderr could be piped into process.stderr:

const {Browser} = require('puppeteer');
const browser = new Browser();
browser.stderr.pipe(process.stderr);
browser.version().then(version => {
  console.log(version);
  browser.close();
});

browser.stdout

<stream.Readable>

A Readable Stream that represents the browser process's stdout. For example, stdout could be piped into process.stdout:

const {Browser} = require('puppeteer');
const browser = new Browser();
browser.stdout.pipe(process.stdout);
browser.version().then(version => {
  console.log(version);
  browser.close();
});

browser.version()

returns: <Promise<string>> String describing browser version. For headless chromium, this is similar to HeadlessChrome/61.0.3153.0. For non-headless, this is Chrome/61.0.3153.0.

Note

the format of browser.version() is not fixed and might change with future releases of the library.

class: Page

Page provides methods to interact with browser page. Page could be thought about as a browser tab, so one Browser instance might have multiple Page instances.

An example of creating a page, navigating it to a URL and saving screenshot as screenshot.png:

const {Browser} = require('puppeteer');
const browser = new Browser();
browser.newPage().then(async page =>
  await page.navigate('https://example.com');
  await page.screenshot({path: 'screenshot.png'});
  browser.close();
});

event: 'console'

<string>

Emitted when a page calls one of console API methods, e.g. console.log or console.dir.

If multiple arguments are passed over to the console API call, these arguments are dispatched in an event.

An example of handling console event:

page.on('console', (...args) => {
  for (let i =0; i < args.length; ++i)
    console.log(`${i}: ${args[i]}`);
});
page.evaluate(() => console.log(5, 'hello', {foo: 'bar'}));

event: 'dialog'

<Dialog>

Emitted when a javascript dialog, such as alert, prompt, confirm or beforeunload, gets opened on the page. Puppeteer can take action to the dialog via dialog's accept or dismiss methods.

event: 'frameattached'

<Frame>

Emitted when a frame gets attached.

event: 'framedetached'

<Frame>

Emitted when a frame gets detached.

event: 'framenavigated'

<Frame>

Emitted when a frame committed navigation.

event: 'load'

Emitted when a page's load event was dispatched.

event: 'pageerror'

<string>

Emitted when an unhandled exception happens on the page. The only argument of the event holds the exception message.

event: 'request'

<Request>

Emitted when a page issues a request. The request object is a read-only object. In order to intercept and mutate requests, see page.setRequestInterceptor

event: 'requestfailed'

<Request>

Emitted when a request is failed.

event: 'requestfinished'

<Request>

Emitted when a request is successfully finished.

event: 'response'

<Response>

Emitted when a response is received.

page.$(selector, pageFunction, ...args)

selector <string> A selector to be matched in the page
pageFunction <function(Element)> Function to be evaluated in-page with first element matching selector
...args <...string> Arguments to pass to pageFunction
returns: <Promise<Object>> Promise which resolves to function return value.

Example:

const outerhtml = await page.$('#box', e => e.outerHTML);

Shortcut for page.mainFrame().$(selector, pageFunction, ...args).

page.$$(selector, pageFunction, ...args)

selector <string> A selector to be matched in the page
pageFunction <function(Element)> Function to be evaluated in-page for every matching element.
...args <...string> Arguments to pass to pageFunction
returns: <Promise<Array<Object>>> Promise which resolves to array of function return values.

Example:

const headings = await page.$$('h1,h2,h3,h4', el => el.textContent);
for (const heading of headings) console.log(heading);

Shortcut for page.mainFrame().$$(selector, pageFunction, ...args).

page.addScriptTag(url)

url <string> Url of a script to be added
returns: <Promise> Promise which resolves as the script gets added and loads.

Adds a <script></script> tag to the page with the desired url. Alternatively, javascript could be injected to the page via page.injectFile method.

page.click(selector)

selector <string> A query selector to search for element to click. If there are multiple elements satisfying the selector, the first will be clicked.
returns: <Promise> Promise which resolves when the element matching selector is successfully clicked. Promise gets rejected if there's no element matching selector.

page.close()

returns: <Promise> Returns promise which resolves when page gets closed.

page.emulate(name, options)

name <string> A name of the device to be emulated. Get the full list of emulated devices via page.emulatedDevices().
options <Object> Emulation parameters which might have the following properties:
- landscape <boolean> Emulates device in the landscape mode, defaults to false.
returns: <Promise> Returns promise which resolves when device is emulated. Can reload the page if switching between mobile and desktop devices.

page.emulatedDevices()

returns: <Array<String>> Returns array of device names that can be used with page.emulate().

page.evaluate(pageFunction, ...args)

pageFunction <function> Function to be evaluated in browser context
...args <...string> Arguments to pass to pageFunction
returns: <Promise<Object>> Promise which resolves to function return value

This is a shortcut for page.mainFrame().evaluate() method.

page.evaluateOnInitialized(pageFunction, ...args)

pageFunction <function> Function to be evaluated in browser context
...args <...string> Arguments to pass to pageFunction
returns: <Promise<Object>> Promise which resolves to function

page.evaluateOnInitialized adds a function which would run on every page navigation before any page's javascript. This is useful to amend javascript environment, e.g. to seed Math.random

page.focus(selector)

selector <string> A query selector of element to focus. If there are multiple elements satisfying the selector, the first will be focused.
returns: <Promise> Promise which resolves when the element matching selector is successfully focused. Promise gets rejected if there's no element matching selector.

page.frames()

returns: <Array<Frame>> An array of all frames attached to the page.

page.goBack(options)

options <Object> Navigation parameters, same as in page.navigate.
returns: <Promise<Response>> Promise which resolves to the main resource response. In case of multiple redirects, the navigation will resolve with the response of the last redirect. If can not go back, resolves to null.

Navigate to the previous page in history.

page.goForward(options)

options <Object> Navigation parameters, same as in page.navigate.
returns: <Promise<Response>> Promise which resolves to the main resource response. In case of multiple redirects, the navigation will resolve with the response of the last redirect. If can not go back, resolves to null.

Navigate to the next page in history.

page.httpHeaders()

returns: <Object> Key-value set of additional http headers which will be sent with every request.

page.injectFile(filePath)

filePath <string> Path to the javascript file to be injected into page.
returns: <Promise> Promise which resolves when file gets successfully evaluated in page.

page.keyboard

returns: <Keyboard>

page.mainFrame()

returns: <Frame> returns page's main frame.

Page is guaranteed to have a main frame which persists during navigations.

page.navigate(url, options)

url <string> URL to navigate page to
options <Object> Navigation parameters which might have the following properties:
- maxTime <number> Maximum navigation time in milliseconds, defaults to 30 seconds.
- waitUntil <string> When to consider navigation succeeded, defaults to load. Could be either:
  - load - consider navigation to be finished when the load event is fired.
  - networkidle - consider navigation to be finished when the network activity stays "idle" for at least networkIdleTimeoutms.
- networkIdleInflight <number> Maximum amount of inflight requests which are considered "idle". Takes effect only with waitUntil: 'networkidle' parameter.
- networkIdleTimeout <number> A timeout to wait before completing navigation. Takes effect only with waitUntil: 'networkidle' parameter.
returns: <Promise<Response>> Promise which resolves to the main resource response. In case of multiple redirects, the navigation will resolve with the response of the last redirect.

The page.navigate will throw an error if:

there's an SSL error (e.g. in case of self-signed certificates).
target URL is invalid.
the maxTime is exceeded during navigation.

page.pdf(options)

options <Object> Options object which might have the following properties:
- path <string> The file path to save the PDF to.
- scale <number> Scale of the webpage rendering. Defaults to 1.
- displayHeaderFooter <boolean> Display header and footer. Defaults to false.
- printBackground <boolean> Print background graphics. Defaults to false.
- landscape <boolean> Paper orientation. Defaults to false.
- pageRanges <string> Paper ranges to print, e.g., '1-5, 8, 11-13'. Defaults to the empty string, which means print all pages.
- format <string> Paper format. If set, takes priority over width or height options. Defaults to 'Letter'.
- width <string> Paper width, accepts values labeled with units.
- height <string> Paper height, accepts values labeled with units.
- margin <Object> Paper margins, defaults to none.
  - top <string> Top margin, accepts values labeled with units.
  - right <string> Right margin, accepts values labeled with units.
  - bottom <string> Bottom margin, accepts values labeled with units.
  - left <string> Left margin, accepts values labeled with units.
returns: <Promise<Buffer>> Promise which resolves with PDF buffer.

The width, height, and margin options accept values labeled with units. Unlabeled values are treated as pixels.

A few examples:

page.pdf({width: 100}) - prints with width set to 100 pixels
page.pdf({width: '100px'}) - prints with width set to 100 pixels
page.pdf({width: '10cm'}) - prints with width set to 10 centimeters.

All possible units are:

px - pixel
in - inch
cm - centimeter
mm - millimeter

The format options are:

Letter: 8.5in x 11in
Legal: 8.5in x 14in
Tabloid: 11in x 17in
Ledger: 17in x 11in
A0: 33.1in x 46.8in
A1: 23.4in x 33.1in
A2: 16.5in x 23.4in
A3: 11.7in x 16.5in
A4: 8.27in x 11.7in
A5: 5.83in x 8.27in

page.plainText()

returns: <Promise<string>> Returns page's inner text.

page.press(key[, options])

key <string> Name of key to press, such as ArrowLeft. See KeyboardEvent.key
options <Object>
- text <string> If specified, generates an input event with this text.
returns: <Promise>

Shortcut for keyboard.down and keyboard.up.

page.reload(options)

options <Object> Navigation parameters, same as in page.navigate.
returns: <Promise<Response>> Promise which resolves to the main resource response. In case of multiple redirects, the navigation will resolve with the response of the last redirect.

page.screenshot([options])

options <Object> Options object which might have the following properties:
- path <string> The file path to save the image to. The screenshot type will be inferred from file extension.
- type <string> Specify screenshot type, could be either jpeg or png. Defaults to 'png'.
- quality <number> The quality of the image, between 0-100. Not applicable to png images.
- fullPage <boolean> When true, takes a screenshot of the full scrollable page. Defaults to false.
- clip <Object> An object which specifies clipping region of the page. Should have the following fields:
  - x <number> x-coordinate of top-left corner of clip area
  - y <number> y-coordinate of top-left corner of clip area
  - width <number> width of clipping area
  - height <number> height of clipping area
returns: <Promise<Buffer>> Promise which resolves to buffer with captured screenshot

page.setContent(html)

html <string> HTML markup to assign to the page.
returns: <Promise> Promise which resolves when the content is successfully assigned.

page.setHTTPHeaders(headers)

headers <Object> Key-value set of additional http headers to be sent with every request.
returns: <Promise> Promise which resolves when additional headers are installed

page.setInPageCallback(name, callback)

name <string> Name of the callback to be assigned on window object
callback <function> Callback function which will be called in puppeteer's context.
returns: <Promise> Promise which resolves when callback is successfully initialized

The in-page callback allows page to asynchronously reach back to the Puppeteer. An example of a page showing amount of CPU's:

const os = require('os');
const {Browser} = require('puppeteer');
const browser = new Browser();

browser.newPage().then(async page =>
  await page.setInPageCallback('getCPUCount', () => os.cpus().length);
  await page.evaluate(async () => {
    alert(await window.getCPUCount());
  });
  browser.close();
});

page.setRequestInterceptor(interceptor)

interceptor <function> Callback function which accepts a single argument of type <InterceptedRequest>.
returns: <Promise> Promise which resolves when request interceptor is successfully installed on the page.

After the request interceptor is installed on the page, every request will be reported to the interceptor. The InterceptedRequest could be modified and then either continued via the continue() method, or aborted via the abort() method.

En example of a naive request interceptor which aborts all image requests:

const {Browser} = require('puppeteer');
const browser = new Browser();

browser.newPage().then(async page =>
  await page.setRequestInterceptor(interceptedRequest => {
    if (interceptedRequest.url.endsWith('.png') || interceptedRequest.url.endsWith('.jpg'))
      interceptedRequest.abort();
    else
      interceptedRequest.continue();
  });
  await page.navigate('https://example.com');
  browser.close();
});

page.setUserAgent(userAgent)

userAgent <string> Specific user agent to use in this page
returns: <Promise> Promise which resolves when the user agent is set.

page.setViewport(viewport)

viewport <Object> An object with two fields:
- width <number> Specify page's width in pixels.
- height <number> Specify page's height in pixels.
returns: <Promise> Promise which resolves when the dimensions are updated.

The page's viewport size defines page's dimensions, observable from page via window.innerWidth / window.innerHeight. The viewport size defines a size of page screenshot (unless a fullPage option is given).

In case of multiple pages in one browser, each page can have its own viewport size.

page.title()

returns: <Promise<string>> Returns page's title.

page.type(text)

text <string> A text to type into a focused element.
returns: <Promise> Promise which resolves when the text has been successfully typed.

page.uploadFile(selector, ...filePaths)

selector <string> A query selector to a file input
...filePaths <string> Sets the value of the file input these paths
returns: <Promise> Promise which resolves when the value is set.

page.url()

returns: <string> Current page url.

This is a shortcut for page.mainFrame().url()

page.userAgent()

returns: <string> Returns user agent.

page.viewport()

returns: <Object> An object with two fields:
- width <number> Page's width in pixels.
- height <number> Page's height in pixels.

page.waitFor(selector)

selector <string> A query selector to wait for on the page.
returns: <Promise> Promise which resolves when the element matching selector appears in the page.

The page.waitFor successfully survives page navigations:

const {Browser} = new require('puppeteer');
const browser = new Browser();

browser.newPage().then(async page => {
  let currentURL;
  page.waitFor('img').then(() => console.log('First URL with image: ' + currentURL));
  for (currentURL of ['https://example.com', 'https://google.com', 'https://bbc.com'])
    await page.navigate(currentURL);
  browser.close();
});

page.waitForNavigation(options)

options <Object> Navigation parameters, same as in page.navigate.
returns: <Promise<Response>> Promise which resolves to the main resource response. In case of multiple redirects, the navigation will resolve with the response of the last redirect.

Shortcut for page.mainFrame().waitFor(selector).

class: Keyboard

Keyboard provides an api for managing a virtual keyboard. The high level api is keyboard.type, which takes raw characters and generates proper keydown, keypress/input, and keyup events on your page.

For finer control, you can use keyboard.down, keyboard.up, and keyboard.sendCharacter to manually fire events as if they were generated from a real keyboard.

An example of holding down Shift in order to select and delete some text:

page.keyboard.type('Hello World!');
page.keyboard.press('ArrowLeft');

page.keyboard.down('Shift');
for (let i = 0; i = 0; i < ' World'.length; i++)
  page.keyboard.press('ArrowLeft');
page.keyboard.up('Shift');

page.keyboard.press('Backspace');
// Result text will end up saying 'Hello!'

keyboard.down(key[, options])

key <string> Name of key to press, such as ArrowLeft. See KeyboardEvent.key
options <Object>
- text <string> If specified, generates an input event with this text.
returns: <Promise>

Dispatches a keydown event.

This will not send input events unless text is specified.

If key is a modifier key, Shift, Meta, Control, or Alt, subsequent key presses will be sent with that modifier active. To release the modifier key, use keyboard.up.

keyboard.modifiers()

returns: <Object>
- Shift <boolean>
- Meta <boolean>
- Control <boolean>
- Alt <boolean>

Returns which modifier keys are currently active. Use keyboard.down to activate a modifier key.

keyboard.press(key[, options])

key <string> Name of key to press, such as ArrowLeft. See KeyboardEvent.key
options <Object>
- text <string> If specified, generates an input event with this text.
returns: <Promise>

Shortcut for keyboard.down and keyboard.up.

keyboard.sendCharacter(char)

char <string> Character to send into the page.
returns: <Promise>

Dispatches a keypress and input event. This does not send a keydown or keyup event.

page.keyboard.sendCharacter('嗨');

keyboard.type(text)

text <string> Text to type into the page
returns: <Promise>

Sends a keydown, keypress/input, and keyup event for each character in the text. This is the suggested way to type printable characters.

page.keyboard.type('Hello World!');

keyboard.up(key)

key <string> Name of key to release, such as ArrowLeft. See KeyboardEvent.key
returns: <Promise>

Dispatches a keyup event.

class: Dialog

Dialog objects are dispatched by page via the 'dialog' event.

An example of using Dialog class:

const {Browser} = require('puppeteer');
const browser = new Browser({headless: false});
browser.newPage().then(async page => {
  page.on('dialog', dialog => {
    console.log(dialog.message());
    dialog.dismiss();
    browser.close();
  });
  page.evaluate(() => alert('1'));
});

NOTE: Chrome Headless currently has issues with managing javascript dialogs, see issue 13

dialog.accept([promptText])

promptText <string> A text to enter in prompt. Does not cause any effects if the dialog's type is not prompt.
returns: <Promise> Promise which resolves when the dialog has being accepted.

dialog.dismiss()

returns: <Promise> Promise which resolves when the dialog has being dismissed.

dialog.message()

returns: <string> A message displayed in the dialog.

dialog.type

<string>

Dialog's type, could be one of the alert, beforeunload, confirm and prompt.

class: Frame

At every point of time, page exposes its current frame tree via the page.mainFrame() and frame.childFrames() methods.

Frame object's lifecycle is controlled by three events, dispatched on the page object:

'frameattached' - fired when the frame gets attached to the page. Frame could be attached to the page only once.
'framenavigated' - fired when the frame commits navigation to a different URL.
'framedetached' - fired when the frame gets detached from the page. Frame could be detached from the page only once.

An example of dumping frame tree:

const {Browser} = new require('.');
const browser = new Browser({headless: true});

browser.newPage().then(async page => {
  await page.navigate('https://www.google.com/chrome/browser/canary.html');
  dumpFrameTree(page.mainFrame(), '');
  browser.close();

  function dumpFrameTree(frame, indent) {
    console.log(indent + frame.url());
    for (let child of frame.childFrames())
      dumpFrameTree(child, indent + '  ');
  }
});

frame.$(selector, pageFunction, ...args)

selector <string> A selector to be matched in the page
pageFunction <function(Element)> Function to be evaluated with first element matching selector
...args <...string> Arguments to pass to pageFunction
returns: <Promise<Object>> Promise which resolves to function return value.

frame.$$(selector, pageFunction, ...args)

selector <string> A selector to be matched in the page
pageFunction <function(Element)> Function to be evaluted for every element matching selector.
...args <...string> Arguments to pass to pageFunction
returns: <Promise<Array<Object>>> Promise which resolves to array of function return values.

frame.childFrames()

returns: <Array<Frame>>

frame.evaluate(pageFunction, ...args)

pageFunction <function> Function to be evaluated in browser context
...args <...string> Arguments to pass to pageFunction
returns: <Promise<Object>> Promise which resolves to function return value

If the function, passed to the page.evaluate, returns a Promise, then page.evaluate would wait for the promise to resolve and return it's value.

const {Browser} = require('puppeteer');
const browser = new Browser();
browser.newPage().then(async page =>
  const result = await page.evaluate(() => {
    return Promise.resolve().then(() => 8 * 7);
  });
  console.log(result); // prints "56"
  browser.close();
});

frame.isDetached()

returns: <boolean>

Returns true if the frame has being detached, or false otherwise.

frame.isMainFrame()

returns: <boolean>

Returns true is the frame is page's main frame, or false otherwise.

frame.name()

returns: <string>

Returns frame's name as specified in the tag.

frame.parentFrame()

returns: <Frame> Returns parent frame, if any. Detached frames and main frames return null.

frame.url()

returns: <string>

Returns frame's url.

frame.waitFor(selector)

selector <string> CSS selector of awaited element,
returns: <Promise> Promise which resolves when element specified by selector string is added to DOM.

Wait for the selector to appear in page. If at the moment of calling the method the selector already exists, the method will return immediately.

class: Request

Whenever the page sends a request, the following events are emitted by puppeteer's page:

'request' emitted when the request is issued by the page.
'response' emitted when/if the response is received for the request.
'requestfinished' emitted when the response body is downloaded and the request is complete.

If request fails at some point, then instead of 'requestfinished' event (and possibly instead of 'response' event), the 'requestfailed' event is emitted.

If request gets a 'redirect' response, the request is successfully finished with the 'requestfinished' event, and a new request is issued to a redirected url.

Request class represents requests which are sent by page. Request implements Body mixin, which in case of HTTP POST requests allows clients to call request.json() or request.text() to get different representations of request's body.

request.headers

<Headers>

Contains the associated Headers object of the request.

request.method

<string>

Contains the request's method (GET, POST, etc.)

request.response()

returns: <Response> A matching Response object, or null if the response has not been received yet.

request.url

<string>

Contains the URL of the request.

class: Response

Response class represents responses which are received by page. Response implements Body mixin, which allows clients to call response.json() or response.text() to get different representations of response body.

response.headers

<Headers>

Contains the Headers object associated with the response.

response.ok

<boolean>

Contains a boolean stating whether the response was successful (status in the range 200-299) or not.

response.request()

returns: <Request> A matching Request object.

response.status

<number>

Contains the status code of the response (e.g., 200 for a success).

response.statusText

<string>

Contains the status message corresponding to the status code (e.g., OK for 200).

response.url

<string>

Contains the URL of the response.

class: InterceptedRequest

InterceptedRequest represents an intercepted request, which can be mutated and either continued or aborted. InterceptedRequest which is not continued or aborted will be in a 'hanging' state.

interceptedRequest.abort()

Aborts request.

interceptedRequest.continue()

Continues request.

interceptedRequest.headers

<Headers>

Contains the Headers object associated with the request.

Headers could be mutated with the headers.append, headers.set and other methods. Must not be changed in response to an authChallenge.

interceptedRequest.isHandled()

returns: <boolean> returns true if either abort or continue was called on the object. Otherwise, returns false.

interceptedRequest.method

<string>