taki
TypeScript icon, indicating that this package has built-in type declarations

3.0.0 • Public • Published

taki

NPM version NPM downloads CircleCI donate

Sponsor

Oxylabs: Innovative Proxy Service to Gather Data at Scale

Install

npm i taki

Built on the top of Google's Puppeteer, for a jsdom/chromy version please visit here.

Usage

const { request } = require('taki')

// Prerender this page to static HTML
// Wait for 1s since this page renders remote markdown file
request({ url: 'https://sao.js.org', wait: 1000 }).then((html) => {
  // serialized html string of target url
  console.log(html)
})

NOTE: You need to call cleanup when you no longer use request:

import { cleanup } from 'taki'

// After fetching..
cleanup()

Custom html selector

By default it returns the html for the entire document, but you can specify a selector to get the html for a specific element.

const { request } = require('taki')

request({ url: 'https://example.com', htmlSelector: '.some-element' }).then(
  (html) => {
    console.log(html)
  }
)

Manually take snapshot

By default taki will take a snapshot of the URL when all resources are loaded, if you have control of the website's source code, you can disable that and manually call window.snapshot:

request({
  url: 'http://my-web.com',
  manually: true,
})

And in your website's source code:

fetchSomeData().then(data => {
  this.setState({ data }, () => {
+    window.snapshot && window.snapshot()
  })
})

Alternatively, choose your own method to invoke when your app is ready to return HTML:

request({
  url: 'http://my-web.com',
  manually: 'iamready',
})

Then call window.iamready() instead of window.snapshot() in your app.

Wait

Wait for specific timeout or a CSS selector to appear in dom.

request({
  url,
  // Wait for 3000 ms
  wait: 3000,
  // Or wait for <div class="comments"></div> to appear
  wait: '.comments',
})

This option will be ignored if manually is set.

Minify

Minify HTML.

request({
  url,
  minify: true,
})

Filter resource

We always abort network requests to following types of resource: stylesheet image media font since they're not required to render the page. In addtion, you can use resourceFilter option to abort specfic type of resource:

request({
  url,
  /**
   * @param {Object} context
   * @param {string} context.type - Resource type
   * @see {@link https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#requestresourcetype}
   * @param {string} context.url - Resource URL
   * @see {@link https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#requesturl}
   * @returns {boolean} Whether to load this resource
   */
  resourceFilter({ type, url }) {
    // Return true to load the resource, false otherwise.
  },
})

You can also use blockCrossOrigin: true shortcut to block all cross-origin requests.

Contributing

  1. Fork it!
  2. Create your feature branch: git checkout -b my-new-feature
  3. Commit your changes: git commit -am 'Add some feature'
  4. Push to the branch: git push origin my-new-feature
  5. Submit a pull request :D

Author

taki © egoist, Released under the MIT License.
Authored and maintained by egoist with help from contributors (list).

Website · GitHub @egoist · Twitter @_egoistlily

Package Sidebar

Install

npm i taki

Weekly Downloads

223

Version

3.0.0

License

MIT

Unpacked Size

14.1 kB

Total Files

5

Last publish

Collaborators

  • egoist
  • rem