the-crawl

Crawler for the-frameworks

Installation

$ npm install the-crawl --save

Usage

'use strict'
 
const { TheCrawl } = require('the-crawl')
 
async function tryExample () {
 
  const pageFetcher = new TheCrawl(
    // Processor for page
    async (url, window) => {
      const { document } = window
      return {
        title: document.title,
        url,
      }
    }, {})
  const googleTopPage = await pageFetcher.process('https://www.google.com/')
  console.log('googleTopPage', googleTopPage)
 
}
 
tryExample().catch((err) => console.error(err))
 

API Guide

• the-crawl@1.0.0
  • create(args)
  • TheCrawl

License

This software is released under the MIT License.

Links

• THE Labo