❤
npm
Sign UpSign In

periodic-crawler
TypeScript icon, indicating that this package has built-in type declarations

1.0.4 • Public • Published

Periodic Crawler

Build Status npm

I do NOT recommend using it in real projects.

npm i periodic-crawler

Exsampls

const periodicCrawler = require('periodic-crawler');
 
async () => {
  const res = await periodicCrawler(['www.example.com/'], {
    proxy: {
      useProxy: false
    },
    settings: {
      pauseMin: 1,
      pauseMax: 2
    }
  });
  console.log(res);
};
 
// in console:
// [
//   {
//     "url": "www.example.com/",
//     "dowDate": "2019-04-01T02:39:55.268Z",
//     "body": "<!doctype html>\n<html>\n<head></head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is established to be used for illustrative examples in documents. You may use this\n    domain in examples without prior coordination or asking for permission.</p>\n    <p><a href=\"http://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>\n"
//   }
// ]

TODO

Readme

Keywords

none

Package Sidebar

Install

npm i periodic-crawler

Repository

github.com/Ganevru/periodic-crawler

Homepage

github.com/Ganevru/periodic-crawler

Weekly Downloads

1

Version

1.0.4

License

MIT

Unpacked Size

13.8 kB

Total Files

15

Last publish

Collaborators

  • ganevru
Try on RunKit
Report malware