flamescraper
TypeScript icon, indicating that this package has built-in type declarations

1.0.11 • Public • Published

Web scraper for humans

Extended and fully async version of scrape-it

npm install --save flamescraper
Features and usage
  • Multiple pages parse per request
  • Recursive subpage loading per each parameter with subpage field
  • Pagination infinity loop, where Next page param detected and next page link is finded
  • You can select for each page: use Puppeteer for SPA sites or default web request
  • If Convert and How is a string => make functions
  • Async PageConvertor callback after each page (in pagination) for bulk processing/save results, but with full control after each step
  • Async stopCallback for stop scraping if results are right for you (for example you scraped next pages last time)

Readme

Keywords

Package Sidebar

Install

npm i flamescraper

Weekly Downloads

8

Version

1.0.11

License

MIT

Unpacked Size

38.7 kB

Total Files

6

Last publish

Collaborators

  • flameart