Mr Spider
Crawl the web politely.
Installation
$ npm i mrspider --save
Example
Middleware
Fetch the page
Parse DOM
Parse Data
Data validation
Data persistence
Features
- Super simple api.
- Streaming architecture allows complete customisation.
- Use the full power of JavaScript giving you great flexibility.
Tests
To run the test suite, first install the dependencies, then run npm test
: