node-spider-man

Features

Transform the page into JSON format response according to the rules
Use fetchQueue to manage the fetchRequests in the order
Support sync and async mode.
- sync: Execute each fetchTask after the previous one finished
- async: Execute each fetchTask after the delayFetch time
Support queue finished callback
Support to PAUSE or CONTINUE the spider's work(>=0.3.0)

npm install node-spider-man --save

var spiderMan = require('node-spider-man');

The demo shows the experiement that we can grab what we want from the internet using node-spider-man

Lisence @Apache-2.0