snapscrape

Snapscrape is a modular scaper that fetchs content from other sites and uses middleware to parse the data

scraping the web with node on request.

var scrapper = require("snapscrape");
 
scrapper.scrape("http://google.com", function(resbody){
    console.log(arguments);
});

cool beans

 
scrapper.use(/*middleware*/);
 
//comes with a htmlparser 
scrapper.use(scrapper.htmlParser);
scrapper.scrape("http://google.com", function(resbody){
    // see googles javascripts 
    console.log(res.html("body").text());
});

the middleware is very similiar to connects middleware

module.exports = function(responsebodynext){
    next(); 
};

response is the response from the server your pulling from

body is the body from server, or aka the goods

next is a function called to pass payload to next middleware function or callback;

probably hooking this into a promise system so that you can have nice errors;

var error = function(){
    console.log("you fail")
}
scrapper.scrape("FAIL", function(resbody){
    
}, err);

and test!