node package manager

jwebquery

An jQuery style web crawler(actually extend jquery).

jwebquery

jQuery like web crawler

Note

It's my first node modules and features are not well tested now.Since it's just a crawler,use it on your own risk.

Feature

  • gzip support
  • http/https support
  • cookie support
  • can set page fetch/download interval,and max parallel download limitations.
  • http proxy support
  • GET method support ,POST is not tested and upload multipart is not supported
  • auto add http head Referer

Example

var jwebquery = require("jwebquery");
(new jwebquery.Page("http://www.stackoverflow.com/")).ready(function(page){
    //get things by css selector 
    //we get every summery text of index page of stackoverflow 
    page.find(".question-summary .summary h3").each(function(index){
//access jquery from jwebquery  
console.log(index,jwebquery.$(this).text());
    })
}).on("error",function(err){
    console.log(err);
})
 
 
//use localhost:8099 as http proxy 
jwebquery.setProxy("http://localhost:8099/");