X Plan Spider
It's the spider framework only for xplan of www.cst.zju.edu.cn.
Attention: You should install rabbitMQ first.
Usage
var XPlanSpider = ;
Pioneer
Create a new pionner
var pioneer = ;
Set the page count function
pioneer;
Attention:
urlWithPage
is some string like"http://foo/bar?page=:page"
. You should write":page"
instead the true page.
Set the list parsing function
pioneer;
Set the url
You should pass all the url with ":page"
.
Eg. there're two types of the system. you should pass:
pioneer;pioneer;
Start the service
pioneerstart"rabbitMQ connection string" "message queue router key" timeout;
Attention:
timeout
is the millisecond. Spider will start pertimeout
milliseconds.
Impl
TODO.
Contributors
- XadillaX
- Waiting for you