spidercheck

basic web crawler

SpiderCheck

basic web crawler

  • 'error' - (error)
  • 'crawl' - (statusCode, url, res)

add a URL to the crawl queue

start crawling