Search results
15 packages found
Priority based Semantic Web Crawler.
- semantic-crawler
- pdf-crawler
- text-crawler
- priority
- priority-crawler
- scraper
- crawling
- spider
- scraping
- simplecrawler
- crawler
- osmosis
- js-crawler
- supercrawler
- View more
Very straigntforward web crawler. Uses EventEmitter. Generates queue statistics and has a basic cache mechanism with extensible backend. This version is forked in order to add the ability to filter by referrer url
SQLite FetchQueue Implementation for Simplecrawler
MongoDB FetchQueue Implementation for Simplecrawler
iSpider is a simple and fast web spider written with Nodejs!
MongoDB queue implementation for simplecrawler
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
web-spider is a simple and fast web spider written with Nodejs!
Forked version of https://github.com/ciffard/node-simplecrawler.
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
MongoDB queue for Node Simple Crawler
A web crawler forked from simplecrawler.
A web crawler forked from simplecrawler.
Very straightforward web crawler. Uses EventEmitter. Based on Simplecrawler but using a distributed queue system.
- simple
- crawler
- spider
- cache
- queue
- simplecrawler
- eventemitter
- spiderhunt
- distributed
- redis
- rabbitmq
- rabbit
- scalable