Search results
15 packages found
Priority based Semantic Web Crawler.
- semantic-crawler
- pdf-crawler
- text-crawler
- priority
- priority-crawler
- scraper
- crawling
- spider
- scraping
- simplecrawler
- crawler
- osmosis
- js-crawler
- supercrawler
- View more
Very straigntforward web crawler. Uses EventEmitter. Generates queue statistics and has a basic cache mechanism with extensible backend. This version is forked in order to add the ability to filter by referrer url
SQLite FetchQueue Implementation for Simplecrawler
iSpider is a simple and fast web spider written with Nodejs!
MongoDB FetchQueue Implementation for Simplecrawler
MongoDB queue implementation for simplecrawler
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
web-spider is a simple and fast web spider written with Nodejs!
Forked version of https://github.com/ciffard/node-simplecrawler.
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
MongoDB queue for Node Simple Crawler
A web crawler forked from simplecrawler.
A web crawler forked from simplecrawler.
Very straightforward web crawler. Uses EventEmitter. Based on Simplecrawler but using a distributed queue system.
- simple
- crawler
- spider
- cache
- queue
- simplecrawler
- eventemitter
- spiderhunt
- distributed
- redis
- rabbitmq
- rabbit
- scalable