❤
npm
Sign UpSign In

Search results

14 packages found

Sort by: Default
  • Default
  • Most downloaded this week
  • Most downloaded this month
  • Most dependents
  • Recently published

simplecrawler

Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.

kbychkov
published version 1.1.9, 5 years ago76 dependents licensed under $BSD-2-Clause
135,810

semantic-crawler

Priority based Semantic Web Crawler.

mehmet.kozan
published version 0.0.2, 7 years ago0 dependents licensed under $MIT
307

web-spider

web-spider is a simple and fast web spider written with Nodejs!

yuexing0921
published version 0.3.1, 8 years ago0 dependents licensed under $MIT
43

simplecrawler-zic

Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.

zicjin
published version 1.2.6, 6 years ago0 dependents licensed under $MIT
39

simplecrawler-mongo-queue

MongoDB queue implementation for simplecrawler

kbychkov
published version 1.0.1, 5 years ago0 dependents licensed under $MIT
34

mongodb-simplecrawler-queue

MongoDB FetchQueue Implementation for Simplecrawler

novikovevgeniy
published version 0.4.1, 5 years ago0 dependents licensed under $MIT
29

ispider

iSpider is a simple and fast web spider written with Nodejs!

yuexing0921
published version 0.0.4, 9 years ago0 dependents licensed under $MIT
25

spiderhunt

Very straightforward web crawler. Uses EventEmitter. Based on Simplecrawler but using a distributed queue system.

donflopez
published version 0.1.2, 9 years ago0 dependents licensed under $BSD-2-Clause
25

sqlite-simplecrawler-queue

SQLite FetchQueue Implementation for Simplecrawler

lemoussel
published version 1.1.0, 5 years ago0 dependents licensed under $MIT
22

simplecrawler-fork

Forked version of https://github.com/ciffard/node-simplecrawler.

james-mckinnon
published version 0.5.5, 9 years ago0 dependents licensed under $BSD-2-Clause
16

jes-spider

A web crawler forked from simplecrawler.

e_tang
published version 0.0.2, 8 years ago0 dependents licensed under $BSD-2-Clause
19

simplecrawler-referrer-filter

Very straigntforward web crawler. Uses EventEmitter. Generates queue statistics and has a basic cache mechanism with extensible backend. This version is forked in order to add the ability to filter by referrer url

davidsinclair
published version 0.3.1-1.2, 8 years ago0 dependents
17

jespider

A web crawler forked from simplecrawler.

atomba
published version 0.0.1, 9 years ago0 dependents licensed under $BSD-2-Clause
12

@gombosg/simplecrawler

Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.

gombosg
published version 1.1.9, 6 years ago0 dependents licensed under $BSD-2-Clause
11