Search results

444 packages found

Crawler is a web spider written with Nodejs. It gives you the full power of jQuery on the server to parse a big number of pages as they are downloaded, asynchronously

published 1.5.0 3 months ago
M
Q
P

A specification compliant robots.txt parser with wildcard (*) matching support.

published 3.0.1 a year ago
M
Q
P

xvideos.com api implementation.

published 1.6.4 8 days ago
M
Q
P

A tiny node module to detect spiders/crawlers quickly and comes with optional middleware for ExpressJS

published 2.0.1 10 months ago
M
Q
P

URL crawler for analysing web content

published 0.2.7 7 months ago
M
Q
P

Crawler is a web spider written with Nodejs. It gives you the full power of jQuery on the server to parse a big number of pages as they are downloaded, asynchronously

published 1.3.1 2 years ago
M
Q
P

spankbang.com api implementation

published 0.0.3 11 days ago
M
Q
P
M
Q
P

NPM Spider solitaire lib

published 1.0.14 a year ago
M
Q
P

Crawler (spider) of site web pages by domain name

published 1.2.3 3 years ago
M
Q
P

Parses the wget spider output into an object

published 2.0.0 8 years ago
M
Q
P

xvideos.com api implementation.

published 1.0.0 2 months ago
M
Q
P

A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.

published 2.0.3 2 years ago
M
Q
P

This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.

published 3.3.0 2 years ago
M
Q
P

betterLoading library NOT

published 1.0.10 a year ago
M
Q
P

Simple WAF to integrate with Node.js web systems

published 1.0.3 4 years ago
M
Q
P

Detect user-agent is a bot/spider/crawler

published 1.1.2 9 months ago
M
Q
P

har download 从 chrome 生成的 har 文件中下载整个网站所有资源

published 1.0.4 7 years ago
M
Q
P

Spidey with the power of Redis

published 1.0.4 a year ago
M
Q
P

Lightweight crawler that works like a real browser

published 1.0.2 8 years ago
M
Q
P