keywords:Crawler - npm search

puppeteer-extra-plugin-stealth

Stealth mode: Applies various techniques to make detection of headless puppeteer harder.

berstend

published 2.10.1 7 days ago

M

Q

P

@nodelib/fs.walk

A library for efficiently walking a directory recursively

mrmlnc

published 1.2.8 a year ago

M

Q

P

crawler

Crawler is a web spider written with Nodejs. It gives you the full power of jQuery on the server to parse a big number of pages as they are downloaded, asynchronously

mike442144

published 1.3.0 2 years ago

M

Q

P

node-iceberg

nodejs lightweight scraper/crawler

manuasir

published 1.7.1 5 years ago

M

Q

P

robots-txt-parser

A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.

cakroyd

published 2.0.3 2 months ago

M

Q

P

usetube

crawl youtube without api key (search videos channels or get all channel/playlist's videos)

valerebron

published 2.1.4 9 months ago

M

Q

P

node-html-crawler

Crawler (spider) of site web pages by domain name

safonovpro

published 1.2.3 a year ago

M

Q

P

extract-email

A simple email extractor for obfuscated emails.

mehmet.kozan

published 1.1.3 5 years ago

M

Q

P

wafjs

Simple WAF to integrate with Node.js web systems

undertuga

published 1.0.3 3 years ago

M

Q

P

spidey-boy

A simple utility for crawling a file system.

jshuttler

published 1.0.1 3 years ago

M

Q

P

inject-fingerprint

Create Internal proxy to inject fingerprint

laissonrs

published 1.1.3 6 months ago

M

Q

P

Yet another node torrent scraper based on x-ray. (Support iptorrents, torrentleech, torrent9, Yyggtorrent, ThePiratebay, torrentz2, 1337x, KickassTorrent, Rarbg, TorrentProject, Yts, Limetorrents, Eztv)

jimmylaurent

published 2.1.4 a year ago

M

Q

P

es6-crawler-detect

This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.

jefferyhus

published 3.3.0 6 months ago

M

Q

P

x-ray-build

A helper that build a x-ray based on a schema

diogoazevedos

published 1.2.0 4 years ago

M

Q

P

dom-reducer

DOM crawler, executing given reducer recursively beginning with provided node.

stormcrows

published 1.0.9 2 years ago

M

Q

P

dom-visitor

Visitor DOM crawler, supporting breadth-first & in-order traversal.

stormcrows

published 1.1.3 2 years ago

M

Q

P

osmosis

Web scraper for NodeJS

rchipka

published 1.1.10 3 years ago

M

Q

P

torrent-indexer

Yet another node.js torrent scraper based on seek-torrent but made especially for movies, series, anime and music (scrape from 1337x, eztv, limetorrents, rarbg, skytorrents, torrentproject, yts and zooqle)