Search results
1054 packages found
The fastest directory crawler & globbing alternative to glob, fast-glob, & tiny-glob. Crawls 1m files in < 1s
A library for efficiently walking a directory recursively
Crawler is a ready-to-use web spider that works with proxies, asynchrony, rate limit, configurable request pools, jQuery, and HTTP/2 support.
A library to recursively retrieve and serialize Notion pages with customization for machine learning applications.
Apify API client for JavaScript
A tiny node module to detect spiders/crawlers quickly and comes with optional middleware for ExpressJS
The unofficial HLTV Node.js API
x-crawl is a flexible Node.js AI-assisted crawler library.
- x-crawl
- nodejs
- typescript
- ts
- javascript
- crawl
- crawler
- spider
- ai
- ai assisted
- ai crawl
- flexible
- control page
- rotate agents
- View more
Asset Crawler for common Web pages
The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.
Dex8 CLI is a command line interface which helps developers to create and run dex8.com skripts (automated serverless scripts).
DOM Document Object Artifact Collector
Collection of patches for puppeteer and playwright to avoid automation detection and leaks. Helps to avoid Cloudflare and DataDome CAPTCHA pages. Easy to patch/unpatch, can be enabled/disabled on demand.
- automation
- bot
- bot-detection
- crawler
- crawling
- chromedriver
- webdriver
- headless
- headless-chrome
- stealth
- captcha
- scraping
- web-scraping
- cloudflare
- View more
Distributed web crawler powered by Headless Chrome
A lightweight web crawler.
The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.
Gracefully handle timeout and network error with auto retry.
- graceful
- retry
- retries
- error
- errors
- handling
- timeout
- ERR_NETWORK
- ERR_CONNECTION
- ERR_SOCKET
- page
- crashed
- goto
- playwright
- View more
Web service and CLI tool for SEO site audit: crawl site, lighthouse all pages, view public reports in browser. Also output to console, json, csv.
linkedin scraper for 2020 website
xvideos.com api implementation.