Search results
223 packages found
x-crawl is a flexible Node.js AI-assisted crawler library.
- x-crawl
- nodejs
- typescript
- ts
- javascript
- crawl
- crawler
- spider
- ai
- ai assisted
- ai crawl
- flexible
- control page
- rotate agents
- View more
A library for scraping manga from various websites.
- manga
- scraping
- nettruyen
- crawl
- manga crawl
- crossover
- web scraping
- manga scraping
- manga crawler
- manga scrapper
- manga-lib
- manga api
- blogtruyen
- blogtruyen.vn
- View more
Fetch special for spider.
Async and sync crawler for json object
This library provides support for traversing objects and their values while providing information on the traversal state, pathing to target values, and the ability to manipulate said pathing to easily move to related values.
A node roguelike
Crawls a Multi-Page Application to a zip file, serve the Multi-Page Application from the zip file. A MPA archiver. Could be used as a Site Generator
Craw bot verifier. Veirfy search engines crawl bots.
A simple crawler made in JavaScript for Node.
This tool helps you to generate lighthouse reports for provided sitemap URL & compare them effortlessly.
SyphonX is a tool that extracts data from HTML data, transforming it into JSON of any shape or size. It combines the power of CSS Selectors and jQuery, Regular Expressions, and Javascript into a declarative template format to elegantly solve the simplest
- automation
- cheerio
- crawl
- crawler
- crawling
- chrome
- dom
- headless
- html
- html2json
- jquery
- parse
- parser
- puppeteer
- View more
[](https://www.npmjs.com/package/recrawl-sync) [](https://github.com/aleclarson/recrawl/actions/workflows/release.yml)
OAI-PMH harvester module for nodejs - typescript refactor
fs.readdir with sync, async, streaming, and async iterator APIs + filtering, recursion, absolute paths, etc.
- fs
- readdir
- async
- promise
- iterator
- generator
- async-iterator
- stream
- event
- event-emitter
- recursive
- deep
- walk
- crawl
- View more
Generates a sitemap for a site.
fs.readdir with sync, async, streaming, and async iterator APIs + filtering, recursion, absolute paths, etc.
- fs
- readdir
- async
- promise
- iterator
- generator
- async-iterator
- stream
- event
- event-emitter
- recursive
- deep
- walk
- crawl
- View more
Walk an entire directory. Fast, simple, and asynchronous.
takes an array of urls and crawls the urls for links
[](https://www.npmjs.com/package/recrawl) [](https://github.com/aleclarson/recrawl/actions/workflows/release.yml) [![codeco
A library for scraping manga from various websites.
- manga
- scraping
- nettruyen
- crawl
- manga crawl
- crossover
- web scraping
- manga scraping
- manga crawler
- manga scrapper
- manga-crawl-lib
- manga api
- blogtruyen
- blogtruyen.vn
- View more