Search results
218 packages found
Web Crawler
Lightweight crawler written in TypeScript using ES6 generators.
Scrap and collect data from urls, html, json and stuff..
An using chrome headless mode to download dynamic page tool.
Spy on files
Scrapoxy is a proxy for scrapers
Scrapes/Crawls the logo from a provided url(s)/website for your Node.js applications.
NodeJS Crawler for Twitter
The script used to create Ripley, the reddit programming index
Lightweight crawler written in TypeScript using ES6 generators.
Utility to crawl an entry URL and produce a static copy of the site.
A node.js module to crawl product reviews from Amazon.
A node.js module to crawl product IDs from Amazon.
This is a function that accepts 3 arguments, "url", "tag" and "output", and writes to a file, in the "output" path, the content of an html "tag", relative to a specific "url".
A module to download images from a given URL
Crawl websites in a streaming fashion
A structural framework for creating good bots
Parses robots.txt files to provide meaningful, useful output as well as reporting syntax errors.