Search results

subscrapetest

subscrape is a very simple tool to download the latest images from a specific sub reddit.

• meme
• reddit
• node
• download
• rip
• memes
• extract
• images
• crawler
• regex
• javascript

voids

published 0.1.0 • 5 years agopublished 0.1.0 5 years ago

floob

Floob is a simple web crawler which can run several plugins to analyze each page.

• crawler

zlypher

published 0.0.4 • 8 years agopublished 0.0.4 8 years ago

copha

a general framework for running custom network tasks.

• copha
• cli
• tools
• log
• logs
• koa
• cron
• nodejs
• framework
• selenium
• crawler
• puppeteer

yxwzaxns

published 0.0.9 • 3 years agopublished 0.0.9 3 years ago

lemming

a simple recursive file crawler

• runner
• crawler
• directory
• promise

mechanoid

published 0.0.4 • 9 years agopublished 0.0.4 9 years ago

jasg

Just Another Sitemap Generator

• sitemap
• crawler

angga2oioi

published 0.0.11 • 4 years agopublished 0.0.11 4 years ago

godless

crawler

• crawler

yneos

published 0.1.69 • 7 years agopublished 0.1.69 7 years ago

website-detail-crawler

A demo package for crawing the basic details of a website

• website
• crawler

nikhil12321

published 1.0.1 • 8 years agopublished 1.0.1 8 years ago

crawler-zdy

crawler

• crawler

zhangdanyang

published 1.0.0 • 7 years agopublished 1.0.0 7 years ago

crawl_fetcher

A web crawler. Automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.

• crawler
• spider

wangnn01

published 1.0.0 • 4 years agopublished 1.0.0 4 years ago

node-crawlers-aa

Send custom response to Facebook and Twitter agents

• Facebook
• crawler
• user-agent
• Twitter

atulawate

published 1.0.0 • 7 years agopublished 1.0.0 7 years ago

node-readability-cheerio

node-readability的cheerio版。支持GBK、GB2312等编码的网页抓取和解析。Turning any web page into a clean view.

• readability
• GBK
• GB2312
• web
• crawler
• cheerio

indooorsman

published 1.0.0 • 9 years agopublished 1.0.0 9 years ago

fb-crawler-check

Check user agent to be a facebook crawler

• facebook
• crawler
• check
• user
• agent

rogovdm

published 0.8.1 • 8 years agopublished 0.8.1 8 years ago

malkovich-malkovich

A lightweight and simple API for web crawling built on chromium puppeteer

• crawling
• crawler
• puppeteer
• web
• scraping

austin_c

published 0.0.1 • 4 years agopublished 0.0.1 4 years ago

kremlin

Modern WebApp Pentesting Suite

• pentest
• fuxzzer
• exploit
• scanner
• crawler
• attack

zacharyiles

published 0.0.5 • 7 years agopublished 0.0.5 7 years ago

bauer-crawler-extract

Plugin for bauer-crawler to extract values from JSON data.

• multi-thread
• multi-core
• process
• fork
• multi-process
• cluster
• crawler
• fetch

yneves

published 0.1.5 • 9 years agopublished 0.1.5 9 years ago

jdistiller

A page scraping DSL for extracting structured information from unstructured XHTML, built on Node.js and jQuery.

• crawler
• jQuery

bcoe

published 2.0.0 • 10 years agopublished 2.0.0 10 years ago

bauer-crawler-pdf-to-text

Plugin for bauer-crawler to convert PDF into text.

• multi-thread
• multi-core
• process
• fork
• multi-process
• cluster
• crawler
• scrape
• scraper

yneves

published 0.1.4 • 9 years agopublished 0.1.4 9 years ago

bauer-crawler-csv-to-json

Plugin for bauer-crawler to convert CSV into JSON.

• multi-thread
• multi-core
• process
• fork
• multi-process
• cluster
• crawler
• scrape
• scraper

yneves

published 0.1.3 • 9 years agopublished 0.1.3 9 years ago

web-htmlparser

moved to web-automation.

• node
• crawler

zyp001a

published 0.0.4 • 10 years agopublished 0.0.4 10 years ago

bauer-plugin-csv-to-json

Plugin for bauer to convert CSV into JSON.

• multi-thread
• multi-core
• process
• fork
• multi-process
• cluster
• crawler
• csv
• json

yneves

published 0.2.2 • 8 years agopublished 0.2.2 8 years ago