387 packages found

    Crawler is a web spider written with Nodejs. It gives you the full power of jQuery on the server to parse a big number of pages as they are downloaded, asynchronously

    published 1.3.0 a year ago
    M
    Q
    P

    A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.

    published 2.0.2 4 months ago
    M
    Q
    P

    Crawler (spider) of site web pages by domain name

    published 1.2.3 2 months ago
    M
    Q
    P

    har download 从 chrome 生成的 har 文件中下载整个网站所有资源

    published 2.1.0 a year ago
    M
    Q
    P

    JSpider 3 is a Chrome DevTools crawler framework that includes full crawler support. JSpider 3 是在 Chrome Devtools 中进行爬虫的爬虫框架, 这个框架包括了完整的爬虫支持。

    published 3.2.3 2 months ago
    M
    Q
    P

    xvideos.com api implementation.

    published 1.2.0 6 months ago
    M
    Q
    P

    Parse and stringify cookies for web spider.

    published 1.0.3 7 months ago
    M
    Q
    P

    This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.

    published 3.1.3 7 months ago
    M
    Q
    P

    Another web crawler.

    published 1.1.0 17 days ago
    M
    Q
    P

    DEX8 SDK is a library which helps developers to create DEX8 crawler, scraper tasks or automated serverless scripts.

    published 2.11.0 9 hours ago
    M
    Q
    P

    xvideos.com api implementation and update.

    published 1.1.21 10 months ago
    M
    Q
    P

    极简网络蜘蛛爬虫,适用任何网站,只需设置一条规则,就可以把你想要网站上的内容整理出来,非常方便,简单!

    published 5.0.10 a year ago
    M
    Q
    P

    A tiny node module to detect spiders/crawlers quickly and comes with optional middleware for ExpressJS

    published 2.0.0 2 years ago
    M
    Q
    P

    A simple and fast crawler tool based on node

    published 1.0.3 2 months ago
    M
    Q
    P

    a crawler cli of comics

    published 1.3.3 a year ago
    M
    Q
    P

    Simple WAF to integrate with Node.js web systems

    published 1.0.3 2 years ago
    M
    Q
    P

    A node.js API for scraping lossless mp3s, albums, EDM tracks, torrents and more with cache support.

    published 1.2.4 20 days ago
    M
    Q
    P

    web spider, support puppeteer, cheerio and so on, include task-queue and dispatcher

    published 2.2.4-preview.1607350101966 10 months ago
    M
    Q
    P

    爬虫工具,根据传入的配置和规则爬取数据

    published 1.1.4-0 a year ago
    M
    Q
    P

    Detect user-agent is a bot/spider/crawler

    published 1.0.1 3 months ago
    M
    Q
    P