23 packages found

    Description

    Parser for XML Sitemaps to be used with Robots.txt and web crawlers

    Keywords

    Publisher

    published 3.1.124 months ago
    M
    Q
    P

    Description

    Easy to use crawling solution for websites.

    Keywords

    Publisher

    published 1.1.08 days ago
    M
    Q
    P

    Description

    Parser for XML Sitemaps to be used with Robots.txt and web crawlers

    Keywords

    Publisher

    published 2.1.19a month ago
    M
    Q
    P

    Description

    Site mapper utilizing puppeteer to generate a xml sitemap

    Keywords

    Publisher

    published 1.0.58 months ago
    M
    Q
    P

    Description

    Crawls through provided website, checking for 200 response, content load, ssl cert errors, and more!

    Keywords

    Publisher

    published 1.1.87 months ago
    M
    Q
    P

    Description

    Search musics on youtube and return your links

    Keywords

    Publisher

    published 3.0.54 months ago
    M
    Q
    P

    Description

    Parser for XML Sitemaps to be used with Robots.txt and web crawlers

    Keywords

    Publisher

    published 2.1.1710 months ago
    M
    Q
    P

    Description

    CLI for download manga and serve it locally.

    Keywords

    Publisher

    published 0.0.67 years ago
    M
    Q
    P

    Description

    A sitemapping tool for crawlers

    Keywords

    Publisher

    published 0.3.14 years ago
    M
    Q
    P

    Description

    Parser for XML Sitemaps to be used with Robots.txt and web crawlers

    Keywords

    Publisher

    published 3.0.23 years ago
    M
    Q
    P

    Description

    Url scraper which takes the text input and finds the links/urls, scraps them using cheerio and will returns an object with original text, parsed text (using npm-text-parser) and array of objects where each object contains scraped webpage's information.

    Keywords

    Publisher

    published 1.0.25 years ago
    M
    Q
    P

    Description

    A simple agent for performing a sequence of http requests in node.js

    Keywords

    Publisher

    published 0.1.210 years ago
    M
    Q
    P

    Description

    Webcrawler script to retrieve the daily menu of the Bern University of Applied Sciences cantina in Biel

    Keywords

    Publisher

    published 1.0.32 years ago
    M
    Q
    P

    Description

    Parser for XML Sitemaps to be used with Robots.txt and web crawlers. (Extended version by mastixmc)

    Keywords

    Publisher

    published 3.2.02 years ago
    M
    Q
    P

    Description

    MyReadingManga WebCrawler. Find By Languages, Genres and Tags.

    Keywords

    Publisher

    published 0.0.102 years ago
    M
    Q
    P

    Description

    Simple framework for crawling/scraping web sites. The result is a tree, where each node is a single request.

    Keywords

    Publisher

    published 1.1.44 years ago
    M
    Q
    P

    Description

    A friendly javascript pre-rendering engine - BETA (UNSTABLE)

    Keywords

    Publisher

    published 1.2.33 years ago
    M
    Q
    P

    Description

    An autonomous webcrawler for indexing robots.txt files.

    Keywords

    Publisher

    published 1.0.35 years ago
    M
    Q
    P

    Description

    A simple webcrawler that prints out the URLs of the pages it encounters. Runs in parallel, up to a limit you specify.

    Keywords

    Publisher

    published 0.1.44 years ago
    M
    Q
    P

    Description

    auto web crawler

    Keywords

    Publisher

    published 0.0.76 years ago
    M
    Q
    P