Have opinions about JavaScript? We want to hear them. Take the 2018 JavaScript Ecosystem Survey »

raspar

1.1.2 • Public • Published

raspar

A simple to use Promise-based web scraper with local caching.

Build Status codecov Dependency Status Known Vulnerabilities bitHound Overall Score NPM Version Greenkeeper badge Latest Documentation

Usage

const raspar = require('raspar');
 
raspar.fetch('http://www.google.com/humans.txt').then(res => {
 
    console.log(res.body);
 
});
const raspar = require('raspar');
 
raspar.fetch(['http://www.google.com/humans.txt']).then(res => {
 
    console.log(res[0].body);
 
});

Options

const raspar = require('raspar');
 
const options = {
    'cacheDirectory': 'temp/cache/',
    'cacheMemory': false,
    'requestOptions': {
        'headers': {
            'User-Agent': 'request'
        },
        'method': 'POST'
    },
    'ttl': 1800
};
 
raspar.fetch('http://www.google.com/humans.txt', options).then(res => { });
Name Description Default Value
cacheDirectory Directory to store cache. temp/cache/
cacheMemory Store cache in memory instead of local files. false
ttl TTL (Time to live) in seconds for local/memory based cache. 1800

Documentation

View full documentation here.

install

npm i raspar

Downloadsweekly downloads

38

version

1.1.2

license

MIT

homepage

github.com

repository

Gitgithub

last publish

collaborators

  • avatar
Report a vulnerability