share-info
This module will scrape metadata from the URL shared
Module in pure javascript for node.js
Crawling supports:
- Configurable pool size and retries
- Control rate limit
- Priority queue of requests
- Parser with Cheerio
Scraping meta information supports:
- the
<title>
tag in the document head meta[name="description"]
tagmeta[property="og"]
tagmeta[name="twitter"]
tag
Installation
This is a Node.js module available through the npm registry.
Before installing, download and install Node.js. Node.js 8.0 or higher is required.
Installation is done using the
npm install
command:
$ npm install share-info
Example
shareInfo.parse(url, html)
var request = ;var url = 'https://github.com/'; ;
//results
Use Crawler
var shareCrawler = gzip: true maxConnections: 2 method: 'GET' timeout: 5000 retries: 3 retryTimeout: 3000 { if error console; else console; //console.log(results.parser.meta); //console.log(results.parser.og); //console.log(results.parser.twitter); //console.log(results.req.context); //console.log(results.res); ; };shareCrawler;shareCrawler;
Tests
Tests are written with mocha
npm test