NairalandScraper

A simple nodejs tool to extract topics(links) from Nairaland's front page and comments relating to certain specified keywords from Nairaland's SERP -- Search Engine Results Page.

Installation

npm install nairaland-scraper

Example

You can pass in an optional object before the callback function to limit the number of scraped data. Defaults to 20.

var NLscraper = require('nairaland-scraper');
 
 
var latest = new NLScraper();
 
latest.getScrapedData({ limit: 10 }, function(err, topics) {
    if (err) { console.log(err) };
    console.log(topics);
    return topics;
})

To scrape data from Nairaland's SERP, make a call to the getSERPScrapedData() method specifying an options object with a specific keyword and board (must be an int, e.g. 0). You can also add a limit params to the options object to limit the number of comments returned.

var latest = new NLScraper();
 
var queryOptions = {
    "q": "fuel",
    "board": 0,
    "limit": 20
}
latest.getSERPScrapedData(queryOptions, function(err, comments) {
    console.log(comments);
    return;
})

License

The MIT License