Scrape the web.
Skyscraper is an easy to use web scraper / spider for extracting arbitrary data from web pages by using simple CSS selectors. Skyscraper is super good when you want to evaluate a number of pages that all follow the same format.
Try it out for ya self!
$ npm install skyscraper --save
Skyscraper is based on simple schemas (we call them blueprints), somewhat inspired by Mongoose. One blueprint describes one type of page and is linked to one Skyscraper instance. After you have set up the blueprint, you can start scraping pages.
/*** Create a new `Skyscraper` instance.*/var skyscraper = ;/*** Set up the blueprint.*/skyscraperblueprintobject/*** Scrape a page.*/skyscraper;
Though we have not reached v1.0.0 yet, we are pretty optimistic about our Sykscraper. Here are the key facts:
Skyscraper is based on cheerio by Matthew Mueller, which is pretty well tested and gets constantly updated. Cheerio is probably the fastest and most forgiving jQuery-esque implementation for Node.js.
We follow the Semantic Versioning rules to spare you the dependency hell.
We heavily use Skyscraper ourself 24/7 for our web-indexing software. If we discover an issue, we want to get it fixed. Instantly.