Needlessly Promiscuous, Modularize!

    casper-base

    0.1.1 • Public • Published

    casper-base | Utils for scraping the web with CasperJs

    Build

    CasperJs

    A personal, simple module of CasperJs for scraping the web!

    Explanation

    Turned out that there are some configurations (and some utils) that I always needed (re)writing a scraper in CasperJs. So I'm trying to wrap every simple and almost full configurable piece of code to start new projects from here and not totally from scratch or with copy-paste.

    Usage

    Install

    You can simply install via NPM, but the require will be a little bigger cause you will run it with CasperJs and not with NodeJs.

    npm install casper-base
    

    Or clone the repo if you want start from here:

    git clone https://github.com/LasaleFamine/casper-base.git
    

    Or simply download the main files:

    • /custom-utils (entire folder)
    • config.json
    • casper_base.js

    And put them in the root directory of your project.

    Configure

    Right now you can change directly the config.json file of the NPM module to change the configuration of your casper.create() method. You can also clone this repo and change the configurations as you want.

    For more information about CasperJs configuration: Casper Options

    {
      "clientScripts":  [],
      "pageSettings": {
          "loadImages":  false,
          "loadPlugins": false
      },
      "logLevel": "warning",
      "verbose": true
    }

    Import

    Simply require for your casper instance the casper_base module.

    If was installed via NPM:

      var casper = require('./node_modules/casper-base/casper_base')

    Or if the file is in your root directory:

      var casper = require('./casper_base')

    Don't forget to leave the custom-utils folder within the main casper_base with the current hierachy.

    Start scraping!

    First install casperjs and phantomjs if you did't yet:

    npm install --save casperjs phantomjs
    

    Write your first scrapy!

     
    var casper = require('./casper_base')
     
    // Simulate error 404
    casper.start('http://google.com/sad')
    casper.run()
     

    Fork it

    Forks are always appreciate! ✈️

    Keywords

    none

    Install

    npm i casper-base

    DownloadsWeekly Downloads

    1

    Version

    0.1.1

    License

    ISC

    Last publish

    Collaborators

    • lasalefamine