Share your code. npm Orgs help your team discover, share, and reuse code. Create a free org »

    nstar-scrape-for-loginpublic

    NStar Bills Table

    Module to interact with the bills table on the NGrid account homepage. This module uses cheerio to parse the account homepage html

    Installation

    npm install -S nstar-bills-table

    Usage

    var billsTable = require('nstar-bills-table')
     
    var $ = 'account homepage html parsed by cheerio'
    var pageNumbers = billsTable.getPageNumbers($)
    var billNumbers = billsTable.getBillNumbers($)

    Methods

    The following methods are supported by the module

    • getPageNumbers
    • getBillNumbers
    • loadNextPage
    • viewBill

    getPageNumbers

    The bills table can have 1 or more pages. The page number appears in the bottom left corner with text that looks like "Page 1 of 3". The getPageNumbers method takes the account homepage html parsed by cheerio as a parameter and returns an object with the page number data

    var billsTable = require('nstar-bills-table')
    var inspect = require('eyespect').inspector()
    var $ = 'account homepage html parsed by cheerio'
    var pageNumbers = billsTable.getPageNumbers($)
    inspect(pageNumbers, getPageNumbers data)

    In the example above, pageNumbers is an object that looks like the following. Note that pageNumbers uses 1-based indexing to match with what appears on the NStar website

    var pageNumbers = {
      currentPageNumber: 1, // 1-based index 
      numPages: 3
    }

    getBillNumbers

    The bills table can have 0 or more bill rows. The getBillNumbers method takes the account homepage html parsed by cheerio as a parameter and returns an array of bill numbers

    var billsTable = require('nstar-bills-table')
    var inspect = require('eyespect').inspector()
    var $ = 'account homepage html parsed by cheerio'
    var billNumbers = billsTable.getBillNumbers($)
    inspect(billNumbers, getBillNumbers data)

    In the example above, billNumbers is an array that looks like the following...

    var billNumbers = [
      "bf3fa73d284759d702564b355a1b29b6",
      "2c32c071f784980f5cf2afa9ef06b5eb"
    ]

    viewBill

    When the scraper finds a new bill in the bills table on the account homepage. That bill must be downloaded and parsed. To download the bill, use the viewBill function. In the bills table, in each row there is a *View Bill button. When you click on this button your browser issues a post request. The viewBill function here emulates that same post request to view the bill.

    When submitting the post requestion the following fields are required. Note that the number of __VIEWSTATE fields can vary. The __VIEWSTATE#, __PREVIOUSPAGE and __EVENTVALIDATION values are extracted from the account homepage html

    {
      "__VIEWSTATEFIELDCOUNT": "dsoifjs",
      "__VIEWSTATE1": "aazbsan2",
      "__VIEWSTATE2": "masdfn0adfsjh2nz0j32",
      "__EVENTVALIDATION": "sd01nsd023mdsf9k32msal"
      "__PREVIOUSPAGE": "asdfasfde",
      "ctl00$Main$lvBills$ctrl0$btViewB.x": "1",
      "ctl00$Main$lvBills$ctrl0$btViewB.y" "1",
     
    }

    The x & y values in the keys ctl00$Main$lvBills$ctrl0$btViewB.x and ctl00$Main$lvBills$ctrl0$btViewB.y keys refer to the pixel location of where the user clicked on the View Bill button. The values of x & y are thus irrelevant for our use here. The important part is the ctrl0 part of the key. The first bill in the table will have ctrl0, then second bill row will have crtl1, third crtl2 and so on.

    When submitting the post form, NStar requires that a valid user agent header by supplied

    loadNextPage

    The bills table on the account homepage is paginated with about 12 bills per page. If there are more then 12 bills available for a a given login, the bills will be split across multiple pages of the bills table. To switch pages, this module issues a post request as required by the NStar server

    When submitting the post request, the following fields are required. Note that the number of __VIEWSTATE fields can vary. The viewstates are extracted from the account homepage

    {
      "__EVENTTARGET": "ctl00$Main$lvBills$viewPager",
      "__EVENTARGUMENT": "2" // the desired page index (1-based indexing) 
      "__VIEWSTATEFIELDCOUNT": "dsoifjs",
      "__VIEWSTATE": "rrtisp3kds92mvmvvzpal3"
      "__VIEWSTATE1": "aazbsan2",
      "__VIEWSTATE2": "masdfn0adfsjh2nz0j32",
      "__EVENTVALIDATION": "sd01nsd023mdsf9k32msal"
      "__PREVIOUSPAGE": "dsdfmxcv09jasdf",
     
    }

    The __EVENTTARGET should always be the same static value of ctl00$Main$lvBills$viewPager. The __EVENTARGUMENT value should match the desired bills table page that the scraper is requesting. Thus to go to the third page, __EVENTARGUMENT = '3'

    As with other requests to the NStar website, when submitting the post form, NStar requires that a valid user agent header be supplied as well

    Keywords

    none

    install

    npm i nstar-scrape-for-login

    Downloadslast 7 days

    2

    version

    1.0.1

    license

    none

    last publish

    collaborators

    • avatar