nstar-scrape-for-login

NStar Bills Table

Module to interact with the bills table on the NGrid account homepage. This module uses cheerio to parse the account homepage html

Installation

npm install -S nstar-bills-table

Usage

var billsTable = require('nstar-bills-table')
 
var $ = 'account homepage html parsed by cheerio'
var pageNumbers = billsTable.getPageNumbers($)
var billNumbers = billsTable.getBillNumbers($)

Methods

The following methods are supported by the module

• getPageNumbers
• getBillNumbers
• loadNextPage
• viewBill

getPageNumbers

The bills table can have 1 or more pages. The page number appears in the bottom left corner with text that looks like "Page 1 of 3". The getPageNumbers method takes the account homepage html parsed by cheerio as a parameter and returns an object with the page number data

var billsTable = require('nstar-bills-table')
var inspect = require('eyespect').inspector()
var $ = 'account homepage html parsed by cheerio'
var pageNumbers = billsTable.getPageNumbers($)
inspect(pageNumbers, getPageNumbers data)

In the example above, pageNumbers is an object that looks like the following. Note that pageNumbers uses 1-based indexing to match with what appears on the NStar website

var pageNumbers = {
  currentPageNumber: 1, // 1-based index
  numPages: 3
}

getBillNumbers

The bills table can have 0 or more bill rows. The getBillNumbers method takes the account homepage html parsed by cheerio as a parameter and returns an array of bill numbers

var billsTable = require('nstar-bills-table')
var inspect = require('eyespect').inspector()
var $ = 'account homepage html parsed by cheerio'
var billNumbers = billsTable.getBillNumbers($)
inspect(billNumbers, getBillNumbers data)

In the example above, billNumbers is an array that looks like the following...

var billNumbers = [
  "bf3fa73d284759d702564b355a1b29b6",
  "2c32c071f784980f5cf2afa9ef06b5eb"
]

viewBill

When the scraper finds a new bill in the bills table on the account homepage. That bill must be downloaded and parsed. To download the bill, use the viewBill function. In the bills table, in each row there is a *View Bill button. When you click on this button your browser issues a post request. The viewBill function here emulates that same post request to view the bill.

When submitting the post requestion the following fields are required. Note that the number of __VIEWSTATE fields can vary. The __VIEWSTATE#, __PREVIOUSPAGE and __EVENTVALIDATION values are extracted from the account homepage html

{
  "__VIEWSTATEFIELDCOUNT": "dsoifjs",
  "__VIEWSTATE1": "aazbsan2",
  "__VIEWSTATE2": "masdfn0adfsjh2nz0j32",
  "__EVENTVALIDATION": "sd01nsd023mdsf9k32msal"
  "__PREVIOUSPAGE": "asdfasfde",
  "ctl00$Main$lvBills$ctrl0$btViewB.x": "1",
  "ctl00$Main$lvBills$ctrl0$btViewB.y" "1",
 
}

The x & y values in the keys ctl00$Main$lvBills$ctrl0$btViewB.x and ctl00$Main$lvBills$ctrl0$btViewB.y keys refer to the pixel location of where the user clicked on the View Bill button. The values of x & y are thus irrelevant for our use here. The important part is the ctrl0 part of the key. The first bill in the table will have ctrl0, then second bill row will have crtl1, third crtl2 and so on.

When submitting the post form, NStar requires that a valid user agent header by supplied

loadNextPage

The bills table on the account homepage is paginated with about 12 bills per page. If there are more then 12 bills available for a a given login, the bills will be split across multiple pages of the bills table. To switch pages, this module issues a post request as required by the NStar server

When submitting the post request, the following fields are required. Note that the number of __VIEWSTATE fields can vary. The viewstates are extracted from the account homepage

{
  "__EVENTTARGET": "ctl00$Main$lvBills$viewPager",
  "__EVENTARGUMENT": "2" // the desired page index (1-based indexing)
  "__VIEWSTATEFIELDCOUNT": "dsoifjs",
  "__VIEWSTATE": "rrtisp3kds92mvmvvzpal3"
  "__VIEWSTATE1": "aazbsan2",
  "__VIEWSTATE2": "masdfn0adfsjh2nz0j32",
  "__EVENTVALIDATION": "sd01nsd023mdsf9k32msal"
  "__PREVIOUSPAGE": "dsdfmxcv09jasdf",
 
}

The __EVENTTARGET should always be the same static value of ctl00$Main$lvBills$viewPager. The __EVENTARGUMENT value should match the desired bills table page that the scraper is requesting. Thus to go to the third page, __EVENTARGUMENT = '3'

As with other requests to the NStar website, when submitting the post form, NStar requires that a valid user agent header be supplied as well