Setups up an http express app for an single scraper. This app allows external services to get the status of the scraper and start new scrapes
Allow node.js based scrapers to connect to a status socket so that external services can query the current status of the scraper and check if it is running or stopped.
When a scraper starts up, it registers a new service with seaport using the role scraperStatusSupplierCode. For example the NGrid Electric scraper has the supplierCode NGE so it would register with the role scraperStatusNGE
Then this module will connect to an axon rep socket on the port returned by seaport.
To query the status of a scraper, get the port of the status socket from seaport and send a status request over http
var inspect = require'eyespect'inspector;var request = require'request'var config = require'nconf'var supplierCode = 'NGE'var seaConfig = configget'seaport'var seaHost = seaConfighostvar seaPort = seaConfigportvar ports = seaportconnectseaHost seaPortvar role = 'scraperStatus' + supplierCode // eg "scraperStatusNGE"var scraperServices = portsqueryrole // an array of servicesvar scraperService = scraperServices0 // use the first one as an examplevar statusHost = scraperServicehostvar statusPort = scraperServiceportvar url = 'http://' + statusHost + ':' + statusPort + '/status'var opts =url: urljson: truerequestoptsif errinspecterr 'error getting status of scraper'returninspectreply 'scraper status reply'
The reply is json and should look like
reply ="supplierCode": "NGE""status": "running""running": true