- allows a data loader to register with it (DONE)
- presents a dashboard (TO DO)
- allows jobs to be re-run earlier than scheduled (TO DO)
Warning - Client loader hasn't run in more than 24 hours (select max(startTime) from LoaderSnapshots where exitcode=<3). Warning - Client loader hasn't run and it is the 1st of the month
Exit Codes - 0, ran fine no errors, all rows loaded (Only sent at end of run) - 1, loader exited with a caught exception at some point during execution (Sent on any caught exception) - 2, loader ran to completion but received errors from the services while executing (Sent at end of run, checking for status of the failures) - 3, loader tried to run but had no files to execute with (Sent at branch of file load). - 4, partial fail on ingestion, but load continued.
We want to know - Is the loader continuing to run on its schedule (Has the loader run in X minutes) - two phase register. - Did the loader register a failure at the end of its run? - Did the loader fail abruptly (i.e. did it register an exception) - What failures did the lodaer register?
- loader will run every so often. Would be good to know a heartbeat (has the loader stopped running for whatever)