jobhub
Move your heavy workloads into fully managed child processes.
Features
- Jobs run in child processes and terminate when they are complete.
- Actively running jobs are centrally managed and tracked.
- Jobs can be optionally configured to:
- Validate job params.
- Quickly handle a job before a child process is forked.
- Be unique, so only one instance of the job runs at a time.
- Be unique by user-generated key, so only one runs per key at a time.
- Send progress updates.
- Jobs that linger after success/failure are auto-terminated.
- Highly customizable using built-in middleware or by directly extending classes.
API Docs and Examples
There are two forms of the API docs:
- Normal API docs - Use this if you using jobhub as-is and not customizing/extending it's functionality.
- Extended API docs - Use this to also view middleware and
protected
classes and methods that you can use to customize/extend jobhub.
Examples are also available in the examples dir.
Quick Start
Install the library using NPM into your existing node project:
npm install jobhub
At a minimum you need to instantiate and start the HubManager, which we will add to a file named index.js
.
The only required configuration for HubManager is the path to the file which defines your jobs.
/* Contents of index.js */ // Import the HubManager from jobhubvar HubManager = HubManager; // Create a new instance of the HubManager, configured to load your jobs from jobs.jsvar hub = jobsModulePath: ; // Startup the hub manager, which will import and register the jobs from jobsModulePath.hubstart;
Defining Jobs
Jobs are defined within the module specified by HubManagerOptions#jobsModulePath with each export following the JobConfig object type definition.
In the Quick start above we defined it as jobs.js
, so let's create that file and define some jobs:
/* Contents of jobs.js */ // The simplest form of a job is a function, which is executed in the child process.exports { var numberOfGremlins = jobparamsnumberOfGremlins; if numberOfGremlins < 1 job; else var hour = ; if hour >= 0 && hour < 5 // Assumes okay to feed after 6am job; else job; ; // TODO: Yikes! What if it goes past midnight while we're feeding? ; }; // If a job is an object, we can add additional configuration.exportsgeGremlinsWet = // For example, let's validate the params. // The validation is performed in both the parent and child process. { if paramsnumberOfGremlins === 0 throw 'No gremlins to get wet!'; } // At a minimum you must include a "run" function, which is executed in the child process. { job; };
Queuing Jobs
We can test out our jobs by adding some more code to index.js
that calls hub.queueJob:
/* Add to the end of index.js */ // Queue a job to be run, which returns an object that lets us track the job.var trackedJob = hub; // We can listen for events, such as when a progress message is received. trackedJob; // TrackedJob instances include then/catch to be Promise-liketrackedJob ;
Express.js Example
A common use case for jobhub is to use it with a HTTP server, such as express.js.
Since you do not want your heavy processing to be done in the same process as your express app, we can use jobhub to run that processing in child processes.
First install express and jobhub.
npm install express jobhub
Second, create our express app in index.js
:
/* Contents of index.js */ var express = ;var HubManager = HubManager; var hub = jobsModulePath: start; // Listen to job lifecycle events.hub; var app = ; app; app; app;
Finally, create the jobs in jobs.js
:
/* Contents of jobs.js */ exports'big-daily-data-load' = // Make sure only one instance runs at a time. unique: true { ; };
Start the server using:
node index.js
We can test the routes using a few HTTP calls. Create a file with the following contents named client.js
:
/* Contents of client.js */ var http = ; ; ; ; ; { http;} { var body = ''; stream;}
Then run the tests on the command line:
node client.js
Change Log
See CHANGELOG.md
License
The MIT License (MIT)
Copyright (c) 2016 Andre Mekkawi <github@andremekkawi.com>
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.