A flexible, REST-based job queue.
At Axisto Media, we had a requirement to run a processor-intensive task (generating images of slides from a PDF file) on a server separate to those running the web application.
We originally tried Kue, but encountered issues using it, and so decided to create a tool that handled failing jobs gracefully.
In order to cater for the different kind of processor-intensive jobs we handle at Axisto, we created a tool that did not require implicit knowledge of our web application - it simply provided a REST API that the web application could interact with, store any kind of JSON data that we needed the job worker to know, and a way for the Job Queue to communicate the state of those jobs back to the web application.
git clone git://github.com/webcast-io/jobukyu cd jobukyu npm install
Then, make a copy of the config.example.js file
npm run config
... and supply your preferred database credentials and port to run the app on.
Boot the server
The Job Queue is accessed via a REST API. There is a single resource called
GET /jobs Lists all of the jobs in the queue, regardless of status POST /jobs Creates a job GET /jobs/new Lists all of the jobs that are yet to be processed GET /jobs/processing Lists all of the jobs that are being processed GET /jobs/completed Lists all of the jobs that have completed GET /jobs/failed Lists all of the jobs that failed GET /jobs/search Search for jobs based on search criteria GET /jobs/:id Shows a job's details PUT /jobs/:id Updates a job PUT /jobs/:id/take Marks a job's status as 'processing', so that no other workers can take it PUT /jobs/:id/release Marks a job's status as 'new', so that other workers can take it PUT /jobs/:id/complete Marks a job's status as 'completed' PUT /jobs/:id/fail Marks a job's status as 'failed' PUT /jobs/:id/retry Marks a job's status from 'failed' to 'new' DELETE /jobs/:id Deletes a job from the queue
The Job Queue can be installed and run from the command line:
Installing npm release:
$ npm install -g jobukyu
Installing git release:
$ npm install -g git://github.com/webcast-io/jobukyu
Running using CLI
Running useing CLI (with a path to a config file)
$ jobukyu --config ./my_custom_config.json