node package manager
Painless code sharing. npm Orgs help your team discover, share, and reuse code. Create a free org »


Thoonk Jobs

What is Thoonk?

Thoonk is a persistent (and fast!) framework for Redis backed live data and objects, such as push feeds, queues, and jobs.

See for more on Thoonk.

What is this?

Thoonk Jobs are Redis-based job queues with support for claiming, stalling, retrying, and retracting individual tasks. Jobs are useful for distributing load, ensuring a task is completed regardless of outages, and keeping long running tasks away from synchronous interfaces.

Publish |
      V |     
    +---------+  Stall>    +-------+
    +---------+  <Retry    +-------+
        |  | ^
    Get |  | Cancel
      V |  |
      +-------+             +--------+
      +-------+    Finish>  +--------+

Multiple Thoonk Job instances (aka workers) may share the same job queue, allowing multiple tasks to be processed in parallel.

When a task becomes available in the queue, a worker must claim it to have sole control over its processing. In the case of multiple workers receiving the available task notification, the first to claim the task wins. Once the task is claimed, if there is some recoverable error such as a lack of required resources, the task must be cancelled to send it back to the queue for other workers to claim it. The number of times that a task has been sent back is tracked to facilitate monitoring.

Tasks can also be stalled and moved to a holding area where they will not be dispatched for workers to claim. Tasks that have been cancelled multiple times are good candidates for stalling so that the repeated issues can be investigated before retrying the task.


npm install thoonk-jobs


var thoonk = require('thoonk').createClient();
var Job = require('thoonk-jobs');
thoonk.registerObject('Job', Job, function () {
    var jobPublisher = thoonk.objects.Job('coolJobChannel');
    jobPublisher.subscribe(function () {
        jobPublisher.publish({whatever: 'you need as json'}, {
            //id: 'customId', 
            //priority: true, // push the job to the front of the queue 
            onFinish: function () {
                console.log('Job completed!');
        }, function () {
            console.log('Job published');
var async = require('async');
var thoonk = require('thoonk').createClient();
var Job = require('thoonk-jobs');
thoonk.registerObject('Job', Job, function () {
    var jobWorker = thoonk.objects.Job('coolJobChannel');
    async.forever(function (next) {
        jobWorker.get(0, function (err, item, id) { 
            if (err) return next();
            item = JSON.parse(item);
            jobWorker.finish(id, 'the results', function (err) {


To prevent accidentally clobbering databases, tests are disabled by default. Set "enabled" to true in test_config.json to enable them, then run:

npm test



Created By

If you like this, follow @fritzy on twitter.