Redis-based simple and safe work queue
A work queue for Node.js producers and consumers that uses Redis.
Example of a client producing work:
var Queue = require'simple-redis-safe-work-queue';client = Queueclient'send-email';clientpushto: 'firstname.lastname@example.org' subject: 'hey there' body: 'yo';
or, with a callback for when the work is pushed:
clientpushto: 'email@example.com' subject: 'hey there' body: 'yo'if err console.error'Error when pushing work into the queue: ' errstack;;
You can also provide some options for the work item you're pushing:
var options =timeout: 120e3 // 120 seconds;clientpushto: 'firstname.lastname@example.org' subject: 'hey there' body: 'yo' optionsif err console.error'Error when pushing work into the queue: ' errstack;;
Client also accepts options as second argument in constructor:
- port: redis port, defaults to 6379
- host: redis host name, defaults to "127.0.0.1"
- password: redis password, defaults to
- redisOptions: additional redis options, defaults to
- defaultTimeout: the default worker timeout, in miliseconds. defaults to 60000 (60 seconds)
Client emmits the following events:
emit('before push', work)- before work is pushed
emit('after push', work)- after work is successfully pushed
emit('error', err)- when a push error happens and a callback wasn't provided
Example of a worker consuming work:
var Queue = require'simple-redis-safe-work-queue';worker = Queueworker'send-email' workFunction;sendEmailemailif err cberr;elseconsole.log'email send successfully, calling back with no errors';cb;;
You can pass some options on the third argument of the worker constructor:
- port: redis port (defaults to 6379)
- host: redis host (defaults to "127.0.0.1")
- password: redis password
- redisOptions: any option allowed by the redis client, defaults to
- maxConcurrency: the maximum pending work units. defaults to 10.
- maxRetries: the maximum number of retries when a work unit errors. defaults to 10.
- popTimeout: the worker pop timeout, after which it retries, in seconds. Defaults to 3 seconds.
- runTimeoutWatchdog: run a timeout watchdog, defaults to
- runStalledWatchdog: run a stalled watchdog, defaults to
- autoListen: worker listens automatically (otherwise you must call
.fetch()when ready for the next message), defaults to
.fetch() method is a non-long-polling version of
.listen(), useful when you want to get a
null response if there are no messages, rather than waiting for one to be pushed. If you're using
autoListen: false then you probably want to use
A worker emits the following events:
emit('ready'): when redis client is ready
emit('listening'): when listening for work
emit('worker error', err): when a worker error occurs
emit('work done', payload): when a worker finishes a piece of work
emit('repush', payload): when a work unit is being repushed after failure
emit('max retries', lastError, payload): when the maximum retries has been reached
Redis 2.6 or greater, with Lua scripting enabled.
Clone this repo, enter the repo directory, start a redis server and run:
$ npm test