This package provides a unified API across a variety of different queue backends.
- listening to new jobs
npm i @desmart/queue
const manager job =const syncConnector =const queue =// syncConnector will dispatch job immediately// in this case it's required to attach listeners before job is pushed to queuequeuequeue
Job contains information regarding task that should be handled:
name(String) name of the job
queue(String) name of the queue on which message had been sent
remove()(Function) remove message from the queue backend; has to be triggered once job is processed
release(delay = 0)(Function) put job back to queue backend; optionally it can be delayed by given number of seconds
creating new job
To create Job instance you can use
.of() static method:
const job =job
queue are optional. By default job will use
Each job received by queue backend will be dispatched to it's handler. Job can have only one dedicated handler.
pushing to queue backend
Every job can be pushed through manager to queue backend:
listening to new jobs
By default manager will not listen for incoming jobs.
To start listening for new jobs it's required to call
Listener will wait for new queue messages, convert them to
Job object and pass it to bound handler.
By default listener will check for messages in
Queue manager uses connectors to handle various queue backends.
This package provides two basic connectors:
syncConnectorfires immediately pushed to queue job
nullConnectordrops all pushed to queue jobs
Each connector has to implement following methods:
onJob(fn)accepts callback which should receive
Jobinstance once new message is retrieved from backend
push(job)push job to queue backend
It's possible to extend behaviour of manager with middlewares.
Middleware is a function which should accept
next callback. It's triggered once a job is fetched from backend and redirected to handler.
Through middleware it's possible to modify job (note that job is immutable), or do some other stuff. Don't forget to call
next once you want to pass control to another middleware.
Every middleware should (if possible) return the result of
next(). Remember also that other middlewares may return a Promise so
async/await may be useful here.
const manager job =const autoCommit =const syncConnector =const queue =queue// each handle will be converted to terminating middleware - add them after all middlewaresqueue
Package comes with some bundled middlewares. They can be imported from
const autoCommit =queue
Waits for job to finish and removes it from queue. If job failed it will be released back to queue.
This will works only when job handler returns a Promise.
Job is released with exponential delay. After first attempt it will be released without a delay, with second attempt it will be delayed by 5 seconds, later by 15 seconds and so on.. By defualt, after multiple fails, job will be delayed by 6 hours.
true] should failed job be released with exponential delay
21600] maximum delay for failed jobs; used only when
exponential == true
const maxAttempts =queue
Removes automatically a job which failed more than
const debug =queue
Small utility which uses debug to print information about processed job status.
This middleware has to be used before
If you're planning to contribute to the package please make sure to adhere to following conventions.
tests & linting
- lint your code using standard; run
npm run lintto check if there are any linting errors
- make sure to write tests for all the changes/bug fixes
We're not looking back! You are encouraged to use all features from ES6.
This package follows functional approach - if possible use pure functions, avoid classes etc.
issues & PR
- try to provide regression test when you find a bug
- share some context on what you are trying to do, with enough code to reproduce the issue