A queue that is dequeued by forked processes

ForkQueue creates a queue where the items are removed via a 'next' message received from child processes. The number of child processes is passed to the constructor along with the full path of the module to run. The processes are spawned via child_process.fork


var Queue = require('forkqueue');
var numWorkers = 5,
    module = 'worker.js';
var queue = new Queue(numWorkers, module);

Add a value to the queue

for (var i = 0; i < 100; i++) {

Puts the values of the array onto the queue

var vals = [];
for (var i = 0; i < 100; i++) {

Wait for the child processes to work through the queue then kill them.


The Queue inherits from EventEmitter. It emits the following events:

  • queue.emit('msg', value) - a value forwarded from a worker
  • queue.emit('error', error) - an error
  • queue.emit('enqueue', value) - the enqueued value
  • queue.emit('dequeue', value) - the dequeued value
  • queue.emit('concat', values) - the list of values to enqueue
  • queue.emit('flush') - the queue is trying to flush outstanding tasks to available workers

Worker modules are spawned with child_process.fork. In order to request a value off the queue, they send a 'next' message to the parent with process.send('next'). The only message sent to them contains the value off the queue. They will exit with 'SIGTERM' sent from the parent after queue.end is called. They can also send arbitrary messages != 'next', that get emitted from the queue.

A simple worker is below.

  process.send({msg: 'some string'});
  process.on('message', function(value) {
    // Do something with value 
    // Tell the parent to return the next value off the queue