YAMG Ain't Message Groups

YAMG = YAMG Ain't Message Groups

YAMG is a task queue which can guarantee processing order of tasks in a group.

YAMG is based on Redis.

YAMG takes a different approach than Message Groups. Instead of making all the tasks of a same group be routed to the same consumer, YAMG route a task to any worker when there's no pre-pending tasks of the same group. This decision can prevent blocking of tasks of group X being routed to a fixed client A while client A is busy processing tasks of group Y. If there's no pre-pending tasks of group X, a task of group X can be dispatched to any client.

Resque can't guarantee the processing order of the tasks, although the task queue is a fifo list. The more workers you have, and the bigger difference of time required by workers to finish each task, the more possible you'll encountering concurrency race.

If you need:

  1. Multiple homogeneous workers processing a task queue.
  2. Tasks of a same group need be performed by order. Which means the actual performing order, not the order dispatching them.
npm install yamg
queue = require('yamg').connect().queue('yr_queue')
# Provide as many parameters as you needed, the last parameter should be a
# callback.
queue.enqueue 'param_1', 'param_2', 'param_3', -> console.log 'queued'

The first element of the 2nd parameter will be used as the group key.

queue = require('yamg').connect().queue('yr_queue')
# The registered handler function should keep the same signature as you just
# enqueued, plus the last two argument should be 2 callbacks for success and
# fail. Calling them is your responsibility!
queue.regist (param_1, param_2, param_3, callback) ->
  # Do your work here
  setTimeout ->
    err = null
    res = {}
    callback err, res
  , 5

See examples folder for a simple demo.

I wish yamg is helpful. Please help:

  1. Being critical
  2. Provide a test
  3. Point out if there's better way to do this