Tusker - Redis Based Distributed Task Locking
Why? Let's look at following scenario
- We are transcoding ogg streams into mp3 realtime
- And oggs are coming as chunks (<15 secs)
- We have deployed several servers and routing is random and stateless
Lets Cordinate this with Tusker
When we are encoding chunks
var tusker = ;var taskManager = tusker; //when we are begin to encode a chunkvar lockname = "" + Math;taskManager; //after we do the encodingtaskManager;
At the end of the ogg stream
var tusker = ;var taskManager = tusker; var info = some_data: 10 ;var options = {}taskManager;
Available options
attempts
- no of attempts before before mark the task as failed
- new attempt can be occurred if task completed with an error or timeout occurred
timeout
- timeout for the task completion in millis
- if this value exceeded new attempt will be occurred
Receive notification after the all related chunks for a stream has encoded
var tusker = ;var taskManager = tusker; taskManager;
Invoking a timeout for locks
We cannot assure every lock we create will be unlocked. If the server died before unlock
, we have a lock which will never be closed. So we need a timeout machanism.
var tusker =
NOTE: We've to invoke this method via a
cron
like tool
Invoking a timeout for task processing
- We need to timeout long running tasks, and allow another attempt to process it
- default timeout is 30 seconds otherwise if it is not defined when task closing via
.close()
var tusker = ;var taskManager = tusker; //one minute timeouttaskManager;
NOTE: We've to invoke this method via a
cron
like tool
Statistics
var tusker = ;var taskManager = tusker; //one minute timeouttaskManager;
Stats object is formatted as shown below:
{ completed: 101, released: 1, locked: 2, failed: 2, processing: 2 }