A little event-based util to make it easier to throttle execution for a batch of jobs.
npm install throttler
var Throttler = ;var throttler ='maxActive' : 50'wait' : 2500;for var i = 0; i < 1000; i++throttler;throttler;
When creating a new Throttler instance you can specify two options:
'maxActive' : 50 // The maximum number of jobs to run at once. Must be specified'wait' : 2500 // The time, in milliseconds, to wait before trying to run more jobs if maxActive jobs are currently running
You then add jobs to the Throttler instance. Each job is simply a function that takes a single parameter: a callback to be invoked when their work is complete. As each job finishes doing its work, it should invoke this callback with any relevant arguments to notify the Throttler instance. As the Throttler works, it will fire off a variety of events that allow you to keep track of its progress, any arguments passed to the callback will be available via these events.
|job-started||Emitted whenever a new job is started|
|job-completed||Emitted whenever a job completes|
|complete||Emitted when all jobs are complete|
Each event will be emitted with a stats object that gives you the current status of all jobs. For the two job-specific events (job-started and job-completed) the stats object will also have a reference to the job, including any arguments passed to the callback when the job completed.
"active" : 2 // The number of actively running jobs"pending" : 16 // The number of pending jobs"complete" : 4 // The number of completed jobs"averageRunTime" : 17565 // The current average for how long each job takes to complete, in milliseconds"job" :"identifier" : "Job #6" // A system-generated job identifer"startTime" : 1363294870424 // The time the job was started
For the job-completed event, the stats object has all of the above properties, but the jobs property will contain additional properties
"active" : 4"pending" : 15"complete" : 5"averageRunTime" : 2002"job" :"identifier" : "Job #4""startTime" : 1363294867922"completedResults" : "some""results" // The args passed to the callback when the job completed"elapsed" : 2984 // The elapsed time to run the job, in milliseconds"complete" : true // Did the job complete, will always be true here
For the final completed event, you'll simply get the non-job specific stats:
"active" : 0"pending" : 0"complete" : 20"averageRunTime" : 16789