task-stream
This is 0.x version. API will change.
npm install task-stream
and process tasks with Node's Stream API and get backpressure and other neat features for free.
Basic Usage
const taskStream = ;// Provide a function that fetches tasks one by one// callback(error, task)const getTask = dbtasks;// Provide a processing function// callback(error)const processTask = {;};// Combine everything together// and start processing tasks.const worker = ;workerstart;// You can also stop selecting new tasks.// Note that this won't stop any ongoing processing,// but make sure that no more tasks are queued.worker;
Notes
getTask
must callcallback(null, null)
when no more task are to be scheduled.
Options
taskStream
accepts optional options
object
that may include one or more of the following props:
readAhead
positive integer specifies how many tasks will be buffered (defaults to0
, sogetTask()
will only get called after all currently queued tasks are processed);parallelTasks
positive integer specifies how many tasks can be processed in parallel (defaults to1
);onError
function will get called on any error;onReadError
function will get called ifgetTask
fails as;onWriteError
function will get called ifprocessTaks
fails;onFinish
function will get called when all the tasks are processed successfuly.
Data Flow
To get a better idea on the data flow, check out test.js
and play around with reader
and writer
options.