A light-weight distributed queue based on redis and RxJS.
__ _ _ __ __ __/ _` || '__|\ \/ /| (_| || | > <\__, ||_| /_/\_\| ||_|
A distributed work queue for node.js based on redis and RxJS.
'qrx' provides reactive work queue implemented with redis and RxJS. The goals of the project:
- Enable shared asynchronous compute resources on top a native node.js technology stack.
- Focus on minimalism
- Ease of installation (redis and on node are easy)
- Practical performance: Approach near raw redis performance levels with queuing primitives
- Leverage Reactive combinator abstraction for modeling asynchronous distributed computing
npm install qrx
qrx requires a redis installation see: http://redis.io/download
[sudo] npm install qrx
- Create a queue with name
- Enable async compute resource
###Simple usage with 1 queue instance
// create a new queue with well known namewq = 'test-wq';// clear any pending work (optional)wqclear;wqenqueue'one';wqenqueue'two';// subscribe for workwqworkObservableSubscribeconsole.log'new work' workObjwork;// callback to mark the work completed or err'dworkObjcallbacknull workObjwork + ' - completed';;// subscribe for completed workwqcompletedObservableSubscribeconsole.log'completed work' completedWork
var wqMaster = 'clean-test2';var WORK_COUNT = 500;console.log'Test WorkCount' WORK_COUNT;forvar i=0; i < WORK_COUNT; i++wqMasterenqueuei;// two slaves serving 1 mastervar workReceived = 0;var slave1 = 'clean-test2';slave1workObservableSubscribeworkReceived++;workObjcallbacknull workObjwork + 3;;var slave2 = 'clean-test2';slave2workObservableSubscribeworkReceived++;workObjcallbacknull workObjwork + 3;;// master get's his workvar completedWorkCount = 0;wqMastercompletedObservableSubscribecompletedWorkCount++;console.log'Completed Work' workItemcompletedWork;
RxObservableFromArray123// ForkMany usageForkMany'test-q'Subscribeconsole.logresult;;var worker = 'test-q';workerworkObservableSubscribeworkItemcallbacknull workItemwork + 1;;
- Work stop singals
- Enable transactional queueing using redis primatives
- Performance optimization
- Flood control on queue restarts
Copyright (c) Loku. All rights reserved. The use and distribution terms for this software are covered by the Eclipse Public License 1.0 (http://opensource.org/licenses/eclipse-1.0.php) which can be found in the file epl-v10.html at the root of this distribution. By using this software in any fashion, you are agreeing to be bound by the terms of this license. You must not remove this notice, or any other, from this software.