Simple Storm-like distributed application, Work in Progress
Simple Storm-like distributed application, implementation, for Node.js. See
Via npm on Node:
npm install simplestorm
Reference in your program:
var simplestorm = require'simplestorm';
You have Spouts (message sources) and Bolts (message processor). An Spout should have a
// ...contextemitmsg; // you can emit a message many times// ....
An spout emit message via its controller, in any of its methods.
A Bolt has a
// Message process// and emit new message(s)contextemitnewmsg;
There is a topology builder:
// Objectsvar spout = ;var downloader = ;var resolver = ;var harvester = ;// Setting Buildervar builder = sscreateTopologyBuilder;buildersetSpout"spout" spout;buildersetBolt"downloader" downloadershuffleGrouping"resolver"shuffleGrouping"spout";buildersetBolt"resolver" resolvershuffleGrouping"harvester";buildersetBolt"harvester" harvestershuffleGrouping"downloader";var topology = buildercreateTopology;
Start a topology:
Stop a topology (WIP):
A topology worker can listen external messages, sent from other topology workers:
If they are many topology workers, they can share its address using a central server. In a process, create a topology server:
var tserver = sscreateTopologyServer;tserverlistenport;
In each topology worker, connect to the topology server:
When a topology worker connects to the central server, it sends its address. The server shares that address with the rest of the topology workers. See Distributed Web Crawler with Server/Workers for a running example.
git clone git://github.com/ajlopez/SimpleStorm.gitcd SimpleStormnpm installnpm test
If you submit a pull request, please be sure to add or update corresponding
test cases, and ensure that
npm test continues to pass.