is a minimal Dataflow programming engine


Is a minimal Dataflow programming engine.

Try dflow now! (on Heroku).

Table Of Contents:

dflow is beta software. The editor is usable but with no css (help me!) and collaborative editing feature is working but buggy (I am proud of my bugs and they will be fixed in version 1.0 :). The engine is almost stable.

Next steps:

  • start using dflow,
  • improve editor look&feel,
  • add more examples using the editor
  • and go for a stable version!

If you have some graphs ready to run on client side, you can install dflow engine with bower

$ bower install dflow

If you are new to dflow, you probably want to try the editor, so you need to install globally to get dflow cli in your path. With npm do

$ npm install dflow -g

However, if you need to require the dflow engine in your package, or you need to browserify it, or even you want to use the dflow cli in your npm scripts, or whatever, you can install dflow locally with

$ npm install dflow

If you want start hacking on dflow run

$ git clone https://github.com/fibo/dflow.git
$ cd dflow
$ npm install
$ npm start

which will clone repo, install deps, and start the dflow cli. Then point your browser to http://localhost:3000 and edit your graph. Note that by opening another browser window you can try the collaborative editing feature, powered by Socket.IO!

Note that npm start will edit a graph in memory, if you want to edit your graph and save it to a file, for instance graph.json, launch

$ npm start -- graph.json

The following simple graph is executed client side by dflow engine.

An example of dflow in a browser context, testing DOM manipulation.

Node.js Stream Playground first example is

var fs = require("fs");
// Read File 
    // Write File 

It is ported to script stream.js which evaluates graph stream.json using few custom functions.

The main advantage of dflow design is that you do not need to write components or plugins to extend it. You can use one of the most powerful JavaScript features: functions. Write your functions or import them from other packages like JQuery or underscore and you are able to use them as tasks and connect them with pipes.

Also every dflow graph is a function itself, so why not packaging it and put it on npm!?

It is really easy: create your dflow graph and save it to a JSON file, index.json for instance; then launch npm init as usual and when prompted for the entry point write index.json.

Simple as that, see packagedGraph as an example.

dflow is MIT licensed.

It is developed in my spare time and, as far as I know, by now I am my only user. By the way, I got a stimulating feedback by Stamplay's founder, who is pretty much interested in using dflow maybe in the future. Grazie Giuliano, I hope you all the best.

I wrote few times a Dataflow engine, the first one was PNI (Perl Node Interface) and the design evolved until I could say confidently that dflow is here to stay.

Use cases I can think about dflow right now are many, but, the possibilities are I.M.H.O. outstanding: from client to server, from JavaScript to cross language, from mono-thread to graphs distributed on a network and, above all, from skilled programmer who implement functions … to artists, genetic engineers, data scientists, etc. that use those functions to create dflow graphs and get results nobody could even imagine.

If this is also your vision or you just want to use dflow, contact me.

My goal is to say to a dflow user:

Mamma mia! Did you achieve that with dflow?