varstream

Stream variables beetween 2 JavaScript threads (client/server, ipc, worker/main thread).

VarStream

VarStream is a data storage and exchange format.

VarStream :

  • is human readable/writeable : no need to be a programmer to create VarStreams.
  • is streamable : No need to wait the datas to be fully loaded to populate/access your program variables.
  • keeps backward references: you can refer to another variable of the stream in the stream itself.
  • merges with no loss: you can easily merge multiple varstreams.
  • is light: due to it's smart optimizations and syntax sugar.
  • is memory efficient: the garbage collector can cleanup memory before the parse ends, backward references prevent data duplication.
  • accept comments: keep your configuration/localization files readable.
  • loves circular references: transmit your variable trees with no hack.

VarStream allows you to configure your projects in a clear and readable way. Since VarStream is merge friendly, it is particularly usefull for loading multilevel configuration files without erasing previously set contents.

Imagine this sample configuration file:

# Server
server.domain=example.com
server.protocols.+=http
server.protocols.+=https
server.databases.+.host=db1.example.com
server.databases.*.username=db1
server.databases.+.host=db2.example.com
server.databases.*.username=db2
server.cache.size=2048
# HTML document
document.scripts.+.uri=//ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js
document.scripts.+.uri=//ajax.googleapis.com/ajax/libs/jqueryui/1.10.3/jquery-ui.min.js

You could easily override some of its contents by loading this specific configuration file next to him:

# Append my custom dev TLD
server.domain+=.local
# Support 1 more protocol
server.protocols.+=ws
# Reset DB and set my local one
server.databases.!.host=localhost
server.databases.*.username=db1
# Increase cache size (8 times)
server.cache.size*=8
# Use local scripts
document.scripts.0.uri=javascript/jquery.js
document.scripts.1.uri=javascript/jquery-ui.js

The same goes for internationalization files. You could load a language file and augment it with a locale file.

VarStreams particularly suits with the JavaScript messaging systems. Communicate through different JavaScript threads (or over the Network) has never been so simple.

This is particularly usefull for data driven applications.

Compared to JSON, VarStreams brings nice formatting with often less weight.

  • test1 : linear.dat [390 bytes] vs linear.json [423 bytes] => 8% smaller
  • test2 : arrays.dat [1244 bytes] vs arrays.json [1178 bytes] => 6% bigger
  • test3 : references.dat [2844 bytes] vs references.json [3314 bytes] => 16% smaller

## How to use With NodeJs :

// Synchronous API 
var cnt = fs.ReadFileSync('test2.dat', {encoding: 'utf-8'});
// Parse VarStream content 
var obj = VarStream.parse(cnt);
// Get an Object content as a VarStream 
cnt = VarStream.stringify(obj);
 
// Streaming 
var VarStream = require('varstream');
var fs = require('fs');
 
var scope = {}; // The root scope 
var myVarStream=new VarStream(scope, 'prop');
// Reading var stream from a file 
fs.createReadStream('test.dat').pipe(myVarStream)
  .on('end', function () {
  // Piping VarStream to a file 
    myVarStream.pipe(fs.createWriteStream('test2.dat'));
    });

In the browser, you can use browserify or directly VarStreamReader and VarStreamWriter constructors.

## CLI Usage VarStream comes with two CLI utilities, to use them, install VarStream globally:

npm install -g varstream
# Convert JSON datas to VarStream 
json2varstream path/to/input.json > path/to/ouput.dat
# Convert VarStreams datas to JSON 
varstream2json path/to/input.dat > path/to/ouput.json

## Contributing/Testing The VarStream JavaScript library is fully tested. If you want to contribute, test your code before submitting, just run the following command with NodeJS dependencies installed :

npm test

## Contributors

  • Nicolas Froidure - @nfroidure

## License This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/