binary-stream-multiplexer

1.0.7 • Public • Published

binary-stream-multiplexer

binary-stream-multiplexer allows you to record multiple binary streams simultaneously, and to play them back later. It multiplexes up to 254 streams to a single, compact binary stream (or file). Later, you can replay a recorded multiplexed file, which will be de-multiplexed back into N streams.

Because of Node.js's excellent streaming interface, virtually all streams in Node look exactly the same (HTTP streams, file streams, serial port streams, ... etc, all have the same interface). If you have an application where you are relying on live streaming data, it can be useful to have the ability to play back that data in real-time (that is, with simulated delays that mimic the original real-world delays of the data). The more streams you have, the more useful this is.

A trivial but real-world example: let's say you have three servers that expose a streaming binary interface. The binary interfaces emit events in a binary protocol at about 15 times per second, or about every 66 milliseconds. You have a dashboard that consumes these streams in real times and shows summary statistics. About every 10 minutes, the dashboard crashes - what to do? Now you can collect a single log of all the streaming input to your application, and replay it as many times as you need to isolate and fix the bug - and you also have a test fixture to run through tests repeatedly!

See test/index.js for examples of how to hook up basic streams and demultiplex them.

Install

Install as a dependency to a project:

npm install --save binary-stream-multiplexer

Install CLI utilities:

npm install --global binary-stream-multiplexer

CLI Usage

You cannot record streams using the CLI; you must use the API to record streams. With a recorded file, you can:

See all options:

$ binary-stream-multiplexer --helper

Display the header of a multiplexed file:

$ binary-stream-multiplexer header /path/to/multiplexed/file

Extract one of the encoded streams by ID:

$ binary-stream-multiplexer extract /path/to/multiplexed/file 3 > binary_for_stream_3.out

API Usage

The following functions are all available when you require('binary-stream-multiplexer'):

recordStreams(streams, filename)

Pass an array of binary streams and a filename. It will start multiplexing the streams' output to the filename. This function returns a stream that you can .end() to end multiplexing.

multiplexStreams(streams)

Similar to recordStreams but doesn't log out to a file - it only returns the output stream.

demultiplexFile(filename, useRealOffsets, callback)

Demultiplexes the binary data at filename.

useRealOffsets: a boolean value indicating whether or not the original timing of the data should be replicated. If so, timeouts will be introduced to the output streams to mimic the timing of the original streams.

callback(err, streams): called with an error or with a list of output streams. The streams are in the same order that they were when you recorded the data with recordStreams.

demultiplexBuffer(buf, useRealOffsets, callback)

Pass in a Buffer of multiplexed data. All other parameters are the same as demultiplexFilename.

demultiplexStream(stream, useRealOffsets, callback)

Pass in a Stream of multiplexed data. All other parameters are the same as demultiplexFilename. This is the primitive function used internally by demultiplexFilename (which uses fs.createReadStream()) and demultiplexBuffer (which creates a stream and writes the entire buffer to it immediately). It can also be used externally.

Binary protocol (V1)

TODO

Caveats

The binary protocol (v1) limits you to a maximum of 254 streams.

Binary output size will be artificially inflated if you have many large gaps in data that are more than 60 seconds long.

License

MIT license. See LICENSE file.

Package Sidebar

Install

npm i binary-stream-multiplexer

Weekly Downloads

0

Version

1.0.7

License

MIT

Last publish

Collaborators

  • sokhealy
  • pcswiftnav
  • snav
  • ljbade
  • ebethon
  • snav-jason