parse-stream
Parse streams of binary data of arbitrary lengths, handling broken/incomplete chunks.
This is useful when procesisng binary data that can be chunked in any way. For example, imagine we're handling some arbitrary IPC format we'll call "jsonIPC" through a net.Socket
.
"jsonIPC" is a fake, simple data format that encodes the length of the JSON string as a 32-bit little-endian uint before the JSON string. By default, net.Socket
may emit 8192 byte chunks. These chunks may contain multiple messages, may be smaller than 8192 bytes, or contain part of a larger message. To illustrate, they may look like this, with |
indicating a break in chunks:
[len32, ...message], [len32, ...message], [len32, ...mes | sage], [len32, ...message]
By defining how to get the length of each message from a stream of binary data, ParseStream
takes care of splitting chunks properly, dealing with:
- Chunks that contain multiple messages
- Chunks that contain partial messages (e.g. 8192 byte chunks, 1MB message)
- Chunks that don't contain enough data to even parse the length
- Return
Infinity
fromgetDataGramLength()
and a larger chunk will be passed back on the next invocation.
- Return
Usage
Notice! Version 2.0 no longer has the
parseDataGram
function, and does not emit'chunkLen'
anymore. Simply pipe yourParseStream
into another transform stream to replicate the old behavior.
const ParseStream = ;const Transform = ; // Get a socket from somewhereconst sock = 'stream'; // Pipe through a ParseStream.sock; const testData = JSON;const testBuf = Buffer;testBuf;testBuf; sock;// Logs: "{foo: 'bar', biff: [1,2,3]}, 'object'"