Chunk/split your stream without eating the splitter char.
node v0.10.x required for the streams2 support.
Split your streams into chunks with an arbitrary-length matcher. Does not consume the
matcher bytes so you can further analyze them or use them as delimiter for parsing. Also
leftover buffer to make connection mode-switching easier to implement.
Chunker is designed to work with very small splitter token such as CRLF or '\0'.
This module exports a
Chunker class. This is simply an instance of a
Transform stream so you can pipe things in and it will only output (emit
readable) a chunk that has all the content from the start of the first chunk until the
end of the specified splitter token (called a
matcher inside the codebase)
For convenience, you can also listen for the chunker's
chunk event. Additionally, you
can inspect the
leftover property to get at the last chunk that has yet to be processed
once the stream ends.
var fs = require'fs' Chunker = require'./index.js';var chunker = matcher: 'what 'input = fscreateReadStream'./input.txt';chunkeron'readable'var chunk;while chunk = chunkerreadconsole.logchunktoString;;processstdinonce'end'console.logchunkerleftovertoString;processstdinpipechunker;
$ cat input.txt | node example.jsdo whatyou want to do not whatyou are told to do and do whatmakes you happy and keep away from whatmakes you sad but do whatyou have to do that is whatresposibility means but do not forget whatmakes your heart tick or you will forgot whatyou were born to be
BSD3 (see LICENSE file)
- Support for longer-length chunker or use a proper state machine string matching algorithm.
- More exhausive streaming tests.