Chop a single stream of data into a series of readable streams with rewind
What's it for
The use case is:
- You are transfering data from a stream to a service/process which needs it in chunks
- Each chunk can be streamed
- If a chunk fails to transfer, you need to be able to "rewind" to send it again
An example is upload to Google Drive with the "resumable" method for large files (the use case that this package was created for).
Usage
Installation
npm install stream-chop
Usage
const ChopStream = ; const inputStream = fs;const chopStream = ; inputStream; // Stream 1st 1024 bytes of data to fileconst subStream1 = chopStream;const outStream1 = fs;subStream1; outStream1;
Methods
`.chunk( start, length )
Get a stream for specified chunk of the input stream.
// Get stream for 1st 256 KiB of input streamconst subStream1 = chopStream;
The chunk is buffered internally, so .chunk()
can be called again with same start
if transferring the chunk fails and it needs to be sent again.
When .chunk()
is called again, any data buffered before the start point is discarded. i.e. You cannot stream 2 chunks concurrently. It must be one at a time.
Calling .chunk()
will cause any existing unfinished chunk stream to emit an error
event and stop streaming.
.clearBuffer( [end] )
Clear data from internal buffer before byte end
.
If end
is undefined, then entire buffer will be cleared.
// Discard any data in bufferchopStream; // Discard first 128 KiB of stream data if it's bufferedchopStream;
.chunk(1000)
automatically calls .clearBuffer(1000)
internally.
Once data has been discarded from the buffer, the stream can no longer be "rewound" to stream that data out again.
Properties
.bufferStart
Returns byte position of start of buffer.
.bufferEnd
Returns byte position of end of buffer.
.bufferLength
Returns number of bytes in buffer.
Tests
Use npm test
to run the tests. Use npm run cover
to check coverage.
Changelog
See changelog.md
Issues
If you discover a bug, please raise an issue on Github. https://github.com/overlookmotel/stream-chop/issues
Contribution
Pull requests are very welcome. Please:
- ensure all tests pass before submitting PR
- add tests for new features
- document new functionality/API additions in README
- do not add an entry to Changelog (Changelog is created when cutting releases)