Get unlimited public & private packages + package-based permissions with npm Pro.Get started »

it-length-prefixed

3.0.0 • Public • Published

it-length-prefixed

Travis (.org) Codecov Dependency Status js-standard-style

Streaming length prefixed buffers with async iterators

Install

npm install it-length-prefixed

Usage

const pipe = require('it-pipe')
const lp = require('it-length-prefixed')
 
const encoded = []
 
// encode
await pipe(
  [Buffer.from('hello world')],
  lp.encode(),
  async source => {
    for await (const chunk of source) {
      encoded.push(chunk.slice()) // (.slice converts BufferList to Buffer)
    }
  }
)
 
console.log(encoded)
// => [Buffer <0b 68 65 6c 6c 6f 20 77 6f 72 6c 64>]
 
const decoded = []
 
// decode
await pipe(
  encoded, // e.g. from above
  lp.decode(),
  async source => {
    for await (const chunk of source) {
      decoded.push(chunk.slice()) // (.slice converts BufferList to Buffer)
    }
  }
)
 
console.log(decoded)
// => [Buffer <68 65 6c 6c 6f 20 77 6f 72 6c 64>]

API

encode([opts])

  • opts: Object, optional
    • poolSize: 10 * 1024: Buffer pool size to allocate up front
    • minPoolSize: 8: The minimum size the pool can be before it is re-allocated. Note: it is important this value is greater than the maximum value that can be encoded by the lengthEncoder (see the next option). Since encoded lengths are written into a buffer pool, there needs to be enough space to hold the encoded value.
    • lengthEncoder: Function: A function that encodes the length that will prefix each message. By default this is a varint encoder. It is passed a value to encode, an (optional) target buffer to write to and an (optional) offset to start writing from. The function should encode the value into the target (or alloc a new Buffer if not specified), set the lengthEncoder.bytes value (the number of bytes written) and return the target.
      • The following additional length encoders are available:
        • int32BE - const { int32BEEncode } = require('it-length-prefixed')

Returns a transform that yields BufferList objects. All messages will be prefixed with a length, determined by the lengthEncoder function.

encode.single(chunk, [opts])

  • chunk: Buffer|BufferList chunk to encode
  • opts: Object, optional
    • lengthEncoder: Function: See description above. Note that this encoder will not be passed a target or offset and so will need to allocate a buffer to write to.

Returns a BufferList containing the encoded chunk.

decode([opts])

  • opts: Object, optional
    • maxLengthLength: If provided, will not decode messages whose length section exceeds the size specified, if omitted will use the default of 147 bytes.
    • maxDataLength: If provided, will not decode messages whose data section exceeds the size specified, if omitted will use the default of 4MB.
    • onLength(len: Number): Called for every length prefix that is decoded from the stream
    • onData(data: BufferList): Called for every chunk of data that is decoded from the stream
    • lengthDecoder: Function: A function that decodes the length that prefixes each message. By default this is a varint decoder. It is passed some data to decode which is a BufferList. The function should decode the length, set the lengthDecoder.bytes value (the number of bytes read) and return the length. If the length cannot be decoded, the function should throw a RangeError.
      • The following additional length decoders are available:
        • int32BE - const { int32BEDecode } = require('it-length-prefixed')

Returns a transform that yields BufferList objects.

decode.fromReader(reader, [opts])

Behaves like decode except it only reads the exact number of bytes needed for each message in reader.

  • reader: Reader: An it-reader
  • opts: Object, optional
    • maxLengthLength: If provided, will not decode messages whose length section exceeds the size specified, if omitted will use the default of 147 bytes.
    • maxDataLength: If provided, will not decode messages whose data section exceeds the size specified, if omitted will use the default of 4MB.
    • onData(data: BufferList): Called for every chunk of data that is decoded from the stream
    • lengthEncoder: Function: See description above.

Returns a transform that yields BufferList objects.

Contribute

PRs and issues gladly accepted! Check out the issues.

License

MIT © 2016 Friedel Ziegelmayer

Install

npm i it-length-prefixed

DownloadsWeekly Downloads

2,473

Version

3.0.0

License

MIT

Unpacked Size

1.93 MB

Total Files

42

Last publish

Collaborators

  • avatar