Nutritious Pumpkin Mash

    it-length-prefixed
    TypeScript icon, indicating that this package has built-in type declarations

    8.0.2 • Public • Published

    it-length-prefixed

    codecov CI

    Streaming length prefixed buffers with async iterables

    Table of contents

    Install

    $ npm i it-length-prefixed

    Usage

    import { pipe } from 'it-pipe'
    import * as lp from 'it-length-prefixed'
    
    const encoded = []
    
    // encode
    await pipe(
      [uint8ArrayFromString('hello world')],
      lp.encode(),
      async source => {
        for await (const chunk of source) {
          encoded.push(chunk.slice()) // (.slice converts Uint8ArrayList to Uint8Array)
        }
      }
    )
    
    console.log(encoded)
    // => [Buffer <0b 68 65 6c 6c 6f 20 77 6f 72 6c 64>]
    
    const decoded = []
    
    // decode
    await pipe(
      encoded, // e.g. from above
      lp.decode(),
      async source => {
        for await (const chunk of source) {
          decoded.push(chunk.slice()) // (.slice converts Uint8ArrayList to Uint8Array)
        }
      }
    )
    
    console.log(decoded)
    // => [Buffer <68 65 6c 6c 6f 20 77 6f 72 6c 64>]

    API

    import {
      encode, decode
    } from 'it-length-prefixed'
    
    import {
      encode
    } from 'it-length-prefixed/encode'
    
    import {
      decode,
      MAX_LENGTH_LENGTH,
      MAX_DATA_LENGTH
    } from 'it-length-prefixed/decode'

    encode([opts])

    • opts: Object, optional
      • lengthEncoder: Function: A function that encodes the length that will prefix each message. By default this is a varint encoder. It is passed a value to encode, an (optional) target buffer to write to and an (optional) offset to start writing from. The function should encode the value into the target (or alloc a new Buffer if not specified), set the lengthEncoder.bytes value (the number of bytes written) and return the target.

    Returns a transform that yields Uint8ArrayList objects. All messages will be prefixed with a length, determined by the lengthEncoder function.

    encode.single(chunk, [opts])

    • chunk: Buffer|Uint8ArrayList chunk to encode
    • opts: Object, optional
      • lengthEncoder: Function: See description above. Note that this encoder will not be passed a target or offset and so will need to allocate a buffer to write to.

    Returns a Uint8ArrayList containing the encoded chunk.

    decode([opts])

    • opts: Object, optional
      • maxLengthLength: If provided, will not decode messages whose length section exceeds the size specified, if omitted will use the default of 147 bytes.
      • maxDataLength: If provided, will not decode messages whose data section exceeds the size specified, if omitted will use the default of 4MB.
      • onLength(len: Number): Called for every length prefix that is decoded from the stream
      • onData(data: Uint8ArrayList): Called for every chunk of data that is decoded from the stream
      • lengthDecoder: Function: A function that decodes the length that prefixes each message. By default this is a varint decoder. It is passed some data to decode which is a Uint8ArrayList. The function should decode the length, set the lengthDecoder.bytes value (the number of bytes read) and return the length. If the length cannot be decoded, the function should throw a RangeError.

    Returns a transform that yields Uint8ArrayList objects.

    decode.fromReader(reader, [opts])

    Behaves like decode except it only reads the exact number of bytes needed for each message in reader.

    • reader: Reader: An it-reader
    • opts: Object, optional
      • maxLengthLength: If provided, will not decode messages whose length section exceeds the size specified, if omitted will use the default of 147 bytes.
      • maxDataLength: If provided, will not decode messages whose data section exceeds the size specified, if omitted will use the default of 4MB.
      • onData(data: Uint8ArrayList): Called for every chunk of data that is decoded from the stream
      • lengthEncoder: Function: See description above.

    Returns a transform that yields Uint8ArrayList objects.

    Contribute

    PRs and issues gladly accepted! Check out the issues.

    License

    Licensed under either of

    Contribution

    Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

    Install

    npm i it-length-prefixed

    DownloadsWeekly Downloads

    14,542

    Version

    8.0.2

    License

    Apache-2.0 OR MIT

    Unpacked Size

    32.6 kB

    Total Files

    23

    Last publish

    Collaborators

    • achingbrain
    • alanshaw