Nostradamus Prophecy Machine

    superbuffer
    TypeScript icon, indicating that this package has built-in type declarations

    1.1.0 • Public • Published

    Superbuffer

    A simple way to serialize JavaScript objects into ArrayBuffers for high compression on the wire.



    IntroductionUsageData TypesCaveats


    Introduction

    Superbuffer makes it easy to serialize and deserialize data from ArrayBuffers matching the interfaces you define. Its Schema-Model API was inspired by typed-array-buffer-schema.

    Superbuffer was designed to balance the readability and flexibility JSON while maximizing compression of your data on the wire. It is especially useful in situations where a high volume and frequency of network traffic between clients is necessary (such as multiplayer networked games).

    Installation

    NPM

    npm i superbuffer
    

    Yarn

    yarn add superbuffer
    

    Usage API

    Superbuffer allows you to define the shape of the data you want to serialize:

    import {Schema, Model, uint32, uint64, int16, string} from 'superbuffer';
    
    const commentSchema = new Schema({
      message: string,
      timestamp: uint64,
      votes: int16,
    });
    
    const postSchema = new Schema({
      id: uint32,
      title: string,
      tags: [string],
      author: {
        id: uint32,
        name: string,
      },
      comments: [commentSchema],
    });
    
    const post = new Model(postSchema);

    If you use Typescript, Superbuffer automatically ensures your data has proper typings:

    import {Model, float32} from 'superbuffer';
    const position = Model.fromSchemaDefinition({
      x: float32,
      y: float32,
    });
    
    // Typescript knows that only objects with type
    // {x: number, y: number} can be serialized by the `position` model
    const buffer = position.toBuffer({x: 32.1, y: 12.3});
    
    // Typescript knows `result` is of type {x: number, y: number}
    const result = position.fromBuffer(buffer);

    Superbuffer can serialize any non-circular JSON structure:

    import {Schema, Model, views, ExtractSchemaObject} from 'superbuffer';
    
    const {int16, int32, uint8, uint32, uint64, float32, boolean, string} = views;
    
    const playerSchema = new Schema({
      id: uint8,
      x: float32,
      y: float32,
      health: uint8,
      alive: boolean,
    });
    
    const inputSchema = new Schema({action: int16, movement: int16});
    
    const listSchema = new Schema({value: int32});
    
    const snapshotModel = Model.fromSchemaDefinition({
      time: uint64,
      sequenceNumber: uint32,
      input: inputSchema,
      messages: [string],
      data: {
        list: [listSchema],
        players: [playerSchema],
      },
    });
    
    type Snapshot = ExtractSchemaObject<typeof snapshotModel>;
    
    const snapshot: Snapshot = {
      time: BigInt(Date.now()),
      sequenceNumber: 438923,
      input: {
        action: -12,
        movement: -4,
      },
      messages: ['hello', 'hi', 'how are you', 'im good, how are you', 'fine thank you'],
      data: {
        list: [{value: 1}, {value: 2}, {value: 3}, {value: 4}, {value: 5}],
        players: [
          {id: 14, x: 145.32, y: 98.1123, health: 99, alive: true},
          {id: 15, x: 218.46, y: -14.0934, health: 100, alive: true},
          {id: 0, x: 3289.554, y: -1432.0, health: 0, alive: false},
        ],
      },
    };
    
    /** Client */
    // JSON.stringify(snapshot) = 430 bytes
    // model.toBuffer(snapshot) = 139 bytes
    // 68% compression!
    const buffer = snapshotModel.toBuffer(snapshot); // Object to ArrayBuffer
    network.emit(buffer);
    
    /** Server */
    network.on('message', (buffer) => {
      const id = Model.getIdFromBuffer(buffer);
      if (id === snapshotModel.schema.id) {
        const result = snapshotModel.fromBuffer(buffer); // ArrayBuffer to object
      }
    });

    Data Types

    Superbuffer's views are mapped to their respective TypedArray countparts:

    Superbuffer View TypedArray Equivalent Value Range Byte Size
    uint8 Uint8Array 0 to 255 1
    uint16 Uint16Array 0 to 65535 2
    uint32 Uint32Array 0 to 4294967295 4
    uint64 Uint64Array 0 to 2^64-1 8
    int8 Int8Array -128 to 127 1
    int16 Int16Array -32768 to 32767 2
    int32 Int32Array -2147483648 to 2147483647 4
    int64 Int64Array -2^63 to 2^63-1 8
    float32 Float32Array 1.2×10^-38 to 3.4×10^38 4
    float64 Float64Array 5.0×10^-324 to 1.8×10^308 8
    string Uint8Array UTF-8 encoding variable
    boolean Uint8Array 0 or 1 1

    Caveats

    • Max array length of 65535 (uint16)
    • Max number of schema ids is 255 (uint8)
    • No " character in strings
    • No null/undefined properties: all properties of the schema definition must be present on an object to be serialized (empty arrays are allowed)

    License

    MIT License

    Install

    npm i superbuffer

    DownloadsWeekly Downloads

    16

    Version

    1.1.0

    License

    MIT

    Unpacked Size

    8.29 kB

    Total Files

    4

    Last publish

    Collaborators

    • danielhzhang