@purplet/serialize
Formerly named
@davecode/serialize
Utilities for binary serialization, used by Purplet, with the goal to cram as much data into Discord's custom_id
as possible. This is done by using Base1114111 (a 2.5byte/char encoding), then a custom set of serializations that work on the bit-level instead of the byte-level, meaning two booleans
will not occupy two separate bytes, and other data that is typically one byte may be spread across multiple.
This package provides functions to use Base1114111 to encode custom ids from Uint8Array
s and a handful of serializers for compressing JSON-like data into those Uint8Array
s. You can either build specialized serializers to go off the shape of your data, or use the generic
serializer as a drop-in replacement for JSON.stringify
.
The naming of this package is due to it originally being a standalone project, but I believe the source code belongs within the Purplet monorepo, so we can maintain it with its primary use case.
custom_id
s
Encoding and Decoding import { encodeCustomId, decodeCustomId } from '@purplet/serialize';
const id = encodeCustomId(new Uint8Array([1, 2, 3, 4, 5]));
const decoded = decodeCustomId(id);
JSON.stringify
via the generic serializer
Alternate to import { serializers as S } from '@purplet/serialize';
const encoded = S.generic.encodeCustomId({ foo: 'bar', number: 32n }); // works with Dates, BigInts, and other types
const decoded = S.generic.decodeCustomId(encoded);
List of serializers built-in to the library
-
u8
: unsigned 8-bit integer -
u16
: unsigned 16-bit integer -
u32
: unsigned 32-bit integer -
u8bi
: unsigned 8-bit integer as abigint
-
u16bi
: unsigned 16-bit integer as abigint
-
u32bi
: unsigned 32-bit integer as abigint
-
u64bi
: unsigned 64-bit integer as abigint
-
u128bi
: unsigned 128-bit integer as abigint
-
s8
: signed 8-bit integer -
s16
: signed 16-bit integer -
s32
: signed 32-bit integer -
s8bi
: signed 8-bit integer as abigint
-
s16bi
: signed 16-bit integer as abigint
-
s32bi
: signed 32-bit integer as abigint
-
s64bi
: signed 64-bit integer as abigint
-
s128bi
: signed 128-bit integer as abigint
-
boolean
:true
orfalse
-
date
: aDate
object -
float
: anumber
stored in IEEE 754 format -
generic
: any json serializable type +Date``BigInt
-
genericArray
: array ofGeneric
-
genericObject
: object withGeneric
values -
number
: any valid javascriptnumber
-
snowflake
: a discord snowflake id as a string -
string
: a string (must be 255 bytes or less)
Functions that return serializers
-
or(A, B)
:A | B
, uses a prefix bit to indicate which serializer to use. -
arrayOf(T)
: array ofT
-
nullable(T)
:T | null
-
constant(value)
: does not emit or read anything, just returns value. useful withor
-
unsignedInt(bytes)
: unsigned int ofbytes
length. due to js limits, bytes must be <31. -
signedInt(bytes)
: signed int ofbytes
length. due to js limits, bytes must be <31. -
unsignedBigInt(bytes)
: unsigned bigint ofbytes
length -
signedBigInt(bytes)
: signed bigint ofbytes
length
Custom serializers
The read
and write
functions passed to BitSerializer
s operate on BitBuffer
s, which keep track of where they are. Reading or writing advances the buffer's position. By default, 256 bytes are allocated, but the caller can increase or decrease it.
export const boolean = new BitSerializer({
// Given a buffer, read from it and return a parsed value.
read(buffer) {
return buffer.read() === 1;
},
// Given a value and a buffer, write the serialized value.
write(value, buffer) {
buffer.write(value ? 1 : 0);
},
// Used by `or` and potentially other serializers to determine if the data matches this.
// If ommitted, will ALWAYS return true
check(value): value is boolean {
return typeof value === 'boolean';
},
});
BitBuffer
API
Constructors:
-
new(bytes: number)
: creates a new buffer withbytes
allocated. -
new(buf: Uint8Array | ArrayBufferLike | ArrayLike<number>)
: creates a new buffer with the contents ofbuf
.
Properties
-
buffer: ArrayBuffer
(read/write): the underlying buffer. -
index: number
(read/write): the current index in the buffer (bits).
Methods
-
seek(index: number)
: sets the buffer's index toindex
(bits) -
read(length: number)
: readslength
bits from the buffer as a signed value. -
write(value: number, length: number)
: writesvalue
aslength
bits to the buffer. -
readBI(length: number)
: readslength
bits from the buffer as abigint
. -
writeBI(value: bigint, length: number)
: writesvalue
aslength
bits to the buffer.
BitSerializer<T>
API
Constructors:
-
new(options: BitSerializerOptions)
: creates a new serializer with the given options. see above.
Methods:
-
read(buffer: BitBuffer)
: reads from the buffer and returns the parsed value. -
write(value: T, buffer: BitBuffer)
: writes the serialized value to the buffer. -
check(value: T)
: returnstrue
if the value can be serialized by this serializer. -
encode(value: T)
: encodes the value into aUint8Array
without needing to useBitBuffer
. -
decode(buf: Uint8Array)
: decodes the value from aUint8Array
without needing to useBitBuffer
. -
encodeCustomId(value: T)
: encodes the value into astring
that can be used as a custom id. Shorthand forencodeCustomId(serializer.encode(...))
-
decodeCustomId(id: string)
: decodes the value from astring
that was created withencodeCustomId
. Shorthand fordecodeCustomId(serializer.decode(...))