brotli-compress
This package packages brotli-wasm
, minifies and inlines it, so that it works consistently
in Node.js and Browser environments. This solves several issues of using brotli-wasm
directly.
Furthermore, this package comes with two module variants: CommonJS and ESM.
On top of this, this library comes with a copy of the original Brotli decompress
implementation by Google, with an optimited API to be 1:1 compatible with
the WASM variant and with TypeScript typing support.
All in all, this package is the Brotli "fire and forget" solution that should work in all JavaScript environments, with all bundlers and ecosystems, including Vite, Rollup, Webpack, Gatsby and Next.js projects as well as Node.js.
This library can yield stellar compression rates at byte-level. e.g. 322% in our unit test case:
If you're looking for even crazier compression rates at "character level", take a look at
my novel algorithm, brotli-unicode
, where the raw Brotli output binary is encoded using
a subset of the Unicode BMP-1 alphabet. It leads to character level compression rates of 500%+.
Setup
As a package for development (Node.js, Browsers):
yarn add brotli-compress
# or
npm i brotli-compress
Usage of the WASM variant
The usage in a Node.js or Browser environment is trivial:
// import size (uncompressed, but minified) / WASM version / max performance: 1.8M
import { compress, decompress } from 'brotli-compress'
const oneBlockInput = 'Hello🤖!'
// it takes a Uint8Array and returns a Uint8Array
const compressed = await compress(oneBlockInput)
// it takes a Uint8Array and returns a Uint8Array
const decompressed = await decompress(compressed)
Please note that the WASM version comes with a whopping size of (minified) 1.8MiB. This is, because the binary is base64 encoded and inlined.
If you prefer maximum performance and memory efficiency over small bundle size, choose the WASM variant. Also, if you need compression, use the WASM version.
Usage of the pure JS variant
If you need a small bundle size, can effort the slowdown and only need decompression, use the hard-written JavaScript decompressor:
// import size (uncompressed, but minified) / JS version / only decompress / slower: 152K
import { decompress } from 'brotli-compress/js'
// please also note that the pure JS variant is synchronous
// for large inputs, you could optimize the execution by moving
// this call into a Worker
// it takes a Uint8Array and returns a Uint8Array
const decompressed = decompress(compressed)
Encoding to Uint8Array and decoding from Uint8Array
For Node.js, you'd be well advised to use the built-in Buffer
package:
import { Buffer } from 'buffer'
const testInput = 'Hello🤖!'
const testInputUint8 = Buffer.from(testInput)
const compressed = await compress(testInputUint8)
const decompressed = await decompress(compressed)
const decompressedString = Buffer.from(decompressed).toString()
For use in browser/frontend, you can either use TextEncoder
and TextDecoder
or use a polyfill like the buffer library.
new TextEncoder().encode(testInput) // returns a Uint8Array
new TextDecoder().decode(compressed) // returns a string from the compressed Uint8Array
Options
The compress
method comes with a second options
parameter.
Quality level
The most common setting is quality
with a scale from 0 to 11.
By default, the quality is set to best quality (11).
const compressed = await compress(Buffer.from('foobar'), { quality: 9 })
A lower quality value makes the output bigger but improves compression time. In 99.9% of the cases, you don't want to change this value.
Custom dictionary
The relevant options here is customDictionary
. You can set this to an Uint8Array string
of tokens which should be part of the a priori
known dictionary. This can be useful
if you have power over both, the sender and the receiver part and if you know exactly
which tokens will be used alot in the input. For example, if you know that you'll
be compressing text, encoded as UTF16/UCS-2 and you know that the content is TypeScript code,
you could include the keywords of the TypeScript language in the custom dictionary.
Please mind, that you need to set the same value for decoding as well.
// with this configuration, "let" must not be encoded in the dictionary and carried as part of the
// payload. The binary tree (huffman coding tree)
const customDictionary = Buffer.from('let')
const compressed = await compress('let foo = 123; let bar = "foo";', { customDictionary })
const decompressed = await decompress(compressed, { customDictionary })
Limitations
There is no streaming compression/decompression yet. It can be simply done by exposing the API from the WASM implementation. If you need that, pls. ping via Issue.
Build
yarn build
Test
yarn test
Licensing
Most of the code of this library is licensed by Apache-2.0.
The original decompress
implementation is licensed by Google under MIT license (src/js.ts).
Contibutors
Package and build configuration plus cross-library implementation, documentation and unit testing, as well as updating the WASM/JS binding has been done by Aron Homberg.
brotli-wasm
that ships with this package inline as been implemented by Tim Perry and contributors.
Direct binding WASM/JS and the respective code extraction idea has been implemented by stefnotch