fastintcompression

    0.0.4 • Public • Published

    FastIntegerCompression

    Build Status

    This is an integer compression library in JavaScript, useful for work on indexes. Given an array of small integers, it produces an ArrayBuffer that uses far fewer bytes than the original (using VByte compression). It assumes a modern JavaScript engine with typed arrays.

    From the compressed data, you can later recover the original array quickly (at a rate of millions of integers per second).

       // var FastIntegerCompression = require("fastintcompression");// if you use node
       var array = [10,100000,65999,10,10,0,1,1,2000,0xFFFFFFFF];
       var buf = FastIntegerCompression.compress(array);
       var back = FastIntegerCompression.uncompress(buf); // gets back [10,100000,65999,10,10,0,1,1,2000]

    By default, non-negative 32-bit integers are expected. If you have signed (negative and positive) 32-bit integers, then you must use distinct functions since we need to code the sign bit:

       // var FastIntegerCompression = require("fastintcompression");// if you use node
       var array = [10,100000,65999,10,10,0,-1,-1,-2000];
       var buf = FastIntegerCompression.compressSigned(array);
       var back = FastIntegerCompression.uncompressSigned(buf); // gets back [10,100000,65999,10,10,0,-1,-1,-2000]

    You can install the library under node with the command line

       npm install fastintcompression

    This code is made available under the Apache License 2.0.

    Suitability

    This library is meant to compress arrays of small integers. It is not meant to compress text documents or arrays of large (or random) integers.

    Performance numbers

    Go to benchmark repository (check the README.md file) and run the benchmark:

    $ node test.js
    Platform: linux 3.13.0-91-generic x64
    Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
    Node version 4.5.0, v8 version 4.5.103.37
     
    input size: 7.813K compressed size: 1000B
    FastIntegerCompression.compress x 337,845 ops/sec ±0.93% (92 runs sampled)
    Fastest is FastIntegerCompression.compress
    FastIntegerCompression.uncompress x 187,694 ops/sec ±0.72% (93 runs sampled)
    Fastest is FastIntegerCompression.uncompress

    These numbers means that we can uncompress 187,694 1000-integer arrays per second. That's 187 millions of integers per second.

    You might also like...

    If you like this library, you might also like

    Install

    npm i fastintcompression

    DownloadsWeekly Downloads

    495

    Version

    0.0.4

    License

    Apache-2.0

    Unpacked Size

    25 kB

    Total Files

    8

    Last publish

    Collaborators

    • lemire