TypeScript icon, indicating that this package has built-in type declarations

    5.0.1 • Public • Published

    Node.js CI NPM version npm downloads coveralls Codacy Badge Language grade: JavaScript Total alerts DeepScan grade Known Vulnerabilities


    A primitive token library used to read and write from a node Buffer. Although it is possible to use this module directly, it is primary designed to be used with strtok3 tokenizer.


    Module: version 5 migrated from CommonJS to pure ECMAScript Module (ESM). JavaScript is compliant with ECMAScript 2019 (ES10).


    npm install --save token-types

    Usually in combination with strtok3:

    npm install --save strtok3

    Using TypeScript you should also install @tokenizer/token as a development dependency:

    npm install --save-dev @tokenizer/token


    import * as strtok3 from 'strtok3';
    import * as token from 'token-types';
    (async () => {
      const tokenizer = await strtok3.fromFile("somefile.bin");
      try {
        const myNumber = await tokenizer.readToken(token.Float32_BE);
        console.log(`My number: ${myNumber}`);
      } finally {
        tokenizer.close(); // Close the file


    Numeric tokens

    node-strtok supports a wide variety of numerical tokens out of the box:

    Token Number Bits Endianness
    UINT8 Unsigned integer 8 n/a
    UINT16_BE Unsigned integer 16 big endian
    UINT16_LE Unsigned integer 16 little endian
    UINT24_BE Unsigned integer 24 big endian
    UINT24_LE Unsigned integer 24 little endian
    UINT32_BE Unsigned integer 32 big endian
    UINT32_LE Unsigned integer 32 little endian
    UINT64_BE Unsigned integer 64 big endian
    UINT64_LE* Unsigned integer 64 little endian
    INT8 Signed integer 8 n/a
    INT16_BE Signed integer 16 big endian
    INT16_LE Signed integer 16 little endian
    INT24_BE Signed integer 24 big endian
    INT24_LE Signed integer 24 little endian
    INT32_BE Signed integer 32 big endian
    INT32_LE Signed integer 32 little endian
    INT64_BE Signed integer 64 big endian
    INT64_LE* Signed integer 64 little endian
    Float16_BE IEEE 754 float 16 big endian
    Float16_LE IEEE 754 float 16 little endian
    Float32_BE IEEE 754 float 32 big endian
    Float32_LE IEEE 754 float 32 little endian
    Float64_BE IEEE 754 float 64 big endian
    Float64_LE IEEE 754 float 64 little endian
    Float80_BE* IEEE 754 float 80 big endian
    Float80_LE* IEEE 754 float 80 little endian

    Other tokens

    String types:

    • Windows-1252
    • ISO-8859-1

    *) The tokens exceed the JavaScript IEEE 754 64-bit Floating Point precision, decoding and encoding is best effort based.

    Custom token

    Complex tokens can be added, which makes very suitable for reading binary files or network messages:

     ExtendedHeader = {
        len: 10,
        get: (buf, off) => {
          return {
            // Extended header size
            size: Token.UINT32_BE.get(buf, off),
            // Extended Flags
            extendedFlags: Token.UINT16_BE.get(buf, off + 4),
            // Size of padding
            sizeOfPadding: Token.UINT32_BE.get(buf, off + 6),
            // CRC data present
            crcDataPresent: common.strtokBITSET.get(buf, off + 4, 31)


    npm i token-types

    DownloadsWeekly Downloads






    Unpacked Size

    24.6 kB

    Total Files


    Last publish


    • borewit