firebolt-sdk
    TypeScript icon, indicating that this package has built-in type declarations

    0.1.3 • Public • Published

    Firebolt Node.js SDK

    Screen Shot 2022-01-10 at 10 56 30 AM

    Installation

    This library is published in the NPM registry and can be installed using any compatible package manager.

    npm install firebolt-sdk --save
    
    # For Yarn, use the command below.
    yarn add firebolt-sdk

    Using the library

    import { Firebolt } from 'firebolt-sdk'
    
    const firebolt = Firebolt();
    
    const connection = await firebolt.connect({
      username: process.env.FIREBOLT_USERNAME,
      password: process.env.FIREBOLT_PASSWORD,
      database: process.env.FIREBOLT_DATABASE,
      engineName: process.env.FIREBOLT_ENGINE_NAME
    });
    
    const statement = await connection.execute("SELECT 1");
    
    // fetch statement result
    const { data, meta } = await statement.fetchResult();
    
    // or stream result
    const { data } = await statement.streamResult();
    
    data.on("metadata", metadata => {
      console.log(metadata);
    });
    
    data.on("error", error => {
      console.log(error);
    });
    
    const rows = []
    
    for await (const row of data) {
      rows.push(row);
    }
    
    console.log(rows)
    

    Contents

    About

    The Firebolt client for Node.js. firebolt-sdk provides common methods for quering Firebolt databases, fetching and streaming results, and engine management.

    firebolt-sdk supports Node.js > v14.

    Documentation

    Usage

    Create connection

    const connection = await firebolt.connect(connectionOptions);

    ConnectionOptions

    type ConnectionOptions = {
      username: string;
      password: string;
      database: string;
      engineName?: string;
      engineEndpoint?: string;
    };

    Test connection

    TODO: write motivation connection can be tested using:

    const firebolt = Firebolt();
    await firebolt.testConnection(connectionOptions)

    which will perform authentication and simple select 1 query

    Engine URL

    Firebolt engine URLs use the following format:

    <engine-name>.<account-name>.<region>.app.firebolt.io
    

    For example: your-engine.your-account.us-east-1.app.firebolt.io. You can find and copy your engine endpoint name in the Firebolt web UI.

    Execute Query

    const statement = await connection.execute(query, executeQueryOptions);

    Execute Query with set flags

    const statement = await connection.execute(query, {
      settings: { query_id: 'hello' }
    });

    ExecuteQueryOptions

    export type ExecuteQueryOptions = {
      parameters:? unknown[];
      settings?: QuerySettings;
      response?: ResponseSettings;
    };

    parameters

    parameters field is used to specify replacements for ? symbol in the query.

    For example:

    const statement = await connection.execute("select ?, ?", {
      parameters: ["foo", 1]
    });

    will produce select 'foo', 1 query

    QuerySettings

    Parameter Required Default Description
    output_format JSON_COMPACT Specifies format of selected data

    You can also use QuerySettings to specify set flags. For example: { query_id: 'hello' }

    ResponseSettings

    Parameter Required Default Description
    normalizeData false Maps each row in response from array format to object
    bigNumberAsString false Hydrate BigNumber as String

    Fetch result

    const { data, meta, statistics } = await statement.fetchResult();

    The Promise API is not recommended for SELECT queries with large result sets (greater than 10,000 rows). This is because it parses results synchronously, so will block the JS thread/event loop and may lead to memory leaks due to peak GC loads.

    It is recommended to use LIMIT in your queries when using the Promise API.

    Stream result

    const { data } = await statement.streamResult();
    const rows: unknown[] = [];
    
    data.on("metadata", metadata => {
      console.log(metadata);
    });
    
    data.on("error", error => {
      console.log(error);
    });
    
    for await (const row of data) {
      rows.push(row);
    }

    Result hydration

    firebolt-sdk maps SQL data types to their corresponding JavaScript equivalents. The mapping is described in the table below:

    Category SQL type JavaScript type Notes
    Numeric INT Number If value cannot be represented by JavaScript Number (determine using Number.isSafeInteger), BigNumber from "bignumber.js" is used
    INTEGER Number
    BIGINT Number
    LONG Number
    FLOAT Number
    DOUBLE Number
    String VARCHAR String
    TEXT String
    STRING String
    Date & Time DATE Date

    Engine management

    Engines can be managed by using the resourceManager object.

    import { Firebolt } from 'firebolt-sdk'
    const firebolt = Firebolt();
    await firebolt.connect(connectionOptions);
    const enginesService = firebolt.resourceManager.engine

    getById

    Returns engine using engine ID and account ID.

    import { Firebolt } from 'firebolt-sdk'
    const firebolt = Firebolt();
    await firebolt.connect(connectionOptions);
    const engine = await firebolt.resourceManager.engine.getById(
      "c8a228ea-93df-4784-99f9-a99368518782",
      "a32b073b-e093-4880-8fd4-3b302b4cf221"
    );

    getByName

    Returns engine using engine name.

    import { Firebolt } from 'firebolt-sdk'
    const firebolt = Firebolt();
    await firebolt.connect(connectionOptions);
    const engine = await firebolt.resourceManager.engine.getByName("engine_name")

    Engine

    Property Type Notes
    id {engine_id: string; account_id: string}
    name string
    endpoint string
    description string
    current_status_summary string

    Start

    Starts an engine.

    import { Firebolt } from 'firebolt-sdk'
    const firebolt = Firebolt();
    await firebolt.connect(connectionOptions);
    const engine = await firebolt.resourceManager.engine.getByName("engine_name")
    await engine.start()

    Stop

    Stops an engine.

    import { Firebolt } from 'firebolt-sdk'
    const firebolt = Firebolt();
    await firebolt.connect(connectionOptions);
    const engine = await firebolt.resourceManager.engine.getByName("engine_name")
    await engine.stop()

    Restart

    Restarts an engine.

    import { Firebolt } from 'firebolt-sdk'
    
    const firebolt = Firebolt();
    await firebolt.connect(connectionOptions);
    const engine = await firebolt.resourceManager.engine.getByName("engine_name")
    await engine.restart()

    Database management

    Databases can be managed by using the resourceManager object.

    import { Firebolt } from 'firebolt-sdk'
    const firebolt = Firebolt();
    await firebolt.connect(connectionOptions);
    const databaseService = firebolt.resourceManager.database

    Database getById

    Returns database using engine ID and account ID.

    import { Firebolt } from 'firebolt-sdk'
    const firebolt = Firebolt();
    await firebolt.connect(connectionOptions);
    const engine = await firebolt.resourceManager.database.getById(
      "c8a228ea-93df-4784-99f9-a99368518782",
      "a32b073b-e093-4880-8fd4-3b302b4cf221"
    );

    Database getByName

    Returns database using database name.

    import { Firebolt } from 'firebolt-sdk'
    const firebolt = Firebolt();
    const database = await firebolt.resourceManager.database.getByName("database_name")

    Database

    Property Type Notes
    id {engine_id: string; account_id: string}
    name string
    description string

    Resource Manager

    It is possible to create resourceManager separately from firebolt client, providing only auth credentials

    import { FireboltResourceManager } from 'firebolt-sdk'
    
    const authOptions = {
      username: process.env.FIREBOLT_USERNAME as string,
      password: process.env.FIREBOLT_PASSWORD as string
    };
    
    const resourceManager = FireboltResourceManager();
    await resourceManager.authenticate(authOptions);
    const engine = await resourceManager.engine.getByName(
      process.env.FIREBOLT_ENGINE_NAME as string
    );

    Recipes

    Streaming results

    The recommended way to consume query results is by using streams.

    For convenience, statement.streamResult also returns meta: Promise<Meta[]> and statistics: Promise<Statistics>, which are wrappers over data.on('metadata') and data.on('statistics').

    const firebolt = Firebolt();
    
    const connection = await firebolt.connect(connectionParams);
    
    const statement = await connection.execute("SELECT 1");
    
    const {
      data,
      meta: metaPromise,
      statistics: statisticsPromise
    } = await statement.streamResult();
    
    const rows: unknown[] = [];
    
    const meta = await metaPromise;
    
    for await (const row of data) {
      rows.push(row);
    }
    
    const statistics = await statisticsPromise
    
    console.log(meta);
    console.log(statistics);
    console.log(rows)

    Custom stream transformers

    To achieve seamless stream pipes to fs or stdout, you can use the Transform stream.

    import stream,  { TransformCallback } from 'stream';
    
    class SerializeRowStream extends stream.Transform {
      public constructor() {
        super({
          objectMode: true,
          transform(
            row: any,
            encoding: BufferEncoding,
            callback: TransformCallback
          ) {
            const transformed = JSON.stringify(row);
            this.push(transformed);
            this.push('\n')
            callback();
          }
        });
      }
    }
    
    const serializedStream = new SerializeRowStream()
    
    const firebolt = Firebolt();
    const connection = await firebolt.connect(connectionParams);
    const statement = await connection.execute("select 1 union all select 2");
    
    const { data } = await statement.streamResult();
    
    serializedStream.pipe(serializedStream).pipe(process.stdout);

    Or use rowParser that returns strings or Buffer:

    const { data } = await statement.streamResult({
      rowParser: (row: string) => `${row}\n`
    });
    
    data.pipe(process.stdout);

    Development process

    Actions before

    Setup env variables

    cp .env.example .env

    Execute tests

      npm test

    License

    Released under Apache License.

    Keywords

    none

    Install

    npm i firebolt-sdk

    DownloadsWeekly Downloads

    2,136

    Version

    0.1.3

    License

    Apache-2.0

    Unpacked Size

    145 kB

    Total Files

    104

    Last publish

    Collaborators

    • firebolt-db-user