This library is published in the NPM registry and can be installed using any compatible package manager.
npm install firebolt-sdk --save
# For Yarn, use the command below.
yarn add firebolt-sdk
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
const connection = await firebolt.connect({
auth: {
client_id: process.env.FIREBOLT_CLIENT_ID,
client_secret: process.env.FIREBOLT_CLIENT_SECRET,
},
account: process.env.FIREBOLT_ACCOUNT,
database: process.env.FIREBOLT_DATABASE,
engineName: process.env.FIREBOLT_ENGINE_NAME
});
const statement = await connection.execute("SELECT 1");
// fetch statement result
const { data, meta } = await statement.fetchResult();
// or stream result
const { data } = await statement.streamResult();
data.on("metadata", metadata => {
console.log(metadata);
});
data.on("error", error => {
console.log(error);
});
const rows = []
for await (const row of data) {
rows.push(row);
}
console.log(rows)
- About
- Documentation
- Usage
- Recipes
The Firebolt client for Node.js. firebolt-sdk provides common methods for quering Firebolt databases, fetching and streaming results, and engine management.
firebolt-sdk supports Node.js > v14
.
const connection = await firebolt.connect(connectionOptions);
type AccessTokenAuth = {
accessToken: string;
};
type ClientCredentialsAuth = {
client_id: string;
client_secret: string;
};
type ConnectionOptions = {
auth: AccessTokenAuth | ServiceAccountAuth;
database: string;
engineName?: string;
engineEndpoint?: string;
account?: string;
};
You can omit engineName
and execute AQL queries on such connection.
Instead of passing client id/secret directly, you can also manage authentication outside of node sdk and pass accessToken when creating the connection
const connection = await firebolt.connect({
auth: {
accessToken: "access_token",
},
engineName: 'engine_name',
account: 'account_name',
database: 'database',
});
Default way of authenticating is with the client credentials
const connection = await firebolt.connect({
auth: {
client_id: 'b1c4918c-e07e-4ab2-868b-9ae84f208d26';
client_secret: 'secret';
},
engineName: 'engine_name',
account: 'account_name',
database: 'database',
});
TODO: write motivation connection can be tested using:
const firebolt = Firebolt();
await firebolt.testConnection(connectionOptions)
which will perform authentication and simple select 1
query
Firebolt engine URLs use the following format:
<engine-name>.<account-name>.<region>.app.firebolt.io
For example: your-engine.your-account.us-east-1.app.firebolt.io
. You can find and copy your engine endpoint name in the Firebolt web UI.
const statement = await connection.execute(query, executeQueryOptions);
const statement = await connection.execute(query, {
settings: { query_id: 'hello' }
});
export type ExecuteQueryOptions = {
parameters:? unknown[];
settings?: QuerySettings;
response?: ResponseSettings;
};
parameters
field is used to specify replacements for ?
symbol in the query.
For example:
const statement = await connection.execute("select ?, ?", {
parameters: ["foo", 1]
});
will produce select 'foo', 1
query
Format Tuple
:
import { Tuple } from 'firebolt-sdk'
const statement = await connection.execute("select ? where bar in ?", {
parameters: [
1,
new Tuple(['foo'])
]
});
namedParameters
field is used to specify replacements for :name
tokens in the query.
For example:
const statement = await connection.execute("select :foo, :bar", {
namedParameters: { foo: "foo", bar: 123 }
});
will produce select 'foo', 123
query
Parameter | Required | Default | Description |
---|---|---|---|
output_format | JSON_COMPACT | Specifies format of selected data |
You can also use QuerySettings to specify set flags.
For example: { query_id: 'hello' }
Parameter | Required | Default | Description |
---|---|---|---|
normalizeData | false | Maps each row in response from array format to object | |
bigNumberAsString | false | Hydrate BigNumber as String |
const { data, meta, statistics } = await statement.fetchResult();
The Promise API is not recommended for SELECT
queries with large result sets (greater than 10,000 rows). This is because it parses results synchronously, so will block the JS thread/event loop and may lead to memory leaks due to peak GC loads.
It is recommended to use LIMIT
in your queries when using the Promise API.
const { data } = await statement.streamResult();
const rows: unknown[] = [];
data.on("metadata", metadata => {
console.log(metadata);
});
data.on("error", error => {
console.log(error);
});
for await (const row of data) {
rows.push(row);
}
firebolt-sdk maps SQL data types to their corresponding JavaScript equivalents. The mapping is described in the table below:
Category | SQL type | JavaScript type | Notes |
---|---|---|---|
Numeric | INT | Number | If value cannot be represented by JavaScript Number (determine using Number.isSafeInteger), BigNumber from "bignumber.js" is used |
INTEGER | Number | ||
BIGINT | Number | ||
LONG | Number | ||
FLOAT | Number | ||
DOUBLE | Number | ||
String | VARCHAR | String | |
TEXT | String | ||
STRING | String | ||
Date & Time | DATE | Date |
Engines can be managed by using the resourceManager
object.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const enginesService = firebolt.resourceManager.engine
Returns engine using engine name.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name")
Property | Type | Notes |
---|---|---|
name |
string |
|
endpoint |
string |
|
current_status_summary |
string |
Starts an engine.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name")
await engine.start()
Stops an engine.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name")
await engine.stop()
Creates an engine.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.create("engine_name");
Attaches an engine to a database.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.attachToDatabase("engine_name", "database_name");
Deletes an engine.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const engine = await firebolt.resourceManager.engine.getByName("engine_name");
await engine.delete();
Databases can be managed by using the resourceManager
object.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const databaseService = firebolt.resourceManager.database
Returns database using database name.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const database = await firebolt.resourceManager.database.getByName("database_name")
Property | Type | Notes |
---|---|---|
name |
string |
|
description |
string |
Creates a database.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const database = await firebolt.resourceManager.database.create("database_name");
Get engines attached to a database.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const database = await firebolt.resourceManager.database.getByName("database_name");
const engines = database.getAttachedEngines();
Deletes a database.
import { Firebolt } from 'firebolt-sdk'
const firebolt = Firebolt();
await firebolt.connect(connectionOptions);
const database = await firebolt.resourceManager.database.getByName("database_name");
await database.delete();
The recommended way to consume query results is by using streams.
For convenience, statement.streamResult
also returns meta: Promise<Meta[]>
and statistics: Promise<Statistics>
, which are wrappers over data.on('metadata')
and data.on('statistics')
.
const firebolt = Firebolt();
const connection = await firebolt.connect(connectionParams);
const statement = await connection.execute("SELECT 1");
const {
data,
meta: metaPromise,
statistics: statisticsPromise
} = await statement.streamResult();
const rows: unknown[] = [];
const meta = await metaPromise;
for await (const row of data) {
rows.push(row);
}
const statistics = await statisticsPromise
console.log(meta);
console.log(statistics);
console.log(rows)
To achieve seamless stream pipes to fs
or stdout
, you can use the Transform
stream.
import stream, { TransformCallback } from 'stream';
class SerializeRowStream extends stream.Transform {
public constructor() {
super({
objectMode: true,
transform(
row: any,
encoding: BufferEncoding,
callback: TransformCallback
) {
const transformed = JSON.stringify(row);
this.push(transformed);
this.push('\n')
callback();
}
});
}
}
const serializedStream = new SerializeRowStream()
const firebolt = Firebolt();
const connection = await firebolt.connect(connectionParams);
const statement = await connection.execute("select 1 union all select 2");
const { data } = await statement.streamResult();
data.pipe(serializedStream).pipe(process.stdout);
Or use rowParser
that returns strings or Buffer:
const { data } = await statement.streamResult({
rowParser: (row: string) => `${row}\n`
});
data.pipe(process.stdout);
cp .env.example .env
npm test
Released under Apache License.