This package has been deprecated

Author message:

AWS now has tools to do most of what this was originally designed to do

aws_athena_client

AWS Nodejs Athena Client

Description

Run multiple SQL like commands against Athena in order, results write to a stream.

Usage Example

AWS credentials are not required and may be provided via ~/.aws/config and ~/.aws/credentials files or via enviromental variables or vi AWS IAM permissions, etc.

The default output is JSON. This works well for SQL like statements list SELELCT but not for results from ATHENA only statements like 'MSCK REPAIR TABLE action'. To see proper results from those or any statement use the request.setFormatJson(false) method.

// EXAMPLE USAGE:
// Requires node 7.6 or greater for native async/await
const Athena = require('aws_athena_client');
 
// Create the Athena client and set the options.
// See https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Athena.html.
// For all other options.
const CLIENT = Athena.Client().setOptions(
    {
        // Optional region and keys, see note above
        // Must be same region schema is in.
        region : 'us-east-1',
        accessKeyId: '<AwsAccessKeyHere>',
        secretAccessKey : '<AwsSecretKeyHere>'
    }
);
 
// Wrap in an async function only if you want to keep calls asynchronous.
// But note that if you do NOT make the calls asyncronously OR throttle the queries,
// then AWS will throttle multiple syncronous Athena queries.
async function exeMultipleQuriesInSequence(queries) {
    for (let i=0; i<queries.length; i++) {
        // Loop through the queries and await for each to finish.
        await CLIENT.request(
            // This is the request object to pass in.
            // See https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Athena.html#startQueryExecution-property
            // For all other options.
            Athena.Request().setQuery(
                {
                     QueryString: queries[i],
                     ResultConfiguration: {
                         OutputLocation: "s3://some_scratch_athena_bucket/"
         			}
                }
            // Set where you want the output to go.
            // Here it just goes to standard out.
            ).setOutput(process.stdout)
            // Set where you want any errors to go.
            // Here it just goes to standard err.
            .setLogging(process.stderr)
        );
    }
}
 
// A list of example queries to run.
const sampleQueries = [
    "CREATE DATABASE IF NOT EXISTS mytestschema;",
    "DROP TABLE IF EXISTS mytestschema.mytesttable;",
    "CREATE EXTERNAL TABLE IF NOT EXISTS mytestschema.mytesttable( `one` string, `two` string, `three` string) PARTITIONED BY ( year int, month int, day int ) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' WITH SERDEPROPERTIES ( 'serialization.format' = '1') LOCATION 's3://somebucket/some/path/' TBLPROPERTIES ('has_encrypted_data'='false');",
    "ALTER TABLE mytestschema.impression ADD IF NOT EXISTS PARTITION (year='2017',month='9',day='25',hour='20') location 's3://somebuckt/some/path/2017/09/25/20';",
    "SELECT * FROM mytestschema.mytesttable;"
];
 
// See the magic happen
exeMultipleQuriesInSequence(sampleQueries);
 

API

Athena = require("aws_athena_client")

Contains two parts the Athena client that can then take one or more Requests each which have a query and an optional output part.

Client = Athena.Client()

Returns an empty client, must call setOptions

Client.setOptions(<{aws options here}>)

See https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Athena.html. All the options that can be set are explained there. Typical values to pass in are:

{
    // or some other region,
    // must be same region schema is in.
    region : 'us-east-1',
    accessKeyId: '<AwsAccessKeyHere>',
    secretAccessKey : '<AwsSecretKeyHere>',
}

Client.request(Request);

Does the actual call to AWS. See Request below.

Request = Athana.Request()

Gets an empty request

Request.setQuery

See https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Athena.html#startQueryExecution-property Typical query is:

{
    QueryString: "SELECT * FROM mytestschema.mytesttable;",
    ResultConfiguration: {
         OutputLocation: "s3://some_scratch_athena_bucket/"
    }
}

Request.setRetryMax

A non zero positive number of the number of times to retry asking AWS should a failure occure. Defaults to 10.

Request.setRetryWait

A non zero positive number of the number of milli seconds to wait before asking the 1st time or any subsequent retries for results. Defaults to 1000.

Request.setOutput

Takes a stream to write the results to. No default, if not set results are not written anywhere.

Request.setLogging

Takes a stream to write the logging and errors to. No default, if not set the logs are not written anywhere.