@cumulus/integration-tests

18.2.0 • Public • Published

@cumulus/integration-tests

This package provides a CLI and functions for testing Cumulus.

⚠️ The documented API of this package will not change without a deprecation warning being provided in earlier releases. Code in this package that is not documented in this README may change without warning, and is not considered part of the package's public API.

About Cumulus

Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.

Cumulus Documentation

Installation

$ npm install @cumulus/integration-tests

API

Modules

Collections
Executions
Granules
Providers
Rules

Collections

Example

const Collections = require('@cumulus/integration-test/Collections');

createCollection(prefix, [overrides]) ⇒ Promise.<Object>

Create a randomized collection using the Cumulus API.

The default collection is very simple. It expects that, for any discovered file, the granule ID is everything in the filename before the extension. For example, a file named gran-1.txt would have a granuleId of gran-1. Filenames can only contain a single . character.

Collection defaults:

  • name: random string starting with collection-name-
  • version: random string starting with collection-version-
  • reportToEms: false
  • granuleId: '^[^.]+$'
  • granuleIdExtraction: '^([^.]+)\..+$'
  • sampleFileName: 'asdf.jpg'
  • files:
    [
      {
        bucket: 'protected',
        regex: '^[^.]+\..+$',
        sampleFileName: 'asdf.jpg'
      }
    ]

Kind: Exported function
Returns: Promise.<Object> - the generated collection

Param Type Description
prefix string the Cumulus stack name
[overrides] Object properties to set on the collection, overriding the defaults

createCollection~addCustomUrlPathToCollectionFiles(collection, customFilePath) ⇒ Array.<Object>

Given a Cumulus collection configuration, return a list of the filetype configs with their url_paths updated.

Kind: inner method of createCollection
Returns: Array.<Object> - a list of collection filetype configs

Param Type Description
collection Object a Cumulus collection
customFilePath string path to be added to the end of the url_path

createCollection~buildCollection(params) ⇒ Object

Update a collection with a custom file path, duplicate handling, and name updated with the postfix.

Kind: inner method of createCollection
Returns: Object - an updated collection

Param Type Description
params Object
params.collection Object a collection configuration
params.customFilePath string path to be added to the end of the url_path
params.duplicateHandling string duplicate handling setting
params.postfix string a string to be appended to the end of the name

createCollection~collectionExists(stackName, collection) ⇒ boolean

Returns true if collection exists. False otherwise.

Kind: inner method of createCollection

Param Type Description
stackName string the prefix of the Cumulus stack
collection Object a Cumulus collection

createCollection~addCollection(stackName, collection) ⇒ Promise.<undefined>

Add a new collection to Cumulus

Kind: inner method of createCollection

Param Type Description
stackName string the prefix of the Cumulus stack
collection Object a Cumulus collection

createCollection~addCollections(stackName, bucketName, dataDirectory, [postfix], [customFilePath], [duplicateHandling]) ⇒ Promise.<Array.<Object>>

Add collections to database

Kind: inner method of createCollection
Returns: Promise.<Array.<Object>> - - collections that were added

Param Type Description
stackName string Cloud formation stack name
bucketName string S3 internal bucket name
dataDirectory string the directory of collection json files
[postfix] string string to append to collection name
[customFilePath] string
[duplicateHandling] string

Executions

Example

const Executions = require('@cumulus/integration-test/Executions');

findExecutionArn(prefix, matcher, [queryParameters], [options]) ⇒ Promise.<string>

Find the execution ARN matching the matcher function

Kind: Exported function
Returns: Promise.<string> - the ARN of the matching execution

Param Type Default Description
prefix string the name of the Cumulus stack
matcher function a predicate function that takes an execution and determines if this is the execution that is being searched for
[queryParameters] Object Optional query parameters
[options] Object
[options.timeout] integer 0 the number of seconds to wait for a matching execution to be found

getExecutionWithStatus(params) ⇒ Promise.<Object>

Wait for an execution to have an expected status and return the execution

Kind: Exported function
Returns: Promise.<Object> - the execution as returned by the GET /executions/<execution-arn> endpoint

Param Type Default Description
params Object
params.prefix string the name of the Cumulus stack
params.arn string the execution ARN to fetch
params.status string the status to wait for
[params.callback] function cumulusApiClient.invokeApifunction an async function to invoke the API Lambda that takes a prefix / user payload
[params.timeout] integer 30 the number of seconds to wait for the execution to reach a terminal state

Granules

Example

const Granules = require('@cumulus/integration-test/Granules');

getGranuleWithStatus(params) ⇒ Promise.<Object>

Wait for a granule to have an expected status and return the granule

Kind: Exported function
Returns: Promise.<Object> - the granule as returned by the GET /granules/<granule-id> endpoint

Param Type Default Description
params Object
params.prefix string the name of the Cumulus stack
params.granuleId string the granuleId of the granule
params.collectionId string the collectionId of the granule
params.status string the status to wait for
params.updatedAt string minimum updatedAt time the granule must have to return
[params.callback] function cumulusApiClient.invokeApifunction an async function to invoke the API Lambda that takes a prefix / user payload
[params.timeout] integer 30 the number of seconds to wait for the execution to reach a terminal state

getGranuleWithStatus~waitForListGranulesResult(params) ⇒ Promise.<Object>

Wait for listGranules to return at least a single value before returning an empty result

Kind: inner method of getGranuleWithStatus
Returns: Promise.<Object> - - results of a successful listGranules

Param Type Description
params Object parameters to listGranules function

Providers

Example

const Providers = require('@cumulus/integration-test/Providers');

createProvider(prefix, [overrides]) ⇒ Promise.<Object>

Create a provider using the Cumulus API

Provider defaults:

  • id: random string starting with provider-
  • protocol: s3
  • globalConnectionLimit: 10

Kind: Exported function
Returns: Promise.<Object> - the generated provider

Param Type Description
prefix string the Cumulus stack name
[overrides] Object properties to set on the provider, overriding the defaults

Rules

Example

const Rules = require('@cumulus/integration-test/Rules');

createOneTimeRule(prefix, [overrides]) ⇒ Promise.<Object>

Create a onetime rule using the Cumulus API

Rule defaults:

  • name: random string starting with rule_
  • rule: { type: 'onetime' }

Kind: Exported function
Returns: Promise.<Object> - the generated rule

Param Type Description
prefix string the name of the Cumulus stack
[overrides] Object properties to set on the rule, overriding the defaults

CLI Usage

Usage: cumulus-test TYPE COMMAND [options]


  Options:

    -V, --version                   output the version number
    -s, --stack-name <stackName>    AWS Cloud Formation stack name (default: null)
    -b, --bucket-name <bucketName>  AWS S3 internal bucket name (default: null)
    -w, --workflow <workflow>       Workflow name (default: null)
    -i, --input-file <inputFile>    Workflow input JSON file (default: null)
    -h, --help                      output usage information


  Commands:

    workflow  Execute a workflow and determine if the workflow completes successfully

For example, to test the HelloWorld workflow:

cumulus-test workflow --stack-name helloworld-cumulus --bucket-name cumulus-bucket-internal --workflow HelloWorldWorkflow --input-file ./helloWorldInput.json

Contributing

To make a contribution, please see our contributing guidelines.


Generated automatically using npm run build-docs

Readme

Keywords

none

Package Sidebar

Install

npm i @cumulus/integration-tests

Weekly Downloads

48

Version

18.2.0

License

Apache-2.0

Unpacked Size

33.6 MB

Total Files

30

Last publish

Collaborators

  • cumuluspublisher
  • nsidc_kovarik
  • jennyhliu
  • npauzenga