TaskCluster-Lib-Testing
Support for testing TaskCluster components.
This module contains a number of utilities that facilitate testing TaskCluster components. It is typically installed as a devDependency, so it is not used in production code.
See the source for detailed documentation.
Sticky Loader
A sticky loader is a thin wrapper around taskcluster-lib-loader
to support
dependency injection. It "remembers" each value it has returned and will return
it again on the next call; it can also have a dependency injected. Use it like
this in helper.js
:
const stickyLoader = ;const load = ; exportsload = ;exportsload;exportsload;
The load.inject(component, value)
method sets a loader overwrite without
attempting to load it. There is a corresponding load.remove(component)
to
remove a component.
The last line of this suiteSetup
is important: it loads the cfg
component
so that the configuration can be edited in-place (stickyLoader has special
support for a component with this name). If cfg
is not loaded, the
load.cfg()
method will not work.
In test scripts:
const load = ; ;
The load.save()
and load.restore()
methods push and pull loader states in a
stack, and are best used in setup/teardown methods to ensure that one suite
does not "pollute" the loader state for the next.
The load.cfg(path, value)
method edits the cfg
component in place, using a
dotted path to specify the config value. The save
and restore
methods are
careful to deep-copy cfg
so that these in-place modifications affect only
the current loader state.
Secrets
This class handles getting secrets for tests, and easily determining what
secrets are available. It integrates with typed-env-config
. Set it up by
in test/helper.js
:
const Secrets = ; exportssecrets = secretName: 'project/taskcluster/testing/taskcluster-foo' 'project/taskcluster/testing/taskcluster-foo/master-only' // provide a stickyLoader instance for use in mockSuite load secrets: pulse: // env - the environment variable by which this secret is set in the config (if any) // cfg - dotted path to the config value containing this secret (if any) // name - name for the secret (used for programmatic access in tests; defaults to env) // mock - value to provide if secret is not set (for mock runs only) env: 'PULSE_USERNAME' cfg: 'pulse.username' name: 'username' mock: 'dummy' env: 'PULSE_PASSWORD' cfg: 'pulse.password' name: 'password' aws: env: 'AWS_ACCESS_KEY_ID' cfg: 'aws.accessKeyId' env: 'AWS_SECRET_ACCESS_KEY' cfg: 'aws.secretAccessKey' ;
If a secret is defined in the loaded configuration, that value will be used even if the env
key is also set.
Secrets should not have any value set in config.yml
(although !env
is OK), or this class will not function properly.
If the system you are testing does not use typed-env-config
, simply do not specify the cfg
properties to the constructor.
You can then call await secrets.setup()
to set up the secrets (reading from cfg
if necessary).
This must be called during Mocha's runtime, so either in a setup function or a test.
It short-circuits multiple calls, so it's safe to call it all over the place.
In fact, mockSuite
(below) will call it for you.
In CI (when $TASK_ID
is set), the setup
method will attempt to fetch the secrets named in secretName
from the secrets service.
It expects the fetch value to be a map from environment variable name to value.
If a fetch fails, it is considered equivalent to fetching an empty map.
This allows, for example, secrets that can only be fetched on pushes to the master branch, and not pull requests.
The secrets object has a few useful methods, all of which can only be called after setup
, and thus only in a setup function or a test:
secrets.have(name)
-- true if the given secret is availablesecrets.get(name)
-- returns an object containing the secret values by name, or throws an error if not avaialble
mockSuite
The secrets.mockSuite
function abstracts away the most common case: running the same tests in a mock and real environment, skipping the real tests if secrets are not available.
It is called as secrets.mockSuite(title, [secrets], async function(mock, skipping) { .. })
in the same location you might call Mocha's suite(..)
.
The secrets
is an array of secret names required to run this suite in a real environment.
The given function should define the suite, and can include setup
, suiteSetup
, and so on.
The mock
parameter is true for the mock version, and false for the real version.
If $NO_TEST_SKIP
is set, mockSuite
will throw an error when secrets are not available.
Note that Mocha continues to run setupSuite
and teardownSuite
functions even after a suite has been skipped.
Mocha does not provide any way to determine if a suite has been skipped.
Use skipping()
to determine if the suite is currently skipping, and avoid doing initialization that will fail.
Note, too, that all modern versions of Mocha have a bug causing nested suites to run anyway, even when the parent suite is skipped. A quick (but unfortunate) way to work around this bug is
;
Usage
// helper.jsconst Secrets stickyLoader = ; const load = ;const secrets = secretName: 'project/taskcluster/testing/taskcluster-ping' secrets: pingdom: name: 'apiKey' env: 'PINGDOM_API_KEY' cfg: 'app.pingdom.apiKey' taskcluster: name: 'clientId' env: 'TASKCLUSTER_CLIENT_ID' cfg: 'taskcluster.credentials.clientId' name: 'accessToken' env: 'TASKCLUSTER_ACCESS_TOKEN' cfg: 'taskcluster.credentials.accessToken' load; exportssecrets = secrets;exportsload = load;
// some_test.jsconst secrets load = ; // for testing by passing secrets to the subject..secrets; // for testing a loader component..secrets;
The test output for the first suite will contain something like
pingdom updates (mock)
✓ updates once
pingdom updates (real)
- updates once
PulseTestReceiver
A utility for tests written in mocha, that makes it very easy to wait for a specific pulse message. This uses real pulse messages, so pulse credentials will be required.
Example:
;
The receiver
object will setup an PulseConnection before all tests and close
the PulseConnection after all tests. This should make tests run faster. All
internal state, ie. the names given to listenFor
and waitFor
will be reset
between all tests.
schemas
Test schemas with a positive and negative test cases.
The method should be called within a suite
, as it will call the mocha test
function to define a test for each schema case.
schemasetOptions
- {} // options to pass to the taskcluster-lib-validate constructorcases
- array of test casesbasePath
- base path for relative pathnames in test cases (defaultpath.join(__dirname, 'validate')
)
Each test case looks like this:
schema: 'https://tc-tests.localhost/svcname/v7/frobnicate-foo.json' // JSON schema identifier to test against path: 'test-file.json' // Path to test file (relative to basePath) success: true || false // true if validation should succeed; false if it should fail
fakeauth
A fake for the auth service to support testing APIs without requiring production credentials, using Nock.
This object intercepts requests to the auth service's authenticateHawk
method
and returns a response based on the given clients
, instead. Note that
accessTokens are not checked -- the fake simply controls access based on
clientId or the scopes in a temporary credential or supplied with
authorizedScopes.
To start the mock, call testing.fakeauth.start(clients, {rootUrl})
in your suite's
setup
method. The first argument has the form
"clientId1": "scope1" "scope2" "clientId2": "scope1" "scope3"
The auth service on the cluster identified by rootUrl
will be faked. When
used to test an API in a microservice, this is same as the root URL for the
fake web server -- http://localhost:1234
or something of that sort.
Call testing.fakeauth.stop()
in your test suite's teardown
method to stop the HTTP interceptor.
Utilities
Sleep
The sleep
function returns a promise that resolves after a delay.
NOTE tests that depend on timing are notoriously unreliable, and suggest poorly-isolated tests. Consider writing the tests to use a "fake" clock or to poll for the expected state.
Poll
The poll
function will repeatedly call a function that returns a promise
until the promise is resolved without errors.
await ; // delay (ms) between tries