qtc-test-runner

qtc-test-runner is the preferred framework for defining automated test plans. It has no opinion on what your tests do or how you write them. It is meant to organize tests in a consistent way and to let a test plan define how the test should be run.

Getting Started

These instructions explain how qtc-test-runner works, how to create a test plan and how to write a plugin.

Installing

Install the sfdx qtc cli. The qtc-test-runner framework does not provide a binary.

Test plans

You define the tests you want to run in a json file and pass this file path to the cli. Example default.json:

{
	"plugins": {},
	"tests": [
		"path-to-my/test.js"
	]
}

There are only 2 properties in each test plan:

plugins: Map of plugins. Plugins implement hooks that are executed during a test run.

tests: Array of test paths. They are executed in order.

Getting Started with Plugins

You can specify plugins in your run plan which allows you to add specific functionality for your test tun. There are several examples of CPQ Test Plugins

Creating a plugin

Plugins are written in JavaScript and extend the Hookable qtc-test-runner class. It is up to you for where you want to create your plugin code. It can be in it's own repo or it can simply be a JavaScript file in your project. It must be requireable from the directory you run the tests and must extend the Hookable qtc-test-runner class. If you're confused on where to create your plugin code you can read more about how require works Or just look at how the CPQ Test Plugins are organized

Here is a very simple plugin that prints out HELLO WORLD + the test file before each test runs:

const Hookable = new require('qtc-test-runner').Hookable; //require in the Hookable extension
const NAME = 'HelloWorld';

class HelloWorld extends Hookable {

	/**
	 * Pass the name of your plugin to the parent Hookable constructor
	 */
	constructor(arg) {
		super(NAME);
		this.arg = arg;
	}

	/**
	 * Before the test, say HELLO WORLD + test
	 */
	beforeTest(test) {
		this.log(this.arg + test);
	}
}

module.exports = HelloWorld;

Add the plugin to your run plan and pass the constructor a value:

{
  "plugins": {
	"path-to/HelloWorldPlugin.js": "HELLO WORLD"
  },
  "tests": [
	"path-to-my/test.js"  ]
}

When the test runner begins it will require the plugins in the order they are defined and pass to the constructor of the plugin whatever you have defined as a value to the plugin key. In this example "HELLO WORLD" is passed to the constructor of your plugin. You can pass all supported JSON values to your constructor.

Reference docs

This gets into more details.

Command Line Arguments

These are process environment variables that can be sourced on your system or run before the test command.

VERBOSE

Optional boolean.

Turns on verbose logs for the framework.

TEST_USER

Optional, but required if you want to use the provided jsforce connection

If you are testing against a Salesforce org you can specify the user name with TEST_USER. This gives you a free jsforce connection the org.

TEST_PASS

Optional, but required if you want to use the provided jsforce connection

If you are testing against a Salesforce org you can specify the password with TEST_PASS. This gives you a free jsforce connection the org.

LOGIN_URL

Optional, defaults to https://test.salesforce.com.

If you are testing against a Salesforce org you can specify the login url with LOGIN_URL.

Plugin Hooks

These are the plugin lifecycle hooks implemented in Hookable. You can implement any and all of these in your plugin. You must call super in your constructor with your plugin name.

beforeRunner()

Runs before any tests are executed.

afterRunner()

Runs after all tests are executed.

beforeTest(test)

Runs before each test and passes the test name.

afterTest(test)

Runs after each test and passes the test name.

onMessage(test, message)

When ipc is enabled (see below) this callback gives you a handle of the test name and the message provided by the child_process. The child_process is the process created from getTestRunCommand().

Plugin Getters

These are getters you can call from you plugin:

getConnection()

Returns a jsforce connection You must pass a user name and password for this to work. See the command line arguments.

getUser()

Returns the user name specified in TEST_USER.

getPass()

Returns the password specified in TEST_PASS.

getLoginUrl()

Returns the login url specified in LOGIN_URL or https://test.salesforce.com if omitted.

getName()

Returns the plugin name specified in the constructor when calling super() in your plugins constructor.

getRunAgain()

Returns the run again property. This defaults to false but a plugin can set it to true.

getTestRunCommand()

Returns the command that will be used to run the test. This defaults to node_modules/.bin/_mocha, but can be set by a plugin.

getSkipTest()

Returns the skip test property. This defaults to false, but can be set to true by a plugin to skip the current test.

ipcEnabled()

Returns the ipc property. Indicates the test child_process should use ipc communication.

getTestRunName(test)

Returns the name of the current test run if defined.

Plugin Setters and void helpers

These are the void functions you can call:

setRunAgain(b)

Plugins can set this to true and all the tests will run again. A plugin could also set it to false when a previous plugin set it to true.

setSkipTest(b)

Plugins can set this to true and the test will be skipped. A plugin could also set it to false when a previous plugin set it to true.

setTestRunCommand(command, commandArgs)

A single plugin in a run plan can set the command that will be run for each test in a child process If multiple plugins set this property an exception will be thrown. The first parameter is the command you want to run. The second parameter is an Array of additional command line arguments. If an Array is not passed an exception will be thrown.

useIpc(b)

Plugins can enable ipc on the child process.

log(message)

Plugins can log verbose messages that will only be printed when running in verbose mode. See the command line argument for VERBOSE.

info(message)

Plugins can log information messages.

setTestRunName(name)

Sets the name of the test run.

Testing this repo

If you want to test this repository please add cpq-test-plugins to the devDependencies of package.json and install it. However, do NOT commit this change as it creates a circular dependency between these 2 packages and newer npm versions cannot handle it.

"cpq-test-plugins": "git+ssh://[e-mail]:Steelbrick/CPQ-Test-Plugins.git"