Exam

Exam is a JavaScript test runner, designed to be fast and easy. Its powerful features are designed to give you everything you need for testing, and more:

• Simple describe and it functions, like Mocha or Jasmine.
• Simple benchmarking with the bench function.
• Simple deferred logging with alert, debug and trace functions.
• Fast assertions with is (or use your own).
• Fast mocking with mock (or use your own).
• Exam console reporter, similar to Mocha's spec reporter.
• Multi-CPU test distribution for faster speeds on large test suites.

Quick Start

Install exam globally, or as a dev dependency:

sudo npm install --global exam
cd PROJECT_DIR
npm install --save-dev exam

Write some tests in test/*.js:

var a = [1, 2, 3]
describe('Array.indexOf()', function () {
  it("returns -1 when a value isn't found", function () {
    is(a.indexOf(0), -1)
    is(a.indexOf(-1), -1)
  })
  it("returns an index when a value is found", function () {
    is(a.indexOf(1), 0)
    is(a.indexOf(2), 1)
  })
})

Run tests:

exam

Run coverage testing, and open reports (if the open command is available):

exam-cover
open coverage/lcov-report/index.html

Test Functions

Exam exposes functions which you can use in your tests. By default, all of these are available on the global scope. To keep global scope clean instead, use the --no-globals or -G command line option.

Suites and Tests

describe(title, fn)

Runs fn as a suite of tests.

ddescribe(title, fn)

Runs fn exclusively. Aliased as describe.only.

xdescribe(title, fn)

Sets up a suite but skips it. Aliased as describe.skip.

it(name, fn)

Runs fn as a test for the named functionality.

iit(name, fn)

Runs fn exclusively. Aliased as it.only.

xit(name, fn)

Sets up a test but skips it. Aliased as it.skip.

bench(name, fn)

Runs fn as a benchmark.

bbench(name, fn)

Runs fn exclusively. Aliased as bench.only.

xbench(name, fn)

Sets up a benchmark but skips it. Aliased as bench.skip.

before(fn)

Runs fn** before a suite.

beforeEach(fn)

Runs fn before each test in a suite.

after(fn)

Runs fn after a suite.

afterEach(fn)

Runs fn after each test in a suite.

setup/teardown

Aliases for before/after.

Timeouts

The this object inside a test function such as describe or it has a timeout method that accepts a number of milliseconds. It resets a timeout that waits for test execution to complete. The default timeout is one second, so if you need longer, you can customize it.

describe('External service', function () {
  it('takes no more than ten seconds', function (done) {
    this.timeout(10000)
    service.makeRequest(function () {
      done()
    })
  })
})

Assertions and Mocks

is(actual, expected)

Asserts strict equality.

mock(object, properties)

Temporarily decorates an object with values from a mock properties object.

unmock([object])

Restores the original properties of a mocked object (or all mocked objects).

Logs

alert(arguments...)

Stores arguments to be logged after the test run finishes.

debug(arguments...)

Stores arguments to be logged after the test run finishes, if called from within a failing test or suite.

trace(arguments...)

Stores arguments and a stack trace to be logged after the test run finishes, if called from within a failing test or suite.

Assertions

You can use exam's builtin is assertion library, or any other assertion library that throws an AssertionError.

is

You can also use is methods to make assertions:

var a = [1, 2, 3]
is.array(a)
is.same(a, [1,2,3])
is.number(a[0])
is.is(a[0], 1)

Each is method also returns is so you can chain, if that's your thing:

var a = [1, 2, 3]
is
  .array(a)
  .same(a, [1,2,3])
  .number(a[0])
  .is(a[0], 1)

Comparisons

is(actual, expected)

The is.is function is also available simply as is, allowing a shorthand strict equality assertion.

var one = 1
is(one, 1)   // No error. 
is(one, '1') // Throws an AssertionError. 

It asserts that actual is equal to expected, and that they are of the same type.

is.not(actual, expected)

Asserts that actual is not equal to expected (or that they are not of the same type).

is.equal(actual, expected)

Asserts that actual is equal to expected, within JavaScript's dynamic type system.

is.notEqual(actual, expected)

Asserts that actual is not equal to expected, within JavaScript's dynamic type system.

is.same(actual, expected) or is.deepEqual(actual, expected)

Asserts that actual is "the same" as expected, meaning that their stringified representations are equal.

is.notSame(actual, expected) or is.notDeepEqual(actual, expected)

Asserts that actual is not "the same" as expected, meaning that their stringified representations are unequal.

is.greater(first, second)

Asserts that the first value is greater than the second.

is.less(first, second)

Asserts that the first value is less than the second.

is.greaterOrEqual(first, second)

Asserts that the first value is greater than or equal to the second.

is.lessOrEqual(first, second)

Asserts that the first value is less than or equal to the second.

Strict Type Checks

is.type(value, expectedType)

Asserts that the value is of the expected type, expressed as a case-sensitive string returned by typeof.

var num = 1
var one = '1'
is.type(num, 'number') // No error. 
is.type(one, 'number') // Throws an AssertionError. 
is.type(num, 'Number') // Throws an AssertionError. 
is.type(one, 'string') // No error. 

is.notType(value, expectedType)

Asserts that the value is not of the expected type, expressed as a case-sensitive string returned by typeof.

is.null(value)

Asserts that the value is null. This is a strictly-typed assertion.

is.notNull(value)

Asserts that the value is not null. This is a strictly-typed assertion.

is.undefined(value)

Asserts that the value is undefined. This is a strictly-typed assertion.

is.notUndefined(value) or is.defined(value)

Asserts that the value is not undefined. This is a strictly-typed assertion.

is.boolean(value)

Asserts that the value is a boolean. This is a strictly-typed assertion, so truthy or falsy values which are not actually true or false will fail this assertion.

is.notBoolean(value)

Asserts that the value is not a boolean. This is a strictly-typed, so true and false are the only values that will fail this assertion.

is.number(value)

Asserts that the value is a number value. This is a strictly-typed assertion, so strings with numeric values will fail this assertion.

is.notNumber(value)

Asserts that the value is a number value. This is a strictly-typed assertion, so numeric values are the only values that will fail this assertion.

is.string(value)

Asserts that the value is a string. This is a strictly-typed assertion.

is.notString(value)

Asserts that the value is not a string. This is a strictly-typed assertion.

is.function(value)

Asserts that the value is a function. This is a strictly-typed assertion.

is.notFunction(value)

Asserts that the value is not a function. This is a strictly-typed assertion.

is.object(value)

Asserts that the value is an object. This is a strictly-typed assertion.

is.notObject(value)

Asserts that the value is not an object. This is a strictly-typed assertion.

Value Checks

is.true(value)

Asserts that the value is the boolean value true.

is.notTrue(value)

Asserts that the value is not the boolean value true.

is.false(value)

Asserts that the value is the boolean value false.

is.notFalse(value)

Asserts that the value is not the boolean value false.

is.truthy(value)

Asserts that the value evaluates to true, meaning the value is either true, a non-zero number, a non-empty string, a function, or a non-null object.

is.falsy(value)

Asserts that the value evaluates to false, meaning the value is either false, 0 (zero), "" (empty string), null, undefined or NaN.

is.nan(value)

Asserts that the value cannot be evaluated as a number. This includes anything that is not a number, a string representation of a number, null (which evaluates to zero), false (which evaluates to zero) or a Date object.

is.notNan(value)

Asserts that the value can be evaluated as a number. This includes numbers, a string representations of numbers, null (which evaluates to zero), false (which evaluates to zero) and Date objects.

Instance Checks

is.instanceOf(value, expectedClass)

Asserts that the value is an instance of the expected class.

is.notInstanceOf(value, expectedClass)

Asserts that the value is not an instance of the expected class.

is.array(value)

Asserts that the value is an instance of the Array class.

is.notArray(value)

Asserts that the value is not an instance of the Array class.

is.date(value)

Asserts that the value is an instance of the Date class.

is.notDate(value)

Asserts that the value is not an instance of the Date class.

is.error(value)

Asserts that the value is an instance of the Error class.

is.notError(value)

Asserts that the value is not an instance of the Error class.

is.regExp(value)

Asserts that the value is an instance of the RegExp class.

is.notRegExp(value)

Asserts that the value is not an instance of the RegExp class.

Advanced

is.in(value, search)

Asserts that value is a string and that it contains a substring search or matches a regular expression search.

is.notIn(value, search)

Asserts that either 1) value is not a string, 2) search is not a string or regular expression, or 3) search is not found in value.

is.lengthOf(value, length)

Asserts that the value (string, array, etc.) has the specified length.

is.notLengthOf(value, length)

Asserts that the value (string, array, etc.) has no length, or a different length than specified.

is.arrayOf(value, expectedTypeOrClass)

Asserts that the value is an array of the specified type or an array of instances of the specified class (depending on whether the second argument is a string).

is.notArrayOf(value, expectedTypeOrClass)

Asserts that the value is not an array, or contains an item that is not of the specified type or an item that is not an instance of the specified class (depending on whether the second argument is a string).

Mocking

You can use exam's builtin mock library, or any other mocking library you like. The mock library exposes 2 globals, mock and unmock:

describe('myConsole', function () {
  it('calls console.log', function (done) {
    mock(console, {
      log: function () {
        unmock(console)
        done()
      }
    })
  })
})

mock(object, mockedProperties)

When mock is used as a function, it accepts 2 objects. Any properties of the second object will be copied onto the first object, and if those properties were already defined on the first, they will be saved so they can be unmocked later.

In addition, mock is an object with several methods for replacing methods with simple functions that create testable output.

mock.ignore()

Returns a function that does nothing.

describe('myConsole', function () {
  it('.log does not throw an error', function () {
    mock(console, {
      log: mock.ignore()
    })
    // This logs nothing, despite calling console.log('a'). 
    myConsole.log('a')
  })
})

mock.count()

Returns a function that increments its value property each time it is called.

describe('myConsole', function () {
  it('.log calls console.log once', function () {
    mock(console, {
      log: mock.count()
    })
    is(console.log.value, 0)
    myConsole.log('a')
    is(console.log.value, 1)
    unmock(console)
  })
})

mock.concat([delimiter])

Returns a function whose first argument is concatenated onto its value property each time it is called.

describe('myConsole', function () {
  it('.log calls console.log', function () {
    mock(console, {
      log: mock.concat()
    })
    is(console.log.value, '')
    myConsole.log('a')
    is(console.log.value, 'a')
    myConsole.log('b')
    is(console.log.value, 'ab')
    unmock(console)
  })
})

If a delimiter is supplied, it will be used to separate the concatenated arguments.

describe('myConsole', function () {
  it('.log calls console.log', function () {
    mock(console, {
      log: mock.concat(',')
    })
    is(console.log.value, '')
    myConsole.log(1)
    is(console.log.value, '1')
    myConsole.log(2)
    is(console.log.value, '1,2')
    unmock(console)
  })
})

mock.args([index])

Returns a function that pushes its arguments into an array each time it is called.

describe('myConsole', function () {
  it('.log calls console.log with multiple arguments', function () {
    mock(console, {
      log: mock.args()
    })
    is.same(console.log.value, [])
    myConsole.log('a')
    is.same(console.log.value, [{0: 'a'}])
    myConsole.log('b', 'c')
    is.same(console.log.value, [{0: 'a'}, {0: 'b', 1: 'c'}])
    unmock(console)
  })
})

If an index is supplied, it only pushes one of the arguments.

describe('myConsole', function () {
  it('.log calls console.log', function () {
    mock(console, {
      log: mock.args(0)
    })
    is.same(console.log.value, [])
    myConsole.log(1)
    is.same(console.log.value, [1])
    myConsole.log(2, 3)
    is.same(console.log.value, [1, 2])
    unmock(console)
  })
})

mock.fs([config][, createNewFs])

Uses mock-fs to create a temporary in-memory file system for fast, reliable tests. If createNewFs is truthy, Node's built-in fs module remains unchanged, otherwise its methods are mocked.

// Replace Node's `fs` with a temporary file system. 
var fs = mock.fs({
  'path/to/fake/dir': {
    'some-file.txt': 'file content here',
    'empty-dir': {} // Empty directory. 
  },
  'path/to/some.png': new Buffer([8, 6, 7, 5, 3, 0, 9])
})
 
// Verify that we can read content. 
var content = fs.readFileSync('path/to/fake/dir/some-file.txt')
is(content.toString(), 'file content here')
 
// Restore Node's built-in file system. 
unmock(fs)

Calling mock.fs sets up a mock file system and returns a reference to Node's built-in fs module, whose methods are now mocked. The resulting file system has two base directories, process.cwd() and os.tmpdir(), plus any directories/files added by the optional config object.

A config object is a nested structure in which:

• Keys are paths, relative to process.cwd().
• Buffer and string values are file contents.
• Plain object values are directories.

To create a file or directory with additional properties (owner, permissions, atime, etc.), use mock.file() or mock.directory().

Caveats:

• Paths should use forward slashes, even on Windows.

• When you use mock.fs without the createNewFs argument, Node's own fs module is modified. If you use it before any other modules that modify fs (e.g. graceful-fs), the mock should behave as expected.

• The following fs functions are overridden: fs.ReadStream, fs.Stats, fs.WriteStream, fs.appendFile, fs.appendFileSync, fs.chmod, fs.chmodSync, fs.chown, fs.chownSync, fs.close, fs.closeSync, fs.createReadStream, fs.createWriteStream, fs.exists, fs.existsSync, fs.fchmod, fs.fchmodSync, fs.fchown, fs.fchownSync, fs.fdatasync, fs.fdatasyncSync, fs.fstat, fs.fstatSync, fs.fsync, fs.fsyncSync, fs.ftruncate, fs.ftruncateSync, fs.futimes, fs.futimesSync, fs.lchmod, fs.lchmodSync, fs.lchown, fs.lchownSync, fs.link, fs.linkSync, fs.lstatSync, fs.lstat, fs.mkdir, fs.mkdirSync, fs.open, fs.openSync, fs.read, fs.readSync, fs.readFile, fs.readFileSync, fs.readdir, fs.readdirSync, fs.readlink, fs.readlinkSync, fs.realpath, fs.realpathSync, fs.rename, fs.renameSync, fs.rmdir, fs.rmdirSync, fs.stat, fs.statSync, fs.symlink, fs.symlinkSync, fs.truncate, fs.truncateSync, fs.unlink, fs.unlinkSync, fs.utimes, fs.utimesSync, fs.write, fs.writeSync, fs.writeFile and fs.writeFileSync.

• Mock fs.Stats objects have the following properties: dev, ino, nlink, mode, size, rdev, blksize, blocks, atime, ctime, mtime, uid, and gid. In addition, all of the is*() methods are provided (e.g. isDirectory() and isFile()).

• Mock file access is controlled based on file mode where process.getuid() and process.getgid() are available (POSIX systems). On other systems (e.g. Windows) the file mode has no effect.

• The following fs functions are not currently mocked (if your tests use these, they will work against the real file system): fs.FSWatcher, fs.unwatchFile, fs.watch, and fs.watchFile.

mock.file(properties)

Creates a mock file. Supported properties:

• content - string|Buffer File contents.
• mode - number File mode (permission and sticky bits). Defaults to 0666.
• uid - number The user id. Defaults to process.getuid().
• git - number The group id. Defaults to process.getgid().
• atime - Date The last file access time. Defaults to new Date(). Updated when file contents are accessed.
• ctime - Date The last file change time. Defaults to new Date(). Updated when file owner or permissions change.
• mtime - Date The last file modification time. Defaults to new Date(). Updated when file contents change.

var old = new Date(1)
mock({
  foo: mock.file({
    content: 'file content here',
    ctime: old,
    mtime: old
  })
})

mock.directory(properties)

Creates a mock directory. Supported properties:

• mode - number Directory mode (permission and sticky bits). Defaults to 0777.
• uid - number The user id. Defaults to process.getuid().
• git - number The group id. Defaults to process.getgid().
• atime - Date The last directory access time. Defaults to new Date().
• ctime - Date The last directory change time. Defaults to new Date(). Updated when owner or permissions change.
• mtime - Date The last directory modification time. Defaults to new Date(). Updated when an item is added, removed, or renamed.
• items - Object Directory contents. Members will generate additional files, directories, or symlinks.

To create a mock filesystem with a directory with the relative path some/dir that has a mode of 0755 and a couple child files, you could do something like this:

mock({
  'some/dir': mock.directory({
    mode: 0755,
    items: {
      file1: 'file one content',
      file2: new Buffer([8, 6, 7, 5, 3, 0, 9])
    }
  })
})

mock.symlink(properties)

Create a mock symlink. Supported properties:

• path - string Path to the source (required).
• mode - number Symlink mode (permission and sticky bits). Defaults to 0666.
• uid - number The user id. Defaults to process.getuid().
• git - number The group id. Defaults to process.getgid().
• atime - Date The last symlink access time. Defaults to new Date().
• ctime - Date The last symlink change time. Defaults to new Date().
• mtime - Date The last symlink modification time. Defaults to new Date().

mock({
  'some/dir': {
    'regular-file': 'file contents',
    'a-symlink': mock.symlink({
      path: 'regular-file'
    })
  }
})

unmock(object)

Restores the properties which belonged to the object prior to being mocked.

Benchmarking

bench(name, fn)

The bench function is similar to describe. It sets up a suite that can contain multiple it calls, declaring implementations to run against each other. The individual tests are run until Exam is 99% certain that it has determined which implementation is fastest, or until a timeout is reached - whichever comes first.

The bench function's this context has properties which can be modified:

• benchTime - number The number of milliseconds to spend running the set of it functions under a bench function. The default is 60000ms (1min), which is inherited from the --bench-time flag.
• z - number The z-score that is required before declaring a winner. The default is 2.576, which equates to 99% confidence.
• stopZ - number The z-score that results in a benchmark halting execution of a test that is clearly far slower than the fastest. The default is 30, which is a ridiculously high z-score.

Running exam

Exam can be run using the command line interface, or by requiring the module.

exam command

Install exam globally (using sudo if your environment requires root access to install a Node binary):

sudo npm install -g exam

Usage

exam [OPTIONS] [PATHS...]

For example, to run "test/a-test.js" and "test/b-test.js" in multiple processes and watch for changes:

exam --multi-process --watch test/a-test test/b-test

If there are no paths arguments, the path is assumed to be test, so test files include all files directly under the test directory.

Options

-h, --help
Show usage information only.

-w, --watch
Keep the process running, watch for file changes, and re-run tests when a change is detected.

-V, --version
Show the version number.

-r, --require <modules>
Require a comma-delimited list of modules.

-R, --reporter <name>
Which library will be used to output results. Options include "console", "tap", "xunit" and "counts". Default: "console".

-G, --no-globals
Do not add functions such as it and describe to the global scope. Instead, add those functions to the exam object.

-v, --recursive
Use the default test directory, or any directories specified as paths, and read their contents recursively, running any encountered files as tests.

-p, --parser <parser>
Which EcmaScript parser will be used to handle syntax errors. Options include "acorn" and "esprima". Default: "acorn".

-b, --bail
Exit after the first test failure.

-a, --assertive
Stop a test after one failed assertion.

-g, --grep <regexp>
Only run files/tests that match a regular expression.

-i, --ignore <regexp>
Exclude files/tests that match a regular expression.

-d, --debug
Run node with the --debug flag.

-m, --multi-process
Spawn child processes, creating a cluster of test runners.

-t, --timeout <millis>
Test case timeout in milliseconds. (Default: 1000)

-s, --slow <millis>
Threshold for a slow test (yellow) warning in milliseconds. (Default: 10)

-S, --very-slow <millis>
Threshold for a very slow (red) warning in milliseconds. (Default: 100)

-A, --hide-ascii
Do not show ASCII art before the test run.

-P, --hide-progress
Do not show passing/failing dots as tests run.

-C, --no-color
Turn off color console logging.

-B, --bench
Run benchmarks along with the test suite.

-T, --bench-time
Run each benchmark for a specified number of milliseconds. (Default: 60000)

exam-cover command

The exam-cover command checks coverage using istanbul. Run exam tests using the istanbul cover command.

Usage

exam-cover [EXAM_ARGS] [-- ISTANBUL_ARGS]

Options

In addition to the exam options above, exam-cover supports arguments for checking coverage (via istanbul check-coverage) after the test suite has been instrumented and executed. Each <minimum> is expressed as a whole number percent, such as 70.

--statements <minimum>
Require at least <minimum> statement coverage. (Default: 0)

--branches <minimum>
Require at least <minimum> branch coverage. (Default: 0)

--functions <minimum>
Require at least <minimum> function coverage. (Default: 0)

--lines <minimum>
Require at least <minimum> line coverage. (Default: 0)

Module

You can also run exam from within your Node application. The module exposes itself as a function that accepts an options object containing the arguments you would pass to the CLI:

var exam = require('exam')
exam({
  paths: ['test'],
  watch: true
})

More on Exam...

• Contributing
• License (ISC)
• Change Log
• Roadmap