Simple Test Framework
Another lightweight testing utility for node.
(NOTE: There's an even simpler way to test. Look up "just-test-it" on npm.)
If you want anything beyond this, STF doesn't do it, and won't at anytime in the future. But, it may be easily combined with other code and libraries to get this feature.
- STF is a library, not a command-line tool.
- STF is a library for testing, not assertions, mock-ups or spies.
- STF tests functionality, not code coverage, stress, etc.
- STF writes results to a stream, not to a server request, database, etc.
- STF writes results in a human-readable format, not HTML, XML, JSON, YAML, etc.
Not that I have anything against any of these features. I just want to keep this library as simple as possibly, while still providing useful features specific to testing.
- Tests can be run async or sync.
- Tests can be nested.
- Async tests make use of domains to catch errors and automatically fail tests.
- Tests can automatically time out if no activity occurs for a certain amount of time.
- Tests can be specifically finished, or they can be told how many checks to expect and finish automatically once this count is reached.
- Subtests can pass/fail based on a single condition, or based on whether a function throws an error or not.
- Comments and errors may be added to the results.
- Comments and errors may be either strings or objects, and objects are inspected to produce output.
- Results output is minimized for completed tests which have no errors or failures.
- The code behind the API is available for more direct access, if the programmer wants it.
To install simple-test-framework into your current node project:
npm install simple-test-framework
To create a test:
var test = ;
To plug your test into your npm package:
Run your test
If you are using a package.json file:
Either way will work.
The primary API is briefly described here. For a more thorough explanation of all public functions, including a few methods and objects which do not appear here because they aren't expected to be used often, please see the source code.
Note that this API, while it works, is considered "unstable", in the sense of the word used by the Node API documentation. It may change at some point in the future.
test = require('simple-test-framework')
function test(name[, options][, body]) [Test]
stringThe name for the test, as output to stream.
objectOptions for controlling the test and output.
functionOptional body for asynchronous test.
Creates a 'root' test object, writing the results to a stream when that
test is finished. Except for writing to a stream, the behavior is the
same as calling
test on a
numberSpecifies the number of milliseconds after which the Test will time out due to no activity. Defaults to 5000, or 5 seconds.
objectSpecifies a writer to write the results to when the test is finished. This object must provide the same API as
ResultWriter(see source code). If not set, a ResultWriter will be used. If specifically set to null, then the test results will not be output.
writable streamSpecifies the stream to write the results of the Test to. This property is ignored if options.writer is set. If not defined, results will be written to
process.stdout. Although a writeable stream is suggested, all that is really expected is an object that contains a
writemethod, that takes a
If a test body is passed to the method, that function will be run
asynchronously, with the new Test object being passed to the function.
Test.test for more details.
If a test body is not passed to the function, then the new Test object will be returned from the function, to be used synchronously.
If no test body is passed to the function, the Test object will be returned by the function.
Creates a subtest on the current test.
stringThe name of the new subtest.
numberOptional number of milliseconds after which the test will time out if there's no activity. Defaults to 5000 milliseconds. Pass 0 to turn this feature off, if you're absolutely certain about what's going to happen.
functionOptional function body for asynchronous test.
After testing code is completed, your test should call the finish method to indicate that it finished normally, or use the finishAfter feature, otherwise it will not end until the test times out.
After a subtest is finished, it's parent test will be notified whether it passed or failed.
asynchronous mode: If a test body is passed to this function, the test will be scheduled to run asynchronously. Even synchronous code in the test body will not happen until a later cycle of the node event loop.
The primary benefit of asynchronous mode, is that unexpected errors occurring during this test will be logged and cause the test to bail. This is true even if the error occurs in asynchronous functions like setTimeout or file system calls.
synchronous mode: If a test body is not passed, then the Test object will be returned from the function, and can be controlled synchronously. The timeout functionality will still work, but any exceptions that occur will not automatically bail out the test.
A test that is already in synchronous mode can be shifted into asynchronous mode by calling a function using Test.run.
Either way, the Test object will be returned from the function, allowing for more complicated behavior.
Creates a simple subtest on the current test that is automatically
finished and passed based on the condition. This is very similar to
assert.ok as it is used in other test frameworks.
booleanIndicates whether the test has passed or failed.
stringThe name of the test.
Usually, an expression is used here to check the value of a variable to determine if the test passed.
Creates a simple subtest based on the success of a function. This is
similar to how
assert.doesNotThrow is used in other test frameworks.
functionA synchronous function that will be called immediately.
stringThe name of the subtest.
The passed function will be called immediately. If it throws an error, the subtest will fail, and the error will be logged to the test. If it does not, then the checkpoint will pass.
If the function calls async functions, errors occurring in them will
not be seen. Use
test to handle async tests like this.
Adds cleanup code to the test.
functionA function that will be called when the test finishes.
This is where you would place code that needs to be run to release resources after a test is finished, whether the test fails or not. More than one cleanup function can be added.
Cleanup functions will be called, in the order they were declared, after the test and all of it's subtests are finished, but before the parent test is notified of this test finishing.
Finishes the test.
stringA reason for finishing a test, if the test is not finishing normally.
Every test must be finished before results can be shown. Finishing a test causes cleanup to occur, and the parent test to be notified of the test's results.
Once this method is called, the parent test is not notified until after all subtests of this test have finished. This ensures that the parent test can know whether this test has passed or failed.
Passing a reason to this method indicates that the test did not finish
normally, meaning that it failed. Any string can be passed to this
method, but the primary use of the reason argument is to indicate
whether the test timed out (
reason = "timeout") or bailed due to
an uncaught exception (
reason = "bail").
Tells the test to automatically finish after the specified number of tests have been seen.
numberThe number of expected tests.
This is similar to the
plan feature of other test frameworks. Once
the total number of items has reached the expected number, the test
will automatically finish itself.
The main use of this feature is to specify the number of checks that
are expected, so that you don't need to explicitly call finish when
all of them have been declared. This is most useful if you need to
initiate subtests in asynchronous code that may get called later than
your call to
Unlike other test frameworks, it is possible to change this after the value is initially set. Just be careful, if the test has already reached the previous number when you change it, the test will already be finished.
Adds a comment message to the results.
ObjectA piece of data that will be stored in the results.
The comment can be any object. If a non-string object is passed, a summary of the object and it's field values will be written to the output. If an Error is passed, it's stack trace will also be written out.
Adds an error message to the results.
ObjectA piece of data that will be stored in the results.
This works almost exactly as
comment, with two primary differences.
First, any errors added to the results causes the test to fail, even
if all subtests passed. Second, the output of the
error will be formatted to stand out more than a comment.
Runs an asynchronous function as part of the test. Normally, this does not need to be called, as it is called automatically by the test method. However, there are cases where it is useful to run it separately from the 'test' method, or even run more than one body on the same test.
If you wish to run multiple bodies on the same test, keep in mind that there is no guarantee of order of which body will run first, and they are all run asynchronously, no matter how synchronous the code inside them is. Test.finish should only be called once, and only when the last body to run is finished.
FunctionFunction body for the asynchronous test.
Returns true if the test was successful, false if not.
A test is successful only if it meets all of the following criteria:
- It is completed (
- It finished normally (no parameter was passed to
- None of it's subtests have failed.
- No error messages were added.
addExpectedwas never called, or the number of tests declared is the same as what was expected tests have been declared.
Returns true if the test is completed, false if not.
A test has completed only if it meets all of the following criteria:
finishhas been called
- All subtests have completed.
addExpectedwas never called, or all expected tests have been declared.
Simple-Test-Framework was designed with a "Keep It Simple" philosophy. The specifics of that philosophy include the following ideals:
- Write Lightweight Code: The less code I write in my test framework, the less potential side effects it can have on your test.
- Avoid Dependencies: The less I depend on other modules, the less I have to worry about those modules breaking and thus breaking this.
- Minimize the API: Too many functions to use makes it more difficult to learn, more difficult to write tests, and less likely you will write good tests.
- Keep Out Feature-creep: The more stuff I put in here that isn't needed for testing, the more complex the code, and the more potential side effects it can have on your test. I almost didn't write the code that wrote the results out to stream because of this.
- Don't Skimp On Goal-Oriented Features: Just because it's a feature doesn't mean it's a creep. I can't really use the library for what it's designed for without the feature, then I probably do want it. This is why I did end up adding code to write out the results.
The MIT License (MIT) Copyright (c) 2013 Neil M. Sheldon Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.