tap14.1.7 • Public • Published
A TAP test framework for Node.js.
Just wanna see some code? Get started!
- Getting started guide
- Built-in test coverage
- Many reporter formats
- Extensive API featuring:
- Command-line interface for running tests (whether they use node-tap or not)
All this is too much to manage in a single README file, so head over to the website to learn more.
Why should you use this thing!? LET ME TELL YOU!
Most frameworks spend a lot of their documentation telling you why they're the greatest. I'm not going to do that.
tutti i gusti sono gusti
Software testing is a software and user experience design challenge that balances on the intersection of many conflicting demands.
Node-tap is based on my opinions about how a test framework should work, and what it should let you do. I do not have any opinion about whether or not you share those opinions. If you do share them, you will probably enjoy this test library.
Test files should be "normal" programs that can be run directly.
That means that it can't require a special runner that puts magic functions into a global space.
Test output should be connected to the structure of the test file that is easy to determine.
That means not unnecessarily deferring test functions until
nextTick, because that would shift the order of
console.logoutput. Synchronous tests should be synchronous.
Test files should be run in separate processes.
That means that it can't use
require()to load test files. Doing
node ./test.jsmust be the exact same sort of environment for the test as doing
test-runner ./test.js. Doing
node test/1.js; node test/2.jsshould be equivalent (from the test's point of view) to doing
test-runner test/*.js. This prevents tests from becoming implicitly dependent on one anothers' globals.
Assertions should not normally throw (but throws MUST be handled nicely).
I frequently write programs that have many hundreds of assertions based on some list of test cases. If the first failure throws, then I don't know if I've failed 100 tests or 1, without wrapping everything in a try-catch. Furthermore, I usually want to see some kind of output or reporting to verify that each one actually ran.
Basically, it should be your decision whether you want to throw or not. The test framework shouldn't force that on you, and should make either case easy.
Test reporting should be separate from the test process, included in the framework, and enabled by default for humans.
The raw test output should be machine-parseable and human-intelligible, and a separate process should consume test output and turn it into a pretty summarized report. This means that test data can be stored and parsed later, dug into for additional details, and so on. Also: nyan cat.
Writing tests should be easy, maybe even fun.
The lower the barrier to entry for writing new tests, the more tests get written. That means that there should be a relatively small vocabulary of actions that I need to remember as a test author. There is no benefit to having a distinction between a "suite" and a "subtest". Fancy DSLs are pretty, but more to remember.
That being said, if you return a Promise, or use a DSL that throws a decorated error, then the test framework should Just Work in a way that helps a human being understand the situation.
Tests should output enough data to diagnose a failure, and no more or less.
Stack traces pointing at JS internals or the guts of the test framework itself are not helpful. A test framework is a serious UX challenge, and should be treated with care.
Test coverage should be included.
Running tests with coverage changes the way that you think about your programs, and provides much deeper insight. Node-tap bundles nyc for this.
It's not enabled by default only because it does necessarily change the nature of the environment a little bit. But I strongly encourage enabling coverage.
Tests should be output in a predictable order.
Even if they are run in parallel, the test output should be consistent.
As of version 10, tap supports parallel tests, which can make your tests run significantly faster if they are I/O bound or if you have multiple cores on your machine. However, even when run in parallel, the output is still serialized.
Tests should not require more building than your code.
Babel and Webpack are lovely and fine. But if your code doesn't require compilation, then I think your tests shouldn't either. Tap is extremely promise-aware, but works on any version of Node.js back to v4.x.
Software testing should help you build software. It should be a security blanket and a quality ratchet, giving you the support to undertake massive refactoring and fix bugs without worrying. It shouldn't be a purification rite or a hazing ritual.
There are many opinions left off of this list! Reasonable people can disagree. But if you find yourself nodding along, maybe tap is for you.