Eden Testing Framework
This is a small testing framework for programs that behave as pure functions, allowing you to define these tests in plain markdown format. As a result, your tests become highly accessible to the public.
What do you mean with "pure functions"?
We mean that, like an algrebraic mathematical function, the program's output is
fully defined by its input. In other words, if you provide your program with the
foo and it outputs
baz, you can be assured that it will always output
What are examples of programs that behave as pure functions?
Good examples of programs that behave as pure functions are parsers and lexers. Yet other, more advanced examples include evaluators that evaluate mathematical functions. To accomodate testing these kinds of programs, this project was started.
How does it work?
The idea is to re-use as much of the already existing tools as possible.
Therefore, test cases are just shell commands that recieve as input some data
and output some data, which is then compared with an expected value. This way,
the test commands are nothing more than
string -> string-functions. It is up
to you to write a small wrapper around whatever it is that you want to test
that provides such a function.
npm i -g ete
$PATH is set up correctly, you can run your tests by simply running
ete in the shell. It will automatically scan for a
test directory and add
all test files to it. The full list of flags and commands that are supported
can be consulted with
There are several example test suites that show you what this library is capable of. They can be found in the test/ directory.
Defining Your Tests
First note that you can put plain Markdown text as you would normally do, and it won't interfere with the tests. Use this to your advantage to create descriptive tests that clearly tell the user what it is that is being tested.
Specifying a Test Command
First of all, you need to explicitly tell
ete how to run the tests. This is
done by creating a header titled
Command somewhere at the top of your file.
It should contain a code block with
shell specified as the language with the
command that you'd like to run, like so:
## Command This is the command that will be used to run tests in this test suite: ```shell my-parser --read-from-stdin ```
If your command to run differs depending on the platform that the user is
ete from, then you can replace
shell with the name of the specific
shell that is available on that platform. Currently, you can choose from the
Adding Test Cases
You define a new test by placing some specific code
block under a
separate heading. You can nest headings however you like.
ete is smart enough
to only take the title of the header closest to the code block as name of the
test. Next, you will want to define a subheader with the text
ete to compare and diff against any code block that appears
Here's an example of a test case that could be used by an s-expression parser
wrapped with a program that dumps the AST as
## Can parse some s-expressions ```scheme foo bar baz (one two three) 'a-symbol 'another-long-symbol ``` ### Result ``` SYMBOL foo SYMBOL bar SYMBOL baz PAIR SYMBOL one PAIR SYMBOL two PAIR SYMBOL three NULL QUOTE SYMBOL a-symbol QUOTE SYMBOL another-long-symbol ```
ete is more of a proof-of-concept than a rigorous testing
framework. Much remains to be done. Here are some ideas:
- Supports other methods of comparison other than ordinary string equality
- Correct handling of multiple code blocks under the same header
- Grouping test cases under their common header (?)
- Programmatic API
Copyright 2017 Sam Vervaeck
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.