It starts an internal web server, then starts a set of web browsers, makes them execute the tests, and finally writes test reports.
- Supports multiple browser instances to run tests in parallel
- Includes instrumentation for code coverage with node-coverage
- Supports PhantomJS for fully headless tests
- Compatible with most other web browsers
- Test results output formats:
- Json file
- JUnit-style single XML file
- JUnit-style set of files, format accepted by Sonar
- Code coverage output formats:
- Supports Aria Templates unit tests and Mocha with either Chai or Expect.js.
- Adding support for other test frameworks is as simple as adding an adapter for that test framework.
If you want to make use of PhantomJS headless testing, you'll additionally need to install PhantomJS and make sure it's in your
- Download phantomjs-1.9.1-windows.zip and extract it.
- Move the contents of
- Check that it works by issuing
Quick setup on 64bit Ubuntu:
cd /usr/local/share/ sudo wget http://phantomjs.googlecode.com/files/phantomjs-1.9.1-linux-x86_64.tar.bz2 sudo tar jxvf phantomjs-1.9.1-linux-x86_64.tar.bz2 sudo ln -s /usr/local/share/phantomjs-1.9.1-linux-x86_64/ /usr/local/share/phantomjs sudo ln -s /usr/local/share/phantomjs/bin/phantomjs /usr/local/bin/phantomjs
If you have 32bit version, replace
i686 in the commands above.
Check that it works by issuing
phantomjs --version in console.
attester [<options>] [<configuration file>]
Here is a sample
.yml configuration file with explanations about each setting:
resources: # Specifies which files will be accessible through the web server:'/': # This configures resources for the root of the web server:- 'src/main/js' # It is possible to specify one or more directories for each path.- 'src/tests' # When requesting a file, the first directory which contains it is used.'/aria': # This configures resources for the /aria path on the server:- 'libraries/aria'tests:# Describes test configuration for each type of test# Only Aria Templates tests are supported currently, but other types of tests will be added in the futurearia-templates:# There are two ways to specify which tests have to be executed:# either by their classpath or by their file path# It is possible to combine both.# With file paths, it is possible to use patterns.classpaths:includes:- MainTestSuiteexcludes:- test.sample.MyUnfinishedTestfiles:rootDirectory: 'src/tests'includes:- 'test/example/*TestCase.js'excludes:- 'test/example/*SpecialTestCase.js'bootstrap : '/aria/bootstrap.js' # Path to the bootstrap file of Aria Templates. This is the default value.extraScripts: # Path to extra scripts to be inserted in the test page just after the Aria Templates bootstrap file.- '/test/testEnvConfig.js'rootFolderPath : '/' # Root folder path (Aria.rootFolderPath variable) This is the default value.debug : true # Enables or disables Aria Templates debug mode (Aria.debug variable). This is the default value.memCheckMode : true # Enables or disables memory check mode (Aria.memCheckMode variable). This is the default value.mocha:files:includes:- 'test/example/*.js'excludes:- 'test/example/*.conf.js'ui: 'bdd' # User-interface (bdd|tdd|exports). Defaults to bddignoreLeaks: false # Optional to ignore global variable leaksglobals: # Optional, allow the given list of global [names]- 'myGlobal'assertion: 'expect' # Assertion library to be included. It could be either expect or chai or any external urlcoverage:files: # Specifies which files will be instrumented for code coveragerootDirectory: 'src/main/js'includes:- '**/*.js'test-reports: # Path for each test report type:json-file: 'report.json'xml-file: 'report.xml'xml-directory: 'reports'coverage-reports: # Path for each coverage test report type:json-file: 'coverage.json'lcov-file: 'coverage.lcov'browsers:# If the browsers section is not present, attester will not make any difference between browsers# and run each test only once (in whatever browser is connected)# However, if the browsers section is present, each test will be run once in each browser listed here.# (and browsers not listed here will have nothing to do if they are connected)- 'PhantomJS'- 'Opera'# It's possible to distinguish browsers by operating systems, read more below- 'Safari on Mac OS X'- 'Chrome on Windows 7'- 'Chrome on Desktop Linux'- 'Chrome on Android'# It is also possible to distinguish several versions of the same browser:- 'IE 7'- 'IE 8'- 'IE 9'- 'IE 10'# Browser version can be also a semver-compliant string; see- 'Firefox 3.6'- 'Firefox >=20'# In the logs, browser will be identified using the name, version and OS as provided. If you want to change that, add an alias at the end using 'as':- 'Chrome 28 as Chrome Stable 28'- 'Chrome 30 as Chrome Canary 30'# You can mix all of that options, and have whitespace as you see fit.- 'Firefox >=25 on Windows 7 as Firefox/Windows'- 'Firefox >=25 on Ubuntu as Firefox/Ubuntu'
Attester uses ua-parser for that and supports operating system families as returned by ua-parser, e.g.:
- 'Windows XP'
- 'Windows Vista'
- 'Windows 7'
- 'Windows 8'
- 'Mac OS X'
- 'Chrome OS'
- 'Windows Phone'
- 'Firefox OS'
- 'BlackBerry OS'
For convenience, two additional values are accepted by Attester:
- 'Desktop Linux' (Linux, but not Android or webOS)
- 'Desktop Windows' (Windows, but not Windows Phone)
It is possible to build a special portion of the configuration object from an external file using
attester --env package.json
This puts the content of
package.json inside the
env section of attester configuration. It is then possible to reference such values with simple templates.
<%= prop.subprop %> Expands to the value of
prop.subprop in the config. Such templates can only be used inside string values, either in arrays or objects
The configuration above generates a coverage report file called
x.y.z is the value taken from
package.json or any other file passed referenced by
Template options can be used both in
json file and the environment file can be of any of the two format.
--phantomjs-path <path> Path to the PhantomJS executable (default:
- Important note: you have to install PhantomJS by yourself if you want to use it. Then, you must either
--phantomjs-pathargument, or add the folder containing
phantomjs.exeto your system's
PATH. Otherwise, you'll get runtime errors.
--phantomjs-instances <number> Number of instances of PhantomJS to start (default:
Additionally, a string
auto can be passed to let the program use the optimal number of instances for best performance (max. 1 per CPU thread).
--robot-browser "<browser name>" Name of the browser to automatically start with the Selenium Java Robot.
It can be:
--run-browser <path> Path to any browser executable to execute the tests. Can be repeated multiple times to start multiple
browsers or multiple instances of the same browser. Each browser is started with one parameter: the URL to open to start tests.
At the end of the tests, all started processes are killed.
--env <path> Path to a
json file containing environment options. See section above.
--ignore-errors When enabled, test errors (not including failures) will not cause this process to return a non-zero value.
--ignore-failures When enabled, test failures (anticipated errors) will not cause this process to return a non-zero value.
--live-results When enabled (which is the default, disable with --no-live-results), it is possible to connect to attester
to get live test results. Note that, when enabled, this feature keeps the full log of test results in memory, which increases
the memory consumption of attester.
--port <number> Port used for the internal web server. If set to
0 (default), an available port is automatically selected.
--server-only Only starts the web server, and configure it for the test campaign but do not start the campaign. This is useful to
run tests manually.
--max-task-restarts <number> Maximum number of times a task can be restarted after being interrupted by a browser disconnection (default: 5).
--task-restart-on-failure When enabled, failing tasks are also automatically restarted as if the browser was disconnected.
This option is especially useful for test campaigns which are not very stable, containing tests which can randomly fail (even if, of course,
having randomly failing tests in a test suite is not recommended!).
--log-level <number> Level of logging: integer from
0 (no logging) to
--colors Uses colors (disable with --no-colors).
--help Displays a help message and exits.
--version Displays the version number and exits.
--json-console When enabled, JSON objects will be sent to
stdout to provide information about tests.
This is used by the junit bridge.
--heartbeats Delay (in ms) for heartbeats messages sent when
--json-console is enabled. Use
0 to disable them.
--task-timeout Maximum duration (in ms) for a test task.
--shutdown-on-campaign-end <boolean> When set to false, will not shut the server down nor exit the process upon finishing the campaign.
Might be useful for debugging.
--predictable-urls <boolean> If true, resources served by the campaign have predictable URLs (campaign1, campaign2...). Otherwise, the campaign part in the URL is campaign ID.
Configuration file options
Any option configurable through the configuration file can also be specified from the command line with the
For example, to configure resources, it is possible to use:
attester --config.resources./ path1/root1 --config.resources./ path2/root2 --config.resources./aria path/to/aria
It's important to understand exactly what is served by attester to a connected browser.
A slave browser receives a page with a single test and the files necessary to run it.
The test page has access to anything that is listed in
From the configuration file
tests is used to build a list of tests for each testing framework
resources simply defines what is accessible by the page, none of these files is automatically added to the page
test there are two parameters that deal with files
filesused to generate the list of test files
extraScriptsresources to be included in the test page
It's worth noticing that
extraScripts must be an absolute path from the server root (what is configured in the resources).
Let's see one example
The block above configures attester to serve any file inside
src folder from the server root, and files from
mocha test. When dispatching them to connected browsers, each will receive a page containing
- an attester adapter to mocha tests
- a single test file
Including source files it's up to the test that can use script loaders like noder, curl, head.js or similar. Using a script loader allows to include in the tested page only what's strictly necessary for the test.
It's also possible to configure attester to include other scripts then the test through
The paths specified in
extraScripts are resources URL. In the example above
sourceCode.js must be physically located inside
The files specified in
extraScripts are included in each and every test.