It starts an internal web server, then starts a set of web browsers, makes them execute the tests, and finally writes test reports.
- Supports multiple browser instances to run tests in parallel
- Includes instrumentation for code coverage with node-coverage
- Supports PhantomJS for fully headless tests
- Compatible with most other web browsers
- Test results output formats:
- Json file
- JUnit-style single XML file
- JUnit-style set of files, format accepted by Sonar
- Code coverage output formats:
- Supports Aria Templates unit tests and Mocha with either Chai or Expect.js.
- Adding support for other test frameworks is as simple as adding an adapter for that test framework.
Dependencies & installation
If you want to make use of PhantomJS headless testing, you'll additionally need to install PhantomJS and make sure it's in your
Installing PhantomJS on Windows
- Download phantomjs-1.9.8-windows.zip and extract it.
- Move the contents of
- Check that it works by issuing
Installing PhantomJS on Ubuntu
Quick setup on 64bit Ubuntu:
cd /usr/local/share/ sudo wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-1.9.8-linux-x86_64.tar.bz2 sudo tar jxvf phantomjs-1.9.8-linux-x86_64.tar.bz2 sudo ln -s /usr/local/share/phantomjs-1.9.8-linux-x86_64/ /usr/local/share/phantomjs sudo ln -s /usr/local/share/phantomjs/bin/phantomjs /usr/local/bin/phantomjs
If you have 32bit version, replace
i686 in the commands above.
Check that it works by issuing
phantomjs --version in console.
attester [<options>] [<configuration file>]
Here is a sample
.yml configuration file with explanations about each setting:
resources: # Specifies which files will be accessible through the web server:'/': # This configures resources for the root of the web server:- 'src/main/js' # It is possible to specify one or more directories for each path.- 'src/tests' # When requesting a file, the first directory which contains it is used.'/aria': # This configures resources for the /aria path on the server:- 'libraries/aria'tests:# Describes test configuration for each type of test# Only Aria Templates tests are supported currently, but other types of tests will be added in the futurearia-templates:# There are two ways to specify which tests have to be executed:# either by their classpath or by their file path# It is possible to combine both.# With file paths, it is possible to use patterns.classpaths:includes:- MainTestSuiteexcludes:- test.sample.MyUnfinishedTestbrowserExcludes: # Allows to specify some excluded classpaths specific to each browserPhantomJS: # Each browser listed in browserExcludes must exactly match the name of a browser in the browsers section- test.sample.MyTestFailingInPhantomJSfiles:rootDirectory: 'src/tests'includes:- 'test/example/*TestCase.js'excludes:- 'test/example/*SpecialTestCase.js'browserExcludes: # Allows to specify some exclusion patterns specific to each browserPhantomJS: # Each browser listed in browserExcludes must exactly match the name of a browser in the browsers section- 'test/example/*NoPhantomTestCase.js'bootstrap : '/aria/bootstrap.js' # Path to the bootstrap file of Aria Templates. This is the default value.extraScripts: # Path to extra scripts to be inserted in the test page just after the Aria Templates bootstrap file.- '/test/testEnvConfig.js'rootFolderPath : '/' # Root folder path (Aria.rootFolderPath variable) This is the default value.debug : true # Enables or disables Aria Templates debug mode (Aria.debug variable). This is the default value.memCheckMode : true # Enables or disables memory check mode (Aria.memCheckMode variable). This is the default value.mocha:files:includes:- 'test/example/*.js'excludes:- 'test/example/*.conf.js'browserExcludes: # Allows to specify some exclusion patterns specific to each browserPhantomJS: # Each browser listed in browserExcludes must exactly match the name of a browser in the browsers section- 'test/example/notWorkingInPhantomJS/*.js'ui: 'bdd' # User-interface (bdd|tdd|exports). Defaults to bddignoreLeaks: false # Optional to ignore global variable leaksglobals: # Optional, allow the given list of global [names]- 'myGlobal'assertion: 'expect' # Assertion library to be included. It could be either expect or chai or any external urlcoverage:files:# If the files section is present, the files specified here will be instrumented by attester for code coverage.rootDirectory: 'src/main/js'includes:- '**/*.js'# Files can be pre-instrumented by node-coverage or the NodeCoverageInstrument visitor of atpackager# In that case, information file(s) produced by the instrumentation tool should be specified in the infoFiles array:infoFiles:- 'coverage-instrumentation.json'test-reports: # Path for each test report type:json-file: 'report.json'xml-file: 'report.xml'xml-directory: 'reports'coverage-reports: # Path for each coverage test report type:json-file: 'coverage.json'lcov-file: 'coverage.lcov'browsers:# If the browsers section is not present, attester will not make any difference between browsers# and run each test only once (in whatever browser is connected)# However, if the browsers section is present, each test will be run once in each browser listed here.# (and browsers not listed here will have nothing to do if they are connected)- 'PhantomJS'- 'Opera'# It's possible to distinguish browsers by operating systems, read more below- 'Safari on Mac OS X'- 'Chrome on Windows 7'- 'Chrome on Ubuntu'- 'Chrome on Android'# It is also possible to distinguish several versions of the same browser:- 'IE 7'- 'IE 8'- 'IE 9'- 'IE 10'# Browser version can be also a semver-compliant string; see- 'Firefox 3.6'- 'Firefox >=20'# In the logs, browser will be identified using the name, version and OS as provided. If you want to change that, add an alias at the end using 'as':- 'Chrome 28 as Chrome Stable 28'- 'Chrome 30 as Chrome Canary 30'# You can mix all of that options, and have whitespace as you see fit.- 'Firefox >=25 on Windows 7 as Firefox/Windows'- 'Firefox >=25 on Ubuntu as Firefox/Ubuntu'# You can also add flags with the 'with' keyword to specify an environment which cannot be automatically detected.# Flags have to be explicitly set in the URL when connecting a browser (e.g.: ).- 'IE 11 on Windows with JAWS as IE-JAWS'- 'IE 11 on Windows with mySpecialFlag'
Attester uses ua-parser-js for browser and operating system detection and supports values as returned by ua-parser-js.
It is possible to build a special portion of the configuration object from an external file using
attester --env package.json
This puts the content of
package.json inside the
env section of attester configuration. It is then possible to reference such values with simple templates.
<%= prop.subprop %> Expands to the value of
prop.subprop in the config. Such templates can only be used inside string values, either in arrays or objects
The configuration above generates a coverage report file called
x.y.z is the value taken from
package.json or any other file passed referenced by
Template options can be used both in
json file and the environment file can be of any of the two format.
--phantomjs-path <path> Path to the PhantomJS executable. Attester does not install PhantomJS automatically. The path to the PhantomJS executable can be specified either in the PHANTOMJS_PATH environment variable, or through this command line option. If none are specified, the
phantomjs binary should be in your
PATH for Attester to find it.
--phantomjs-instances <number> Number of instances of PhantomJS to start (default:
Additionally, a string
auto can be passed to let the program use the optimal number of instances for best performance (max. 1 per CPU thread).
--run-browser <path> Path to any browser executable to execute the tests. Can be repeated multiple times to start multiple
browsers or multiple instances of the same browser. Each browser is started with one parameter: the URL to open to start tests.
At the end of the tests, all started processes are killed.
--launcher-config <launcher config> Path to an attester-launcher configuration file. Browsers configured in this file are automatically started and stopped when needed. Can be repeated multiple times to pass multiple configuration files to attester-launcher.
--env <path> Path to a
json file containing environment options. See section above.
--ignore-errors When enabled, test errors (not including failures) will not cause this process to return a non-zero value.
--ignore-failures When enabled, test failures (anticipated errors) will not cause this process to return a non-zero value.
--live-results When enabled (which is the default, disable with --no-live-results), it is possible to connect to attester
to get live test results. Note that, when enabled, this feature keeps the full log of test results in memory, which increases
the memory consumption of attester.
--host <host> Host used for the internal web server.
--port <number> Port used for the internal web server. If set to
0, an available port is automatically selected.
--public-host <host> Public host of the internal web server, which, when defined, is used instead of --host to create the URL to be called from browsers.
--public-port <number> Public port of the internal web server, which, when defined, is used instead of --port to create the URL to be called from browsers.
--server-only Only starts the web server, and configure it for the test campaign but do not start the campaign. This is useful to
run tests manually.
--max-task-restarts <number> Maximum number of times a task can be restarted after being interrupted by a browser disconnection (default: 5).
--task-restart-on-failure When enabled, failing tasks are also automatically restarted as if the browser was disconnected.
This option is especially useful for test campaigns which are not very stable, containing tests which can randomly fail (even if, of course,
having randomly failing tests in a test suite is not recommended!).
--log-level <number> Level of logging: integer from
0 (no logging) to
--colors Uses colors (disable with --no-colors).
--help Displays a help message and exits.
--version Displays the version number and exits.
--json-console When enabled, JSON objects will be sent to
stdout to provide information about tests.
This is used by the junit bridge.
--heartbeats Delay (in ms) for heartbeats messages sent when
--json-console is enabled. Use
0 to disable them.
--task-timeout Maximum duration (in ms) for a test task.
--shutdown-on-campaign-end <boolean> When set to false, will not shut the server down nor exit the process upon finishing the campaign.
Might be useful for debugging.
--predictable-urls <boolean> If true, resources served by the campaign have predictable URLs (campaign1, campaign2...). Otherwise, the campaign part in the URL is campaign ID.
Configuration file options
Any option configurable through the configuration file can also be specified from the command line with the
For example, to configure resources, it is possible to use:
attester --config.resources./ path1/root1 --config.resources./ path2/root2 --config.resources./aria path/to/aria
It's important to understand exactly what is served by attester to a connected browser.
A slave browser receives a page with a single test and the files necessary to run it.
The test page has access to anything that is listed in
From the configuration file
tests is used to build a list of tests for each testing framework
resources simply defines what is accessible by the page, none of these files is automatically added to the page
test there are two parameters that deal with files
filesused to generate the list of test files
extraScriptsresources to be included in the test page
It's worth noticing that
extraScripts must be an absolute path from the server root (what is configured in the resources).
Let's see one example
The block above configures attester to serve any file inside
src folder from the server root, and files from
mocha test. When dispatching them to connected browsers, each will receive a page containing
- an attester adapter to mocha tests
- a single test file
Including source files it's up to the test that can use script loaders like noder, curl, head.js or similar. Using a script loader allows to include in the tested page only what's strictly necessary for the test.
It's also possible to configure attester to include other scripts then the test through
The paths specified in
extraScripts are resources URL. In the example above
sourceCode.js must be physically located inside
The files specified in
extraScripts are included in each and every test.