Gagarin is a mocha based testing framework designed to be used with Meteor. It can spawn multiple instances of your meteor application and run the tests by executing commands on both server and client in realtime. In other words, Gagarin allows you to automate everything you could achieve manually with
meteor shell, browser console and a lot of free time. There's no magic. It just works.
Gagarin is also useful when you need more refined control over the meteor processes and test fancy things, e.g. the behavior of your app on server restarts or when you have multiple app instances writing to the same database. To our knowledge, this is currently not achievable with Velocity - the official Meteor testing framework.
First, install the cli-tool with
npm install -g gagarin
Put your tests in
tests/gagarin/ directory, e.g.
Finally, in your project root directory run:
We recommend running tests in
verbose mode, at least before
gagarin --help if you need more options.
It is possible to use more advanced directory structures. You can, for example, place
your test files in local package folders, without registering them in
See Directory structure for more details.
Gagarin is still under heavy development and a new release is published almost
every week. Some parts of the API change over time and we can't guarantee backward compatibility before we reach
1.0.0. Minor version changes may contain breaking changes. Though, we put a lot of effort to reduce the risk of breaking old tests. Gagarin has it's own test suite with more than 250 test cases, which BTW are very good source of examples.
0.4.10 gagarin can also run on windows platform. The only additional requirement is that it has to be run within the elevated prompt. This is because during the application build process we need to create a symbolic link to
which requires administrative rights. We are working hard to find a decent workaround (look here). Any help and suggestions are welcome.
Compatibility with various node versions
Gagarin should play nicely with node
0.12.x. On the other hand, there are known compatibility issues with
0.11.x so we don't recommend using that particular version. It only applies to the cli-tool though. Your meteor application will always be run with the
node from the development bundle corresponding to your current meteor release. Please keep this in mind if you are using any kind of continuous integration system, because it basically means that the appropriate version of meteor dev-tools will need to be downloaded before the tests can be run.
server object created by
meteor() helper no longer has the
location property. To make sure the
browser starts on the proper location, you need to pass
server as the first argument, so
var server = ;var client = ; // before 0.4.0 you would use server.location here
How is it different from Velocity?
Gagarin is external to meteor. It only takes care of spawning your meteor processes and allows you to execute chunks of source code in your app environment from within your test suite and that's it. On the other hand, Velocity will deeply integrate with your app by making your test cases an integral part of your app source code, but only in a special type of builds called mirrors. This is very clever because your tests will run as fast as they possibly can. The only drawback of using velocity is that you don't have a satisfactory control over your meteor processes. In most situations this is acceptable but there are some very specific scenarios when this is not sufficient. In those cases Gagarin is probably a good choice. Gagarin tests will run a little bit slower because the source code is send to your app through DDP, but in most situations in which you would need Gagarin, this is totally acceptable because the bottleneck of your tests speed is usually somewhere else. Another advantage of using DDP is that the tests can be potentially executed on a deployed application which may be useful in some specific cases.
How is it different from Laika?
The truth is Gagarin originates from Laika. In some sense, one may think of it as Laika 2.0, though it's not backward compatible. The main advantages of using Gagarin rather then Laika are the following:
- it does not depend on
- it does not depend on injected source code, so the test runner does not have to rebuild your app each time you run the tests
- the communication with client is done through a real web driver API, which means that your tests can visit any web page and are not bound to your app's routes
- it does not depend on external mongo processes; the tests runner is clever enough to find mongo executable within your meteor development bundle
Gagarin is a simple test runner built on top of mocha. In it's essence is very similar to laika, though it's much more flexible, up-to-date and compatible with the latest Meteor versions. Currently it's implemented as a custom
mocha interface, which simply extends the standard
bdd ui. This may change in the future if there's a demand to support other testing frameworks.
Gagarin consists of two parts:
gagarin npm module and
anti:gagarin meteor package.
The first one should be installed globally on your system, while the second one should be
added to your meteor application. In order to work properly, the versions of the two guys
The minimal setup
Please start by installing the cli tool:
npm install -g gagarin
If you try to run
gagarin command in your meteor project directory you should receive an error message telling that there are no tests to run. Let's fix it by creating a dummy test in
This time, everything should work fine and your test should pass. Please note that prior to running
the tests scenarios Gagarin builds your application as well. Should the build fail
you will be notified accordingly. In case of problems, it's always good to try
gagarin --verbose mode for better insight.
Gagarin will always look for your test scenarios in
tests/gagarin/ directory. To alter this behavior pass a custom path as the first parameter, e.g.
For more details see Directory structure and
What about the
If you forgot to add it manually,
gagarin cli-tool will make sure to add the right version to your project.
If the dummy test passed you should notice that indeed the
is listed in
The role of the smart package is adding some backdoor functionality, similar to
meteor shell, for testing purposes. But don't worry - it's only activate when
GAGARIN_SETTINGS environment variable is present. For safety, double check it's not there in your production environment.
While we advise to place your tests in
tests/gagarin to maintain compatibility
with other (Velocity) testing frameworks, you're free to create a directory structure
that fits your needs. Gagarin uses node-glob
to find your test files based on a glob pattern.
By default, gagarin assumes you have placed your files inside
providing a glob as second argument, more advanced path structures are possible.
Example: tests under tests/gagarin
Example: tests directly under ./tests
Example: package only structure
Let's say you maintain a 'package only' app, and would like to place your tests
under the package they are testing. So your app lacks a global
and looks somewhat similar to:
client server packages blog-acl tests authorized.js guests.js blog-comments tests comments.js
To run all tests found in local packages, you can provide a glob pattern like:
Or if you have nested folders inside the
Example: multiple extensions
Perhaps you have both
.js files and
.coffee files that you wish to run?
Example: global tests folder combined with package specific tests We can imagine you're having both global tests, and package specific tests in their own package directory. No problem, just run gagarin with a glob like:
gagarin *pattern* --verbose if you need to find out witch tests files are
being found by your pattern.
Some shells may expand wildcards. This will make
gagarin load only the first
test file matching the pattern. If you experience this problem,
please wrap your pattern within
Writing simple tests
The simplest possible test suite may look like this:
In the above example
meteor is a global function provided by the framework, which you can use to spawn new meteor instances.For your convenience we've also exposed
expect from the good old chai.
Please note that the function passed to the
server.execute routine is the only part that
is executed within the server environment. The rest of the code is totally external to your application.
This technique has a lot of advantages but it also have one major drawback. The functions which are passed
to the server do not share their scope with the other ones. In particular, the following code
a = 0;;
will throw an "undefined variable" error. Instead you should pass
a as an argument
Testing with browser
Gagarin makes it really easy to coordinate tests for client and server. For example
This idea originated from Laika, but we decided to go for a more promise-oriented API. Basically speaking, you can use the
browser function to spawn as many clients as you want. The only requirement is that you have a webdriver running somewhere. By default, gagarin will try to find webdriver at port
9515 (chromedriver default). You can customize the webdriver url by providing the corresponding option for the cli tool:
gagarin --webdriver http://localhost:9515
If you're testing locally, we recommend using chromedriver which can be downloaded from here. After unpacking the executable the only thing you need to do is to run it in the background. By default the process will listen on port
9515. This behavior can be altered by specifying the port explicitly
Other webdrivers can be used as well. If you plan to use phantomjs and GhostDriver please note that due to a BUG in GhostDriver all browser sessions will share the same cookie jar, which may be problematic in test scenarios when multiple concurrent users need to be created.
A part of webdriver API is exposed and ready to use within you client promise chain. For example:
The full list of supported methods is:
Additionally we've implemented a bunch of useful helpers, which you can use to simplify your tests.
Testing with Selenium WebDriver
We recommend using selenium
2.45.0 along with Firefox 36 or 34.
Please note that webdriver is broken in Firefox 35, so don't even try to use that one.
Also keep in mind that
selenium is usually much slower than
chromedriver so consider
using larger timeouts values when you switch to Firefox.
Lets assume that you have a copy of
selenium-server-standalone-*.jar available at
/path/to/selenium.jar. First start a selenium "hub" with the following command:
java -jar /path/to/selenium.jar
Selenium server should be listening on port
4444 by default.
Then run your Gagarin tests specifying
gagarin --webdriver http://localhost:4444/wd/hub
Please note the
/wd/hub suffix! If you only try to connect at port
4444 the webdriver will not respond.
We've been testing Gagarin with
chrome (38) and
firefox (34, 36).
At this moment we cannot guarantee it will work with other browsers.
There are a few issues which we are aware of and we are working hard to minimize their impact on the user experience.
Mongo throwing 100 error
Sometimes, when you interrupt your tests with
CTRL-C the test runner will fail to clean-up
the mongo process running in the background. If you see this error, first try to find the process
and kill it. If it does not help go to
.gagarin/local/db and delete the
Tests fail due to timeouts
Depending on the environment and the system performance,
the tests may require timeout values different from the default ones.
Please keep this in mind and see
gagarin --help for available customization options.
In case of problems, it's generally good to run tests in
to have a better picture of what's going on.
before all timeout
If you run your tests in
--verbose mode you would notice that this is happening
because sometimes webdriver fails to respond within a reasonable time window
when we create a new browser session. This may also depend on your system resources.
It's totally external to Gagarin, so the only thing we can do is to report
this error properly.
Since we don't have a comprehensive documentation yet, please consider the following set of simple examples as a current API reference.
Scope of the a local variable
It's good to keep in mind that the code which is intended to be executed on either server or client is passed as a string. Of course it does not have an immediate access to your local variable scope. In particular, things like:
var a = 1;;
will throw "a is undefined". Trying to set
a = 1; will throw as well because the code is implicitly executed in
strict mode, which does not allow introducing new variables to the global scope.
Passing arguments to client and server code
If you don't need to modify the variables within your "remote" code then probably the easiest way to overcome the problem described above is to pass local scope variables as arguments:
var a = 1;;
Note that this construction will already allow you to do anything you want with your local variables, because you can always update them within
then, after your client/server computation is done. However, it's not very convenient in more complicated scenarios.
To simplify the interaction between client and server code, we've added an affordance to reuse the declared closure in all three environments: test scope, server and client. To this end, you need to explicitly provide a list of variables to be synced as well as an accessor function:
var a = 1 b = 2 c = 0;// this is a hack :);
Now this code should work without problems:
The only reserved variable name for closures is
$, which you probably would not like to use for other reasons.
Asynchronous test cases
On both server and client you can also use asynchronous scripts:
The second argument to the
reject, so if you want to throw asynchronously:
If you want to pass additional variables to
promise method, do it like this:
Waiting for conditions
There's also a useful helper to wait for conditions. Again, it works on both server and client:
To test the package locally make sure that a webdriver is listening on port
then simply run the tests with the following command
in the project root directory. Additionally you can use
to display information about all possible options. For example, to use a different webdriver location, you can specify it with
./test.js --webdriver http://localhost:4444/wd/hub
For testing purposes it's sometimes useful to install a version of the npm package from a specific branch.
npm install -g anticoders/gagarin#develop
If you're developing Gagarin locally please remember that
anti:gagarin package is an integral
part of the testing framework and it has to be consistent with the
gagarin node module.
The easiest way to achieve this is to put a symbolic link inside the
within your project root, i.e.
path/to/your/project/packages/anti:gagarin -> path/to/repos/gagarin
Copyright (C) 2015 Tomasz Lenarcik, http://apendua.com