Serial Jasmine

A simple module which allows you to run jasmine tests serially.

Jasmine normally runs spec files in parallel. If you have a use case where you would like to ensure that the files are run one at a time, in sequence, you can use serial-jasmine.

Normal usage is from the command line but it can also be used programatically which allows you to capture information about the tests if you want to further process that information. This is useful if you want to create a dashboard for checking health status.

The module exposes one method, runTasks, which returns a promise. That promise is resolved when all tests are complete. It does not reject if a test fails. You can determine the status of an individual test by examining the data in the resolved promise.

Because of a deficiency in jasmine, jasmine.execute cannot be called multiple times. It does not clean up after itself and residual information from previous calls corrupts subsequent runs. In order to overcome this we fork a new process for each test. The implications of that are that it requires about 30 msec to spin up the process and uses about 10mb of memory. Therefor this module is not designed to run 1000's of tests sequentially.

Note: The module uses generators so works in node Argon or greater or with node v.12+ with the --harmony flag

Installation

npm install serial-jasmine --save-dev (or -g if you are going to run from command line)

Usage##

1. from the command line (requires a global install):

serial-jasmine [--reporter <path to custom reporter>] [--sj ] [--silent] [--failures] -- <path to test files may be a glob>

1. reporter - we will use require(reporter). If that returns an object we add it as a reporter. If it returns a function we invoke the function with no argumrents. That function should obviously return an object that provides the methods jasmine expects. See jasmine documentation for details.
2. sj if present we use the default serial-jasmine reporter. If absent we use the jasmine default.
3. silent - suppress console output. Really makes no sense from command line but is allowed for completeness
4. failures - suppress information about passing tests. If all tests pass only the name of the test file is output.

NOTE: The -- before the file list is not a typo. If you don't add it your first test file will be assigned to one of the flags which I am sure you don't want.

If you just want the default jasmine reporter but want you tests run serially just use this:

serial-jasmine <path to test files>

Programatically##

The parameters for runTasks:

1. taskFiles - the files to test may be a glob.
2. The reporter - usually null for programatic unless you want things consoled out
3. saveData - collects data from tests and returns it. Should be true in any use case I can imagine.
4. silent - suppress all console output. Defaults to false
5. failures - only shows failure information. Ignored if silent is true.

var sj=require("serial-jasmine');
var taskFiles=["./test/foo.spec.js","./test/bar.spec.js"];
sj.runTasks(taskFiles,null,true,true,false)
    .then((results)=>{
   		//Do what you want with data
    })
    .catch((err)=>console.log(err));

The parameters for runTasks:

1. taskFiles - array of file names of task files.
2. reporter -- path to a custom reporter use null to use jasmine default
3. saveData - if true information on tests will be returned in resolved promise. (See information below).

If saveData is true and no custom reporter is provided the serial-jasmine default reporter will be used. If saveData is true and a custom reporter is provided that reporter will be used If saveData is false and no customer reporter is provided the default jasmine reporter will be used but no useful information will be returned on promise resolution. I have trouble imagining why you would be doing things programatically and not want the data back though.

Sample output (command line usage)

$> serial-jasmine --sj -- serial-jasmine/spec/jasmine_examples/PlayerSpec.js 
File: spec/jasmine_examples/PlayerSpec.js
        Suite: Player
                Test: should be able to play a Song Status: failed
                        Reason: TypeError: expect(...).toBePlaying is not a function
                        Stack: at Object.<anonymous> (/Volumes/DataDrive/donnievbitbucket/serial-jasmine/spec/jasmine_examples/Play
                Test: tells the current song if the user has made it a favorite Status: passed
                Suite: when song has been paused
                        Test: should indicate that the song is currently paused Status: failed
                                Reason: TypeError: expect(...).not.toBePlaying is not a function
                                Stack: at Object.<anonymous> (/Volumes/DataDrive/donnievbitbucket/serial-jasmine/spec/jasmine_examples/Play
                        Test: should be possible to resume Status: passed
                Suite: #resume
                        Test: should throw an exception if song is already playing Status: passed

        
$> serial-jasmine  serial-jasmine/spec/jasmine_examples/PlayerSpec.js 
Started
FF...

Failures:
1) Player should be able to play a Song
  Message:
    TypeError: expect(...).toBePlaying is not a function
  Stack:
    TypeError: expect(...).toBePlaying is not a function
        at Object.<anonymous> (/Volumes/DataDrive/donnievbitbucket/serial-jasmine/spec/jasmine_examples/PlayerSpec.js:17:20)

2) Player when song has been paused should indicate that the song is currently paused
  Message:
    TypeError: expect(...).not.toBePlaying is not a function
  Stack:
    TypeError: expect(...).not.toBePlaying is not a function
        at Object.<anonymous> (/Volumes/DataDrive/donnievbitbucket/serial-jasmine/spec/jasmine_examples/PlayerSpec.js:30:26)

5 specs, 2 failures
Finished in 0.016 seconds

Data returned on promise resolution

A hash is returned with each test file being a key. For each file a hash is constructed with two keys, success and results. Success should always be true because that just signifies that running the test (which is promise based succeeded. It has nothing to do with whether the tests in the file where successful. The results hash contains information returned by jasmine's event emitter as the test proceeds. See the jasmine documentation for more information.

Each test file has a hash with two attributes:

1. the file name
2. an array of suites.

A suite is a hash with attributes:

1. suite: the suite name
2. parentSuite: For non nested suites this will be null
3. suiteInfo: the information returned by jasmine
4. nestedSuites: an array of subsuites
5. tests: an array of tests

Each test is a hash with the attributes:

1. test: the test name
2. suite: the suite it is in.
3. testInfo: the information returned by jasmine

{ 'test/testBasic3.spec.js': 
   { success: true,
     results: 
      { file: 'test/testBasic3.spec.js',
        suites: 
         [ { suite: 'third test Suite',
             parentSuite: null,
             suiteInfo: 
              { id: 'suite1',
                description: 'third test Suite',
                fullName: 'third test Suite',
                failedExpectations: [],
                status: 'finished' },
             nestedSuites: [],
             tests: 
              [ { test: 'third test',
                  suite: 'third test Suite',
                  testInfo: 
                   { id: 'spec0',
                     description: 'third test',
                     fullName: 'third test Suite third test',
                     failedExpectations: 
                      [ { matcherName: 'toBe',
                          message: 'Expected true to be false.',
                          stack: 'Error: Expected true to be false.\n    at stack (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1482:17)\n    at buildExpectationResult (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1452:14)\n    at Spec.Env.expectationResultFactory (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:583:18)\n    at Spec.addExpectationResult (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:324:34)\n    at Expectation.addExpectationResult (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:533:21)\n    at Expectation.toBe (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1406:12)\n    at Object.<anonymous> (/Volumes/DataDrive/donnievbitbucket/serial-jasmine/test/testBasic3.spec.js:8:16)\n    at attemptAsync (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1819:24)\n    at QueueRunner.run (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1774:9)\n    at QueueRunner.execute (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1762:10)',
                          passed: false,
                          expected: false,
                          actual: true },
                        { matcherName: 'toEqual',
                          message: 'Expected \'one\' to equal \'seven\'.',
                          stack: 'Error: Expected \'one\' to equal \'seven\'.\n    at stack (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1482:17)\n    at buildExpectationResult (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1452:14)\n    at Spec.Env.expectationResultFactory (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:583:18)\n    at Spec.addExpectationResult (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:324:34)\n    at Expectation.addExpectationResult (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:533:21)\n    at Expectation.toEqual (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1406:12)\n    at Object.<anonymous> (/Volumes/DataDrive/donnievbitbucket/serial-jasmine/test/testBasic3.spec.js:9:17)\n    at attemptAsync (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1819:24)\n    at QueueRunner.run (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1774:9)\n    at QueueRunner.execute (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1762:10)',
                          passed: false,
                          expected: 'seven',
                          actual: 'one' } ],
                     passedExpectations: [],
                     pendingReason: '',
                     status: 'failed' 
                   } 
                } 
              ] 
           } 
         ]
      }
   },
  'test/testBasic.spec.js': 
   { success: true,
     results: 
      { file: 'test/testBasic.spec.js',
        suites: 
         [ { suite: 'firstSuite',
             parentSuite: null,
             suiteInfo: 
              { id: 'suite1',
                description: 'firstSuite',
                fullName: 'firstSuite',
                failedExpectations: [],
                status: 'finished' },
             nestedSuites: 
              [ { suite: 'Nested Suite',
                  parentSuite: 'firstSuite',
                  suiteInfo: 
                   { id: 'suite2',
                     description: 'Nested Suite',
                     fullName: 'firstSuite Nested Suite',
                     failedExpectations: [],
                     status: 'finished' },
                  nestedSuites: 
                   [ { suite: 'Nested Nested Suite',
                       parentSuite: 'Nested Suite',
                       suiteInfo: 
                        { id: 'suite3',
                          description: 'Nested Nested Suite',
                          fullName: 'firstSuite Nested Suite Nested Nested Suite',
                          failedExpectations: [],
                          status: 'finished' },
                       nestedSuites: [],
                       tests: 
                        [ { test: 'Nested nested suite test',
                            suite: 'Nested Nested Suite',
                            testInfo: 
                             { id: 'spec2',
                               description: 'Nested nested suite test',
                               fullName: 'firstSuite Nested Suite Nested Nested Suite Nested nested suite test',
                               failedExpectations: [],
                               passedExpectations: 
                                [ { matcherName: 'toBe',
                                    message: 'Passed.',
                                    stack: '',
                                    passed: true },
                                  { matcherName: 'toEqual',
                                    message: 'Passed.',
                                    stack: '',
                                    passed: true } ],
                               pendingReason: '',
                               status: 'passed' 
                             } 
                          }
                        ] 
                     } 
                   ],
                  tests: 
                   [ { test: 'first test nested suite',
                       suite: 'Nested Suite',
                       testInfo: 
                        { id: 'spec1',
                          description: 'first test nested suite',
                          fullName: 'firstSuite Nested Suite first test nested suite',
                          failedExpectations: [],
                          passedExpectations: 
                           [ { matcherName: 'toBe',
                               message: 'Passed.',
                               stack: '',
                               passed: true },
                             { matcherName: 'toEqual',
                               message: 'Passed.',
                               stack: '',
                               passed: true } ],
                          pendingReason: '',
                          status: 'passed' 
                        } 
                     } 
                   ] 
                 } 
              ],
             tests: 
              [ { test: 'firstTest',
                  suite: 'firstSuite',
                  testInfo: 
                   { id: 'spec0',
                     description: 'firstTest',
                     fullName: 'firstSuite firstTest',
                     failedExpectations: [],
                     passedExpectations: 
                      [ { matcherName: 'toBe',
                          message: 'Passed.',
                          stack: '',
                          passed: true },
                        { matcherName: 'toEqual',
                          message: 'Passed.',
                          stack: '',
                          passed: true } ],
                     pendingReason: '',
                     status: 'passed' 
                   } 
                } 
              ] 
           },
           { suite: 'Second Suite',
             parentSuite: null,
             suiteInfo: 
              { id: 'suite4',
                description: 'Second Suite',
                fullName: 'Second Suite',
                failedExpectations: [],
                status: 'finished' },
             nestedSuites: [],
             tests: 
              [ { test: 'second suite test',
                  suite: 'Second Suite',
                  testInfo: 
                   { id: 'spec3',
                     description: 'second suite test',
                     fullName: 'Second Suite second suite test',
                     failedExpectations: [],
                     passedExpectations: 
                      [ { matcherName: 'toBe',
                          message: 'Passed.',
                          stack: '',
                          passed: true },
                        { matcherName: 'toEqual',
                          message: 'Passed.',
                          stack: '',
                          passed: true } ],
                     pendingReason: '',
                     status: 'passed' 
                   } 
                 } 
              ] 
            }
         ]
      }
   },
  'test/testBasic2.spec.js': 
   { success: true,
     results: 
      { file: 'test/testBasic2.spec.js',
        suites: 
         [ { suite: 'second test',
             parentSuite: null,
             suiteInfo: 
              { id: 'suite1',
                description: 'second test',
                fullName: 'second test',
                failedExpectations: [],
                status: 'finished' },
             nestedSuites: [],
             tests: 
              [ { test: 'secondtestIt',
                  suite: 'second test',
                  testInfo: 
                   { id: 'spec0',
                     description: 'secondtestIt',
                     fullName: 'second test secondtestIt',
                     failedExpectations: 
                      [ { matcherName: 'toBe',
                          message: 'Expected true to be false.',
                          stack: 'Error: Expected true to be false.\n    at stack (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1482:17)\n    at buildExpectationResult (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1452:14)\n    at Spec.Env.expectationResultFactory (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:583:18)\n    at Spec.addExpectationResult (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:324:34)\n    at Expectation.addExpectationResult (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:533:21)\n    at Expectation.toBe (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1406:12)\n    at Object.<anonymous> (/Volumes/DataDrive/donnievbitbucket/serial-jasmine/test/testBasic2.spec.js:8:16)\n    at attemptAsync (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1819:24)\n    at QueueRunner.run (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1774:9)\n    at QueueRunner.execute (/Users/donvawter/.nvm/versions/node/v4.2.1/lib/node_modules/serial-jasmine/node_modules/jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1762:10)',
                          passed: false,
                          expected: false,
                          actual: true } ],
                     passedExpectations: [],
                     pendingReason: '',
                     status: 'failed' 
                   }
                 } 
              ] 
           } 
         ] 
      } 
   } 
}