run-verify

Proper test verifications

$ npm install --save-dev run-verify

Table of Content

• run-verify

  • Table of Content
  • expect Test Verifications
    • Verifying Events and callbacks without Promise
    • Verifying with Promisification
    • Verifying with run-verify
      • Using runVerify with done
      • Using Promisified asyncVerify
  • Verifying Expected Failures
    • Verifying Failures with callbacks
    • Verifying Failures with Promise
    • Verifying Failures with run-verify

• checkFunc

  • 0 Parameter
  • 1 Parameter
  • 2 Parameters

• APIs

  • runVerify
  • asyncVerify
  • runFinally
  • runTimeout
  • runDefer
  • wrapCheck
  • wrapCheck decorators and shortcuts
    • expectError
    • expectErrorHas
    • expectErrorToBe
    • withCallback
    • onFailVerify
  • wrapVerify
  • wrapAsyncVerify

• License

• run-verify

  • Table of Content
  • expect Test Verifications
    • Verifying Events and callbacks without Promise
    • Verifying with Promisification
    • Verifying with run-verify
      • Using runVerify with done
      • Using Promisified asyncVerify
  • Verifying Expected Failures
    • Verifying Failures with callbacks
    • Verifying Failures with Promise
    • Verifying Failures with run-verify

• checkFunc

  • 0 Parameter
  • 1 Parameter
  • 2 Parameters

• APIs

  • runVerify
  • asyncVerify
  • runFinally
  • runTimeout
  • runDefer
  • wrapCheck
  • wrapCheck decorators and shortcuts
    • expectError
    • expectErrorHas
    • expectErrorToBe
    • withCallback
    • onFailVerify
  • wrapVerify
  • wrapAsyncVerify

• License

expect Test Verifications

Verifying Events and callbacks without Promise

For test runner that doesn't have built-in expect utility, if not all code/libraries you use are promisified, then expect in a test that involves async events doesn't work well.

For example, the expect failure below would be out of band as an UncaughtException and the test runner can't catch and report it normally:

it("should emit an event", done => {
  foo.on("event", data => {
    expect(data).to.equal("expected value");
    done();
  });
});

test runners generally watch for uncaught errors, but it doesn't always work well and the stack trace may be all confusing.

See below for discussions on some common patterns for writing tests that need to verify results from async events and callbacks, and how run-verify helps with them.

The first and obvious solution is you need to enclose verifications in try/catch:

it("should emit an event", done => {
  foo.on("event", data => {
    try {
      expect(data).to.equal("expected value");
      done();
    } catch (err) {
      done(err);
    }
  });
});

However, it gets very messy like callback hell when a test deals with multiple async events.

Even with a test runner that takes a Promise as a return result, the same thing must be done:

it("should emit an event", () => {
  return new Promise((resolve, reject) => {
    foo.on("event", data => {
      try {
        expect(data).to.equal("expected value");
        resolve();
      } catch (err) {
        reject(err);
      }
    });
  });
});

Verifying with Promisification

The test verification can be written nicely with promisification like:

const promisifiedFooEvent() => new Promise(resolve => foo.on("event", resolve));

So the verification is now like this:

it("should emit an event", done => {
  return promisifiedFooEvent()
    .then(data => {
      expect(data).to.equal("expected value");
    })
    .then(done)
    .catch(done);
});

The .then(done).catch(done) can be avoided if the test runner takes a Promise as return result:

it("should emit an event", () => {
  return promisifiedFooEvent().then(data => {
    expect(data).to.equal("expected value");
  });
});

It's even nicer if async/await is supported:

it("should emit an event", async () => {
  const data = await promisifiedFooEvent();
  expect(data).to.equal("expected value");
});

Verifying with run-verify

But if you prefer not to wrap with promisification or facing a complex scenario, run-verify always allows you to write test verification nicely:

Using runVerify with done

Using runVerify if you are using the done callback from the test runner:

const { runVerify } = require("run-verify");

it("should emit an event", done => {
  runVerify(
    next => foo.on("event", next),
    data => expect(data).to.equal("expected value"),
    done
  );
});

Using Promisified asyncVerify

Using asyncVerify if you are returning a Promise to the test runner:

const { asyncVerify } = require("run-verify");

it("should emit an event", () => {
  return asyncVerify(
    next => foo.on("event", next),
    data => expect(data).to.equal("expected value")
  );
});

Verifying Expected Failures

When you need to verify that a function actually throws an error, you can do:

it("should throw", () => {
  expect(() => foo("bad input")).to.throw("bad input passed");
});

However, this gets a bit trickier for async functions which can invoke callback or reject with an error.

See below for some common patterns on how to verify async functions return errors and how run-verify helps.

Verifying Failures with callbacks

it("should invoke callback with error", done => {
  foo("bad input", err => {
    if (err) {
      try {
        expect(err.message).includes("bad input passed");
        done();
      } catch (err2) {
        done(err2);
      }
    }
  });
});

Verifying Failures with Promise

For promise it is tricky, but the pattern I commonly use is to have a .catch that saves the expect error and then verify it in a .then:

it("should reject", () => {
  let error;
  return promisifiedFoo("bad input")
    .catch(err => {
      error = err;
    })
    .then(() => {
      expect(error).to.exist;
      expect(error.message).includes("bad input passed");
    });
});

With async/await, it can be done very nicely using try/catch:

it("should reject", async () => {
  try {
    await promisifiedFoo("bad input");
    throw new Error("expected rejection");
  } catch (err) {
    expect(err.message).includes("bad input passed");
  }
});

Verifying Failures with run-verify

run-verify has an expectError decorator to mark a checkFunc is expecting to return or throw an error:

Example that uses a done callback from the test runner:

const { expectError, runVerify } = require("run-verify");

it("should invoke callback with error", done => {
  runVerify(
    expectError(next => foo("bad input", next)),
    err => expect(err.message).includes("bad input passed"),
    done
  );
});

Example that returns a Promise to the test runner:

const { expectError, asyncVerify } = require("run-verify");

it("should invoke callback with error", () => {
  return asyncVerify(
    expectError(next => foo("bad input", next)),
    err => expect(err.message).includes("bad input passed")
  );
});

Example when everything is promisified:

const { expectError, asyncVerify } = require("run-verify");

it("should invoke callback with error", () => {
  return asyncVerify(
    expectError(() => promisifiedFoo("bad input")),
    err => expect(err.message).includes("bad input passed")
  );
});

checkFunc

runVerify takes a list of functions as checkFunc to be invoked serially to run the test verification.

Each checkFunc can take 0, 1, or 2 parameters.

0 Parameter

() => {};

• Assume to be a sync function
• But if it's intended to be async, then it should return a Promise
  • The Promise's resolved value is passed to next checkFunc.

1 Parameter

(next | result) => {};

With only 1 parameter, it gets ambiguous whether it wants a next callback or a sync/Promise function taking a result.

runVerify does the following to disambiguate the checkFunc's single parameter:

• It's expected to be the next callback if:
  • the parameter name starts with one of the following:
    • next, cb, callback, or done
    • The name check is case insensitive
  • The function is decorated with the withCallback decorator
• Otherwise it's expected to take the result from previous checkFunc
  • And its behavior is treated the same as the 0 parameter checkFunc
• A native AsyncFunction is always expected to take the result and returns a Promise.

ie:

async result => {};

2 Parameters

(result, next) => {};

This is always treated as an async function taking the result and a next callback:

• result - result from previous checkFunc
• next - callback to invoke the next checkFunc

APIs

runVerify

runVerify(...checkFuncs, done);

The main API, params:

name	description
checkFuncs	variadic list of functions to invoke to run tests and verifications
done	done(err, result) callback after verification is done or failed

• See details about checkFunc.

Each checkFunc is invoked serially, with the result from one passed to the next, depending on its parameters.

done is invoked at the end, but if any checkFunc fails, then done is invoked immediately with the error.

asyncVerify

asyncVerify(...checkFuncs);

The promisified version of runVerify. Returns a Promise.

Make sure no done callback is passed as the last parameter.

runFinally

runFinally(finallyFunc);

Create a callback that's always called.

• The finally callback can return a Promise.
• If any of them throws or rejects, then done is called with the error.
• They can appear in any order and there can be multiple of them.

ie:

runVerify(
  runFinally(() => {}),
  () => {
    // test code
    return "foo";
  },
  runFinally(() => return promiseCleanup()),
  result => {
    // expect result === "foo
  },
  done
)

runTimeout

runTimeout(ms);

Set a timeout in ms milliseconds for the test.

You can have multiple of these but only the last one has effect.

example:

const { asyncVerify, runTimeout } = require("run-verify");

it("should verify events", () => {
  return asyncVerify(
    runTimeout(50),
    next => foo.on("event1", msg => next(null, msg)),
    msg => expect(msg).equal("ok"),
    runTimeout(20),
    next => bar.on("event2", msg => next(null, msg)),
    msg => expect(msg).equal("done")
  );
});

runDefer

runDefer([ms]);

Create a defer object for waiting on events.

• ms - optional timeout in ms milliseconds for this defer.

Returns: the defer object with these methods:

• resolve(result) - resolve the defer object: resolve("OK").
• reject(error) - reject with error: reject(new Error("fail")).
• wait([ms]) - Wait for the defer object.
• clear() - Put resolved defer back into pending status.

NOTES:

• All registered defer must resolve for the test to complete.
• Any rejection not waited on will fail the test immediately.
• You can decorate wait with expectError.

example:

Explicitly wait on the defer objects:

const { asyncVerify, runDefer } = require("run-verify");

it("should verify events", () => {
  const defer = runDefer();
  const defer2 = runDefer();

  return asyncVerify(
    () => foo.on("event1", msg => defer.resolve(msg)),
    // explicitly wait for defer before continuing with the test
    defer.wait(50),
    msg => expect(msg).equal("ok"),
    () => bar.on("event2", msg => defer2.resolve(msg)),
    // explicitly wait for defer before continuing with the test
    defer2.wait(20),
    msg => expect(msg).equal("done")
  );
});

Just put defer anywhere as long as they resolve:

const { asyncVerify, runDefer } = require("run-verify");

it("should verify events", () => {
  const defer = runDefer();
  const defer2 = runDefer();

  return asyncVerify(
    // just telling runVerify that there are two defer events that must
    // resolve for the test to finish, but you can't verify on their results.
    defer,
    defer2,
    () => foo.on("event1", msg => defer.resolve(msg)),
    () => bar.on("event2", msg => defer2.resolve(msg))
  );
});

wrapCheck

wrapCheck(checkFunc);

Wrap a checkFunc with these decorators:

• expectError, expectErrorHas, expectErrorToBe
• withCallback
• onFailVerify

For example:

runVerify(wrapCheck(next => foo("bad input", next)).expectError.withCallback, done);

wrapCheck decorators and shortcuts

expectError

expectError(checkFunc);

Shortcut for:

wrapCheck(checkFunc).expectError;

Decorate a checkFunc expecting to throw or return Error. Its error will be passed to the next checkFunc.

This uses wrapCheck internally so withCallback is also available after:

expectError(() => {}).withCallback;

expectErrorHas

expectErrorHas(checkFunc, msg);

Shortcut for:

wrapCheck(checkFunc).expectErrorHas(msg);

Decorate a checkFunc expecting to throw or return Error with message containing msg. Its error will be passed to the next checkFunc.

expectErrorToBe

expectErrorToBe(checkFunc, msg);

Shortcut for:

wrapCheck(checkFunc).expectErrorToBe(msg);

Decorate a checkFunc expecting to throw or return Error with message to be msg. Its error will be passed to the next checkFunc.

withCallback

withCallback(checkFunc);

Shortcut for:

wrapCheck(checkFunc).withCallback;

Decorate a checkFunc that takes a single parameter to expect a next callback for that parameter.

This uses wrapCheck internally so expectError is also available after:

withCallback(() => {}).expectError;

onFailVerify

onFailVerify(checkFunc);

Shortcut for:

wrapCheck(checkFunc).onFailVerify;

Decorate a checkFunc that will be called with err if the checkFunc right before it failed.

It's skipped if the checkFunc right before it passed.

• Its returned value will be ignored.
• Any exceptions from it will be caught and used as the new error for failing the test.

Example:

return asyncVerify(
  () => {
    throw new Error("oops");
  },
  onFailVerify(err => {
    console.log("test failed with", err);
  })
);

wrapVerify

wrapVerify(...checkFuncs, done);

• Returns a function that wraps runVerify.
• The new function takes a single parameter and pass it to the first checkFunc.

wrapAsyncVerify

wrapAsyncVerify(...checkFuncs);

The promisified version of wrapVerify

License

Licensed under the Apache License, Version 2.0

---