Noble Programming Mantra

    perturb
    TypeScript icon, indicating that this package has built-in type declarations

    0.3.1 • Public • Published

    perturb Build Status experimental

    pərˈtərb

    verb

    1. make (someone) anxious or unsettled.

      "I'm perturbed about the quality of our unit tests."

    2. subject (a system, moving object, or process) to an influence tending to alter its normal or regular state or path.

      "This library perturbs our source code to check unit test strength."

    Installation

    For global command line use

    npm i -g perturb

    Within a project for programmatic use or for package.json scripts

    npm i perturb

    About

    perturb is a mutation testing framework for JavaScript projects. It helps determine the quality of unit tests.

    "Mutation testing is used to design new software tests and evaluate the quality of existing software tests. Mutation testing involves modifying a program in small ways." Source

    Perturb takes your source code, parses the AST, generates mutations, and runs your test suite against them. If all your tests pass when run against a mutant, you probably missed a test.

    Mutation testing is different from and generally more comprehensive than code coverage metrics. Unit tests are one way way of specifying the behavior of software. If a mutation is not covered by a unit test, then that aspect of the program is unspecified.

    Stability Disclaimer

    This project is in rapid development, and should be considered experimental. It "works" -- sort of. This repository contains two working examples. First, I pulled in the Node.js core EventEmitter library plus its associated tests. If you clone the library and run make example-events you'll note that the unit tests on that library have a lot of blind spots (308 mutations covered of 466 total as of this writing). Second, you can run make dogfood to run perturb on it's own source code. Frankly, the actual test coverage of this library is pretty poor, and it's still riddled with bugs.

    Usage

    At this very moment, the library is basically unusable, since it's npm installation is broken. Cloning it and running it directly should (mostly) work.

    This library is intended (eventually) to be used primarily by the command line. However, the exiting CLI modules are badly out of date with the core library, so you need to call the library directly. Luckily this is pretty easy -- the CLI will just provide an interface for constructing the root configuration object, which is all the main perturb function needs.

    const perturb = require("perturb");
    cosnt config = { /* ... */ };
    perturb(config).then(function (results) {
      // ...
    });

    Configuration parameters will be documented (eventually), but there is enough to get going in script/run.js and src/make-config.ts. The trickiest thing will be figuring out how to get perturb to correctly match up a source file with it's test file(s). The simplest possible arrangement is a single directory with all the source files and another single directory with all the test files, where their internal structure mirrors one another. This library's tests are written that way. If your code fits that pattern it should be relatively simple to get up and running. More advanced matching patterns are available, but aren't documented at this point.

    Keywords

    none

    Install

    npm i perturb

    DownloadsWeekly Downloads

    23

    Version

    0.3.1

    License

    MIT

    Unpacked Size

    625 kB

    Total Files

    265

    Last publish

    Collaborators

    • nickbottomley