confidencejs

A light-weight JavaScript library to help you make sense of your A/B test results.

Confidence.js

Confidence.js is a light-weight JavaScript library to help you make sense of your A/B Test results. Given an A/B test (or "Split Test") result set, Confidence.js will tell you if a statistical "winner" can be determined.

Download confidence.js here

Include confidence.js in your HTML.

<script src="path/to/confidence.js"></script>

You're all ready! Start testing... with confidence.

We have many examples in the API Reference. Here's a simple one to get you started:

// Create a new test result 
var myConfidence = new Confidence();
 
// Add variant A results 
myConfidence.addVariant({
  id: 'A',
  name: 'Variant A',
  conversionCount: 2500,
  eventCount: 3000
});
 
// Add variant B results 
myConfidence.addVariant({
  id: 'B',
  name: 'Variant B',
  conversionCount: 1500,
  eventCount: 3000
});
 
// Get results, using Z-Test method 
zTestResult = myConfidence.getResult();
/*
{
  hasWinner: true,
  hasEnoughData: true,
  winnerID: 'A',
  winnerName: 'Variant A',
  confidencePercent: 95.00,
  confidenceInterval: { min: 82, max: 84.67 },
  readable: 'With 95% confidence, the true population
    parameter of the "Variant A" variant will
    fall between 82% and 84.67%.'
}
*/

Confidence.js provides two methods for evaluating an A/B test:

  • The Z-Test Method
  • Chi Square Test and Marascuillo's Procedure

Selecting an appropriate method depends on your data set and use case - both have advantages.

Learn how each method works here.

npm install
npm test
  • implement "Confidence to beat baseline" comparison (in progress)
  • add removeVariant function

Found a bug? Create an issue here on GitHub!

For general questions, tweet me @jessicaraygun.

Developed and maintained by Jessica Thomas, Data Scientist @ sendwithus.com, with guidance and tutelage from Statistics Mastermind Emily Malcolm.