Table of contents
- API howto
As always, we support both Node.js (CommonJS/
require as well as ES modules/
import) and browser (ES modules/
import as well as IIFE for
$ npm install --save ebisu-js
Then, in your code, if you use CommonJS and
var ebisu = require('ebisu-js');
If you use ES modules and
import, do this instead:
import * as ebisu from 'ebisu-js';
If you use TypeScript, the above will just work.
ebisu.min.js available on your webserver and load it in a
script tag (5.6 KB uncompressed, 3 KB after gzip), then in your HTML:
This makes the
ebisu object available in the top-level. (I also recommend you include the sourcemap to help debugging in the browser.)
If you want to avoid polluting your global namespace with this variable, you can use the minified ES module: make
ebisu.min.mjs available on your webserver (and ideally its sourcemap), and in your HTML:
<script type="module"> import * as ebisu from './ebisu.min.mjs'; </script>
The above files are ES6+ and support modern browsers, only because I'm lazy and ESbuild only supports ES6+. If you need to support older browsers, please get in touch and I'll be happy to make an ES5-compatible build.
Let’s start working immediately with code and we’ll explain as we go.
It’s important to know that Ebisu is a very narrowly-scoped library: it aims to answer just two questions:
- given a set of facts that a student is learning, which is the most (or least) likely to be forgotten?
- After the student is quizzed on one of these facts, how does the result get incorporated into Ebisu’s model of that fact’s memory strength?
Ebisu doesn’t concern itself with what these facts are, what they mean, nor does it handle storing the results of reviews. The external quiz app, at a minimum, stores a probability model with each fact’s memory strength, and it is this model that Ebisu transforms into predictions about recall probability or into new models after a quiz occurs.
Create a default model to assign newly-learned facts:
var defaultModel = ebisu.defaultModel(24); // Also ok: `ebisu.defaultModel(24, 4)` or even `ebisu.defaultModel(24, 4, 4)`. console.log(defaultModel);
This returns a three-element array of numbers: we’ll call them
[a, b, t].
These three numbers describe the probability distribution on a fact’s recall probability. Specifically, they say that,
24 hours after review, we believe this fact’s recall probability will have a
Beta(a, b) = Beta(4, 4) distribution, whose histogram looks like this, for a hundred thousand samples:
The above distribution only applies 24 hours after learning the fact. Ebisu can transform this distribution given any time horizon though. That is, given you think the recall probability follows
Beta(4, 4) a day after a fact is learned, here are the recall probability after just six hours, and after four days:
You can also tune
ebisu.defaultModel(24, 4) will explicitly initialize
b to 4. In words, the lower the
a=b, the less sure you are that
t=24 is the halflife. In pictures, here are the histograms for 1.5, 4, and 12:
I recommend using
a=b around 2-4: this is loose and allows the quiz data to aggressively guide the scheduling. You don't want to go lower than 1 because
a=b=1 is the flat distribution: you have no confidence that 24 hours is the halflife, and the math gets weird.
We use the Beta distribution, and not some other probability distribution on numbers between 0 and 1, for statistical reasons that are indicated in depth in the Ebisu math writeup.
This should give you some insight into what those three numbers,
[4, 4, 24] mean, and why you might want to customize them—you might want the half-life to be just two hours instead of a whole day, in which case you’d set
Predict current recall probability:
Given a set of models for facts that the student has learned, you can ask Ebisu to predict each fact’s recall probability by passing in its model and the currently elapsed time since that fact was last reviewed or quizzed via
var model = ebisu.defaultModel(24); var elapsed = 1; // hours elapsed since the fact was last seen var predictedRecall = ebisu.predictRecall(model, elapsed, true); console.log(predictedRecall);
This function efficiently calculates the mean of the histogram of recall probabilities in the interactive demo above (but it uses math, not histograms!). It's a bit faster if you let the third argument be
false, in which case this function returns log-probabilities.
In either case, a quiz app can call this function on each fact to find which fact is most in danger of being forgotten—that’s the one with the lowest predicted recall probability.
Update a recall probability model given a quiz result:
Suppose your quiz app has chosen a fact to review, and tests the student. It's time to update the model with the quiz results. Ebisu supports a rich set of quiz types:
- of course we support the binary quiz, i.e., pass/fail.
- We also support Duolingo-style quizzes where the student gets, e.g., 2 points out of a max of 3. This is called the binomial case (and of course plain binary quizzes are a special case of the binomial with a max of 1 point).
- The most complex quiz type we support is called the noisy-binary quiz and lets you separate the actual quiz result (a pass/fail) with whether the student actually remembers the fact, by specifying two independent numbers:
Probability(passed quiz | actually remembers), or $q_1$ in the derivation below, is the probability that, assuming the student actually remembers the fact, they got the quiz right? This should be 1.0 (100%), especially if your app is nice and lets students change their grade (typos, etc.), but might be less if your app doesn’t allow this. Second, you can specify
Probability(passed quiz | actually forgot), or $q_0$ that is, given the student actually forgot the fact, what’s the probability they passed the quiz? This might be greater than zero if, for example, you provided multiple-choice quizzes and the student only remembered the answer because they recognized it in the list of choices. Or consider a foreign language reader app where users can read texts and click on words they don’t remember: imagine they read a sentence without clicking on any words—you’d like to be able to model the situation where, if you actually quizzed them on one of the words, they would fail the quiz, but all you know is they didn’t click on a word to see its definition.
updateRecall function handles all these cases. It wants an Ebisu model (output by
defaultModel for example), the number of
successes out of
total points, and the time
elapsed. Let's illustrate the simple binary case here:
var model = ebisu.defaultModel(24); var successes = 1; var total = 1; var elapsed = 10; var newModel = ebisu.updateRecall(model, successes, total, elapsed); console.log(newModel);
The new model is a new 3-array with a new
[a, b, t]. The Bayesian update magic happens inside here: see here for the gory math details.
For the plain binary and binomial cases,
successes is an integer between 0 and
For the noisy-binary case,
total must be 1 and
successes can be a float going from 0 to 1 (inclusive).
successes < 0.5, the quiz is taken as a failure, whereas if
successes > 0.5, it's taken as a success.
Probability(passed quiz | actually remembers)is called
q1in the Ebisu math derivation and is taken to be
max(successes, 1-successes). That is, if
successesis =0.1 or 0.9, this conditional probability
q1is the same, 0.9.
- The other probability
Probability(passed quiz | actually forgot)is called
q0in the Ebisu derivation and defaults to
1-q1, but can be customized: it's passed in as another argument after the elapsed time.
The following code snippet illustrates how to use the default
q0 and how to specify it:
var model = ebisu.defaultModel(24); var successes = 0.95; var total = 1; var elapsed = 96; var updated1 = ebisu.updateRecall(model, successes, total, elapsed); // default q0 var q0 = 0.2; var updated2 = ebisu.updateRecall(model, successes, total, elapsed, q0); // compare halflives of these two cases console.log([updated1, updated2].map(m => ebisu.modelToPercentileDecay(m))) // [ 34.76462629898456, 28.024103424692004 ]
Both updates model a successful quiz with
q1 = 0.95. But the first defaulted
q0 to the complement of
q1, i.e., 0.05, as plausible. The second explicitly set a higher
q0. The result can be seen in the halflife of the resultant models: 35 hours versus 28 hours.
This code snippet also illustrates another function in the API, which we look at next.
Model to halflife:
Sometimes it's useful to convert an Ebisu model (an
[a, b, t] array) into the halflife it represents. We did this above to compare the result of choosing different
q0 for the soft-binary case.
ebisu.modelToPercentileDecay accepts a model and optionally a percentile, and uses a golden section root finder to find the time needed for the model's recall probability to decay to that percentile:
var model = ebisu.defaultModel(24); console.log(ebisu.modelToPercentileDecay(model)); // 23.999553931988203 console.log(ebisu.modelToPercentileDecay(model, 0.1)); // 99.331489589545 console.log(ebisu.modelToPercentileDecay(model, 0.9)); // 3.375025317798656
Manual halflife override:
It happens. You initialized a model and you updated it with some quizzes, but your initial halflife was wrong. Your student tells your quiz app that it's just not the right halflife, and they want to see this fact more or less frequently. Ebisu gives you a function to accurately deal with this:
ebisu.rescaleHalflife takes an
[a, b, t] model and a
scale argument, a positive number, and returns a new model with the same probability distribution on recall probability but scaled to
t * scale.
The following two code snippets let you say "I want to see this fact twice as often" versus "I'm seeing this fact ten times too often":
var model = ebisu.defaultModel(24); // I forgot this fact! Its halflife is half what I thought: var newModel = ebisu.rescaleHalflife(model, 0.5); // I know this fact! Its halflife is ten times what you think var newModel2 = ebisu.rescaleHalflife(model, 10);
That’s it! That’s the entire API:
ebisu.defaultModel(t: number, a = 4.0, b = a): Modelif you can’t bother to create a 3-array (a
Model). The units of
tare the units of all other time inputs—you decide if you want to deal with hours, days, etc.
ebisu.predictRecall(prior: Model, tnow: number, exact = false): numberreturns the recall log-probability given a model and the time elapsed since the last review or quiz. If
exact, you get linear probability (between 0 and 1) instead of log-probability (-∞ to 0).
ebisu.updateRecall(prior: Model, successes: number, total: number, tnow: number, q0?: number): Modelto update the model after a quiz session with
totalstatistically-independent trials exercising the fact, and time after its last review. See above for the noisy-binary case where
0 <= successes <= 1is a float and optionally
q0. Returns a new model.
Two bonus functions:
ebisu.modelToPercentileDecay(model: Model, percentile = 0.5, tolerance = 1e-4): numberto find the half-life (time for recall probability to decay to 50%), or actually, any percentile-life (time for recall probability to decay to any percentile).
tolerancetunes how accurate you want the result to be.
ebisu.rescaleHalflife(prior: Model, scale = 1): Modelwill return a new
Modelwith the halflife scaled by
This is a TypeScript library. For a one-shot compile, run
npm run compile. You can also run the TypeScript compiler in watch mode with
npm run compile-watch.
tape for tests: after compiling, run
npm test. This consumes
test.json, which came from the Ebisu Python reference implementation.
We use ESbuild to create CommonJS (for Node
require), ES modules (for Node and browsers'
import), and an IIFE (for browsers'
npm run build will generate all three.
The version of this repo matches the Python reference’s version up to minor rev (i.e., Python Ebisu 1.0.x will match Ebisu.js 1.0.y). See the Python Ebisu changelog.
rescaleHalflife, and changes to
updateRecall so that it always rebalances.
I use gamma.js, one of substack’s very lightweight and very useful modules.
We also use this fine golden section minimization routine from the wonderful Scijs package.