pickpick
An A/B testing engine
To use this engine, create one or more experiments and stick them in a container. Each experiment is composed of a bunch of variations which are randomly picked/served to you. Once you obtain a variation, use it to render some content or make some sort of a decision.
The container's job is to select the correct experiments for each visitor
based on each experiment's targeting expression. Thusly, to get a variation one would call pick()
twice, once on the container and once on the experiment.
Further information about the targeting experssion syntax can be found here: pickpick-targeting-compiler
Also, please take a look at our examples
example
npm i -S pickpick
Let's say we have a website with two pages buy
and index
and we want to run 3 experiments:
- on the
buy
page testcolor button
- on the
buy
page testprice
- on the
index
page testtext
const Experiment ExperimentContainer = // first create the experiments: let e1 = Experiment let e2 = Experiment let e3 = Experiment // now create a container:let experiments = e1 e2 e3let container = ExperimentContainer // simulate a visitor that needs a determination about which variation of which experiment he gets:let visitor = page: 'index' for let i = 0; i < 10; i++ let experiment = container if !experiment // no experiment that targets this user // handle this with defaults console else console let variation = experiment console
API
Table of Contents
Experiment
An A/B test experiment contains one or more variations and a definition of targeting.
Experiments are serializable and can be created using classes from this engine or object literals. For example:
const Experiment = const e1 = Experiment
Parameters
$0
Object$0.name
$0.id
$0.variations
(optional, default[]
)$0.targeting
(optional, defaultTargeting.default()
)$0.userData
pick
randomly select one variation from the Variations set
Returns Variant the value contained within the selected variation
match
check if this experiment matches the input targeting
Parameters
targeting
Object
Returns Boolean
add
add another variation to this experiment
Parameters
iterator
iterate over the variations contained in this experiment
Targeting
Targeting
Parameters
expression
String see pickpick-targeting-compiler for more detailsuserEnvironment
Object (optional, default{}
)
match
check if the input data is matched by this targeting instance
Parameters
inputTargeting
Object is normally a simple js object
Returns Boolean
expression
access this Targeting's expression
Returns String
iterator
iterate over the features that participate in the targeting
has
check if a feature is part of this targeting instance
Parameters
feature
String a name of a feature, e.ggeo
Returns Boolean
Variation
A variation attaches weight to a piece of data. Variations are used in Experiments and ExperimentContainers
Parameters
$0
Object$0.object
$0.weight
(optional, default1
)
ExperimentContainer
lib/ExperimentContainer.js:38-239
Contains one or more experiments and routes traffic evenly to each of them based on their targeting. The following is an example of using a container to host several experiments, pick on thats appropriate for a single visitor's targeting and then access a variation from the selected experiment:
const ExperimentContainer Experiment = const experiments = Experiment Experiment Experiment const container = ExperimentContainer let experiment = containerif experiment let variation = experiment // do something with the variation data else console
Parameters
__seed
number just for testing / predictable engine results
add
lib/ExperimentContainer.js:77-102
Add an experiment to this container. Inside a container experiments must have unique ids. This method can accept different kinds of experiment expressions:
- an instance of Experiment:
container
- An instance of Variation where it's object is an Experiment: ```js container.add(Variation.create(Experiment.create(...))) ````
- An instance of Variation where it's object is an Expriment defined as an object literal:
container
- A variation object literal wrapping an experiment object literal, this is useful in deserialization scenarios:
container
Parameters
experiments
...any
pick
lib/ExperimentContainer.js:131-151
The pick method accepts a targeting object and randomly selects an experiment from a set of experiments that match the targeting specification.
By default, selection is random and even, however, bias can be applied by specifying a weight when adding an experiment to the container (see ExperimentContainer.add())
Weights are considered at the moment of selection from the current set of matching experiments, therefor, careful planning of targeting is required to achieve accurate traffic distribution betwee experiments.
For example, consider two experiments, E1
, that targets { geo: 'US', page: '*' }
and E2
that targets
{ geo: 'US', page: 'index.html' }
. If both had the weight 1
, given the following stream
of visitors:
{ geo: 'US', page: 'sale.html' }
{ geo: 'US', page: 'index.html' }
{ geo: 'US', page: 'sale.html' }
{ geo: 'US', page: 'index.html' }
Then it is more likely that E1
will receive more traffic than E2
since E1
competes with E2
evenly on index.html
page but not on sale.html
Parameters
targeting
Targeting
Returns Experiment an experiment that matches this targeting or null if none is found.
targetingFeatures
lib/ExperimentContainer.js:159-161
An iterator over all the targeting features from all the experiments added to this container
Returns Iterator
iterator
lib/ExperimentContainer.js:175-177
iterate over all the experiments in this container:
let container = ExperimentContainer.create(...)
for (let experiment of container) {
console.log(experiment.id)
}
Returns ObjectIterator
has
lib/ExperimentContainer.js:184-188
check if this container contains the specified experiment
Parameters
experiment
Expriment
Returns Boolean
hasId
lib/ExperimentContainer.js:195-201
check if this container contains an experiment using an id
Parameters
experimentId
String
Returns Boolean
toJSON
lib/ExperimentContainer.js:208-215
serialize this container with all it's experiments
Returns Object