speedy
Tiny benchmark utility
If you need to benchmark some sort of code or if you are writing a module and want to see how well it performs in comparison with older versions, you can use speedy
. Use it for rapid prototyping and informal benchmarks. Don't use it for serious jobs like benchmarking HTTP servers.
This module doesn't check for errors so make sure your code doesn't break before running the benchmark.
The benchmark is formed by tests and each test is formed by samples (default 3). Each sample is executed with a fixed amount of time (default 1000ms). An arithmetic mean and a standard error is calculated with these samples.
A recursive loop is used to measure the speed of the code. There are basically 2 ways to make a recursive loop: nextTick
and setImmediate
. nextTick
cannot loop indefinitely because it has a maximum call stack limit (process.maxTickDepth
) and setImmediate
is slower than nextTick
(see examples/nexttick-vs-setimmediate-vs-setTimeout.js) and produces inconsistent benchmark results. The solution is to use an hybrid approach (found in the node-bench source code):
var async = { var i = 0; return { if i++ < 100 process; else ; i = 0; }}; async { //Code};
This worked for me and produces consistent benchmark results. But it has a problem. Cannot execute codes that make an excessive use of nextTick in a recursive way because the process.maxTickDepth
limit can be reached. In Node.js v0.12 this limit will be removed.
Objects
A Speedy
instance is returned when the module is required.
Events
Methods
- Speedy#run([name][, fn][, options]) : Speedy
- Speedy#samples([n]) : undefined | Number
- Speedy#timeout([n]) : undefined | Number
Emitted when the benchmark finishes. The callback receives an array. Each index stores an object with he result of each test.
Properties:
- raw - Array
Stores all the sample results. - name - String
Stores the name of the function, if any. It is like the result of thetest
event.
For example, a baseline benchmark with default options (samples 3, timeout 1000):
speedy ;
Emitted when a sample has been executed. The callback receives a number between (0, 1]. 1 means that the benchmark is finished.
This event is emitted before the test
and end
events.
Emitted for each test. The callback receives an object with the following properties:
- name - String
The name of the function. If the function is anonymous this property is not available. - raw - Array
Array with the sample results. The number of samples can be changed with the samples() function.
This event is emitted before the end
event.
Speedy#run([name][, fn][, options]) : Speedy
Executes the benchmark. The results are displayed with an arithmetic mean of operations per second followed by the standard error in percentage. The error should be less than 1%, typically less than 0.5%. If the error is greater than 1% then something went wrong and the mean is not as accurate as it can be.
Options:
-
output - WritableStream
By default the results will be sent to the stdout but any writable stream can be used.//The output is sent to a filespeedy;
Asynchronous benchmarking
Simply execute the callback when you are ready to iterate again. The asynchronous benchmarking is a little less precise than the synchronous version.
speedy;
Ways to run the benchmark and its results
Anonymous function:
speedy; /*<value>*/
Named function:
speedy; /*fn <value>*/
Using the name parameter:
speedy; /*fn <value>*/
Batch:
speedy; /*a <value>b <value>c <value>*/
Speedy#samples([n]) : undefined | Number
Changes or returns the number of samples per test. With more samples the final result will be more stable. An arithmetic mean is calculated with all the samples. Default is 3.
speedy;
Speedy#timeout([n]) : undefined | Number
Changes or returns the execution time per sample, in milliseconds. Higher values tend to imply more precise results. Default is 1000.
speedy;