benchmark each release of a node module published on npm against each other release using a small command line tool
A command line tool to examine your code's performance over various releases.
- Downloads all releases
- runs the specified js file against all versions
- specialized to read ops/sec from stdout
Example usage for testing the
redis package, and running the file
multi_bench.js against each release:
npmbench redis multi_bench.jsOps/sec***** v0.0.1******* v0.0.2******** v0.0.3**** v0.0.4********** v0.0.5
- Output from each run of your js file will be put into a file named
- These are then parsed into json files which are put at
- makes it easy to run your own analysis of the numbers after all the benches have been run, or just grep through it :)
Why is this nice? Makes it easy to write your own parser for the output.
- skip running a given bench if there is already a .txt file there for it
- graph the graph with something fancier than asterisks on the command line.
- split out the parsing stage so people can supply their own output parser (too much work?)
- actually write the thing that aggregates all the json blobs into one graphable json blob