Neoclassical Programming Multitude
    Wondering what’s next for npm?Check out our public roadmap! »

    @thi.ng/bench
    TypeScript icon, indicating that this package has built-in type declarations

    2.1.0 • Public • Published

    bench

    npm version npm downloads Twitter Follow

    This project is part of the @thi.ng/umbrella monorepo.

    About

    Benchmarking utilities w/ various statistics & formatters (CSV, Markdown etc.).

    Though no public API change (only additions), since v2.0.0 this library internally attempts to use high-res ES BigInt timestamps (in Node via process.hrtime.bigint()). If BigInt is not available in the target environment, timestamps are still only sourced via Date.now().

    Status

    STABLE - used in production

    Search or submit any issues for this package

    Related packages

    Installation

    yarn add @thi.ng/bench
    // ES module
    <script type="module" src="https://unpkg.com/@thi.ng/bench?module" crossorigin></script>
    
    // UMD
    <script src="https://unpkg.com/@thi.ng/bench/lib/index.umd.js" crossorigin></script>

    Package sizes (gzipped, pre-treeshake): ESM: 1.37 KB / CJS: 1.47 KB / UMD: 1.47 KB

    Dependencies

    Usage examples

    Several demos in this repo's /examples directory are using this package.

    A selection:

    Screenshot Description Live demo Source
    Fork-join worker-based raymarch renderer Demo Source

    API

    Generated API docs

    import { timed, bench, benchmark } from "@thi.ng/bench";
    
    // test functions
    const fib = (n) => n > 2 ? fib(n - 1) + fib(n - 2) : n > 0 ? 1 : 0;
    
    const fib2 = (n) => {
        const res = [0, 1];
        for(let i = 2; i <= n; i++) {
            res[i] = res[i - 1] + res[i - 2];
        }
        return res[n];
    };
    
    // measure single execution time
    timed(() => fib(40));
    // 714ms
    // 102334155
    timed(() => fib2(40));
    // 0ms
    // 102334155
    
    // measure 1mil iterations (default)
    bench(() => fib(10), 1e6);
    // 395ms
    // 55
    
    bench(() => fib2(10), 1e6);
    // 53ms
    // 55

    Benchmarking with statistics

    The benchmark() function executes a number of warmup runs, before executing the main measurement and producing a number of useful statistics: mean, median, min/max, 1st/3rd quartile, standard deviation (as percentage)...

    See api.ts for configuration options.

    Also see the formatting section below for other output options. This example uses the default format...

    benchmark(() => fib(40), { title: "fib", iter: 10, warmup: 5 });
    // benchmarking: fib
    //         warmup... 3707.17ms (5 runs)
    //         executing...
    //         total: 7333.72ms, runs: 10
    //         mean: 733.37ms, median: 733.79ms, range: [728.58..743.43]
    //         q1: 730.98ms, q3: 735.03ms
    //         sd: 0.54%
    
    // also returns results:
    // {
    //   title: "fib",
    //   iter: 10,
    //   total: 7333.72402,
    //   mean: 733.372402,
    //   median: 733.794194,
    //   min: 728.5808,
    //   max: 743.432538,
    //   q1: 730.980115,
    //   q3: 735.025314,
    //   sd: 0.542200865574415
    // }

    Benchmark suites

    Multiple benchmarks can be run sequentially as suite (also returns an array of all results):

    b.suite(
        [
            { title: "fib2(10)", fn: () => fib2(10) },
            { title: "fib2(20)", fn: () => fib2(20) },
            { title: "fib2(30)", fn: () => fib2(30) },
            { title: "fib2(40)", fn: () => fib2(40) },
        ],
        { iter: 10, size: 100000, warmup: 5, format: b.FORMAT_MD }
    )
    
    // |                   Title|    Iter|    Size|       Total|    Mean|  Median|     Min|     Max|      Q1|      Q3|     SD%|
    // |------------------------|-------:|-------:|-----------:|-------:|-------:|-------:|-------:|-------:|-------:|-------:|
    // |                fib2(10)|      10|  100000|       54.34|    5.43|    5.15|    4.40|    8.14|    4.84|    6.67|   20.32|
    // |                fib2(20)|      10|  100000|      121.24|   12.12|   12.13|   11.73|   12.91|   11.93|   12.35|    2.61|
    // |                fib2(30)|      10|  100000|      152.98|   15.30|   14.51|   13.93|   20.77|   14.35|   16.35|   12.65|
    // |                fib2(40)|      10|  100000|      164.79|   16.48|   15.60|   15.01|   19.27|   15.42|   18.80|    9.34|

    Same table as actual Markdown:

    Title Iter Size Total Mean Median Min Max Q1 Q3 SD%
    fib2(10) 10 100000 54.34 5.43 5.15 4.40 8.14 4.84 6.67 20.32
    fib2(20) 10 100000 121.24 12.12 12.13 11.73 12.91 11.93 12.35 2.61
    fib2(30) 10 100000 152.98 15.30 14.51 13.93 20.77 14.35 16.35 12.65
    fib2(40) 10 100000 164.79 16.48 15.60 15.01 19.27 15.42 18.80 9.34

    Output formatting

    The following output formatters are available. Custom formatters can be easily defined (see source for examples). Formatters are configured via the format option given to benchmark() or suite().

    • FORMAT_DEFAULT - default plain text formatting
    • FORMAT_CSV - Comma-separated values (w/ column header)
    • FORMAT_MD - Markdown table format

    Authors

    Karsten Schmidt

    If this project contributes to an academic publication, please cite it as:

    @misc{thing-bench,
      title = "@thi.ng/bench",
      author = "Karsten Schmidt",
      note = "https://thi.ng/bench",
      year = 2018
    }

    License

    © 2018 - 2021 Karsten Schmidt // Apache Software License 2.0

    Install

    npm i @thi.ng/bench

    DownloadsWeekly Downloads

    73

    Version

    2.1.0

    License

    Apache-2.0

    Unpacked Size

    63.4 kB

    Total Files

    28

    Last publish

    Collaborators

    • avatar