Nocturnally Psychologizing Millipede

    @thi.ng/bench
    TypeScript icon, indicating that this package has built-in type declarations

    3.1.7 • Public • Published

    bench

    npm version npm downloads Twitter Follow

    This project is part of the @thi.ng/umbrella monorepo.

    About

    Benchmarking utilities w/ various statistics & formatters (CSV, Markdown etc.).

    Though no public API change (only additions), since v2.0.0 this library internally attempts to use high-res ES BigInt timestamps (in Node via process.hrtime.bigint()). If BigInt is not available in the target environment, timestamps are still only sourced via Date.now().

    Status

    STABLE - used in production

    Search or submit any issues for this package

    Related packages

    Installation

    yarn add @thi.ng/bench

    ES module import:

    <script type="module" src="https://cdn.skypack.dev/@thi.ng/bench"></script>

    Skypack documentation

    For Node.js REPL:

    # with flag only for < v16
    node --experimental-repl-await
    
    > const bench = await import("@thi.ng/bench");
    

    Package sizes (gzipped, pre-treeshake): ESM: 1.35 KB

    Dependencies

    Usage examples

    Several demos in this repo's /examples directory are using this package.

    A selection:

    Screenshot Description Live demo Source
    Doodle w/ K-nearest neighbor search result visualization Demo Source
    Poisson-disk shape-aware sampling, Voronoi & Minimum Spanning Tree visualization Demo Source
    Minimal Markdown to Hiccup to HTML parser / transformer Demo Source
    Parser grammar livecoding editor/playground & codegen Demo Source
    Interactive pixel sorting tool using thi.ng/color & thi.ng/pixel Demo Source
    Full umbrella repo doc string search w/ paginated results Demo Source
    Fork-join worker-based raymarch renderer (JS/CPU only) Demo Source

    API

    Generated API docs

    import { timed, bench, benchmark } from "@thi.ng/bench";
    
    // test functions
    const fib = (n) => n > 2 ? fib(n - 1) + fib(n - 2) : n > 0 ? 1 : 0;
    
    const fib2 = (n) => {
        const res = [0, 1];
        for(let i = 2; i <= n; i++) {
            res[i] = res[i - 1] + res[i - 2];
        }
        return res[n];
    };
    
    // measure single execution time
    timed(() => fib(40));
    // 714ms
    // 102334155
    timed(() => fib2(40));
    // 0ms
    // 102334155
    
    // measure 1mil iterations (default)
    bench(() => fib(10), 1e6);
    // 395ms
    // 55
    
    bench(() => fib2(10), 1e6);
    // 53ms
    // 55

    Benchmarking with statistics

    The benchmark() function executes a number of warmup runs, before executing the main measurement and producing a number of useful statistics: mean, median, min/max, 1st/3rd quartile, standard deviation (as percentage)...

    See api.ts for configuration options.

    Also see the formatting section below for other output options. This example uses the default format...

    benchmark(() => fib(40), { title: "fib", iter: 10, warmup: 5 });
    // benchmarking: fib
    //         warmup... 3707.17ms (5 runs)
    //         executing...
    //         total: 7333.72ms, runs: 10
    //         mean: 733.37ms, median: 733.79ms, range: [728.58..743.43]
    //         q1: 730.98ms, q3: 735.03ms
    //         sd: 0.54%
    
    // also returns results:
    // {
    //   title: "fib",
    //   iter: 10,
    //   total: 7333.72402,
    //   mean: 733.372402,
    //   median: 733.794194,
    //   min: 728.5808,
    //   max: 743.432538,
    //   q1: 730.980115,
    //   q3: 735.025314,
    //   sd: 0.542200865574415
    // }

    Benchmark suites

    Multiple benchmarks can be run sequentially as suite (also returns an array of all results):

    b.suite(
        [
            { title: "fib2(10)", fn: () => fib2(10) },
            { title: "fib2(20)", fn: () => fib2(20) },
            { title: "fib2(30)", fn: () => fib2(30) },
            { title: "fib2(40)", fn: () => fib2(40) },
        ],
        { iter: 10, size: 100000, warmup: 5, format: b.FORMAT_MD }
    )
    
    // |                   Title|    Iter|    Size|       Total|    Mean|  Median|     Min|     Max|      Q1|      Q3|     SD%|
    // |------------------------|-------:|-------:|-----------:|-------:|-------:|-------:|-------:|-------:|-------:|-------:|
    // |                fib2(10)|      10|  100000|       54.34|    5.43|    5.15|    4.40|    8.14|    4.84|    6.67|   20.32|
    // |                fib2(20)|      10|  100000|      121.24|   12.12|   12.13|   11.73|   12.91|   11.93|   12.35|    2.61|
    // |                fib2(30)|      10|  100000|      152.98|   15.30|   14.51|   13.93|   20.77|   14.35|   16.35|   12.65|
    // |                fib2(40)|      10|  100000|      164.79|   16.48|   15.60|   15.01|   19.27|   15.42|   18.80|    9.34|

    Same table as actual Markdown:

    Title Iter Size Total Mean Median Min Max Q1 Q3 SD%
    fib2(10) 10 100000 54.34 5.43 5.15 4.40 8.14 4.84 6.67 20.32
    fib2(20) 10 100000 121.24 12.12 12.13 11.73 12.91 11.93 12.35 2.61
    fib2(30) 10 100000 152.98 15.30 14.51 13.93 20.77 14.35 16.35 12.65
    fib2(40) 10 100000 164.79 16.48 15.60 15.01 19.27 15.42 18.80 9.34

    Output formatting

    The following output formatters are available. Custom formatters can be easily defined (see source for examples). Formatters are configured via the format option given to benchmark() or suite().

    • FORMAT_DEFAULT - default plain text formatting
    • FORMAT_CSV - Comma-separated values (w/ column header)
    • FORMAT_MD - Markdown table format

    Authors

    Karsten Schmidt

    If this project contributes to an academic publication, please cite it as:

    @misc{thing-bench,
      title = "@thi.ng/bench",
      author = "Karsten Schmidt",
      note = "https://thi.ng/bench",
      year = 2018
    }

    License

    © 2018 - 2022 Karsten Schmidt // Apache Software License 2.0

    Install

    npm i @thi.ng/bench

    DownloadsWeekly Downloads

    477

    Version

    3.1.7

    License

    Apache-2.0

    Unpacked Size

    44.2 kB

    Total Files

    24

    Last publish

    Collaborators

    • thi.ng