Performance measuring utilities for DefinitelyTyped.
This is not currently designed for public consumption, so docs, tests, and usage help are very light. The utility is used only in Azure Pipelines for DefinitelyTyped and TypeScript. There are three main functions that run in CI:
Every night, see what packages on DefinitelyTyped haven’t been benchmarked with the current version of TypeScript since their last change, and benchmark them. The CLI entrypoint
getPackagesToBenchmarkqueries the Cosmos DB instance to see what packages need to be benchmarked, then generates a file specifying what to run. In another phase, a large number of build agents read that file with the
benchmarkentrypoint and benchmark the packages in parallel. After a new TypeScript release, it takes several weeks of running this nightly to cover all of DefinitelyTyped.
On every PR to DefinitelyTyped, pull the benchmarks for changed packages. If they haven’t been run or aren’t up-to-date, run them on
master. Then, benchmark the PR, and compare. Post the summary of the results in a comment on the PR. This is done with the
Every night, find existing benchmarks that have been run with the latest TypeScript version, run them with
typescript@master, and compare. This is done with the
compareTypeScriptCLI entrypoint. Results are uploaded to the
typeScriptComparisonscollection in the database.
The package registers a
definitely-not-slow upon installation:
npx definitely-not-slow compare --package react/v16 --upload false
Installation and publishing
This utility isn’t published to npm; rather, a GitHub workflow produces branches that include the compiled code:
npm install github:andrewbranch/definitely-not-slow#production # Published on every tag npm install github:andrewbranch/definitely-not-slow#beta # Published on every push to master
It can be useful to queue a Pipelines build manually against a real DefinitelyTyped PR with the variables
DNS_VERSION=beta (definitely-not-slow—bad acronym, sorry) to test changes to definitely-not-slow on real code.