yarn-nixpkgs

A prototype tool for automating packaging of Node.js applications for NixOS / Nixpkgs.

This tool is specifically for distribution packaging. If you are instead looking for ways to use Nix as a build tool for a Node.js project, may I suggest yarn-plugin-nixify instead.

Usage

This tool can be run directly with npx. No need to clone the repository or download anything. Only a Node.js installation is required.

npx yarn-nixpkgs prettier ./out

The above packages the latest version of prettier on npm, and produces output in ./out.

You can now build the package using Nix.

nix-build ./out

And then run a binary from the result:

./result/bin/prettier --help

Details

Output consists of several files:

• package.nix is the main Nix expression for building the package. It contains fetch derivations for all dependencies, and a derivation to produce the installation that brings it all together. It can be included in an overlay with pkgs.callPackage ./path/to/package.nix { }, which also allows customizing the Node.js version.

• default.nix is a small wrapper for package.nix that allows quick testing with nix-build.

• pnp.cjs is a Nix-specific PnP loader used by package.nix. This loader can be shared between all builds produced with the same version of yarn-nixpkgs.

• pnp.data.json is the PnP data generated by Yarn and used by package.nix. The PnP loader hooks Node.js module loading functions, and uses this data to locate the actual dependencies.

• yarn.lock is the regular lockfile. Not necessary for the Nix build, but leaving this file around and rerunning the same yarn-nixpkgs command will speed up the fetch step. (All packages are still updated to their latest version.)

Implementation

Yarn-nixpkgs essentially runs yarn install in a temporary workspace with just the one dependency. Using the data structures prepared by Yarn, we then generate all necessary output.

The main trick is that yarn-nixpkgs replaces all fetchers in Yarn with custom implementations, so that it can produce results that can be matched exactly in Nix fetch derivations, without creating circular dependencies on the Node.js ecosystem. These fetchers also don't use the regular Yarn cache, but a custom cache that integrates with the Nix store.

Other than that, the PnP loader is also special. First, PnP data is transformed so that packages can be loaded directly from the Nix store. Second, the loader is patched so that it can be shared between all installations. Unlike regular PnP loaders, it expects an environment variable NIX_YARN_PNP_BASE to point to a directory with pnp.data.json.

Comparison

Currently, yarn-nixpkgs has some open issues that make it alpha quality:

• Does not yet work with packages that have build steps. These will need to be generated as Nix derivations, but that does mean the result can be shared between packages. (Will also have to account for one package with multiple dependency trees, because of different peer dependencies.)

• Only able to fetch from the npm registry for now. No support for git dependencies or otherwise. Not impossible, but each of these would have to generate a different kind of Nix derivation, and it's not clear how many packages rely on other types of fetch methods.

• ESM support has not yet been implemented.

• Not well tested.

Compared to node2nix, for example, some advantages:

• Dependencies are ZIP files on disk, greatly reducing disk space and inode usage.

• Installation is fast. Essentially, the package installation itself just copies pnp.data.json and creates wrappers for binaries.

• Dependencies can be shared on disk. However, Nix does have the ability to hardlink similar files in the store, which may have the same effect.

But also some disadvantages:

• Limited to PnP installations. This can in theory break some packages, but whether this is an issue or not in practice has not been tested. An alternative could be to build a pnpm-like structure of node_modules directories on disk, which still allows sharing between packages.

• Nix expressions contain precalculated store paths. There are probably places where yarn-nixpkgs doesn't support a store path other than /nix/store. But I'm also not sure if it's frowned upon in general.

• The PnP loader is essentially a binary blob. It may be possible to build the loader from source, but that has not yet been investigated.

• Single purpose. Yarn-nixpkgs probably doesn't work for all node2nix use-cases.

Hacking

Install dependencies:

yarn

Build TypeScript:

# Add `-w` to watch for changes.
yarn tsc

Try your local build:

yarn yarn-nixpkgs prettier ./out
nix-build ./out
./result/bin/prettier --help