cacache is a Node.js library for managing
local key and content address caches. It's really fast, really good at
concurrency, and it will never give you corrupted data, even if cache files
get corrupted or manipulated.
It was originally written to be used as npm's local cache, but can just as easily be used on its own.
$ npm install --save cacache
Table of Contents
const cacache =const fs =const tarball = '/path/to/mytar.tgz'const cachePath = '/tmp/my-toy-cache'const key = 'my-unique-key-1234'// Cache it! Use `cachePath` as the root of the content cachecacacheconst destination = '/tmp/mytar.tgz'// Copy the contents out of the cache and into their destination!// But this time, use stream instead!cacacheget// The same thing, but skip the key index.cacacheget
- Extraction by key or by content address (shasum, etc)
- Subresource Integrity web standard support
- Multi-hash support - safely host sha1, sha512, etc, in a single cache
- Automatic content deduplication
- Fault tolerance (immune to corruption, partial writes, process races, etc)
- Consistency guarantees on read and write (full data verification)
- Lockless, high-concurrency cache access
- Streaming support
- Promise support
- Pretty darn fast -- sub-millisecond reads and writes including verification
- Arbitrary metadata storage
- Garbage collection and additional offline verification
- Thorough test coverage
- There's probably a bloom filter in there somewhere. Those are cool, right? 🤔
The cacache team enthusiastically welcomes contributions and project participation! There's a bunch of things you can do if you want to contribute! The Contributor Guide has all the information you need for everything from reporting bugs to contributing entire new features. Please don't hesitate to jump in if you'd like to, or even ask us questions if something isn't clear.
All participants and maintainers in this project are expected to follow Code of Conduct, and just generally be excellent to each other.
Please refer to the Changelog for project history details, too.
cacache includes a complete API in English, with the same features as other
translations. To use the English API as documented in this README, use
require('cacache/en'). This is also currently the default if you do
require('cacache'), but may change in the future.
cacache also supports other languages! You can find the list of currently
supported ones by looking in
./locales in the source directory. You can use
the API in that language with
Want to add support for a new language? Please go ahead! You should be able to
./locales/en.json and fill them in. Translating the
README.md is a bit more work, but also appreciated if you get around to it. 👍🏼
Lists info for all entries currently in the cache as a single large object. Each
entry in the object will be keyed by the unique index key, with corresponding
get.info objects as the values.
cacache// Output'my-thing':key: 'my-thing'integrity: 'sha512-BaSe64/EnCoDED+HAsh=='path: '.testcache/content/deadbeef' // joined with `cachePath`time: 12345698490size: 4023948metadata:name: 'blah'version: '1.2.3'description: 'this was once a package but now it is my-thing''other-thing':key: 'other-thing'integrity: 'sha1-ANothER+hasH='path: '.testcache/content/bada55'time: 11992309289size: 111112
Lists info for all entries currently in the cache as a single large object.
cacachels// Outputkey: 'my-thing'integrity: 'sha512-BaSe64HaSh'path: '.testcache/content/deadbeef' // joined with `cachePath`time: 12345698490size: 13423metadata:name: 'blah'version: '1.2.3'description: 'this was once a package but now it is my-thing'key: 'other-thing'integrity: 'whirlpool-WoWSoMuchSupport'path: '.testcache/content/bada55'time: 11992309289size: 498023984029...
Returns an object with the cached data, digest, and metadata identified by
data property of this object will be a
Buffer instance that
presumably holds some data that means something to you. I'm sure you know what
to do with it! cacache just won't care.
integrity is a Subresource
string. That is, a string that can be used to verify
data, which looks like
If there is no content identified by
key, or if the locally-stored data does
not pass the validity checksum, the promise will be rejected.
get.byDigest may be used for identical behavior, except lookup
will happen by integrity hash, bypassing the index entirely. This version of the
function only returns
data itself, without any wrapper.
This function loads the entire cache entry into memory before returning it. If
you're dealing with Very Large data, consider using
// Look up by keycache// Output:metadata:thingName: 'my'integrity: 'sha512-BaSe64HaSh'data: Buffer#<deadbeef>size: 9320// Look up by digestcacheget// Output:Buffer#<deadbeef>
Returns a Readable Stream of the cached data identified by
If there is no content identified by
key, or if the locally-stored data does
not pass the validity checksum, an error will be emitted.
integrity events will be emitted before the stream closes, if
you need to collect that extra data about the cached entry.
get.stream.byDigest may be used for identical behavior,
except lookup will happen by integrity hash, bypassing the index entirely. This
version does not emit the
integrity events at all.
// Look up by keycacheget// Outputs:metadata: ...integrity: 'sha512-SoMeDIGest+64=='// Look up by digestcachegetstream
key in the cache index, returning information about the entry if
key- Key the entry was looked up under. Matches the
integrity- Subresource Integrity hash for the content this entry refers to.
path- Filesystem path relative to
cacheargument where content is stored.
time- Timestamp the entry was first added on.
metadata- User-assigned metadata associated with the entry/content.
cacacheget// Outputkey: 'my-thing'integrity: 'sha256-MUSTVERIFY+ALL/THINGS=='path: '.testcache/content/deadbeef'time: 12345698490size: 849234metadata:name: 'blah'version: '1.2.3'description: 'this was once a package but now it is my-thing'
Looks up a Subresource Integrity hash in the cache. If content
exists for this
integrity, it will return an object, with the specific single integrity hash
that was found in
sri key, and the size of the found content as
size. If no content exists for this integrity, it will return
cacacheget// Outputsri:source: 'sha256-MUSTVERIFY+ALL/THINGS=='algorithm: 'sha256'digest: 'MUSTVERIFY+ALL/THINGS=='options:size: 9001cacacheget// Outputfalse
Inserts data passed to it into the cache. The returned Promise resolves with a
digest (generated according to
opts.algorithms) after the
cache entry has been successfully written.
Returns a Writable
Stream that inserts
data written to it into the cache. Emits an
integrity event with the digest of
written contents when it succeeds.
cacache.put functions have a number of options in common.
Arbitrary metadata to be attached to the inserted key.
If provided, the data stream will be verified to check that enough data was
passed through. If there's more or less data than expected, insertion will fail
If present, the pre-calculated digest for the inserted content. If this option
if provided and does not match the post-insertion digest, insertion will fail
algorithms has no effect if this option is present.
Hashing algorithms to use when calculating the subresource integrity
for inserted data. Can use any algorithm listed in
'お任せします' to pick a random hash algorithm on each insertion. You
may also use any anagram of
'modnar' to use this feature.
Currently only supports one algorithm at a time (i.e., an array length of
1). Has no effect if
opts.integrity is present.
If provided, cacache will do its best to make sure any new files added to the
cache use this particular
gid combination. This can be used,
for example, to drop permissions when someone uses
sudo, but cacache makes
no assumptions about your needs here.
If provided, cacache will memoize the given cache insertion in memory, bypassing any filesystem checks for that key or digest in future cache fetches. Nothing will be written to the in-memory cache unless this option is explicitly truthy.
opts.memoize is an object or a
Map-like (that is, an object with
set methods), it will be written to instead of the global memoization
Reading from disk data can be forced by explicitly passing
memoize: false to
the reader functions, but their default will be to read from memory.
Clears the entire cache. Mainly by blowing away the cache directory itself.
Removes the index entry for
key. Content will still be accessible if
requested directly by content address (
Removes the content identified by
integrity. Any index entries referring to it
will not be usable again until the content is re-added to the cache with an
Configure the language/locale used for messages and errors coming from cacache.
The list of available locales is in the
./locales directory in the project
Interested in contributing more languages! Submit a PR!
Completely resets the in-memory entry cache.
Returns a unique temporary directory inside the cache's
tmp dir. This
directory will use the same safe user assignment that all the other stuff use.
Once the directory is made, it's the user's responsibility that all files within
are made according to the same
opts.uid settings that would be
passed in. If not, you can ask cacache to do it for you by calling
tmp.fix(), which will fix all tmp directory permissions.
If you want automatic cleanup of this directory, use
Creates a temporary directory with
tmp.mkdir() and calls
with it. The created temporary directory will be removed when the return value
cb() resolves -- that is, if you return a Promise from
cb(), the tmp
directory will be automatically deleted once that promise completes.
The same caveats apply when it comes to managing permissions for the tmp dir's contents.
For content verification and addressing, cacache uses strings following the
That is, any time cacache expects an
integrity argument or option, it
should be in the format
One deviation from the current spec is that cacache will support any hash
algorithms supported by the underlying Node.js process. You can use
crypto.getHashes() to see which ones you can use.
Generating Digests Yourself
If you have an existing content shasum, they are generally formatted as a
hexadecimal string (that is, a sha1 would look like:
5f5513f8822fdbe5145af33b64d8d970dcf95c6e). In order to be compatible with
cacache, you'll need to convert this to an equivalent subresource integrity
string. For this example, the corresponding hash would be:
If you want to generate an integrity string yourself for existing data, you can use something like this:
const crypto =const hashAlgorithm = 'sha512'const data = 'foobarbaz'const integrity =hashAlgorithm +'-' +crypto
You can also use
ssri to have a richer set of functionality
around SRI strings, including generation, parsing, and translating from existing
Checks out and fixes up your cache:
- Cleans up corrupted or invalid index entries.
- Custom entry filtering options.
- Garbage collects any content entries not referenced by the index.
- Checks integrity for all content entries and removes invalid content.
- Fixes cache ownership.
- Removes the
tmpdirectory in the cache and all its contents.
When it's done, it'll return an object with various stats about the verification process, including amount of storage reclaimed, number of valid entries, number of entries removed, etc.
opts.uid- uid to assign to cache and its contents
opts.gid- gid to assign to cache and its contents
opts.filter- receives a formatted entry. Return false to remove it. Note: might be called more than once on the same entry.
echo somegarbage >> $CACHEPATH/content/deadbeef
Date representing the last time
cacache.verify was run on