Fast, reliable and smart caching system heavily based on streams.
Made with obelisk.js.
stores is a fast, reliable, smart caching system heavily based on streams.
stores deals transparently with cache stampede or cache miss storm for you. Basically, it ensures that only one request fetches the resource to be cached. Other ones are put in a queue and served ASAP, when the resource has been fetched or cached.
stores aims to be a solid caching facility that allows you to easily implement your custom backing store.
That said, it comes with a balanced
filesystem store and an express middleware.
npm install stores --save --production
You're in hurry, I can understand that.
Here is an example on how you can cache processed images easily:
This code ensures that you only process a given image once, and serve the cached version to all others.
It implicitly use the
FileStore to cache the image to the filesystem.
slot represents a stream pointing to a cache bucket that the
FileStore has automatically created.
This provided middleware allows you to configure a store globally.
This mandatory argument can be:
stringthat defines which type of store you want to use (for now only
constructorthat inherits from
Note that you can register custom stores and then use the
storesMyPreciousStore = MyPreciousStore;;
options will configure the store. All stores accepts those values:
maxPending: (default: 100), defines how many requests can be enqueued while a resource is being cached (see [hit hot]).
writesRetries: (default: 3), defines how many times the store will try to write to a cache bucket that fails (see [sealed buckets].
Note that each implementation can add additional values for their specific needs (i.e. [file store options]).
Optional, this callback allows the store to fetch missing data and store it. It is called on a cache miss, with the following arguments:
req: the network request.
Writablestream that will persist data to cache. It is allocated by the store. You must
next: next middleware in the stack. It's useful if something went wrong and you want to abort the request asap.
Note that if you don't specify a
fetch callback here, then you must specify it via
As seen previously, you can instanciate a store by yourself for some additional flexibility. As a user, you only are interested in the public
Creates a store with the given options.
var store = writesRetries: 0 ;
This tells the store how to fetch a missing resource from the cache. If you haven't specified it via the
stores middleware, you still can do it here globally.
var store = ;store;
The main advantage of specifying it this way, is that you can change the
fetch method whenever you want during the lifecycle of your application.
This is the most important method. It will try to fetch a resource from the cache or call
It can be useful if for some reason, you prefer using
stores in one of your existing route of middleware, or when you want to add additional logic before using it.
var store = ;app;
Note that if you already have specified a fetch method globally, you can then omit the
As you may already guess, it uses the filesystem as storage medium. It creates a balanced directory structure which ensures performances are always the best, even if your cache is growing fast.
Basically each cache entry is associated with a strong hash (
sha256). This hash is used to create the path to the cache file. This path is composed of multiple subdirectories in order to ensure there is not more that 256 entries in each directory.
Creates a filesystem store with the given options. Available options are common ones plus the following ones:
root: (default: current directory), root directory of the cache structure.
depth: (default: 4), number of subdirectories less 1.
var store = root: '/var/cache/www' depth: 2;
To implement a custom store, you have to inherit from the
This object provides two methods,
_lock that are respectively needed to fetch the original resource and to lock a new cache bucket. A cache bucket can be seen as the physical location where your cached resource will be
stored. It can be a file, a memory chunk, a REDIS key, a S3 bucket, or whatever you want.
You also need be aware that
stores uses [graceful-fs-stream] (
gfs) as dependency.
gfs slightly changes the behavior of
fs.createWriteStream by opening / creating the underlying file on first read or write. The main advantage is that instead of throwing an error, those function will emit an
error event instead.
If you do still want to use standard versions in your project, use