Stable Cache
This lib makes extensive use of a concept called producer. A producer is the source of truth of a cached value. Producers are only called when needed to set a cache, and are usually not needed when there's a cache hit.
The proposal of this library is to make it easier to manage producers and how keys are fetched/saved to redis.
Features:
- there's easily configurable resilience against producer failures:
- timeout
- retry with exponential backoff, and decorrelated jitter
- circuit breaker
- optionally set redis ttl for produced values
- optionally ignoring cached values and forcing values refresh
- optionally refreshing keys in background. Useful when you want keys to be updated before their ttl would evict them.
- optionally producing values in background, while returning the cached value, even on cache misses
- builtin RTA for prometheus
Usage
Cache instance
If the circuitBreaker
config is not provided this cache instance simply will not have a circuit breaker flow, and will still be usable.
I recommend to instantiate a cache for each service you consume, or else the circuit breaker would be shared. Which in turn could mean that a failing service would open the circuit for working services too.
const Redis = ;const Cache = ; const redis = port: 6379 host: 'redis' // localhost; const cache = redis // required to inject a redis client options: // everything is optional from here name: 'someService' // name of the service the cache will handle, useful for RTA circuitBreaker: // circuit breaker config for the service // percentage of failing producer requests to trigger the circuit breaker threshold: 02 // time window to be considered by the threshold, in milliseconds duration: 10000 // attempt to half open the circuit after this time in milliseconds halfOpenAfter: 20000 // don't open the circuit if there's less than this amount of RPS. // This avoids opening the circuit, during failures in low load periods. minimumRps: 5 ;
Cache.get method
Only the key
argument is required. Other options are used to augment the producer flow. See more usage examples at lib/examples/cache-get-usage.js
Cache get options are:
NOTE: circuit breaker policy is configured only when instantiating the
Cache
class.
Option | Description | Default Behavior | Default Value |
---|---|---|---|
producer |
A function that will be called when the source of the truth of the key is needed, has the signature function(): Promise<string> |
No producer to call | null |
ttl |
Time in milliseconds for the redis key ttl, in case the producer resolves | No ttl set | null |
producerRetry |
Object configuration for the exponential backoff | No retry | null |
producerRetry.maxDelay |
Max amount of delay between retry attempts, in milliseconds | - | 30000 |
producerRetry.maxAttempts |
Max amount of retries for the producer | - | 10 |
producerRetry.initialDelay |
Initial delay, in milliseconds before the retry | - | 128 |
producerTimeout |
Timeout in milliseconds for the producer. Even if there's a timeout, if the producer eventually resolves, the key will be set in background so it would be best if you configure a greater timeout on the producer itself, together with this config | No timeout | null |
returnEarlyFromCache |
Whether to return the cached value, and make the producer call on background, on cache miss | Await the producer on cache miss | false |
overrideCache |
Whether to ignore cached values and request producer, regardless of cache hits | Don't ignore cache hits, and call producer only if needed | false |
shouldRefreshKey |
A callback that if returns true , calls the producer and sets the key on background. Has the signature function(key, currentTTL, options): boolean |
No automatic refresh of keys | null |
Example of all the configs:
{ return { ; };} cache;
Cache.set method
Sets a key with a value, and receives an optional ttl for the key.
Cache set options:
Option | Description | Default Behavior | Default Value |
---|---|---|---|
ttl |
Time in milliseconds for the redis key ttl | No ttl set | null |
Example:
// sets `some^key` with value `a-value` with ttl of 30 secondscache;
Prometheus Exporter
There's an exporter for prometheus that exposes the following metrics
cache_results_count
: counts cache hits/misses, with labels:['service', 'operation', 'result']
cache_operations_duration_seconds
: histogram for cache RT, with labels:['service', 'operation']
cache_operations_count
: counts cache operations made, with labels:['service', 'operation']
producer_operations_duration_seconds
: histogram with RT for producer calls, with labels:['service']
producer_operations_result_count
: counts the successes/errors for producer calls, with labels:['service', 'result']
producer_circuit_break_state
: gauge for the state of a service circuit breaker. A value of0
means the circuit is closed (working normally), and a value of1
means the circuit is open (fail fast is activated).
What labels mean?
service
: is the cachename
option- cache
operation
: type of redis operation, e.g.set
,get
. - cache
result
: cachehit
ormiss
- producer
result
:success
, orerror
Usage
Prometheus exporter options for the constructor are:
Option | Description | Default Behavior | Default Value |
---|---|---|---|
prefix |
Prefix to be added to every exported metric | No prefix added | '' |
registers |
Array of prometheus registers to which metrics should be exported | Use default prom-client register |
[Prometheus.register] |
cacheBuckets |
Array of numbers, representing the histogram buckets for cache RT | Use default value | Prometheus.exponentialBuckets(0.05, 2, 8) |
producerBuckets |
Array of numbers, representing the histogram buckets for producer RT | Use default value | Prometheus.exponentialBuckets(0.1, 2, 8) |
Example:
const Prometheus = ;const PrometheusExporter = ; const exporter = prefix: 'my_app_' registers: Prometheusregister cacheBuckets: Prometheus producerBuckets: Prometheus; exporter; // start collecting metrics