node package manager


cache feeds

manger - cache feeds

The manger Node package provides caching for RSS and Atom formatted XML feeds. It provides an interface to query entries by feed and time.

null | undefined

An optional String().

String() | void()

One metadata object per XML feed.

  • author str()
  • copyright str()
  • feed str()
  • id str()
  • image str()
  • language str()
  • link str()
  • payment str()
  • subtitle str()
  • summary str()
  • title str()
  • ttl str()
  • updated Number()

A related resource of an entry().

  • href str()
  • length Number()
  • type str()

An sanitized HTML String() with tags limited to: 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'blockquote', 'p', 'a', 'ul', 'ol', 'li', 'b', 'i', 'strong', 'em', 'code', 'br', 'div', 'pre'.


An individual entry.

  • author str()
  • enclosure enclosure() | void()
  • duration Number() | null The value of the <itunes:duration> tag in seconds or null.
  • feed str()
  • id str()
  • image str()
  • link str()
  • subtitle str()
  • summary html() | void()
  • title str()
  • updated str()

A query to get a feed or entries of a feed in a time range between and since.

Options for a Manger instance.

  • cacheSize Number() | 8 * 1024 * 1024 Passed to levelup()
  • objectMode Boolean() | false Provide objects instead of buffers
  • name String() The name of the file system directory for the database
  • opts opts()

The manger module exports a single function that returns a new cache object (an instance of Manger). To access the Manger class require('manger').

const manger = require('manger')
const cache = manger('/tmp/manger.db')

If opts has objectMode set to true, results are read as Object types, instead of Buffer, moulding valid JSON.

manger leverages the lexicographical key sort order of LevelDB. The keys are designed to stream feeds or entries in time ranges between now and some user defined point in the past.

The distinction between feed and entries might be unclear. A feed models the metadata of an RSS or Atom feed (title, author, published, etc.), while entries are the actual items in the feed. These are detached to not repeatedly transmit feed metadata—after all manger tries to reduce round-trips.

A Transform stream that transforms queries or URLs to entries.

  • write(Buffer() | String() | query())
  • read() Buffer() | entry()

A stream that transforms queries or URL strings to feeds.

  • write(query() | String())
  • read() Buffer() | String() | feed()

A Readable stream of URLs of all feeds currently cached.

  • read() Buffer() | str()

Updates all ranked feeds and returns a stream that emits feed URLs of updated feeds. This, of course, could produce a resource heavy operation! Feeds are updated ordered by their popularity, using the rank index, therefore flushCounter must have been invoked before this method can take any effect.

  • read() str()

A stream of URLs sorted by rank (highest first).

  • read() Buffer() | String() | null

Resets the ranks index.

  • cb Function(error) The callback
    • error Error() | void() The possible error
  • cb Function(error, count) | void() The callback
    • error Error() | void() The possible error
    • count Number() Total number of cached feeds

manger keeps an in-memory count of how many times feeds have been accessed. This function flushes the counter to disk, updating the ranks index.

Applies callback cb without arguments if a feed with this url is cached.

  • url String() The URL of the feed
  • cb Function(error) | void() The callback
    • error Error() | void() The possible error

Attempts to remove a feed matching the url from the cache and applies callback without error if this succeeds.

  • url String() The URL of the feed
  • cb Function(error) | void() The callback
    • error Error() | void() The possible error

The manger module decorates the exported Manger constructor with two convenience functions for querying the cache.

A failable factory function returning a valid query() or void().

This stream transforms JSON to queries which can be piped to feeds() and entries() streams. The expected JSON input format:

  { "url": "" },
  { "url": "",
    "since": 1433083971124 },
  { "url": "",
    "since": "May 2015" },

Where "since" can be anything Date() is able to parse.

To make piping easier—by not breaking pipes—the queries stream emits a 'warning' event instead of an 'error' event if it encounters an invalid query. General stream errors, of course, are still emitted nevertheless.

With npm, do:

$ npm install manger

MIT License