Wondering what’s next for npm?Check out our public roadmap! »

    bmcrusher-node

    1.3.0 • Public • Published

    bmcrusher-node

    For transferring crushed data between app script and node using various plugins.

    Installation

    yarn add bmcrusher-node
    

    the crusher

    This is a key value store client, with various back ends as plugins. Values are compressed, and if necessary spread over multiple physical records in the backend store depending on its capabilities. Different backends are supported by plugins.

    This is the Node client. Alternative clients allow you to share data across multiple platforms. In addition to the Node client, there is also an Apps Script client which has a set of plugins for multilple backends. So sharing data between Apps Script and Node is just a matter of using the appropriate plugin.

    There are only 3 methods in a client once intialized to a back end plugin.

    put

    Put a value. If it's too big it'll compress it and then split it into pieces. An optional expiry time is available to limit the lifetime of a kv pair. If the back end supports automatic lifecycle housekeeping (for example redis, Apps Script cache service and google cloud storage) expired items will be automatically removed sometime after expiry. Irrespective of the back end capabilities, items are never returned if they have expired.

    The key should be a string, and the value can be either a string or a stringifiable object.

    crusher.put(key, someValue  [,expiryTimeInSeconds])
    

    get

    Get a value. If it's in pieces it will reconstitute it to the original. If it's expired or doesn't exist, it'll return null. If the original input was an object, the object will be recreated.

    crusher.get(key)
    

    remove

    Remove a value

    crusher.remove(key)
    

    Coming soon

    Other values such as blobs etc will be supported shortly too. That will allow entire files such as images to be cached and reconstitued between platforms, as will various file conversions (for example between Google Sheets and Excel, or Google Docs and PDF). Watch this space for additional plugins to support these capabilities.

    Plugins

    Google Apps Script has a selection of supported plugins

    some of which are specific to Apps Script platform

    • CacheService
    • PropertyService

    and others which are generic and are/will be implemented for Node as well

    • Upstash (redis/graphql)
    • Github
    • Drive
    • Google cloud storage
    • One Drive

    For details on Apps Script implementations, see https://ramblings.mcpher.com/apps-script/apps-script-cache-crusher/

    Some of the Node plugins will be implemented and built in to this module, but you can easily build your own for alternative backends.

    CrusherPluginUpstashService

    Uses Upstash as a redis backend.

    For setting up Upstash and Apps script see https://ramblings.mcpher.com/apps-script/apps-script-cache-crusher/upstash/

    You'll need an Upstash account and credentials.

    Node usage

    First get your upstash credentials. Choose the appropriate credential depending on whether you are reading or read/writing.

    const upstashrw = "xxx";
    const upstashr = "xxx";
    

    Initialize the crusher

    This is a similar pattern and options as described in the Apps Script writeup in https://ramblings.mcpher.com/apps-script/apps-script-cache-crusher/upstash/.

    At a minumum you should provide a token service function that returns your upstash key. I also recommend a prefix to be applied to cache keys in case you want to use the same Upstash store for something else at some point.

    const { CrusherPluginUpstashService } = require("bmcrusher-node");
    const { upstashrw } = require("./private/secrets");
    
    const crusher = new CrusherPluginUpstashService().init({
      tokenService: () => upstashrw,
      prefix: "/crusher/store"
    });
    
    

    Now you can use the the standard crusher.get, crusher.put and crusher.remove methods.

    CrusherPluginGitService

    Uses Github as a backend.

    For setting up Github and Apps script see https://ramblings.mcpher.com/apps-script/apps-script-cache-crusher/github/

    You'll need a github token. The simplest is to create a personal token, since this is to be server based. For a client version, you'd want to create an oauth flow. Here's the docs for how to create a personal access token https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token

    Node usage

    First get your token, created with repo scope.

    const crusherGit = "gxxxxxxxx";
    

    Initialize the crusher

    This is a similar pattern and options as described in the Apps Script writeup in https://ramblings.mcpher.com/apps-script/apps-script-cache-crusher/github/

    At a minumum you should provide a token service function that returns your github token. I also recommend a prefix to be applied to cache keys in case you want to use the same repo for something else at some point.

    const { CrusherPluginGitService } = require("bmcrusher-node");
    const { crusherGit } = require("./private/secrets");
    
    const crusher = new CrusherPluginGitService().init({
      tokenService: () => crusherGit,
      prefix: "store",
      repo: "-crusher-store",
      owner: "brucemcpherson"
    });
    
    

    Now you can use the the standard crusher.get, crusher.put and crusher.remove methods.

    CrusherPluginGcsService

    Uses Google Cloud Storage as a backend.

    For setting up Github and Apps script see https://ramblings.mcpher.com/cacheplugins/apps-script-and-node-gcs/

    You'll need a service account .json file with enough scope to write to the cloud storage bucket you're using for this. See https://ramblings.mcpher.com/apps-script/apps-script-cache-crusher/gcs/

    Node usage

    First get your service account credentials using whichevery method you prefer. I usually do something like this

    const _credentials = require("./bmcrusher-test-xxxxxxx.json");
    const getGcpCreds = () => _credentials;
    

    Initialize the crusher

    This is a similar pattern and options as described in the Apps Script writeup in https://ramblings.mcpher.com/apps-script/apps-script-cache-crusher/gcs/

    At a minumum you should provide a token service function that returns your service account credentials. I also recommend a prefix to be applied to cache keys in case you want to use the same bucket for something else at some point.

    const { CrusherPluginGcsService } = require("bmcrusher-node");
    const { getGcpCreds } = require("./private/secrets");
    
    const gcsCrusher = new CrusherPluginGcsService().init({
      tokenService: () => getGcpCreds(),
      prefix: "/crusher/store",
      bucketName: "bmcrusher-test-bucket-store",
    });
    
    

    Now you can use the the standard crusher.get, crusher.put and crusher.remove methods.

    Keywords

    none

    Install

    npm i bmcrusher-node

    DownloadsWeekly Downloads

    10

    Version

    1.3.0

    License

    MIT

    Unpacked Size

    70.7 kB

    Total Files

    14

    Last publish

    Collaborators

    • avatar