gemfire

    1.0.1 • Public • Published

    Node GemFire

    Build Status

    This project brings Pivotal GemFire an in-memory database to NodeJS.

    What is GemFire

    GemFire is a highly avaible database that utilizes RAM/Disk/CPU in a distributed manner to provide a near-linear scale out performance. GemFire's main data access pattern key-value while also supporting a rich query language that allows object traversal of the values in the query.

    GemFire Features

    Predictable Low Latency

    Even during huge peaks in concurrent access, your users can count on predictable low latency because of the in-memory, horizontally scalable architecture of GemFire.

    Elastic Scale-Out

    Easily scale out horizontally to meet the most demanding peaks in usage. Scale back down again gracefully when the peaks are behind you. Keep your steady state runtime costs down.

    Real-Time Event Notifications

    Your applications can subscribe to real-time events to react to changes immediately. GemFire’s continuous queries notify your application when new data is available, which reduces the overhead on your SQL database.

    High Availability and Business Continuity

    No more downtime from os, vm, physical system failures, datacenter-wide outages, or software upgrades. GemFire nodes automatically fail-over in the event of a node-level outage, while its highly efficient WAN replication protocol enables multi-datacenter deployment for business continuity.

    Durability

    Eliminate data loss with GemFire’s in-memory data replication and a shared-nothing persistence architecture, providing unmatched data resiliency.

    Cloud-Ready

    Available as a service on Pivotal Cloud Foundry, enable on-demand access to GemFire's high performance, highly available, event-driven data grid on the leading cloud-native application platform.

    Learn more about GemFire

    GemFire Storage Patterns

    Common GemFire Data Storage Patterns

    Caching

    In-memory databases like GemFire allow applications to take advantage of a computers RAM to offer extreme performance over traditional disk based systems. Even if those disk based systems are built on flash storage, in-memory provides a extreme number of input/output operations per second (IOPS).

    System of Record

    GemFire has a mature append only disk persistence capability. This allows for applications that need durability in thier architecture.

    Cache Aside

    In the cache aside pattern the application instance controls how data is written to the cache. Typically the application checks to see if the key/value is in the system. If there is a miss the application retrieves the value from an operational data store. Then stores the key/value for the next time it is needed.

    Read/Write Through

    By utilizing this pattern we can encapsulate the operational data store access within the GemFire servers. This allows the application to focus on value added behavior. Also allows GemFire to prevent operational concerns like the thundering herd problem.

    This pattern can enable change data capture patterns in a synchronous manner.

    Write Behind

    In GemFire it has a powerful queuing mechanism that is highly avaible and durable that operates at in-memory performance and scale. This allows GemFire to replicate data actions to an operational data store with a policy based on time or number of operations - which ever comes first.

    This pattern can enable change data capture patterns in an asynchronous manner.

    Refresh Ahead or Hot Cache

    In this pattern we either write an application that bulk loads the data into GemFire. As an alternative we can utilize the GemFire servers load the data in a parallel scale out manner using GemFire functions.

    Node GemFire Usage

    Writing a GemFire NodeJS application we first need to connect to the GemFire database. In the following example we use a GemFire Cache Factory to programmatically create the connection parameters to the GemFire servers. Followed up with creating a CACHING_PROXY connection to a region.

    var gemfire = require('gemfire');
    // Create a Cache Factory so we can configure the connection details
    var cacheFactory = gemfire.createCacheFactory();
    // Add a locator which allows the system to dynamically discover GemFire data servers
    cacheFactory.addLocator(host, parseInt(port));
    // Instantiate the GemFire system connection.
    var cache = cacheFactory.create();
    // Create the region to do storage operations.   Caching Proxy allows the client to use some of its memory to locally cache entries.
    var region = cache.createRegion("myRegion", {type: "CACHING_PROXY"});
    // Perform some data access using the asynchronous method - the synchronous version is called region.putSync(key, value);
    region.put('foo', { bar: ['baz', 'qux'] }, function(error) {
      region.get('foo', function(error, value) {
        console.log(value); // => { bar: ['baz', 'qux'] }
      });
    });

    For more information, please see the API Summary.

    If you are using Cloud Foundry a sample can be found here.

    Supported platforms

    • Ubuntu 14.04 64-bit based platforms / ubuntu:trusty / cflinuxfs2 - Other 64-bit Linux platforms may work.

    Tested NodeJS runtime versions

    • 8.11.3

    Tested GemFire versions

    • pivotal-gemfire-9.5.1
    • pivotal-gemfire-native-client-9.2.0

    Installation

    GemFire Client Libraries

    1. Download GemFire 9.2.0 Native Client Driver: Pivotal Network Download.
    2. Follow the installation instructions here: GemFire Native Client Installation

    Installing the NPM package

    Note that for the time being, if you want to be able to use the precompiled binary, you'll need to set NODE_TLS_REJECT_UNAUTHORIZED=0 when running npm install. Otherwise, npm install will fallback to compiling from source, which may only work on certain platforms.

    $ cd /my/node/project
    $ NODE_TLS_REJECT_UNAUTHORIZED=0 npm install --save gemfire
    

    Configuring the GemFire client

    By default the GemFire client will look for a geode.properties file in the current working directory. This property file allows the developer to specify a variety of configuration settings.

    As an alternative we can specify a properties file by passing the location to the gemfire.createCacheFactory("/path/to/geode.properties") function.

    For a full list of properties please review the the GemFire Properties Documentation.

    Sample Properties File

    Here is an example file that turns off statistics collection, sets the "warning" log level, and redirects the log output to a file.

    statistic-sampling-enabled=false
    log-level=warning
    log-file=log/gemfire.log
    

    GemFire Node JS Driver Development

    To enable linux development the team uses a virtual environment to enable windows or mac developers to contribute. To simplify setting up the linux guest OS we use vagrant to provision the VM and install the appropriate OS and tools.

    Prerequisites

    Install Vagrant 2.x or later to your choice of virtualization.

    Setup

    Before we run vagrant up for the first time we have to download GemFire from Pivotal Network. Download the following packages from Pivotal Network and copy to <project>/tmp directory

    • pivotal-gemfire-9.5.1.zip
    • pivotal-gemfire-native-9.2.0-build.10-Linux-64bit.tar.gz

    Once those are in place we can provision the vm and install all of the tools and GemFire by running vagrant up in the project directory.

    To build the VM:

    $ cd <project home>    
    $ vagrant up
    

    Developer workflow

    After the vagrant is all setup the project home is mounted in the VM as /vagrant. This enables the developer to make edits on the Host or on the VM since the directory is shared between environments.

    To ssh into the VM use the vagrant command:

    $ vagrant ssh
    

    Change to the project home once ssh'd into the VM:

    $ cd /vagrant
    

    Build the Node Module and run the tests use grunt

    $ grunt
    

    GemFire Server Management

    The GemFire server should be automatically started for you as part of the above tasks. If you need to restart it manually, use the following:

    $ vagrant ssh
    $ grunt server:restart # or server:start or server:stop
    

    Contributing

    Please see CONTRIBUTING.md for information on how to submit a pull request.

    License

    BSD, see LICENSE for details

    For dependency licenses, see doc/dependencies.md

    Keywords

    none

    Install

    npm i gemfire

    DownloadsWeekly Downloads

    0

    Version

    1.0.1

    License

    BSD

    Unpacked Size

    139 kB

    Total Files

    42

    Last publish

    Collaborators

    • pivotal