Nuclear Pumpkin Mayhem

    yildiz

    9.1.0 • Public • Published

    Yildiz Graph Database

    yildiz

    Thin graph database layer on top of Google Bigtable.

    Build Status Version MIT License node Swagger

    Intro

    YildizDB acts as a highly scaleable HTTP layer in front of Google Bigtable. It helps you store billions of relations between nodes in edges and resolve them in milliseconds. A small access layer helps you manage multiple namespaces with ease. YildizDB scales to hundrets of Terabytes. YildizDB excells in $N:N$ non nested relationships.

    Features

    • simple namespacing (table separation) with prefixes
    • simple access management through tokens
    • multi-tenancy through table prefixes (with access management)
    • high read and write performance
    • fast read access under heavy write load
    • able to handle billions of edges and nodes
    • scales beyond Terabytes
    • lightweight deployments (small Node.js footprint)
    • ttl feature for all resources
    • HTTP Open API
    • Kubernetes HELM charts

    Build for high throughput $N:N$

    • Highly async API based on fastify
    • Thin layer on top of Google's GRPC Bigtable API
    • Hashing and translating all string identifiers into integer representations via murmurhash3 automatically

    Fast random access to a node's edge data $1:N$

    • Multiple complex caching layers
    • Custom Bigtable cache table speed up by Google Memorystore (Redis)
    • All Memorystore hits also speed up by In-Memory store
    • Fetch job that keeps active nodes refreshed in cache

    Available clients

    • Node.js Client
    • Any Http Client can be used to access the HTTP-Interface

    Usage

    • You will need a Google Cloud Project with a running Bigtable cluster
    • Additionally YildizDB requires a Memorystore (or Redis) instance
    • Configure ./config/bigtable.json accordingly
    • Install and start: npm install -g yildiz and yildizdb -p 3058 -l ./config/bigtable.json
    • A word on configuration can be found here
    • Use right alongside your code
    • Spawn server via http interface

    Deployment

    • YildizDB is designed to be deployed as simple platform service
    • it requires Node.js > 9.x.x and Redis > 3.x
    • It simply requires its npm module as well as a config file that describes the connections to Bigtable and Memorystore (Redis)
    • It scales best with an HTTP load balancer in front of it e.g. NGINX
    • We run and scale it very successfully in Google's Kubernetes Engine
    • We also offer HELM charts
    • Read some more about the config file here

    Metrics & Monitoring

    • Yildiz exposes Prometheus Metrics @ /admin/metrics
    • Read more about it here

    Developing YildizDB

    Developing YildizDB with Google Bigtable backend

    • Configure ./config/bigtable.json accordingly
    • Run tests via yarn test
    • Start via yarn http

    Documentation

    Disclaimer

    • This project is not affiliated with Google
    • License is MIT see

    Install

    npm i yildiz

    DownloadsWeekly Downloads

    9

    Version

    9.1.0

    License

    MIT

    Unpacked Size

    738 kB

    Total Files

    212

    Last publish

    Collaborators

    • krystianity
    • rjmasikome