Nuns Playing Monopoly

    @thi.ng/rstream-query
    TypeScript icon, indicating that this package has built-in type declarations

    2.1.8 • Public • Published

    rstream-query

    npm version npm downloads Twitter Follow

    This project is part of the @thi.ng/umbrella monorepo.

    About

    @thi.ng/rstream based triple store & reactive query engine with declarative query specs related to Datalog / SPARQL. Inserted triples / facts are broadcast to multiple indexing streams and any query subscriptions attached to them. This enables push-based, auto-updating query results, which are changing each time upstream transformations & filters have been triggered.

    Triples are 3-tuples of [subject, predicate, object]. Unlike with traditional RDF triple stores, any JS data types can be used as subject, predicate or object (though support for such must be explicitly enabled & this feature is currently WIP).

    Current features

    • Dynamic & declarative dataflow graph construction via high-level data specs and/or functions
    • Entirely based on stream abstractions provided by @thi.ng/rstream
    • All data transformations done using dynamically composed tranducers
    • Query optimizations
    • Extensive re-use of existing sub-query results (via subscriptions)
    • Interim result de-duplication / dataflow gates
    • Push-based, auto-updating query results

    Status

    STABLE - used in production

    Search or submit any issues for this package

    This project is currently still in early development and intended as a continuation of the Clojure based thi.ng/fabric, this time built on the streaming primitives provided by @thi.ng/rstream.

    Installation

    yarn add @thi.ng/rstream-query

    ES module import:

    <script type="module" src="https://cdn.skypack.dev/@thi.ng/rstream-query"></script>

    Skypack documentation

    For Node.js REPL:

    # with flag only for < v16
    node --experimental-repl-await
    
    > const rstreamQuery = await import("@thi.ng/rstream-query");
    

    Package sizes (gzipped, pre-treeshake): ESM: 2.74 KB

    Dependencies

    Usage examples

    Several demos in this repo's /examples directory are using this package.

    A selection:

    Screenshot Description Live demo Source
    Triple store query results & sortable table Demo Source

    API

    Generated API docs

    import { TripleStore, asTriples } from "@thi.ng/rstream-query";
    import { trace } from "@thi.ng/rstream";
    
    // create store with initial set of triples / facts
    const store = new TripleStore([
        ["london", "type", "city"],
        ["london", "part-of", "uk"],
        ["portland", "type", "city"],
        ["portland", "partOf", "oregon"],
        ["portland", "partOf", "usa"],
        ["oregon", "type", "state"],
        ["usa", "type", "country"],
        ["uk", "type", "country"],
    ]);
    
    // alternatively, convert an object into a sequence of triples
    const store = new TripleStore(asTriples({
        london: {
            type: "city",
            partOf: "uk"
        },
        portland: {
            type: "city",
            partOf: ["oregon", "usa"]
        },
        oregon: { type: "state" },
        uk: { type: "country" },
        usa: { type: "country" },
    });
    
    // compile the below query spec into a dataflow graph
    // pattern items prefixed w/ "?" are query variables
    
    // this query matches the following relationships
    // using all currently known triples in the store
    // when matching triples are added or removed, the query
    // result updates automatically...
    
    // currently only "where" and bounded "path" sub-queries are possible
    // in the near future, more query types will be supported
    // (e.g. optional relationships, pre/post filters etc.)
    store.addQueryFromSpec({
        q: [
            {
                // all "where" subqueries are joined (logical AND)
                where: [
                    // match any subject of type "city"
                    ["?city", "type", "city"],
                    // match each ?city var's "part-of" relationships (if any)
                    ["?city", "partOf", "?country"],
                    // matched ?country var must have type = "country"
                    ["?country", "type", "country"]
                ]
            }
        ],
        // `bind` is an (optional) query post-processor and
        // allows injection of new variables into the result set
        // here we create a new var "answer" whose values are derived from
        // the other two query vars
        bind: {
            answer: (res) => `${res.city} is located in ${res.country}`
        },
        // another post-processing step, only keeps "answer" var in results
        select: ["answer"]
    })
    .subscribe(trace("results"))
    // results Set {
    //   { answer: 'london is located in uk' },
    //   { answer: 'portland is located in usa' } }
    
    // helper fn to insert new city relationship to the store
    const addCity = (name, country) =>
        store.into([
            [name, "type", "city"],
            [name, "partOf", country],
            [country, "type", "country"],
        ]);
    
    addCity("berlin", "germany");
    // results Set {
    //     { answer: 'london is located in uk' },
    //     { answer: 'portland is located in usa' },
    //     { answer: 'berlin is located in germany' } }
    
    addCity("paris", "france");
    // results Set {
    //     { answer: 'london is located in uk' },
    //     { answer: 'portland is located in usa' },
    //     { answer: 'berlin is located in germany' },
    //     { answer: 'paris is located in france' } }

    After setting up the above query and its internal transformations, the generated dataflow topology then looks as follows:

    graphviz output

    • The blue nodes are TripleStore-internal index stream sources, emitting changes when new triples are added
    • The left set of red nodes are the sub-queries of the above where clause, responsible for joining the individual (S)ubject, (P)redicate and (O)bject sub-queries.
    • The results of these are then further joined (right red node) & transformed to produce the final solution set and post-process it

    Btw. The diagram has been generated using @thi.ng/rstream-dot and can be recreated by calling store.toDot() (for the above example)

    The source code for the above example is here

    (Many) more features forthcoming...

    Authors

    Karsten Schmidt

    If this project contributes to an academic publication, please cite it as:

    @misc{thing-rstream-query,
      title = "@thi.ng/rstream-query",
      author = "Karsten Schmidt",
      note = "https://thi.ng/rstream-query",
      year = 2018
    }

    License

    © 2018 - 2022 Karsten Schmidt // Apache Software License 2.0

    Install

    npm i @thi.ng/rstream-query

    DownloadsWeekly Downloads

    401

    Version

    2.1.8

    License

    Apache-2.0

    Unpacked Size

    64.1 kB

    Total Files

    20

    Last publish

    Collaborators

    • thi.ng