Naughty Praying Mantis

    TypeScript icon, indicating that this package has built-in type declarations

    0.1.0-alpha.0 • Public • Published

    Linked Data Event Stream snapshots package


    This package facilitates creating a snapshot of a versioned Linked Data Event Stream (LDES).

    What is a snapshot

    A snapshot of an LDES at time t is an LDES where only the most recent versions until time t are present.

    This indicates that for each version, only one remains in the snapshot.

    One of the benefits of creating a snapshot is that it captures the total state of all members at time t while being smaller in size.

    A downside, however, is that by throwing away all older versions, it loses the history of that LDES.

    Version materialized snapshot

    This package also supports to create the version materialization of a snapshot.

    Note: This package uses @treecg/version-materialize-rdf.js as a basis for the version materializations.

    How to create a snapshot

    First, you have to install the package.

    npm install @treecg/ldes-snapshot

    To create a snapshot of an LDES, there are two options.

    The first option is to create a snapshot on a N3 Store. This way of creating a snapshot is straightforward. Load the ldes in a Store and then create the snapshot. As simple as that! However, there is a downside: This approach will only work for an LDES that can be completely be loaded into memory. Which means it only works for small LDESs.

    The second option is to create a snapshot streamingwise using a Transform. As input a stream of members is required. This stream is then piped through the transformer which will, when the stream stops, emit a stream of snapshot members (which are version materialized). While working with streams might be a little more difficult, it allows to create a snapshot of a bigger LDESs as an LDES does not have to be loaded in memory.

    🔧 Configuring the snapshot

    Configuration for creating a snapshot is done by giving an options object (which has the ISnapshotOptions interface ).

    This object has the following parameters:

    parameter name default value description
    date new Date() A JavaScript Date object; the snapshot will be created until this timestamp
    snapshotIdentifier The identifier of the snapshot
    ldesIdentifier The identifier of the LDES of which you want to create a snapshot from (always required)
    versionOfPath The ldes:versionOfPath of the LDES (which is required in the Transform)
    timeStampPath The ldes:timestampPath of the LDES (which is required in the Transform)
    materialized false When true, the snapshot will be materialized

    Creating a snapshot using an N3 Store

    Below is an example of how to use this package. As LDES, the example from version materialization in the LDES specification is used.

    const Snapshot = require('@treecg/ldes-snapshot').Snapshot;
    const rdfParser = require("rdf-parse").default;
    const storeStream = require("rdf-store-stream").storeStream;
    const streamifyString = require('streamify-string');
    const ldesString = `
    @prefix dct: <> .
    @prefix ldes: <> .
    @prefix tree: <> .
    @prefix xsd: <> .
    @prefix owl: <> .
    @prefix rdfs: <> .
    @prefix ex: <> .
    ex:ES1 a ldes:EventStream;
           ldes:versionOfPath dct:isVersionOf;
           ldes:timestampPath dct:created;
           tree:member [
               dct:isVersionOf <A> ;
               dct:created "2020-10-05T11:00:00Z"^^xsd:dateTime;
               owl:versionInfo "v0.0.1";
               rdfs:label "A v0.0.1"
           ], [
               dct:isVersionOf <A> ;
               dct:created "2020-10-06T13:00:00Z"^^xsd:dateTime;
               owl:versionInfo "v0.0.2";
               rdfs:label "A v0.0.2"
    // have the above ldes as a N3.Store using rdf-parse.js (
    const textStream = streamifyString(ldesString);
    const quadStream = rdfParser.parse(textStream, {contentType: 'text/turtle'});
    const store = await storeStream(quadStream);
    // load the ldes store
    const snapshot = new Snapshot(store);
    // create the snapshot at a given time
    const snapshotCreated = await snapshot.create({
        date: new Date("2020-10-05T12:00:00Z"),
        ldesIdentifier: "",
        materialized: true

    When converting the store back to string, the following output is achieved

    const Writer = require("n3").Writer
    const writer = new Writer();
    <> <> <> .
    <> <> <> .
    <> <> "2020-10-05T12:00:00.000Z"^^<> .
    <> <> <A> .
    <A> <> <n3-0> .
    <A> <> "v0.0.1" .
    <A> <> "A v0.0.1" .
    <A> <> "2020-10-05T11:00:00"^^<> .

    Which is equivalent as the following (when prefixes are added):

    ex:snapshot <> tree:Collection .
    ex:snapshot ldes:versionMaterializationOf ex:ES1 .
    ex:snapshot ldes:versionMaterializationUntil "2020-10-05T12:00:00.000Z"^^xsd:dateTime .
    ex:snapshot tree:member <A> .
    <A> dct:hasVersion <n3-0> .
    <A> owl:versionInfo "v0.0.1" .
    <A> rdfs:label "A v0.0.1" .
    <A> dct:created "2020-10-05T11:00:00"^^xsd:dateTime .

    Creating a snapshot using streams

    const SnapshotTransform = require('@treecg/ldes-snapshot').SnapshotTransform
    const Readable = require("stream").Readable
    // Note: uses the store defined in the first option of creating a snapshot
    const members = store.getObjects(null, '', null)
    const memberStream = new Readable({
        objectMode: true,
        read() {
            for (let member of members) {
                    id: member,
                    quads: store.getQuads(member, null, null, null)
    const snapshotOptions = {
        date: new Date(),
        ldesIdentifier: "",
        snapshotIdentifier: "",
        versionOfPath: "",
        timestampPath: "",
        materialized: true
    const snapshotTransformer = new SnapshotTransform(snapshotOptions)
    const memberStreamTransformed = memberStream.pipe(snapshotTransformer)
    const Writer = require("n3").Writer
    const writer = new Writer();
    memberStreamTransformed.on('metadata', quads => {
    memberStreamTransformed.on('data', ({id, quads}) => {
    console.log(`member: ${id.value}`)
    memberStreamTransformed.on('end', () => {

    Which will output the following

    <> <> <> .
    <> <> <> .
    <> <> "2022-03-09T11:47:10.870Z"^^<> .
    member: A
    <A> <> <n3-1> .
    <A> <> "2020-10-06T13:00:00Z"^^<> .
    <A> <> "v0.0.2" .
    <A> <> "A v0.0.2" .

    Feedback and questions

    Do not hesitate to report a bug.

    Further questions can also be asked to Wout Slabbinck (developer and maintainer of this repository).


    npm i @treecg/ldes-snapshot

    DownloadsWeekly Downloads






    Unpacked Size

    86.9 kB

    Total Files


    Last publish


    • ajuvercr
    • woutslabbinck
    • ddvlanck
    • dexagod
    • pietercolpaert
    • hdlva
    • julianrojas87
    • brechtvdv
    • kasperzutterman