hdt

Native bindings to access HDT compressed triple files.

HDT for Node.js

HDT (Header Dictionary Triples) is a compressed format for RDF triples.
The hdt npm package for Node.js brings fast access to HDT files through C bindings.

Install the library by adding hdt to your package.json or executing

$ npm install hdt

Then require the library.

var hdt = require('hdt');

Open an HDT document with hdt.fromFile, which takes filename and callback arguments. The HDT document will be passed to the callback. Close the document with close.

hdt.fromFile('./test/test.hdt', function (error, hdtDocument) {
  // Don't forget to close the document when you're done
  hdtDocument.close();
});

Search for triples with search, which takes subject, predicate, object, options, and callback arguments. Subject, predicate, and object can be IRIs or literals, represented as simple strings. If any of these parameters is null or a variable, it is considered a wildcard. Optionally, an offset and limit can be passed in an options object, selecting only the specified subset.

The callback returns an array of triples that match the pattern. A third parameter indicates an estimate of the total number of matching triples.

hdt.fromFile('./test/test.hdt', function (error, hdtDocument) {
  hdtDocument.searchTriples('http://example.org/s1', null, null, { offset: 0, limit: 10 },
    function (error, triples, totalCount) {
      console.log('Approximately ' + totalCount + ' triples match the pattern.');
      triples.forEach(function (triple) { console.log(triple); });
      hdtDocument.close();
    });
});

Retrieve an estimate of the total number of triples matching a pattern with count, which takes subject, predicate, object, and callback arguments.

hdt.fromFile('./test/test.hdt', function (error, hdtDocument) {
  hdtDocument.countTriples('http://example.org/s1', null, null,
    function (error, totalCount) {
      console.log('Approximately ' + totalCount + ' triples match the pattern.');
      hdtDocument.close();
    });
});

In an HDT file that was generated with an FM index, you can search for literals that contain a certain substring.

hdt.fromFile('./test/literals.hdt', function (error, hdtDocument) {
  hdtDocument.searchLiterals('b', { offset: 0, limit: 5 },
    function (error, literals, totalCount) {
      console.log('Approximately ' + totalCount + ' literals contain the pattern.');
      literals.forEach(function (literal) { console.log(literal); });
      hdtDocument.close();
    });
});

The standalone utility hdt allows you to query HDT files from the command line.
To install system-wide, execute:

sudo npm install -g hdt

Specify queries as follows:

hdt dataset.hdt --query '?s ?p ?o' --offset 200 --limit 100 --format turtle

Replace any of the query variables by an IRI or literal to match specific patterns.

To build the module from source, follow these instructions:

git clone https://github.com/RubenVerborgh/HDT-Node.git hdt
cd hdt
git submodule init
git submodule update
npm install
npm test

If you make changes to the source, do the following to rebuild:

node-gyp build && npm test

The Node.js bindings for HDT are written by Ruben Verborgh.

This code is copyrighted by Ruben Verborgh and released under the GNU Lesser General Public License. It uses the HDT C++ Library, released under the same license.