npm

Does your artifact manager get in the way? Join us on Oct. 8 at 10am PT, to discuss how npm can help.Sign up »

rdf-test-suite

1.10.3 • Public • Published

RDF-Test-Suite.js

Build Status Coverage Status npm version Greenkeeper badge

This tool executes the RDF and SPARQL test suites with any given system.

It can either output human-readable test results to the console, or it can output machine-readable reports in the EARL vocabulary.

Currently, the following test suites are supported:

Not all RDF test suites are supported at the moment. However, this package is fully modular, so that support for different test cases can be implemented easily.

Planned support:

Installation

Either install it globally:

$ yarn global add rdf-test-suite

Or locally (as a dev dependency):

$ yarn add --dev rdf-test-suite

Usage

After installing, the rdf-test-suite script will become available.

This script requires some kind of engine as first argument, and a test suite manifest URL as second argument.

The engine is a JavaScript file that can handle the test suite. The interface of this engine depends on the manifest. For example, the SPARQL 1.1 test suite requires an IQueryEngine, while the RDF/XML test suite requires a IParser.

Basic execution

The following command executes the SPARQL 1.1 test suite on the engine myengine.js:

$ rdf-test-suite myengine.js http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl

An example of what this myengine.js looks like is available for the Comunica SPARQL engine.

This command will output something like this:

...
✔ syn-bad-pname-04 (http://www.w3.org/2009/sparql/docs/tests/data-sparql11/syntax-query/manifest#test_pn_bad_04)
✔ syn-bad-pname-05 (http://www.w3.org/2009/sparql/docs/tests/data-sparql11/syntax-query/manifest#test_pn_bad_05)
✔ syn-bad-pname-06 (http://www.w3.org/2009/sparql/docs/tests/data-sparql11/syntax-query/manifest#test_pn_bad_06)
✔ syn-bad-pname-07 (http://www.w3.org/2009/sparql/docs/tests/data-sparql11/syntax-query/manifest#test_pn_bad_07)
✔ syn-bad-pname-08 (http://www.w3.org/2009/sparql/docs/tests/data-sparql11/syntax-query/manifest#test_pn_bad_08)
✔ syn-bad-pname-09 (http://www.w3.org/2009/sparql/docs/tests/data-sparql11/syntax-query/manifest#test_pn_bad_09)
✔ syn-bad-pname-10 (http://www.w3.org/2009/sparql/docs/tests/data-sparql11/syntax-query/manifest#test_pn_bad_10)
✔ syn-bad-pname-11 (http://www.w3.org/2009/sparql/docs/tests/data-sparql11/syntax-query/manifest#test_pn_bad_11)
✔ syn-bad-pname-12 (http://www.w3.org/2009/sparql/docs/tests/data-sparql11/syntax-query/manifest#test_pn_bad_12)
✔ syn-bad-pname-13 (http://www.w3.org/2009/sparql/docs/tests/data-sparql11/syntax-query/manifest#test_pn_bad_13)
✔ syn-pp-in-collection (http://www.w3.org/2009/sparql/docs/tests/data-sparql11/syntax-query/manifest#test_pp_coll)
✖ 105 / 268 tests succeeded!

Test filtering

With the -t option, you can filter tests that should be executed based on a regex that will be matched with the test URI.

For example:

$ rdf-test-suite myengine.js http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl \
  -t test_pn_bad_0

Summarized output

By default, the printed test results are verbose, and print expanded views of failed tests.

You can change this to a more compact view using -o summary.

For example:

$ rdf-test-suite myengine.js http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl \
  -o summary

EARL output

Test results are human readable by default. This can be changed to output machine-readable reports in the EARL vocabulary in the Turtle serialization. These reports can be published to report the compliance of engines to certain specifications, like on W3C’s RDF Test Curation Community Group.

As EARL reports require some metadata on your engine, you will need to provide a properties file via the -p argument.

This properties file can look something like this:

{
  "applicationBugsUrl": "https://github.com/comunica/comunica/issues",
  "applicationDescription": "A Comunica engine for SPARQL query evaluation over heterogeneous interfaces",
  "applicationHomepageUrl": "http://comunica.linkeddatafragments.org/",
  "applicationNameFull": "Comunica SPARQL",
  "applicationNameNpm": "@comunica/actor-init-sparql",
  "applicationUri": "https://www.npmjs.com/package/@comunica/actor-init-sparql",
  "authors": [
    {
      "homepage": "https://www.rubensworks.net/",
      "name": "Ruben Taelman",
      "uri": "https://www.rubensworks.net/#me"
    }
  ],
  "licenseUri": "http://opensource.org/licenses/MIT",
  "reportUri": null,
  "specificationUris": [
      "http://www.w3.org/TR/sparql11-query/"
  ]
}

The properties file can point to a non-existing file, in which case it will be auto-generated from the package.json file in the current working directory.

For example:

$ rdf-test-suite myengine.js http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl \
  -o earl -p earl-meta.json

Restricting execution to certain specifications

Some test suites (like the one of SPARQL 1.1) contains tests for multiple specifications. If you only want to test your system for a single specification, then you can define this with the -s parameter.

For example, the following command executes the SPARQL 1.1 test cases that apply to the http://www.w3.org/TR/sparql11-query/ specification.

$ rdf-test-suite myengine.js http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl \
  -s http://www.w3.org/TR/sparql11-query/

Enabling HTTP caching

By default, rdf-test-suite will look up all required manifest files. If this takes too much time, or if you don't have internet connection, then you can enable HTTP caching with the -c argument, so that all files only have to be looked up once.

$ rdf-test-suite myengine.js http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl \
  -c path/to/cache/

If you don't provide a caching value after the -c, then the directory will default to .rdf-test-suite-cache/.

Ignore exit code

When there are failing tests, rdf-test-suite will exit with code 1 instead of 0. This can be useful in Continuous Integration tools.

If you want to disable this behaviour, you can add the -e flag to force an exit code 0.

$ rdf-test-suite myengine.js http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl \
  -e

Pass custom options to the engine

With the optional -i option, a JSON string with arguments can be passed to the engine. These arguments will become available in both IQueryEngine, and IParser.

$ rdf-test-suite myengine.js http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl \
  -i '{ "myProperty": "myValue" }'

Test timeouts

By default, all tests are allowed to run 3000 ms. Using the -d option, you can change this value.

$ rdf-test-suite myengine.js http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl \
  -d 5000

Map URLs to local files

In cases where you are developing your own test manifests, it may be useful to intercept certain URL lookups and return local files instead.

In order to achieve this, the -m option can be used, with a mapping defined with the following pattern: URL~PATH.

$ rdf-test-suite myengine.js http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl \
  -m http://w3c.github.io/rdf-tests/sparql11/data-sparql11/~/path/to/my/files/

Supported test suites

Manifest Specification Interface Entry manifest
SPARQL 1.1 tests SPARQL 1.1 Query IQueryEngine http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl
SPARQL 1.1 tests SPARQL 1.1 Update http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl
SPARQL 1.1 tests SPARQL 1.1 Results CSV/TSV http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl
SPARQL 1.1 tests SPARQL 1.1 Results JSON http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl
SPARQL 1.1 tests SPARQL 1.1 Federated Query http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl
SPARQL 1.1 tests SPARQL 1.1 Entailment http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl
SPARQL 1.1 tests SPARQL 1.1 Service Description http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl
SPARQL 1.1 tests SPARQL 1.1 Protocol http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl
SPARQL 1.1 tests SPARQL 1.1 HTTP RDF Update http://w3c.github.io/rdf-tests/sparql11/data-sparql11/manifest-all.ttl
RDF/XML Syntax Tests RDF 1.1 XML Syntax IParser http://w3c.github.io/rdf-tests/rdf-xml/manifest.ttl
N-Triples Tests RDF 1.1 N-Triples IParser http://w3c.github.io/rdf-tests/ntriples/manifest.ttl
N-Quads Tests RDF 1.1 N-Quads IParser http://w3c.github.io/rdf-tests/nquads/manifest.ttl
Turtle Tests RDF 1.1 Turtle IParser http://w3c.github.io/rdf-tests/turtle/manifest.ttl
TriG Tests RDF 1.1 TriG IParser http://w3c.github.io/rdf-tests/trig/manifest.ttl
JSON-LD Test Suite JSON-LD 1.0 IParser https://w3c.github.io/json-ld-api/tests/toRdf-manifest.jsonld
JSON-LD Test Suite JSON-LD 1.0 ISerializer https://w3c.github.io/json-ld-api/tests/fromRdf-manifest.jsonld
RDFa Test Suite RDFa 1.1 ISerializer http://rdfa.info/test-suite/test-cases/rdfa1.1/html5/manifest.ttl

License

This software is written by Ruben Taelman.

This code is released under the MIT license.

install

npm i rdf-test-suite

Downloadsweekly downloads

139

version

1.10.3

license

MIT

homepage

github.com

repository

Gitgithub

last publish

collaborators

  • avatar
Report a vulnerability