New Power Manual

    shex

    1.0.0-alpha.24 • Public • Published

    NPM Version Build Status Coverage Status ShapeExpressions Gitter chat https://gitter.im/shapeExpressions/Lobby DOI

    shex.js

    shex.js javascript implementation of Shape Expressions (try online)

    This shex npm package is a meta-package which includes the top-level @shexjs/ packages.

    install

    npm install --save shex

    validation

    You can validate RDF data using the bin/validate executable or the lib/ShExValidation library described below.

    validation executable

    Validate something in HTTP-land:

    ./node_modules/shex/bin/validate \
        -x http://shex.io/examples/Issue.shex \
        -d http://shex.io/examples/Issue1.ttl \
        -s http://shex.io/examples/IssueShape \
        -n http://shex.io/examples/Issue1

    That validates node http://shex.io/examples/Issue in http://shex.io/examples/Issue1.ttl against shape http://shex.io/examples/IssueShape in http://shex.io/examples/Issue.shex. The result is a JSON structure which tells you exactly how the data matched the schema.

    {
      "type": "test",
      "node": "http://shex.io/examples/Issue1",
      "shape": "http://shex.io/examples/IssueShape",
      "solution": {
    ...
      }
    }

    Had we gotten a null, we'd know that the document was invalid with respect to the schema. See the ShExJ primer for a description of ShEx validation and the ShExJ specification for more details about the results format.

    relative resolution

    validate's -n and -s arguemtns are evaluated as IRIs relative to the (first) data and schema sources respectively. The above invocation validates the node <Issue1> in http://shex.io/examples/Issue1.ttl. This and the shape can be written as relative IRIs:

    ./node_modules/shex/bin/validate \
        -x http://shex.io/examples/Issue.shex \
        -d http://shex.io/examples/Issue1.ttl \
        -s IssueShape \
        -n Issue1

    validation library

    Parsing from the old interwebs involves a painful mix of asynchronous callbacks for getting the schema and the data and parsing the data (shorter path below):

    var shexc = "http://shex.io/examples/Issue.shex";
    var shape = "http://shex.io/examples/IssueShape";
    var data = "http://shex.io/examples/Issue1.ttl";
    var node = "http://shex.io/examples/Issue1";
    
    var http = require("http");
    var shex = require("shex");
    var n3 = require('n3');
    
    // generic async GET function.
    function GET (url, then) {
      http.request(url, function (resp) {
        var body = "";
        resp.on('data', function (chunk) { body += chunk; });
        resp.on("end", function () { then(body); });
      }).end();
    }
    
    var Schema = null; // will be loaded and compiled asynchronously
    var Triples = null; // will be loaded and parsed asynchronously
    function validateWhenEverythingsLoaded () {
      if (Schema !== null && Triples !== null) {
        console.log(shex.Validator.construct(Schema).validate(Triples, node, shape));
      }
    }
    
    // loaded the schema
    GET(shexc, function (b) {
      // callback parses the schema and tries to validate.
      Schema = shex.Parser(shexc).parse(b)
      validateWhenEverythingsLoaded();
    });
    
    // load the data
    GET(data, function (b) {
      // callback parses the triples and tries to validate.
      var db = n3.Store();
      n3.Parser({baseIRI: data, format: "text/turtle"}).parse(b, function (error, triple, prefixes) {
        if (error) {
          throw Error("error parsing " + data + ": " + error);
        } else if (triple) {
          db.addQuad(triple)
        } else {
          Triples = db;
          validateWhenEverythingsLoaded();
        }
      });
    });

    See? That's all there was too it!

    OK, that's miserable. Let's use the ShExLoader to wrap all that callback misery:

    const shexc = "http://shex.io/examples/Issue.shex";
    const data = "http://shex.io/examples/Issue1.ttl";
    const node = "http://shex.io/examples/Issue1";
    
    const ShExLoader = require("@shexjs/loader");
    const { ctor: RdfJsDb } = require('@shexjs/neighborhood-rdfjs')
    const ShExValidator = require("@shexjs/validator");
    ShExLoader.load({shexc: [shexc]}, {turtle: [data]}).then(function (loaded) {
        var db = RdfJsDb(loaded.data);
        var validator = ShExValidator.construct(loaded.schema, db, { results: "api" });
        var result = validator.validate([{node: node, shape: ShExValidator.start}]);
        console.log(result);
    });

    Note that the shex loader takes an array of ShExC schemas, either file paths or URLs, and an array of JSON schemas (empty in this invocation) and an array of Turtle files.

    conversion

    As with validation (above), you can convert by either executable or library.

    conversion executable

    ShEx can be represented in the compact syntax

    PREFIX ex: <http://ex.example/#>
    <IssueShape> {                       # An <IssueShape> has:
        ex:state (ex:unassigned            # state which is
                  ex:assigned),            #   unassigned or assigned.
        ex:reportedBy @<UserShape>        # reported by a <UserShape>.
    }
    

    or in JSON:

    { "type": "schema", "start": "http://shex.io/examples/IssueShape",
      "shapes": {
        "http://shex.io/examples/IssueShape": { "type": "shape",
          "expression": { "type": "eachOf",
            "expressions": [
              { "type": "tripleConstraint", "predicate": "http://ex.example/#state",
                "valueExpr": { "type": "valueClass", "values": [
                    "http://ex.example/#unassigned", "http://ex.example/#assigned"
              ] } },
              { "type": "tripleConstraint", "predicate": "http://ex.example/#reportedBy",
                "valueExpr": { "type": "valueClass", "reference": "http://shex.io/examples/UserShape" }
              }
    ] } } } }

    You can convert between them with shex-to-json:

    ./node_modules/shex/bin/shex-to-json http://shex.io/examples/Issue.shex

    and, less elegantly, back with json-to-shex.

    conversion by library

    As with validation, the ShExLoader wrapes callbacks and simplifies parsing the libraries:

    var shexc = "http://shex.io/examples/Issue.shex";
    
    var shex = require("shex");
    shex.Loader.load({shexc: [shexc]}, null).then(function (loaded) {
        console.log(JSON.stringify(loaded.schema, null, "  "));
    });

    There's no actual conversion; the JSON representation is just the stringification of the parsed schema.

    local files

    Command line arguments which don't start with http:// or https:// are assumed to be file paths. We can create a local JSON version of the Issues schema:

    ./node_modules/shex/bin/shex-to-json http://shex.io/examples/Issue.shex > Issue.json

    and use it to validate the Issue1.ttl as we did above:

    ./node_modules/shex/bin/validate \
        -j Issue.json \
        -d http://shex.io/examples/Issue1.ttl \
        -s http://shex.io/examples/IssueShape \
        -n http://shex.io/examples/Issue1

    Of course the data file can be local as well.

    Happy validating!

    materialize

    Materialize is used to transform from a source schema to a target schema after validation is done.

    The syntax is:

    materialize `-t <target schema>`|-h [-j `<JSON Vars File>`] [-r `<RDF root IRI>`]

    Materialize reads the output from the validate tool from STDIN and maps it to the specified target schema.

    If supplied, a JSON vars file will be referenced to fill in constant values not specified from the source. This is useful in assigning default fields to the target when there is no equivalent value in the source schema and source data.

    Here is an example of a simple JSON vars file:

    {
      "urn:local:Demographics:constSys": "System",
    }

    If this vars file content is used, then any time a variable in the target file with value "urn:local:Demographics:constSys" is seen, the value "System will be substituted.

    The RDF root IRI specifies the root node from which all nodes in the schema will descend. The default root if none is specified is: tag:eric@w3.org/2016/root

    Here are some examples:

    materialize -h
    validate -x source_schema.shex -l data.jsonld -s ProblemShape | materialize -t target_schema.shex -j vars.json
    cat problem.val | materialize -t target_schema.shex -j vars.json -r http://hl7.org/fhir/shape/problem

    ShEx2 features

    ShEx IMPORT Demo (with relative IRIs):

    1. open a browser window (we'll call validator) with https://rawgit.com/shexSpec/shex.js/main/doc/shex-simple.html
    2. open another browser window (we'll call viewer) with https://shex.io/shexTest/main/viewer?validation
    3. wait 'till viewer loads and look for "3circRefS1-IS2-IS3-IS3" (near the bottom)
    4. drag the "#3circRefS1-IS2-IS3-IS3" cell (or the ✓ to the left of it) to the right of the QueryMap area of validator
    5. click on the long label under "Manifest:", then the long label under "Passing:" and validate.

    It should validate, which involves the IMPORT of 3circRefS2-IS3 and 3circRefS3. 3circRefS2-IS3 also IMPORTs 3circRefS3 which shows that IMPORT is idempotent (has a side effect only the first time).

    lerna monorepo

    This repo uses lerna to manage multiple NPM packages. These packages are located in packages/*:

    Here are some commands you'll need:

    • building/testing
      • lerna bootstrap -- look for all packages in packages/*
      • npm ci -- to install root node_modules per package-lock.json
      • lerna list -- in case you ever wonder what packages flashed past your eyes
    • adding an NPM package (promise-worker) to one of our managed packages (webapp)
      • lerna add promise-worker --scope=@shexjs/webapp
    • remove a package you aren't using it after all:
      • edit e.g. the package.json (e.g. packages/shex-webapp/package.json) to remove the dependency
      • lerna bootstrap --scope @shexjs/webapp --no-ci --force-local (why)
      • if it's in node_modules, npm remove promise-worker
    • adding a dev package
      • shex.js follows the advice of lerna docs to "hoist" all dev dependencies to the root package (hoist:true in lerna.config). Lerna moves devDeps required in more than one package (e.g. the webpack deps in the webapp and extension-map packages) to the root so they won't appear in e.g. packages/extension-map/node_modules.
      • in principle, this should work: lerna add -dev pseudo-worker --scope=@shexjs/webapp
      • but it doesn't seem to so instead: (cd packages/shex-webapp && npm install --save-dev pseudo-worker)

    Install

    npm i shex

    DownloadsWeekly Downloads

    27

    Version

    1.0.0-alpha.24

    License

    MIT

    Unpacked Size

    14.9 kB

    Total Files

    3

    Last publish

    Collaborators

    • ericprud