schema-mapper-spec

Schema mapper defines a specification for data and is specifically designed to make storing data in different data stores less painful (especially for the ones that need a schema to be more useful).

It provides:

• A definition for a Project containing a name, a version and schemas.
• A definition for a Schema that is easy to write by humans but also easy to parse for computers.
• A definitions for Changes that describe modifications made to a project.
• A definition for DataInfo which describes a transport format for data and includes information about the project and schema.

A Project groups a bunch of schemas and versions them.

A Schema describes the type of the data and the columns it has.

Changes represent individual modifications to the schema. These changes can be used to transform a DataInfo object from a old version to a newer one and because we can easily invert changes because they contain the previous value, you can transform new data to data that is compatible with older versions of the schema as well.

A DataInfo object describes data and includes schema, project and version information.

Using the specifications

There are a couple of ways to use this specification.

• Validate your schemas, data and types by using the JSON-schemas for the different types using tools like:
  • tv4
  • is-my-json-valid
  • JSON-schema validators are implemented in other languages as well
• Validate your schemas, data and types by using the validator provider by this package.
  • Node API
  • CLI
• Automatically get changes by comparing 2 Projects using schema-mapper-differ
• Transform DataInfo objects by applying changes by using schema-mapper-tranformer

Validation

npm install --save schema-mapper-validator

Usage:

var validator = require('schema-mapper-validator');
 
var dataInfoValidationResult = validator.validateDataInfo(dataInfo);
var schemaValidationResult = validator.validateProject(project);
var schemaValidationResult = validator.validateSchema(schema);
var changesValidationResult = validator.validateChanges(changes);
console.log(dataInfoValidationResult, schemaValidationResult, changesValidationResult);

Types

Below is a specification of the types

ProjectCollection

A ProjectCollection groups projects together. It is an object where the key is the id of the project and the value is a Project.

Project

The project is an object that contains:

• A name property, that will for example be used for determining the name of a database.
• A version property, that will for example be used for determining the name of a database.
• A schemas property that contains the schemas of the project.

Name	Type	Description
name	String	The name of the project
version	Number	The version of the project
schemas	Object	An object of schemas where the key is the unique id for the schema and the value is an object of type Schema

SchemaCollection

A SchemaCollection groups schemas together. It is an object where the key is the id of the schema and the value is a Schema.

Schema

The schema is an object that contains:

• A name property, that will for example be used for determining the name of a database table.
• A columns property that describe the properties of the data.

Name	Type	Description
name	String	The name of the schema
columns	Object	An object of columns where the key is the unique id for the column and the value is an object of type Column

Example:

{
  "name": "users",
  "columns": {
    "1": {
      "name": "id",
      "type": "uuid"
    },
    "2": {
      "name": "name",
      "type": "string"
    },
    "3": {
      "name": "email",
      "type": "string"
    },
    "4": {
      "name": "location",
      "type": "point"
    }
  }
}

Columns

Columns describe all the properties of data. It is an object where the key is the id of the column and the value is a Column.

Column

A column describes a single property of data.

Name	Type	Description
name	String	The name of the column
type	String	The type of the column, which can be any ColumnType

Example:

{
  "name": "id",
  "type": "uuid"
}

ColumnType

The column type is a string indicating the type of the value in the data. Here is a table to see what a the different types are and what their data format looks like.

Column type	JSON type	Example
uuid	String	"583814cd-b90a-4341-8825-ab42d1125666"
uuid[]	Array	["583814cd-b90a-4341-8825-ab42d1125666"]
string	String	"Hello"
string[]	Array	["Hello", "World"]
text	String	"I can be very long"
text[]	Array	["It's just text", "Yo!"]
point	Object (GeoJSON Point)	{ "type": "Point", "coordinates": [50, 5] }
point[]	Object (GeoJSON MultiPoint)	{ "type": "MultiPoint", "coordinates": [ [100.0, 0.0], [101.0, 1.0] ] }
linestring	Object (GeoJSON LineString)	{ "type": "LineString", "coordinates": [ [100.0, 0.0], [101.0, 1.0] ] }
linestring[]	Object (GeoJSON MultiLineString)	{ "type": "MultiLineString", "coordinates": [ [ [100.0, 0.0], [101.0, 1.0] ], [ [102.0, 2.0], [103.0, 3.0] ] ] }
polygon	Object (GeoJSON Polygon)	{ "type": "Polygon", "coordinates": [ [ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ] ] }
polygon[]	Object (GeoJSON MultiPolygon)	{ "type": "MultiPolygon", "coordinates": [ [[[102.0, 2.0], [103.0, 2.0], [103.0, 3.0], [102.0, 3.0], [102.0, 2.0]]], [[[100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0]], [[100.2, 0.2], [100.8, 0.2], [100.8, 0.8], [100.2, 0.8], [100.2, 0.2]]] ] }
date	String	"2015-08-03"
date[]	Array	["2015-08-03", "2016-09-04"]
datetime	String	"2015-08-03 12:42:33"
datetime[]	Array	["2015-08-03 12:42:33", "2015-08-03 12:42:49"]
float	Number	15.04
float[]	Array	[14.04, 15.04]
integer	Number	420
integer[]	Array	[1, 2, 3]
boolean	Boolean	true
boolean[]	Array	[ true, false ]
json	Object	{ test: true, number: 1 }
json[]	Array	[{ test: true }, {test: false}]

DataInfo

An object describing a piece of data. This object is used as input for the transformer. The transformer will apply changes to this object and modify the name, version and data if needed.

Name	Type	Description
projectId	String	A string indicating the id of the project
projectName	String	A string indicating the name of the project
projectVersion	Number	A string indicating the version of the project
schemaId	String	A string indicating the id of the schema
schemaName	String	A string indicating the name of the schema
item	Object	The item that conforms to the schema

Example:

{
  "projectId": "1",
  "projectName": "demo",
  "projectVersion": 3,
  "schemaId": "1",
  "schemaName": "users",
  "data": {
    "id": "088ea6d5-fd14-4460-aed0-1a4b0ceed555",
    "name": "Koen"
  }
}

Changes

Changes describe a set of changes, changes are usually grouped a versioning mechachnism that spans across / locks the individual schema versions. The data format is a simple array, containing changes as described below.

ProjectCreateChange

TODO

ProjectTagChange

TODO

ProjectRenameChange

TODO

ProjectRemoveChange

TODO

SchemaCreateChange

This change indicates that a schema is created.

Name	Type	Description
change	String	A string indicating the type of change
projectId	String	The id of the project
schemaId	String	The id of the schema
schema	String	A Schema

Example:

{
  "change": "schema.create",
  "projectId": "1",
  "schemaId": "1",
  "schema": {
    "name": "users",
    "columns": {
      "1": {
        "name": "id",
        "type": "uuid"
      }
    }
  }
}

SchemaRemoveChange

This change indicates that a schema is removed.

Name	Type	Description
change	String	A string indicating the type of change
projectId	String	The id of the project
schemaId	String	The id of the schema
oldSchema	String	The old schema

Example:

{
  "change": "schema.remove",
  "schemaId": "1",
  "oldSchema": {
    "name": "users",
    "columns": {}
  }
}

SchemaRenameChange

This change indicates that a schema is renamed.

Name	Type	Description
change	String	A string indicating the type of change
schemaId	String	The id of the schema
name	String	The name of the schema
oldName	String	The old name of the schema

Example:

{
  "change": "schema.rename",
  "schemaId": "1",
  "name": "users",
  "oldName": "user"
}

ColumnCreateChange

This change indicates that a column is added to a schema.

Name	Type	Description
change	String	A string indicating the type of change
schemaId	String	The id of the schema
columnId	String	The id for the column
column	String	The properties of the Column

Example:

{
  "change": "column.create",
  "schemaId": "1",
  "columnId": "1",
  "column": {
    "name": "id",
    "type": "uuid"
  }
}

ColumnRemoveChange

This change indicates that a column is removed from a schema.

Name	Type	Description
change	String	A string indicating the type of change
schemaId	String	The id of the schema
columnId	String	The id for the column
oldColumn	Object	The properties of the old Column

Example:

{
  "change": "column.remove",
  "schemaId": "1",
  "columnId": "2",
  "oldColumn": {
    "name": "name",
    "type": "string"
  }
}

ColumnRenameChange

This change indicates that a column is removed from a schema.

Name	Type	Description
schemaId	String	The id of the schema
change	String	A string indicating the type of change
columnId	String	The id for the column
name	String	The new name of the column
oldName	String	The old name of the column

Example:

{
  "change": "column.rename",
  "schemaId": "1",
  "columnId": "1",
  "name": "id",
  "oldName": "user_id"
}

ColumnTypechangeChange

This change indicates that the type of a column is changed.

Name	Type	Description
change	String	A string indicating the type of change
schemaId	String	The id of the schema
columnId	String	The id for the column
columnName	String	The name of the column
type	String	The new type of the column
oldType	String	The old type of the column

Example:

{
  "change": "column.typechange",
  "schemaId": "1",
  "columnId": "1",
  "type": "integer",
  "oldType": "uuid"
}

API docs

API Docs

Licence

MIT