Swish

Swish is an API for querying JSON-based document stores. The API is based on JSON Schema for queries (with JSON Pointer for partial results), JSON Patch for updating (with convenience methods for search-by-example and JSON Merge Patch updates).

The idea is for different implementations/back-ends to be used, uncoupling the back-end store from the querying interface. This means the same API can be used with different backing databases, or even safely in the browser (by providing an HTTP back-end that talks to a server component).

Implementations will often translate from JSON Schema into another query language (such as MySQL, given a suitable ORM.). Some implementations may require a schema for the data (e.g. ORM-based ones) or other configuration (such as connection information), but the final object should hold the same API.

API v0.3

Queries/searches are described using a JSON Schema for the constraints.

Most methods have a ...ByExample() variant, which use an "example object" instead of a schema. This requires an exact match on any fields (or sub-fields) that are defined in the example, e.g. {"id": 123} or {"author": {"userId": 423}}. See .exampleToSchema() below for more information.

All documents returned by this API must be separate objects, regardless of caching or any other factors. Updating a fetched object should not have any effect on identically fetched objects, or the underlying data store.

db.search(schema, ?options, function (error, results, ?continueOptions) {...})

db.searchByExample(example, ?options, function (error, results, ?continueToken) {...})

Returns a list of documents matching the provided schema.

If continueOptions is not null, it indicates that more results are available (see options.limit). Calls to .search() using continueOptions as the options will provide the next set of results.

Options in continueOptions do not have to be standard or even sensible, so users of the API should not attempt to inspect, merge or otherwise inspect the value. If limit was supplied in the initial options, then this behaviour (if not the exact property) should be maintained through continueOptions.

db.create(record, ?options, function (error, newRecord) {...})

Add a record. The new record (as inserted) is returned in the callback.

Some database stores auto-assign a field (e.g. AUTOINCREMENT in MySQL). Implementations supporting this may use it to populate a field in the record. The auto-assigned field is therefore filled out in the newRecord result.

db.createMultiple(recordList, ?options, function (error, newRecords) {...})

Same as db.create(), but with multiple values.

Even in the case of an error, newRecords should still be returned containing the records that succeeded, in case they need to be cleaned up. Users of the API should still check it exists first.

db.replace(schema, value, ?options, function (error, changeCount) {...})

db.replaceByExample(example, value, ?options, function (error, changeCount) {...})

Replaces all matching entries with copies of value.

var newValue = {
    id: 500,
    title: "Title",
    description: "Description of object"
};
db.replaceByExample({id: 500}, newValue, function (error, updateCount) {...});

db.update(schema, mergeObj, ?options, function (error, changeCount) {...})

db.updateByExample(schema, mergeObj, ?options, function (error, changeCount) {...})

Updates all matching entries using the fields from mergeObj, which should be a JSON Merge Patch object.

var mergeObj = {
    specificField: "this field is changed",
    otherField: null // this field is removed,
    nestedObj: {
        subField: true // this subfield is changed
    }
};
db.updateByExample({id: 500}, mergeObj, function (error, updateCount) {...});

NOTE: A mergeObj of null will remove the entry, and be equivalent to a .remove() call.

db.patch(schema, patch, ?options, function (error, changeCount) {...})

db.patchByExample(example, patch, ?options, function (error, changeCount) {...})

Updates all matching entries using a JSON Patch (RFC 6902).

/* Schema matches all items with quantity <= 10 */
var schema = {
    "type": "object",
    "properties": {
        "quantity": {"type": "integer", "maximum": 10}
    }
};
/* Changes the "status" property */
var patch = [
  {"op": "replace", "path": "/status", "value": "Almost out of stock!"},
];
db.patch(schema, patch, function (error, updateCount) {...});

NOTE: A patch may remove the entire entry, e.g.: [{"op": "remove", "path": ""}]. This is equivalent to a .remove() call.

db.remove(schema, patch, ?options, function (error, removedCount) {...})

db.removeByExample(example, patch, ?options, function (error, removedCount) {...})

Removes all matching entries.

This is exactly equivalent to db.update(schema, null, ...) or db.patch() with a patch that removes the entire entry.

Options - the options object

All of the above method have an optional options argument. The following options are defined:

Search options

options.limit

The maximum number of results to return. (Applies to searches only)

Implementations may provide a default limit for this. To disable the limit and return all results, set options.limit to false.

options.offset

The offset at which to start - can be used in combination with options.limit for paging. Default is 0.

options.path

Sometimes, you don't need the whole record returned (e.g. you might want just the title).

If options.path is a string, this is interpreted as a JSON Pointer path. For each matching record, the specified sub-value from within the document is returned. The empty path "" would return the whole record (default behaviour).

If options.path is an object or array, then it is interpreted as a set of JSON Pointer paths. Each string value is interpreted as a JSON Pointer path, and the corresponding value in in the record is mapped to the appropriate sub-value. Any object or array properties are themselves mapped, and so on.

// Just get the user's name
userDb.searchByExample({id: 400}, {path: '/name'}, function (error, names) {
    ...
});
// List page IDs and titles only
pageDb.search({}, {path: ['/id', '/title']}, function (error, results) {
    if (error) throw error;
    results.forEach(function (pair) {
        var id = pair[0], title = pair[1];
        ...
    });
});

options.sort

A "sort specifier" can be in two forms:

• an object {"path": "/foo/bar", "direction": "asc"}
• a string: "asc/foo/bar" (this is unambiguous because non-empty JSON Pointer paths always begin with /)

The standard direction specifiers are asc and desc, however + and - can be substituted. If the specifier is omitted (no prefix at all), it defaults to asc.

options.sort can either be a single sort specifier, or an array of sort specifiers.

The "base class" (see below) normalises this option, so it is always presented as a list of objects, with direction specifiers as asc/desc.

Simplified implementation using the "base class"

While an implementation could just provide all the above methods, a "base class" is defined (the core swish module) that provides some argument-juggling and generates patches/schemas from examples/merges/etc.

Therefore, when extending this class, only the following methods are necessary:

• ._search(schema, options, callback)
• ._createMultiple(recordList, options, callback)
• ._patch(schema, patch, options, callback)

These (private) methods are called by the public methods in the API.

It is important to note that ._patch must support deletion of the whole value when such a patch is provided. The static method .patchIsRemove() may be useful for implementing this.

Optional methods

The following methods are not necessary to provide, but will be used if defined:

• ._create(record, options, callback)
• ._replace: function (schema, value, options, callback)
• ._update: function (schema, mergeObj, options, callback)
• ._remove: function (schema, options, callback)

By default, ._replace(), ._update() and ._remove() generate a suitable JSON Patch and then forward the call to ._patch().

By default, ._create() just calls ._createMultiple(), wrapping and unwrapping arguments to fit.

Static methods

The swish module/class also provides some static helper methods:

swish.exampleToSchema(exampleObj)

This is the function used by the ...ByExample() variants to generate schemas from the provided examples, defined by the following (recursive) rules:

• objects - type is set to "object", and each property defined in the example object is added to required. A corresponding schema is added to properties using the property value as an example
• arrays - items in the array are non-exclusive alternatives, represented using anyOf. If every schema in the array contains enum (and nothing else) then they are merged into a single enum clause for tidiness.
• other values - specified exactly using a single-length array in enum. type is not set.

swish.mergeToPatch(merge, ?pathPrefix)

This is the function used by the .update*() methods to convert a JSON Merge Patch into a JSON Patch.

swish.patchIsRemove(patch)

This function is not used by this module, but may be convenient for implementations.

It determines whether a given patch would remove the record completely (as opposed to altering it, or replacing it with null).

swish.implementation(?constructor, ?prototype)

This function is not used by this module, but may be convenient for implementations.

It creates a subclass extending the base class. The constructor (if supplied) should be a constructor function. If prototype is supplied (and is an object), then

var swish = require('swish');
 
var MySwishClass = swish.implementation(function (config) {
    this.config = config;
}, {
    _search: function (schema, options, callback) {...},
    _create: function (document, options, callback) {...},
    _patch: function (schema, newValue, options, callback) {...}
});
 
// Usage: var store = new MySwishClass({...});

Database schemas, unique keys, indexes, etc.

This API doesn't account for the store having a particular schema, or unique keys, or indexes.

Parameters such as this should be part of the implementation's configuration data. Implementations may need to supply extra functionality for their use, e.g. methods for adding a database index - however, it's encouraged that these methods use similar patterns to the rest of the API, for example using JSON Pointers instead of MySQL column names.