@travetto/model-query

Data Model Querying

Datastore abstraction for advanced query support.

Install: @travetto/model-query

npm install @travetto/model-query

# or

yarn add @travetto/model-query

This module provides an enhanced query contract for Data Modeling Support implementations. This contract has been externalized due to it being more complex than many implementations can natively support. In addition to the contract, this module provides support for textual query language that can be checked and parsed into the proper query structure.

Contracts

Query

This contract provides the ability to apply the query support to return one or many items, as well as providing counts against a specific query.

Code: Query

export interface ModelQuerySupport {
  /**
   * Executes a query against the model space
   * @param cls The model class
   * @param query The query to execute
   */
  query<T extends ModelType>(cls: Class<T>, query: PageableModelQuery<T>): Promise<T[]>;
  /**
   * Find one by query, fail if not found
   * @param cls The model class
   * @param query The query to search for
   * @param failOnMany Should the query fail on more than one result found
   */
  queryOne<T extends ModelType>(cls: Class<T>, query: ModelQuery<T>, failOnMany?: boolean): Promise<T>;
  /**
   * Find the count of matching documents by query.
   * @param cls The model class
   * @param query The query to count for
   */
  queryCount<T extends ModelType>(cls: Class<T>, query: ModelQuery<T>): Promise<number>;
}

Crud

Reinforcing the complexity provided in these contracts, the Query Crud contract allows for bulk update/deletion by query. This requires the underlying implementation to support these operations.

Code: Query Crud

export interface ModelQueryCrudSupport extends ModelCrudSupport, ModelQuerySupport {
  /**
   * A standard update operation, but ensures the data matches the query before the update is finalized
   * @param cls The model class
   * @param data The data
   * @param query The additional query to validate
  */
  updateOneWithQuery<T extends ModelType>(cls: Class<T>, data: T, query: ModelQuery<T>): Promise<T>;
  /**
   * Update/replace all with partial data, by query
   * @param cls The model class
   * @param query The query to search for
   * @param data The partial data
   */
  updateByQuery<T extends ModelType>(cls: Class<T>, query: ModelQuery<T>, data: Partial<T>): Promise<number>;
  /**
   * Delete all by query
   * @param cls The model class
   * @param query Query to search for deletable elements
   */
  deleteByQuery<T extends ModelType>(cls: Class<T>, query: ModelQuery<T>): Promise<number>;
}

Facet

With the complex nature of the query support, the ability to find counts by groups is a common and desirable pattern. This contract allows for faceting on a given field, with query filtering.

Code: Query Facet

export interface ModelQueryFacetSupport extends ModelQuerySupport {
  /**
   * Facet a given class on a specific field, limited by an optional query
   * @param cls The model class to facet on
   * @param field The field to facet on
   * @param query Additional query filtering
   */
  facet<T extends ModelType>(cls: Class<T>, field: ValidStringFields<T>, query?: ModelQuery<T>): Promise<{ key: string, count: number }[]>;
}

Suggest

Additionally, this same pattern avails it self in a set of suggestion methods that allow for powering auto completion and type-ahead functionalities.

Code: Query Suggest

export interface ModelQuerySuggestSupport extends ModelQuerySupport {
  /**
   * Suggest instances for a given cls and a given field (allows for duplicates with as long as they have different ids)
   *
   * @param cls The model class to suggest on
   * @param field The field to suggest on
   * @param prefix The search prefix for the given field
   * @param query A query to filter the search on, in addition to the prefix
   */
  suggest<T extends ModelType>(cls: Class<T>, field: ValidStringFields<T>, prefix?: string, query?: PageableModelQuery<T>): Promise<T[]>;
  /**
   * Suggest distinct values for a given cls and a given field
   *
   * @param cls The model class to suggest on
   * @param field The field to suggest on
   * @param prefix The search prefix for the given field
   * @param query A query to filter the search on, in addition to the prefix
   */
  suggestValues<T extends ModelType>(cls: Class<T>, field: ValidStringFields<T>, prefix?: string, query?: PageableModelQuery<T>): Promise<string[]>;
}

Implementations

Service	Query	QueryCrud	QueryFacet
Elasticsearch Model Source	X	X	X
MongoDB Model Support	X'	X'	X'
SQL Model Service	X'	X'	X'

Querying

One of the complexities of abstracting multiple storage mechanisms, is providing a consistent query language. The query language the module uses is a derivation of mongodb's query language, with some restrictions, additions, and caveats. Additionally, given the nature of typescript, all queries are statically typed, and will catch type errors at compile time.

General Fields

• field: { $eq: T } to determine if a field is equal to a value
• field: { $ne: T } to determine if a field is not equal to a value
• field: { $exists: boolean } to determine if a field exists or not, or for arrays, if its empty or not
• field: T to see if the field is equal to whatever value is passed in`

General Single Valued Fields

• field: { $in: T[] } to see if a record's value appears in the array provided to $in
• field: { $nin: T[] } to see if a record's value does not appear in the array provided to $in

Ordered Numeric Fields

• field: { $lt: number } checks if value is less than
• field: { $lte: number } checks if value is less than or equal to
• field: { $gt: number } checks if value is greater than
• field: { $gte: number } checks if value is greater than or equal to

Ordered Date Fields

• field: { $lt: Date | RelativeTime } checks if value is less than
• field: { $lte: Date | RelativeTime } checks if value is less than or equal to
• field: { $gt: Date | RelativeTime } checks if value is greater than
• field: { $gte: Date | RelativeTime } checks if value is greater than or equal to

Note: Relative times are strings consisting of a number and a unit. e.g. -1w or 30d. These times are always relative to Date.now, but should make building queries more natural.

Array Fields

• field: { $all: T[]] } checks to see if the records value contains everything within $all

String Fields

• field: { $regex: RegExp | string; } checks the field against the regular expression

Geo Point Fields

• field: { $geoWithin: Point[] } determines if the value is within the bounding region of the points
• field: { $near: Point, $maxDistance: number, $unit: 'km' | 'm' | 'mi' | 'ft' } searches at a point, and looks out radially

Groupings

• { $and: [] } provides a grouping in which all sub clauses are require,
• { $or: [] } provides a grouping in which at least one of the sub clauses is require,
• { $not: { } } negates a clause A sample query for User's might be:

Code: Using the query structure for specific queries

import { ModelQuerySupport } from '@travetto/model-query';
import { User } from './user';

export class UserSearch {
  service: ModelQuerySupport;

  find() {
    return this.service.query(User, {
      where: {
        $and: [
          {
            $not: {
              age: {
                $lt: 35
              }
            }
          },
          {
            contact: {
              $exists: true
            }
          }
        ]
      }
    });
  }
}

This would find all users who are over 35 and that have the contact field specified.

Query Language

In addition to the standard query interface, the module also supports querying by query language to facilitate end - user queries.This is meant to act as an interface that is simpler to write than the default object structure. The language itself is fairly simple, boolean logic, with parenthetical support.The operators supported are:

• <, <= - Less than, and less than or equal to
• >, >= - Greater than, and greater than or equal to
• !=, == - Not equal to, and equal to
• ~ - Matches regular expression, supports the i flag to trigger case insensitive searches
• !, not - Negates a clause
• in, not-in - Supports checking if a field is in a list of literal values
• and, && - Intersection of clauses
• or, || - Union of clauses All sub fields are dot separated for access, e.g. user.address.city.A query language version of the previous query could look like:

Code: Query language with boolean checks and exists check

not (age < 35) and contact != null

A more complex query would look like:

Code: Query language with more complex needs

user.role in ['admin', 'root'] && (user.address.state == 'VA' || user.address.city == 'Springfield')

Regular Expression

When querying with regular expressions, patterns can be specified as 'strings' or as /patterns/. The latter allows for the case insensitive modifier: /pattern/i. Supporting the insensitive flag is up to the underlying model implementation.

Custom Model Query Service

In addition to the provided contracts, the module also provides common utilities and shared test suites.The common utilities are useful for repetitive functionality, that is unable to be shared due to not relying upon inheritance(this was an intentional design decision).This allows for all the Data Model Querying implementations to completely own the functionality and also to be able to provide additional / unique functionality that goes beyond the interface. To enforce that these contracts are honored, the module provides shared test suites to allow for custom implementations to ensure they are adhering to the contract's expected behavior.

Code: MongoDB Service Test Configuration

import { Suite } from '@travetto/test';

import { ModelQuerySuite } from '@travetto/model-query/support/test/query';
import { ModelQueryCrudSuite } from '@travetto/model-query/support/test/crud';
import { ModelQueryFacetSuite } from '@travetto/model-query/support/test/facet';
import { ModelQueryPolymorphismSuite } from '@travetto/model-query/support/test/polymorphism';
import { ModelQuerySuggestSuite } from '@travetto/model-query/support/test/suggest';
import { ModelQueryCrudSupport } from '@travetto/model-query/src/service/crud';
import { ModelQuerySuggestSupport } from '@travetto/model-query/src/service/suggest';
import { ModelQueryFacetSupport } from '@travetto/model-query/src/service/facet';
import { Config } from '@travetto/config';
import { Injectable } from '@travetto/di';

import { QueryModelService } from './query-service';

@Config('model.custom')
class CustomModelConfig { }

@Injectable()
class CustomModelService extends QueryModelService implements ModelQueryCrudSupport, ModelQueryFacetSupport, ModelQuerySuggestSupport {
}

@Suite()
export class CustomQuerySuite extends ModelQuerySuite {
  serviceClass = CustomModelService;
  configClass = CustomModelConfig;
}

@Suite()
export class CustomQueryCrudSuite extends ModelQueryCrudSuite {
  serviceClass = CustomModelService;
  configClass = CustomModelConfig;
}

@Suite()
export class CustomQueryFacetSuite extends ModelQueryFacetSuite {
  serviceClass = CustomModelService;
  configClass = CustomModelConfig;
}

@Suite()
export class CustomQueryPolymorphismSuite extends ModelQueryPolymorphismSuite {
  serviceClass = CustomModelService;
  configClass = CustomModelConfig;
}

@Suite()
export class CustomQuerySuggestSuite extends ModelQuerySuggestSuite {
  serviceClass = CustomModelService;
  configClass = CustomModelConfig;
}