sync-server

    2.0.0-beta.7 • Public • Published

    SyncServer

    Code Climate

    Synopsis

    A small node server which uses NeDB to write data to the disk. The server can be used with a client for example SyncClient to save change sets which can later be synchronized with other devices. The server was made to work with the ISyncProtocol and Dexie.Syncable. It supports the poll pattern using AJAX and the react pattern using nodejs-websocket.

    Installation and usage

    Install globally using npm:

    npm install -g sync-server

    Before using the server it has to be initialized with:

    sync-server init

    The init action must be executed in an empty directory which will later be used to store the data. This folder represents a Database. During initialization a config.json file is create with the default server configuration.

    You can start the server with:

    sync-server start --path INIT/DIRECTORY/PATH

    The --path flag must be given the path to the directory in which init was called.

    Default settings

    These settings are written in a file called config.json in the directory in which init was called. The config file is split into 4 sections: db, logging, server, and sync.

    db

    Setting name Value Description
    connector 'NeDB' The database used to store the data. Currently only NeDB is supported
    opts {} Options for the database. These depend on the selected connector

    NeDB Options

    The sync-server supports the following NeDB options:

    • inMemoryOnly
    • timestampData
    • corruptAlertThreshold

    The NeDB README contains more information about these options.

    logging

    Setting name Value Description
    errorLogFileName "error.log" File name for the error log. Contains information about exceptions and rejected promises
    accessLogFileName "access.log" File name for the access log. Contains information about the requests made against the server

    server

    Setting name Value Description
    requestSizeLimit "100kb" Request size limit for body-parser
    port 3000 Server port. Must be a non-privileged port
    protocol "http" Protocol used by the server. "http", "https", "ws" or "wss"
    https {} This object contains the paths for the files needed by https
    wss {} This object contains the paths for the files needed by wss
    cors {} You can use this object to configure CORS

    sync

    Setting name Value Description
    partialsThreshold 1000 If we have more than 1000 changes to send to the client, send only the first 1000 and partial: true

    Node.js Version

    You need to use a new version of Node.js as the code uses ES2015 features which are not available in Node.js versions < 6.0.0.

    Caveat

    In case the server encounters an uncaughtException or an unhandledRejection it will write to the log and exit with status code 1. This should normally not happen, if it does happen please open an issue with the information from the error log.

    Protocols

    The server supports 4 different protocols: http, https, ws and wss. The http and https protocols can be used for the poll pattern where the server and client communicate via HTTP requests. The ws and wss protocols can be used for the react pattern where server and client communicate via WebSockets. Per default the http protocol is used. For https and wss you have to at least provide certificates. See below on how to configure those.

    Configuring HTTPS

    In order to use HTTPS you need to set the protocol to "https" and add paths for the certificate in the https object. The attributes key and cert or for pfx are required. For example:

    {
      "server": {
        "protocol": "https",
        "https": {
          "key": "key_filename.pem",
          "cert": "cert_filename.pem"
        }
      }
    }

    The files must be in the same directory as the server's config file. You can also specify other options allowed by Node.js for a https server.

    Configuring WebSockets (WS)

    In order to use WebSockets you need to set the protocol to "ws".

    Configuring Secure WebSockets (WSS)

    In order to use WSS you need to set the protocol to "wss" and add paths for the certificate in the wss object. The attributes key and cert or for pfx are required. For example:

    {
      "server": {
        "protocol": "wss",
        "wss": {
          "key": "key_filename.pem",
          "cert": "cert_filename.pem"
        }
      }
    }

    The files must be in the same directory as the server's config file. You can also specify other options allowed by Node.js for a tls server and options allowed by nodejs-websocket.

    API for the poll pattern

    Synchronization

    • URL: /
    • Method: POST
    • ContentType: application/json. This header must be set, otherwise the server will not be able to parse the data
    • Params: JSON with
      • baseRevision: number (It is set to 0 if it is not defined)
      • changes: Array (The ChangeObj is described below)
      • clientIdentity: number (The server generates one if it is not defined)
      • syncedRevision: number (It is set to 0 if it is not defined)
      • requestId: any
      • partial: boolean (If true this is a partial synchronization. Default is false)
    • Return: JSON object
      • If the synchronization was successful
        • success: true
        • changes: Array (The ChangeObj is described below)
        • currentRevision: number
        • clientIdentity: number (The newly generated clientIdentity or the one that was provided by the client)
        • partial: boolean (This is a partial synchronization. The partialsThreshold number defines when we only send a partial synchronization)
        • requestId: any (requestId sent by the client)
      • If the synchronization failed
        • success: false
        • errorMessage: string
        • requestId: any (requestId sent by the client)
      • In both cases the status code is set to 200

    Online check

    Can be used to check if the server is online.

    • URL: /check
    • Method: HEAD
    • Params: None
    • Return: Headers

    API for the react pattern

    Currently the WebSocket server only supports sending and receiving text messages. Binary is not supported.

    Request Messages

    The server can receive 3 message types: clientIdentity, subscribe and changes.

    Response Messages

    The server can respond with 4 message types: clientIdentity, ack, changes and error.

    Requests

    clientIdentity

    This must be the first message sent.

    • Params: JSON with
      • type: "clientIdentity"
      • clientIdentity: number (The server generates one if it is not defined)
    • Server responds with clientIdentity

    subscribe

    This must be the second message sent. It is needed to setup callbacks to inform the client about changes made by other clients. You need to wait on the clientIdentity response before subscribing to make sure that the server saved the clientIdentity.

    • Params: JSON with
      • type: "subscribe"
      • syncedRevision: number (It is set to 0 if it is not defined)
    • Server responds with changes or error

    changes

    • Params: JSON with
      • type: "changes"
      • baseRevision: number (It is set to 0 if it is not defined)
      • changes: Array (The ChangeObj is described below)
      • partial: boolean (If true this is a partial synchronization. Default is false)
      • requestId: any
    • Server responds with ack or error
    • This event would trigger a changes or error message for all other connected clients

    Responses

    ack

    • Params: JSON object
      • type: "ack"
      • requestId: any (The ID sent by the client)

    clientIdentity

    • Params: JSON object
      • type: "clientIdentity"
      • clientIdentity: number (The newly generated clientIdentity or the one that was provided by the client)

    changes

    • Params: JSON object
      • type: "changes"
      • changes: Array (The ChangeObj is described below)
      • currentRevision: number
      • partial: boolean (This is a partial synchronization. The partialsThreshold number defines when we only send a partial synchronization)

    error

    • Params: JSON
      • type: "error"
      • errorMessage: string
      • requestId: any (Only sent if the changes request caused an error)

    ChangeObj

    There are 3 types of ChangeObj. See also Dexie.Syncable.IDatabaseChange

    CREATE

    Object with:

    • type: 1
    • obj: Object (The object to add to the database. Must also contain the key, but does not have to use the key property )
    • key: any (The unique ID of the object. Is also contained in obj)
    • table: string (The name of the table to which the object belongs to)

    UPDATE

    Object with:

    • type: 2
    • mods: Object (Contains only the modifications made to the object with the given key)
    • key: any (The unique ID of the object. Is also contained in obj)
    • table: string (The name of the table to which the object belongs to)

    DELETE

    Object with:

    • type: 3
    • key: any (The unique ID of the object we want to delete)
    • table: string (The name of the table to which the object belongs to)

    Running the tests

    The following commands can be execute to run the tests.

    npm install
    npm test

    TODO

    • cleanup changes table -> Can only do that after Dexie.Syncable supports the clear flag
    • Add E2E tests

    Contributing

    If you feel you can help in any way, be it with documentation, examples, extra testing, or new features please open an issue or pull request. If you have any questions feel free to open an issue with your question.

    License

    MIT License

    Most files in the sync directory where copied from the Dexie Websockets Sample and are under the Apache 2 license. Look at the individual file for more details.

    Install

    npm i sync-server

    DownloadsWeekly Downloads

    2

    Version

    2.0.0-beta.7

    License

    MIT

    Unpacked Size

    144 kB

    Total Files

    61

    Last publish

    Collaborators

    • nponiros