@tanglemesh/server-utilities

server-utilities

A small package with basic server utilities for creating a express server with ease

Generate OpenAPI 3.0.0 Documentation and Postman collection

Use this utility to generate ready to use openAPI definition or a importable postman collection. The endpoint for getting the openAPI docs is /api.json. To receive the postman collection just open /collection.json.

const DocsRouter = require ("@tanglemesh/server-utilities").DocsRouter;

const express = require ('express');
const path = require ('path');
const app = express ();

// Add Docs-Routes
app.use (DocsRouter (serviceName, serviceHost, routerPath = path.resolve (__dirname, '../**/router.js'), apiVersion = "1.0.0", modelsPath, modelSuffix = ".model.js", excludeProperties = ['_deleted', '_created', '_updated'], customApiDefinition = {}, filterDefinition = (definition) => definition));

…

Error handling

To create errors that are resolveable by express, just use our DefaultError class. To resolve this error you can simply call the method .resolveResponse (response) with the response object and return it. Now you will get easily printed an error message.

const DefaultError = require ("@tanglemesh/server-utilities").DefaultError;

const error = new DefaultError ("Some error message", "error-code-15", 500, {
    message: "Some more error details",
});

Helpers

Encryption / Hashing helper

Use this helper to en/decrypt values or hash values for example passwords.

const EncryptionHelper = require ("@tanglemesh/server-utilities").EncryptionHelper ("encryption secret");

const hash = EncryptionHelper.hash ("value");
const isPasswordCorrect = EncryptionHelper.compareHash ("value", "hash");

const encrypted = EncryptionHelper.encrypt ("value");
const decrypted = EncryptionHelper.decrypt (encrypted);
const repeatableHash = EncryptionHelper.repeatableHash ("value");

Resposne helper

Use this helper to arrange responses and use the parameters of the validation middleware.

const ResponseHelper = require ("@tanglemesh/server-utilities").ResponseHelper (errorReporting = false, filterAttributes = [
    '_created',
    '_updated',
    '_deleted',
], timeoutMilliseconds = 1000 * 60);

// Add status route to your express-app
ResponseHelper.AddStatusHandling (app, serviceName, serviceHost, servicePort, serviceVersion, status, mode = "production", route = "/", statusVariables = {}, statusCode = 200);

// Use ResponseHelper.Arrange in addition to validator-middleware to parse user-input and send to a controller-method
…
router.post ("/", 
    ValidatorMiddleware.String ("body", "name"),
    …
    ResponseHelper.Arrange (SomeController.doSomething, "someOtherParam"), //doSomething (parameters, name, someOtherParam)
);
…

// Add error handling to your app
ResponseHelper.AddExceptionHandling (app, errorReporting = false);

Database transactions

If you also want to work with sequelize-transactions, you must attatch your initialized sequelize-instance to the global namespace:

global.responseHelper.database = new Sequelize (…);

If the global.responseHelper.database is not set, there will be no managed automatic transaction management! So please check, that you've implemented your own way of database-transactions.

Savepoints and Rollbacks

To create a safepoint in your controller, simply call:

// Commit the transaction
await ResponseHelper.commit ();

// Create new transaction
await ResponseHelper.transaction ();

// Rollback / Revert transaction
await ResponseHelper.rollback ();

JWT (JSON-Web-Token) helper

Use this helper to create and verify JWTs and read content of the token.

const JWTHelper = require ("@tanglemesh/server-utilities").JWTHelper;

const token = JWTHelper.createToken (secret, payload, issuer, audience, authentificationAlgirithm = "HS512", expirationMilliseconds = 1000 * 60 * 15, receiver = null, content = {}, subject);
const valid = await JWTHelper.verifyToken (secret, token, serviceName, audience);

const payload = JWTHelper.getPayload (token);
const content = JWTHelper.getContent (token);

CSRF helper

const CSRFHelper = require ("@tanglemesh/server-utilities").CSRFHelper;

const secret = await CSRFHelper.createSecret ();
const token = CSRFHelper.createToken (secret);
const valid = CSRFHelper.validateToken (secret, token);

Generator helper

const GeneratorHelper = require ("@tanglemesh/server-utilities").GeneratorHelper;

const randomString = await GeneratorHelper.randomString (12);
const randomInteger = await GeneratorHelper.randomInteger (0, 12);
const randomPhrase = await GeneratorHelper.randomMixed (12);
const anonymizedString = await GeneratorHelper.anonymizeString ("Some string to anonymize", prefixPercent = .3, suffixPercent = .2);

TwoFactor helper

const TwoFactorHelper = require ("@tanglemesh/server-utilities").TwoFactorHelper;

const secret = TwoFactorHelper.createSecret ();
const dataURI = await TwoFactorHelper.createQRCodeDataURI ("secret", "label", "issuer");
const valid = TwoFactorHelper.verifyTwoFactorToken ("secret", "token", 2);

Base64 helper

const Base64Helper = require ("@tanglemesh/server-utilities").Base64Helper;

const mimeType = Base64Helper.getBase64MimeType("base64string");
const filePath = Base64Helper.saveBase64File ("base64string", "filePath");
const base64DataUriString = Base64Helper.readFileInBase64 ("filePath");

HTTPClient helper

const HTTPClientHelper = require ("@tanglemesh/server-utilities").HTTPClientHelper;

const client = new HTTPClientHelper (apiBaseUrl, apiPath = "/", headers = {}, timeout = 15000, httpsSettings = {}); // returns client
const requestResponse = await client.request (method = "GET", path = "/", params = {}, data = {}, httpsSettings = {}); // returns [error, responseData, response]
const getResponse = await client.get (path = "/", params = {}, httpsSettings = {}); // returns [error, responseData, response]
const postResponse = await client.post (path = "/", params = {}, data = {}, httpsSettings = {}); // returns [error, responseData, response]
const putResponse = await client.put (path = "/", params = {}, data = {}, httpsSettings = {}); // returns [error, responseData, response]
const patchResponse = await client.patch (path = "/", params = {}, data = {}, httpsSettings = {}); // returns [error, responseData, response]
const deleteResponse = await client.delete (path = "/", params = {}, httpsSettings = {}); // returns [error, responseData, response]

//Response: [error, data, response]

S3 Helper

Use this helper to upload and download files from and to AWS S3.

const S3Helper = require ("@tanglemesh/server-utilities").S3Helper (accessKeyId, secretAccessKey, region = "eu-central-1");

// use the aws sdk directly

await S3Helper.uploadFile (Bucket, Key, FilePath) // Returns { Bucket: "", Key: "", Location: "" };
await S3Helper.downloadFile (Bucket, Key, FilePath); // Returns { Bucket: "", Key: "", Location: "" };
await S3Helper.streamFile (Bucket, Key); // Returns <Stream>
await S3Helper.existsFile (Bucket, Key); // Returns true|false
await S3Helper.copyBucketFile (Bucket, ActualKey, CopyKey); // Returns { Bucket: "", Key: "" };
await S3Helper.renameBucketFile (Bucket, ActualKey, NewKey); // Returns { Bucket: "", Key: "" };
await S3Helper.deleteBucketFile (Bucket, Key); // Returns { Bucket: "", Key: "" };
await S3Helper.listBucketFiles (Bucket, MaxKeys = 1000); // Returns [{ Bucket: "", Key: "", Location: "" }, …];

// or use our BucketFile class

const bucketFile = new S3Helper.BucketFile (bucket, key, filePath = null);
bucketFile.FilePath;
bucketFile.Bucket;
bucketFile.Key;

bucketFile.toJSON (); //{Bucket: "xx", Key: "xx"}
bucketFile.toString (); //"{Bucket: "xx", Key: "xx"}"
await bucketFile.clearLocal (); //remove local copy of file
await bucketFile.downloadFile (filePath = null); //download local copy of file (default location: project-root/tmp)
await bucketFile.uploadFile (); //upload local-copy/replacement
await bucketFile.streamFile (); //return null or a stream
await bucketFile.existsFile (); // returns true/false, if file is available in S3

CRUD helper

Use this helper to create routes and controllers for a simple CRUD implementation for eg. admin access to the models. You can configure the routes with many configuration settings, so you can also customize the controllers in a way you like.

The CRUD helper will work together with the DocsRouter and will automatically appear in the api-specification of the service.

To create a new CRUD implementation for a specific Sequelize model, you need to create a new Object of the CRUD helper:

const CRUDHelper = require ("@tanglemesh/server-utilities").CRUDHelper;

new CRUDHelper (expressApp, sequelizeModel, "/models/sequelizeModel");

You can pass 4 parameters to the constructor. The first one is the app or a router of express. To this router the methods will be attatched. The second argument is the sequelizeModel for which the CRUD methods should be implemented. The third argument describes the route, the CRUD methods should be applied to. In the above example you can list all instances of the sequelizeModel by calling GET: /models/sequelizeModel. More details on what methods will be available can be seen in the api-documentation that will be generated for you.

To customize the CRUD implementation even more, you can pass the fourth argument to the constructor config:

const CRUDHelper = require ("@tanglemesh/server-utilities").CRUDHelper;

new CRUDHelper (expressApp, sequelizeModel, "/models/sequelizeModel", {
            // Route config
            middlewares: [],

            // Pagination config
            start: 0,
            limit: 25,
            maxLimit: 100,
            defaultOrder: 'asc:id',

            // Filtering config
            searchMethod (q) {
                return {
                    // where: {
                    //     [Op.or]: {
                    //         name: q,
                    //         title: q,
                    //     }
                    // },
                    // include: [],
                }
            },

            // Configure the functionality set
            listable: true,
            readable: true,
            creatable: true,
            updateable: false,
            patchable: false,
            deletable: false,
            
            // Attribtues config
            readonlyAttributes: ['id'],
            hiddenAttributes: ['_deleted'],
            notHiddenOnChange: [], //these attributes will not be hidden on creation, even if listed in the hiddenAttributes
            listAttributes: [], //if empty, list all, otherwise only the listed attributes on listMethod

            attributeGenerators: {
                id (request, response) {
                    return uuid ();
                },
            },
            attributeValidators: {
                // _created (value, config, parameters, request, response) {
                //     // …
                //     // throw new DefaultError ("");
                // },
                // Validator-Middleware Methods can also be used here!
                // name: ValidatorMiddleware.ValidationEnum,
            },
            attribtueSetters: {
                // _updated (value, parameters, request, response) {
                //     return new DateTime ();
                // },
            },
            attributeGetters: {
                // _file (value, instance) {
                //     return downlaodedFile;
                // },
            },
        });

Above you can see the default configuration for the CRUDhelper. But you can simply overwrite any of the default values with your own configuration.

Deep Merge helper

This helper can be used to merge deep object structures without overwriting complete keys or stuff like that.

const DeepMergeHelper = require ("@tanglemesh/server-utilities").DeepMergeHelper;

const mergedObject = DeepMergeHelper ({
    target: {
        deeper: {
            yes: true,
            no: false,
        }
    }
}, {
    target: {
        deeper: {
            maybe: 1,
        },
    },
    someMore: {
        test: "string",
    },
});

User-Agent helper

This helper can be used to parse and fetch client-data like it's ip-address.

const UserAgentHelper = require ("@tanglemesh/server-utilities").UserAgentHelper;

const userAgent = UserAgentHelper.getUserAgent (request);
const ipAddress = UserAgentHelper.getClientIpAddress (request);
const platform = UserAgentHelper.getPlatform (request); // returns name, version, layout, os, description, product, manufacturer

Retryer helper

This helper can be used to retry an action until it succeeds or a maximum of attempts has been executed.

const RetryerHelper = require ("@tanglemesh/server-utilities").RetryerHelper;

const result = await RetryerHelper (handler = () => {
    if (Math.rand () <= .5) {
        throw new Error ("Error happened");
    }
    return "success";
}, retries = 10, retryInterval = 1000 * 5, errorHandler = (error) => {});

Cache helper

This helper provides you very basic and simple caching functionality. You can store any value you want together with a specific key. You can provide a TTL (time to live) at creation or use the default TTL.

const CacheHelper = require ("@tanglemesh/server-utilities").CacheHelper;

// Set a value
CacheHelper.setValue (path = "some.path.or.key.you.want", {
    name: "Some value ;)",
    number: 323.23
}, ttlMs = CacheHelper.TTL_MS);

// Get a value
CacheHelper.getValue (path = "some.path.or.key.you.want");

The getValue method will return the value or null if this key does not exist or the ttl has been expired.

You can also change the default TTL or Clear-Interval depending on your requirements:

const CacheHelper = require ("@tanglemesh/server-utilities").CacheHelper;

// Set default TTL
CacheHelper.TTL_MS = 1000 * 60 * 10; // default value is 10 minutes

// Set default Clear-Interval
CacheHelper.CLEAR_INTERVAL_MS = 1000 * 60 * 1; // default value is 1 minute

Middlewares

IP-Address Checker

Use this middleware to protect your api / service from clients not listed in your ip whitelist.

const IPWhitelistMiddleware = require ("@tanglemesh/server-utilities").IPWhitelistMiddleware ([
    "192.168.178.0/24",
    "::1",
    …
], isLogginEnabled = false);

…
router.post ("/",
    IPWhitelistMiddleware,
    …
);
…

Use this middleware to protect your api / service from clients listed in your ip blacklist.

const IPBlacklistMiddleware = require ("@tanglemesh/server-utilities").IPBlacklistMiddleware ([
    "192.168.178.0/24",
    "::1",
    …
], isLogginEnabled = false);

…
router.post ("/",
    IPBlacklistMiddleware,
    …
);
…

Validator middleware

Validate user input to be in the correct format and type, so that your controller can be sure to get valid parameters. For exact details, what types and validators are available just have a look into the middlewares/validator.middleware.js file.

Types available:

• Custom specify your own validator function
• String - { minLen: null, maxLen: null }
• Float - { min: null, max: null, decimals: null, allowStringValue: false }
• Integer - { min: null, max: null, allowStringValue: false }
• Enum - { enumValues: [] }
• Boolean - { allowStringBooleans: false, allowNumberBooleans: false }
• DateTime - { format: "YYYY-MM-SS HH:mm:ss", isBefore: null, isAfter: null }
• Email - { }
• Base64File checks if a base64 string is valid - { mimeTypes: null, minFileSize: null, maxFileSize: null }
• Optional set parameters that can be optional, but if set validate them with the following validator middlewares - {removeNullValues: true, removeZeroValues: false, removeEmptyStrings: false }
• Array - { minValues: null, maxValues: null, valueValidationMethod: null, valueValidationConfig: {} }
• Object - { minValues: null, maxValues: null, valueValidationMethod: null, valueValidationConfig: {} }

const ValidatorMiddleware = require ("@tanglemesh/server-utilities").ValidatorMiddleware;

…
router.post ("/",
    ValidatorMiddleware.String ("body", "name", {
        minLen: 6,
        maxLen: 64,
    }, error = null, errorCode = "some-unique-error-code", statusCode = 400),
    
    ValidatorMiddleware.Custom ("body", "name",
    async (parameter, config, parameters, request, response) => { return true; },
    "This name is already in use.", "some-unique-error-code", config = {}, statusCode = 400),
    …
);
…

Authentification middleware

This middleware checks a special access token, if this is valid and authorizes the client to access the route. There are two ways of authenticate a client. A client can be a server or app using only the api key or clients that additionally need a csrf-protection to keep them secure.

const AuthentificationMiddleware = require ("@tanglemesh/server-utilities").AuthentificationMiddleware (cookieSecure = true);

// 1. way, only working with api-key
…
router.post ("/",
    AuthentificationMiddleware.Authentificate ("secret", "serviceName", "audience"),
    …
);
…

// 2. way, working also with csrf-protection
app.use (AuthentificationMiddleware.cookieParser ());
…
router.post ("/",
    AuthentificationMiddleware.Authentificate ("secret", "serviceName", "audience", withCookies = true),
    …
);
…

If you are working with a client only using the api key, you can create one with the JWTHelper. If you working with a browser-client that also needs csrf, you can use the following method to prepare a response, to set the needed cookies, when the user has authentificated before (eg. with mail, password, 2FA):

const CorsMiddleware = require ("@tanglemesh/server-utilities").CorsMiddleware;

// if origins is empty, all origins will be allowed access
// there are some default allowedHeaders, needed to work with other server-utilities, but for custom headers, you need to specify them here
// corsConfiguration: custom configurations for the npm-package 'cors' (https://www.npmjs.com/package/cors)

…
router.post ("/",
    CorsMiddleware (origins = [], allowedHeaders = {}, corsConfiguration = {}),
    …
);
…

CORS middleware

This middleware enables CORS for a specific set of routes you specify.

const CorsMiddleware = require ("@tanglemesh/server-utilities").CorsMiddleware (origins = [], allowedHeaders = {}, corsConfiguration = {});

//Make all routes CORS-enabled
app.options('*', CorsMiddleware ()) // include before other routes

// Only make specific route(s) CORS-enabled
…
router.post ("/",
    CorsMiddleware ()
    …
);
…

The CorsMiddleware has 3 properties:

    CorsMiddleware (origins = [], allowedHeaders = [], corsConfiguration = {});

The origins array takes multiple arguments that can be specific origins as string or a RegExp. Also you can define which headers can be used by the routes. With the last attribute you can configure the CORSMiddleware even more. All options available can be found in this dependency package.

Captcha middleware

This middleware is used to protect routes with google's reCaptcha.

• reCaptchaSecret is the re-captcha secure token
• checkCaptchaToken is the flag to define if the token value should be checked or not (eg. useful in local environments)
• captchaTokenWhitelist is an optional array to define some static captcha tokens that can be used instead of "real" captcha tokens (eg. useful for doing test requests without having a frontend)

const CaptchaMiddleware = require ("@tanglemesh/server-utilities").CaptchaMiddleware (reCaptchaSecret, checkCaptchaToken = true, captchaTokenWhitelist = []);

// Only make specific route(s) CORS-enabled
…
router.post ("/",
    CaptchaMiddleware.Resolve,
    …
);
…

Pagination middleware

This pagination is used to enable simple pagination of the requested source.

const PaginationMiddleware = require ("@tanglemesh/server-utilities").PaginationMiddleware;

…
router.post ("/",
    PaginationMiddleware (maxItemsPerPage = 100, defaultItemsPerPage = 25),
    …
);
…

With this middleware the user can simply send their pagination requests with the following pattern: /items?start=25&limit=50. If the limit is bigger than maxItemsPerPage it will automatically be set to the configured value. Also start can not be smaller than 0. If limit is not provided, the limit will automatically be set to the value of defaultItemsPerPage.

In your controller, you can simply use the validated integer-values request.paginationMiddleware.start and request.paginationMiddleware.limit. These values will always be valid integers in your specified range!

(request, response) => {
    const items = await loadItems (request.paginationMiddleware.start, request.paginationMiddleware.limit);
    return {
        count: items.length,
        start: request.paginationMiddleware.start,
        limit: request.paginationMiddleware.limit,
        results: [
            ...items
        ],
    }
}

Rate-Limit middleware

This rate-limit middleware is used to limit the requests to your public api. This middleware will count requests from the ip's and limit after the client exceeds your configured limit.

const RateLimitMiddleware = require ("@tanglemesh/server-utilities").RateLimitMiddleware;

…
router.get ("/",
    RateLimitMiddleware (limitIpRequests = 250, limitTimeframe = 1000 * 60 * 10, delayMsOnLimitExceed = null),
    …
);
…

You can specify the max requests allowed in a specific time-frame. If you also set the delayMsOnLimitExceed parameter, the client's requests will not be blocked, but slowed down. Each additional request that exceeds the client's limit, will multiply the delayMsOnLimitExceed milliseconds. Eg. 1 request slowed down by 500ms, second one by 1000ms, …

const RateLimitMiddleware = require ("@tanglemesh/server-utilities").RateLimitMiddleware;

…
router.get ("/",
    RateLimitMiddleware (limitIpRequests = 250, limitTimeframe = 1000 * 60 * 10, delayMsOnLimitExceed = 500),
    …
);
…

Default model to use for Mysql-Sequelize models

This model can be used as parent model-class to extend your new models with basic attributes and configuration.

const Sequelize = require('sequelize');
const { DefaultModel, DefaultOptions } = require ("@tanglemesh/server-utilities");

const Model = Database.define ('model', {
    ...DefaultModel,
    someOtherComponent: {
        type: Sequelize.STRING (5),
        allowNull: false,
        validate: {
            len: {
                args: [5, 5],
                msg: "The someOtherComponent is invalid! Must have a length of 5 characters!",
            },
        },
    },
}, {
    ...DefaultOptions,
});

module.exports = Model;

Service worker

This class can be used to create your own service workers that will do stuff in a regular interval you can specify! You can use a interval in milliseconds by setting intervalMsOrCron to any integer value or you can also use cron to specify more complex time-based excecution schedules. For example https://crontab.guru/ helps you to set up your proper cron configuration. In order to provide cron functionality we used the cron npm package.

const ServiceWorker = require ("@tanglemesh/server-utilities").ServiceWorker;

class SomeExampleServiceWorker extends ServiceWorker {

    async executionPrecondition () {
        if (somethingNotOkay === true) {
            return false; //do nothing in this iteration, because some preconditions are not okay
        }
        return true; //now execute executionHandler
    }

    async executionHanlder () {
        return; // implement execution handler
    }

}

const myServiceWorker1 = new SomeExampleServiceWorker (intervalMsOrCron = 1000 * 60, initialRun = false); // execute executionHandler every minute
const myServiceWorker2 = new SomeExampleServiceWorker (intervalMsOrCron = "0 0 0 * * *", initialRun = false); // execute executionHandler every night at 0:00

You can set the initialRun to true, if you want to execute the service-worker directly after initialization of your service-worker.

You have three methods to stop, pause and continue the processing of your service-worker. You can execute these methods from your service-worker itself or from outside using the service-worker object that you receive after constructing your service worker.

await myServiceWorker1.stop (); // completely stops the service-worker without any chance to resume

await myServiceWorker2.pause (); // pauses the execution of your handlers
setTimeout (async () => {
    await myServiceWorker2.continue (); // continues to execute the handlers
});

Note: When using pause and continue the interval stays the same. So regardless of using cron or interval on construction the timeout stays the same. So by calling pause the interval stays the same but only ignores your handler. By continuing the original interval still runs and the execution will begin whenever the original interval is reached.