Next Perpendicular Moonlanding

    data-elevator

    4.0.0 • Public • Published

    npm version

    DATA ELEVATOR

    The data elevator is an easy to use and very flexible utility for migrating data sources. Every floor is a migration and the current level the current active migration.

    QUICKSTART

    STEP 1: Install

    npm install data-elevator
    

    STEP 2: Construct a new data elevator for the project.

    node ./node_modules/data-elevator construct
    

    STEP 3: Add a new migration.

    node ./data-elevator/elevator add "add phone number to users"
    

    STEP 4: Enter you migration code in the generated floor file.

    see: ./data-elevator/floors/
    

    STEP 5: Move the elevator to migrate data.

    node ./data-elevator/elevator move top
    node ./data-elevator/elevator move ground
    node ./data-elevator/elevator move 2
    

    CONFIGURATION

    • levelControllerConfig.fileName: Name of the file to store the current elevator level in
    var config = {
        levelControllerConfig: {
            fileName: "current_level.json"
        }
    }
    

    FEATURES

    • Floor templates: Base template for all migration floors.
    • Custom level controllers: Easily implement custom storage of the current migration level in any kind of data source like MongoDb, Postgres or a custom webservice.
    • Custom loggers: Implement your own logger.
    • onInitialize: Initialize any kind of custom component needed during the migrations process on beforehand.

    EXTENSIONS

    Some custom level controller implementations are already available in NPM:

    • data-elevator-elasticsearch (npm, github) - store elevator migration levels in elasticsearch
    • data-elevator-mongodb (npm, github) - store elevator migration levels in mongodb
    • data-elevator-mysql (npm, github) - store elevator migration levels in mysql
    • data-elevator-postgres (npm, github) - store elevator migration levels in postgres
    • data-elevator-sqlite3 (npm, github) - store elevator migration levels in sqlite3

    COMMANDS

    CONSTRUCT

    Construct a new data elevator in you project. In principle this command is only performed once per project.

    node ./node-modules/data-elevator construct
    

    ADD

    A new floor file will be created in which data migrations can be implemented. It is recommended to use the '--name' parameters for easier identification of the purpose of a floor.

    node ./<working-dir>/elevator add <name>
    
    Examples:
        node ./data-elevator/elevator add "migrating users"
    

    MOVE

    Elevator will move and perform the migrations for each floor passed by. Use 'ground' to move to ground floor and 'top' to move to the top floor.

    node ./<working-dir>/elevator move <floor>
        
    Examples:
        node ./data-elevator/elevator move top
        node ./data-elevator/elevator move 5
        node ./data-elevator/elevator move ground
    

    STATUS

    Display the last action of the elevator.

    node ./<working-dir>/elevator status
    

    LIST

    Display a list of all floors.

    node ./<working-dir>/elevator list
    

    OPTIONAL PARAMETERS

    The parameters can be applied to any command

    --<parameter_name> (<alias>) <description>     
    
    Parameters:
        --config-dir=  (-c=) Data elevator config dir (default=./data-elevator)
        --working-dir= (-w=) Data elevator working dir (def=./data-elevator)
        --verbose      (-v)  Verbose mode
    

    FLOOR TEMPLATE

    When a new floor is added the file 'floor-template.js' from the working directory is used as the template. Alterations to floor template are added to new floors. The minimal template contains at least the 'onUp' and 'onDown' function.

    module.exports = {
        /**
         * Data transformation that need to be performed when migrating the data up
         * @param floorWorkerParameters - instance of FloorWorkerParameters
         * @param callback(error) - If an error is returned then all the subsequent migration will not be handled
         */
        onUp : function(floorWorkerParameters, callback) {
            return callback(null);
        }, 
        /**
         * Data transformation that need to be performed when migrating the data down
         * @param floorWorkerParameters - instance of FloorWorkerParameters
         * @param callback(error) - If an error is returned then all the subsequent migration will not be handled
         */
        onDown : function(floorWorkerParameters, callback) {
            return callback(null);
        }
    }
    

    FloorWorkerParameters

    The FloorWorkerParameters gives access to the current configuration, the logger and the current floor object.

    class FloorWorkerParameters {
        constructor(config, logger, floor) {
            this.config = config;
            this.floor = floor;
            this.logger = logger;
        }
    }
    

    CUSTOM STUFF

    All the custom stuff can be implemented in '/elevator.js'.

    /**
     * Elevator
     * Data elevator
    **/
    var ElevatorBase = require('data-elevator/lib/elevator-engine/elevator-base');
    var ConsoleLogger = require('data-elevator/lib/logger/console-logger');
    var FileLevelController = require('data-elevator/lib/level-controllers/file-level-controller');
    
    class Elevator extends ElevatorBase {
        /**
        * Initialize custom components after all migrations have been applied
        * @param callback(error)
        */
        onInitialize(callback) {
            return callback(null);
        };
    
        /**
        * Uninitiailze custom components after all migrations have been applied
        * @param callback(error)
        */
        onUnInitialize(callback) {
            return callback(null);
        }
    }
    
    var elevator = new Elevator(new ConsoleLogger(false), FileLevelController);
    
    //Run the elevator
    elevator.run(function(error) { });
    

    Initializing custom components

    The method onInitialize an onUnInitialize can be overwritten to initialize custom components that need to be accessed during the migrations but only need to be initialized ones.

    Custom logger

    The first parameter of the constructor the the Elevator class can take a custom logger. If this parameter is null the ConsoleLogger will be used. A custom logger must be derived from BaseLogger ('data-elevator/lib/logger/base-logger.js') and must implement all methods of the BaseLogger.

    Custom level controller

    The custom level controller is what makes the data elevator extremely flexible. It provides the opportunity to store the current level of the elevator in any place needed just by plugging in your own level controller. So if there is the need to store the elevator level in a Postgres database of even a custom webservice just build a custom level controller and plug it in.

    A custom level controller needs to be derived from BaseLevelController and needs to implement the methods 'saveCurrentLevel' and 'retrieveCurrentLevel'. A simple example of a custom level controller is shown below. (Note: we left out all the error handling to keep the example short and to the point)

    Optionally the getManaul function can be implement to display a custom manual when the help command is executed.

    /**
     * MyLevelController
    **/
    
    var fs = require('fs')
    var BaseLevelController = require('data-elevator/lib/level-controllers/base-level-controller');
    var Level = require('data-elevator/lib/level-controllers/level');
    
    class MyLevelController extends BaseLevelController {
        /**
        * Save the current level
        * @param level
        * @param callback(error)
        */
        saveCurrentLevel(level, callback) {
            fs.writeFileSync(config.levelControllerConfig.levelFilePath, JSON.stringify(level));
            return callback(null);
        }
    
        /**
        * Retrieve the current level
        * @param callback(error, level)
        */
        retrieveCurrentLevel(callback) {
            var json = fs.readFileSync(filePath, "utf8");
            return callback(error, Level.fromJson(json));
        }
    }
    
    module.exports = MyLevelController;
    

    After creating the custom level controller it needs to be plugin in to the elevator constuctor ('/elevator.js'):

    var ElevatorBase = require('data-elevator/lib/elevator-engine/elevator-base');
    var ConsoleLogger = require('data-elevator/lib/logger/console-logger');
    var MyLevelController = require('<path-to>/my-level-controller.js');
    
    class Elevator extends ElevatorBase {
    }
    
    var elevator = new Elevator(new ConsoleLogger(false), MyLevelController);
    
    //Run the elevator
    elevator.run(function(error) { });
    

    RUNNING FROM CODE

    The elevator can also run from code.

    var ElevatorBase = require('data-elevator/lib/elevator-engine/elevator-base.js');
    var ConsoleLogger = require('data-elevator/lib/logger/console-logger');
    var FileLevelController = require('data-elevator/lib/level-controllers/file-level-controller');
    
    /**
     * Constructor
     */
    var elevator = new ElevatorBase(new ConsoleLogger(false), FileLevelController);
    
    elevator.runCommand({command: 'help'}, function(error) {});
    elevator.runCommand({command: 'add', name: 'update users', workingDir: '.\data-elevator'}, function(error) {});
    elevator.runCommand({command: 'move', floor: 2,'workingDir: '.\data-elevator'}, , function(error) {});
    elevator.runCommand({command: 'status', workingDir: '.\data-elevator'} , function(error) {});
    

    Install

    npm i data-elevator

    DownloadsWeekly Downloads

    15

    Version

    4.0.0

    License

    MIT

    Unpacked Size

    96 kB

    Total Files

    36

    Last publish

    Collaborators

    • kaasdude
    • cacadu