@rxdi/xmigrate
TypeScript icon, indicating that this package has built-in type declarations

0.7.51 • Public • Published

@rxdi/xmigrate

Build Status Coverage Status

Migration library for Mongodb and Mongoose written in TypeScript

Features

  • Simple UI/UX
  • Rollback support
  • Templates for migrations
  • TypeScript/JavaScript compatible
  • async/await configuration loader
  • Mongoose and Mongodb compatibility
  • ACID transactions provided by MongoDB
  • error and success logs for up/down migrations
  • Infinite rrror log with append NodeJS streaming technique
  • 100% TypeScript support with JIT compilation provided by @gapi/cli and ParcelJS

Installation

Using binary

wget https://github.com/rxdi/xmigrate/raw/master/dist/xmigrate-linux

Give it permission to execute

chmod +x xmigrate-linux
./xmigrate up|down|create|etc

Using NodeJS

npm i -g @rxdi/xmigrate

Configuration

Automatic configuration

xmigrate init

Manual configuration

You can create a xmigrate.js file where you execute the xmigrate command:

export default async () => {
  return {
    changelogCollectionName: 'migrations',
    migrationsDir: './migrations',
    defaultTemplate: 'es6',
    typescript: true,
    outDir: './.xmigrate',
    /* Custom datetime formatting can be applied like so */
    // dateTimeFormat: () => new Date().toISOString(),
    logger: {
      folder: './migrations-log',
      up: {
        success: 'up.success.log',
        error: 'up.error.log',
      },
      down: {
        success: 'down.success.log',
        error: 'down.error.log',
      },
    },
    mongodb: {
      url: `mongodb://localhost:27017`,
      databaseName: 'test',
      options: {
        useNewUrlParser: true,
      },
    },
  };
};

Commands

First time run

xmigrate init

Creating migration

xmigrate create "my-migration"

Creating migration with template.

Templates to choose: es5, es6, native, typescript. By default xmigrate create "my-migration" executes with es6 template

xmigrate create "my-migration" --template (es5|es6|native|typescript)
xmigrate create "my-migration" --template typescript

Up migrations

Will execute all migrations which have the status PENDING

xmigrate up

Up migrations with rollback down to current errored migration

xmigrate up --rollback

Down migrations

Will execute migrations one by one starting from the last created by timestamp

xmigrate down

Status of migrations

xmigrate status

Will print inside the console a table. When there is a PENDING flag these migrations were not running against the current database.

🖥️  Database: test

💿  DBCollection: migrations

🗄️  LoggerDir: ./migrations-log

📁  MigrationsDir: migrations

👷  Script: xmigrate status

┌─────────┬───────────────────────────┬────────────────────────────┐
│ (index) │         fileName          │         appliedAt          │
├─────────┼───────────────────────────┼────────────────────────────┤
│    0    │ '20190725160010-pesho.js''2019-07-25T16:07:27.012Z' │
│    1    │ '20190725160011-pesho.js''PENDING'          │
│    2    │ '20190725160012-pesho.js''PENDING'          │
│    3    │ '20190725160013-pesho.js''PENDING'          │
└─────────┴───────────────────────────┴────────────────────────────┘

🔥  There are 3 migrations with status 'PENDING', run 'xmigrate up' command!

Migration templates

Native mongo driver template

module.exports = {
  async up(client) {
    await client
      .db()
      .collection('albums')
      .updateOne({ artist: 'The Beatles' }, { $set: { blacklisted: true } });
    await client
      .collection('albums')
      .updateOne({ artist: 'The Doors' }, { $set: { stars: 5 } });
  },

  async down(client) {
    await client
      .db()
      .collection('albums')
      .updateOne({ artist: 'The Doors' }, { $set: { stars: 0 } });
    await client
      .collection('albums')
      .updateOne({ artist: 'The Beatles' }, { $set: { blacklisted: false } });
  },
};

ES5 template

module.exports = {
  async up(client) {
    return ['UP'];
  },

  async down(client) {
    return ['DOWN'];
  },
};

ES6 template

export async function up(client) {
  return ['Up'];
}
export async function down(client) {
  return ['Down'];
}

Typescript template

(Optional) type definitions for mongodb and mongoose

npm install @types/mongodb @types/mongoose -D
import { MongoClient } from 'mongodb';

export async function up(client: MongoClient) {
  await client
    .db()
    .collection('albums')
    .updateOne({ artist: 'The Beatles' }, { $set: { blacklisted: true } });

  await client
    .db()
    .collection('albums')
    .updateOne({ artist: 'The Doors' }, { $set: { stars: 5 } });
}

export async function down(client: MongoClient) {
  await client
    .db()
    .collection('albums')
    .updateOne({ artist: 'The Doors' }, { $set: { stars: 0 } });

  await client
    .db()
    .collection('albums')
    .updateOne({ artist: 'The Beatles' }, { $set: { blacklisted: false } });
}

TypeScript migrations

To be able to run migrations with TypeScript you need to set typescript: true inside xmigrate.js and install @gapi/cli globally

Install @gapi/cli for runtime build using glob

npm i -g @gapi/cli

Command that will be run internally

npx gapi build --glob ./1-migration.ts,./2-migration.ts

After success transpiled migration will be started from ./.xmigrate/1-migration.js Before exit script will remove artifacts left from transpilation located inside ./.xmigrate folder

Rollback

When executing command xmigrate up --rollback this will trigger immediate rollback to DOWN migration on the current crashed migration The log will look something like this:

🖥️  Database: test

💿  DBCollection: migrations

🗄️  LoggerDir: ./migrations-log

📁  MigrationsDir: migrations

👷  Script: xmigrate up


🔥  Status: Operation executed with error
🧨  Error: {"fileName":"20190725160011-pesho.js","migrated":[]}
📨  Message: AAA


🙏  Status: Executing rollback operation xmigrate down
📁  Migration: /migrations/20190725160011-pesho.js


🚀  Rollback operation success, nothing changed if written correctly!

Logs

By default logs will append streaming content for every interaction made by migration

Down migration success Log

🚀 ********* Thu Jul 25 2019 11:23:06 GMT+0300 (Eastern European Summer Time) *********

{
  "fileName": "20190723165157-example.js",
  "appliedAt": "2019-07-25T08:23:06.668Z",
  "result": [
    {
      "result": "DOWN Executed"
    }
  ]
}

Down migration error log

🔥 ********* Thu Jul 25 2019 03:28:48 GMT+0300 (Eastern European Summer Time) *********

{
  "downgraded": [],
  "errorMessage": "AAA",
  "fileName": "20190724235527-pesho.js"
}

Up migration success log

🚀 ********* Thu Jul 25 2019 11:23:24 GMT+0300 (Eastern European Summer Time) *********

{
  "fileName": "20190723165157-example.js",
  "appliedAt": "2019-07-25T08:23:24.642Z",
  "result": [
    {
      "result": "UP Executed"
    }
  ]
}

Up migration error log

🔥 ********* Thu Jul 25 2019 03:39:00 GMT+0300 (Eastern European Summer Time) *********

{
  "migrated": [],
  "errorMessage": "AAA",
  "fileName": "20190724235545-pesho.js"
}

TypeScript configuration

When you change your configuration file to xmigrate.ts it will automatically transpile to ES5 and will be loaded

This type of configuration requires @gapi/cli to be installed since we will transpile configuration with it!

npm i -g @gapi/cli
import { Config } from '@rxdi/xmigrate';

export default async (): Promise<Config> => {
  return {
    changelogCollectionName: 'migrations',
    migrationsDir: './migrations',
    defaultTemplate: 'typescript',
    typescript: true,
    outDir: './.xmigrate',
    logger: {
      folder: './migrations-log',
      up: {
        success: 'up.success.log',
        error: 'up.error.log',
      },
      down: {
        success: 'down.success.log',
        error: 'down.error.log',
      },
    },
    mongodb: {
      url: `mongodb://localhost:27017`,
      databaseName: 'test',
      options: {
        useNewUrlParser: true,
      },
    },
  };
};

Everytime xmigrate.ts is loaded timestamp is checked whether or not this file is changed

This information is saved inside .xmigrate/config.temp like a regular Date object

This will ensure that we don't need to transpile configuration if it is not changed inside xmigrate.ts file

Basically this is caching to improve performance and user experience with TypeScript configuration

API Usage

import { Container, setup } from '@rxdi/core';

import {
  MigrationService,
  GenericRunner,
  LogFactory,
  ConfigService,
  LoggerConfig,
  Config,
} from '@rxdi/xmigrate';

const config = {
  changelogCollectionName: 'migrations',
  migrationsDir: './migrations',
  defaultTemplate: 'typescript',
  typescript: true,
  outDir: './.xmigrate',
  logger: {
    folder: './migrations-log',
    up: {
      success: 'up.success.log',
      error: 'up.error.log',
    },
    down: {
      success: 'down.success.log',
      error: 'down.error.log',
    },
  },
  mongodb: {
    url: 'mongodb://localhost:27017',
    databaseName: 'test',
    options: {
      useNewUrlParser: true,
    },
  },
};

setup({
  providers: [
    GenericRunner,
    LogFactory,
    ConfigService,
    {
      provide: Config,
      useValue: config,
    },
    {
      provide: LoggerConfig,
      useValue: config.logger,
    },
  ],
}).subscribe(async () => {
  const template = `
import { MongoClient } from 'mongodb';

export async function up(client: MongoClient) {
  return true
}
export async function down(client: MongoClient) {
  return true
}
`;

  const migrationService = Container.get(MigrationService);

  // Create migration with template
  const filePath = await migrationService.createWithTemplate(
    template as 'typescript',
    'pesho1234',
    { raw: true, typescript: true },
  );
  console.log(filePath);

  // Up migration
  await migrationService.up();
  process.exit(0);

  // Down migration
  await migrationService.down();
  process.exit(0);

  // Status
  await migrationService.status();

  process.exit(0);
}, console.error.bind(console));

Minimal configuration

export default async () => {
  return {
    defaultTemplate: 'typescript',
    outDir: './.xmigrate',
    typescript: true,
    mongodb: {
      url: 'mongodb://localhost:27017',
      databaseName: 'test',
      options: {
        useNewUrlParser: true,
      },
    },
  };
};

Performance tests

Running 600 migrations takes less than 15 seconds in TypeScript compiled right down to Javascript ES5.

Check this video inside IPFS network

Link is not working at the moment...

Readme

Keywords

none

Package Sidebar

Install

npm i @rxdi/xmigrate

Weekly Downloads

2

Version

0.7.51

License

MIT

Unpacked Size

171 kB

Total Files

62

Last publish

Collaborators

  • rxdi