dyna
Mograte
A simple little utility that provides DynamoDB context to store migrations state in a DynamoDB table, and migrate tables and seed data for your application -- up and down.
It's so simple that when it migrates up, it really just reads files in a particular order, reads records from a DynamoDB table, and then pairs them together. If a given file hasn't been run, this will run them, and add a record to the DynamoDB table. When it migrates down, any existing records are deleted from the DynamoDB table. Bingo bango wango tango. That's all she wrote. Bob's your uncle.
Installation
$ npm install -g mograte
... or ...
$ npm install mograte --save-dev
Getting Started
If you installed as a global npm package, you should call
$ mograte init
If you installed as a local project dependency, you should call
$ npx mograte init
You will be taken through a series of questions in an interactive shell in order to generate the
.mogrc
file that will be used to drive your migrations behaviors.You may select between the following formats:
.js
,.ts
,.json
, orpackage.json
.Make a note: generated configuration will be called
mogConfig
in the rest of this document.If you installed as a package dependency, prefix all calls mentioned in this doc with
npx
.
Your mogConfig
will look something like this when you're done:
export default {
language: 'ts', // determines the format of the migrations generated
migrationsDir: './src/db/migrations', // determines where to read and write migrations to/from
migrationsTable: 'migrations', // name of the DynamoDB table to write migrations info to
awsConfig: {
profile: 'SomeCoolProfileName', // mutually exclusive with other properties in same hash; may also be read from process.env.AWS_PROFILE
accessKeyId: '*******', // may also be read from process.env.AWS_ACCESS_KEY_ID
secretAccessKey: '*******', // may also be read from process.env.AWS_SECRET_ACCESS_KEY;
region: '**-****-*', // may also be read from process.env.AWS_REGION
},
user: () => require('os').userInfo().username || 'unknown' // in json formats, this will be a string, but we will eval it for you. This is useful for keeping track of who ran a given migration. Customize or tailor it to your needs (ie. process.env.GITHUB_ACTOR in GH Actions)
}
Usage
Usage: mograte [options] [command]
Options:
-h, --help output usage information
Commands:
init Initalize the migrations tool in a project
list List migrations and their status
create <name> Create a new migration
delta Gets the number of un-run migrations for reversion purposes
up [name | number] Migrate up to a given migration by id, filename, or number
down [name | number] Migrate down to a given migration by id, filename, or number
help display help
Creating Migrations
To create a migration, execute mograte create <title>
with an optional title. You should make the meaningful/atomic so that when you're reasoning about the migrations over time, you have an idea of what you're creating and/or tearing down.
When you ran mograte init
, you selected a language for your project: js
or ts
. You also chose a directory to contain your migrations (migrationDir
). If you look in your migrationDir
, you will have a file for your chosen language, what contains two functions: up
and down
. You should use them to create any data stuffs you want in your DynamoDB database and/or tear down any such stuffs. They will resemble the following...
JS example:
'use strict'
module.exports.up = async (context) => {
}
module.exports.down = async (context) => {
}
TS example:
import { MigrationContext } from 'mograte';
export default {
up: async (context: MigrationContext): Promise<any> => {
},
down: async (context: MigrationContext): Promise<any> => {
}
}
Dynamo Context
Each up
and down
migration function is passed a context
object when it executes. The context object has some useful abstractions for DynamoDB create/delete/read behaviors. Its contents will help you on your way. Use it to crud DynamoDB tables and items until your heart finds peace.
For example:
$ mograte create add-pets
$ mograte create add-owners
The first call creates ./{mogConfig.migrationDir}/{timestamp in milliseconds}-add-pets.{mogConfig.language}
, which we can populate:
const fs = require('fs');
const TableName = 'pets';
exports.up = async (context) => {
await context.table.createAsync({ TableName, ...someTableDefinition });
const petSeed = JSON.parse(fs.readFileSync('./path/to/some/pets.file'));
await context.item.writeAsync(petSeed, TableName);
}
exports.down = async (next) => {
await context.table.deleteAsync({ TableName, ...someTableDefinition });
}
The second creates ./{mogrc.migrationDir}/{timestamp in milliseconds}-add-owners.{mogrc.language}
, which we can populate:
const fs = require('fs');
const TableName = 'owners';
exports.up = async (next) => {
await context.table.createAsync({ TableName, ...someTableDefinition });
const pets = await context.item.getAllAsync({ someQuery: 'here', thatIs: 'awsScanParams' });
const petOwnersSeed = JSON.parse(fs.readFileSync('./path/to/some/pet-owners.file'));
for (let i = 0; i < pets.length; i++) {
const pet = pets[i];
const petOwner = (petOwnersSeed.filter((po) => po.name === pet.ownerName) || [])[0];
petOwner.petIds = [...(petOwner.petIds || []), pet.id];
}
await context.item.writeAsync(petOwnersSeed, TableName);
}
exports.down = async (next) => {
await context.table.deleteAsync({ TableName, ...someTableDefinition });
}
Running Migrations
When first running the migrations, all will be executed in sequence.
$ mograte
up : 1316027432511-add-pets.js
up : 1316027432575-add-owners.js
migration : complete
Subsequent attempts will simply output ...No work... Migrations are up to date
because the tables and other artifacts are presumed created until you mograte down
$ mograte
...No work...
Migrations are up to date
If we were to create another migration using mograte create
, and then execute migrations again, we would execute only those not previously executed:
$ mograte
up : 1316027433455-coolest-owner.js
You can also run migrations incrementally by specifying a migration or the number of migration it is
$ mograte up 1316027433425-coolest-pet.js # or mograte up 3
up : 1316027432511-add-pets.js
up : 1316027432575-add-owners.js
up : 1316027433425-coolest-pet.js
migration : complete
This will run migrations up to (and including) 1316027433425-add-owners.js
. Similarly you can run down-migrations to (and including) a specific migration, instead of migrating all the way down. When migrating down, remember the migrations are run in reverse order. Also, you may think running mograte down 1
repeatedly will move the cursor back to each subsequent migration. That is only true if the migration file is also deleted; otherwise, mograte looks at the most recent migration file, sees there is no migration record for it in the database, and considers the work done.
$ mograte down 1316027433425-add-owners.js # or mograte down 2
down : 1316027433425-coolest-pet.js
down : 1316027432575-add-owners.js
migration : complete
Any time you want to see the current state of the migrations, you can run mograte list
to see an output like:
$ mograte list
1316027432511-add-pets.js migrated on May 2, 2022, 1:57:46 PM by firstnamelastname
1316027433425-coolest-pet.js migrated on May 2, 2022, 1:57:46 PM by crazygithubhandle
Rolling Migrations Back
Before you migrate up, you can run mograte delta
, which will give you the number of migrations that have not been migrated. This number can be used to rollback migrations in case they fail or don't pass any post-migration tests you may have written.
$ set MIGRATION_DELTA=$(mograte delta)
$ mograte up
up : 1316027432511-add-pets.js
up : 1316027432575-add-owners.js
migration : complete
# do some smoke testing - discover you migrated some data wrong
$ mograte down $MIGRATION_DELTA
down : 1316027432575-add-owners.js
down : 1316027432511-add-pets.js
migration : complete