DEPRECATION NOTICE
This project is abandoned.
mongolfier
A Simple MySQL to MongoDB Migration Tool
NOTE: not "montgolfier" but "mongolfier". ;)
install and run
local install and run
$ npm install mongolfier
$ node ./node_modules/.bin/mongolfier --help
global install and run
$ npm install -g mongolfier
$ mongolfier --help
simple migration with bulk-copy
To copy all rows from users
table on test
mysql database
into users
collection on test
mongo db.
$ mongolfier -s mysql://root@localhost/test -o mongodb://localhost/test -b users
NOTE use -d
flag to test WITHOUT real insert
operations:
$ mongolfier -s mysql://root@localhost/test -o mongodb://localhost/test -b -d users
NOTE use -e
flag to remove all existing documents in the collection:
$ mongolfier -s mysql://root@localhost/test -o mongodb://localhost/test -b -e users
complex migration with template(EXPERIMENTAL)
For complex migration,
you need to write a config file and template files and use -c
option:
$ mongolfier -c config.json
usage
$ mongolfier -help
Usage: mongolfier [OPTIONS] [COLLECTION_NAME ...]
Options:
-h, --help output usage information
-V, --version output the version number
-c, --config [FILE] path to config file(js or json)
-s, --mysql [URL] mysql connection url
-o, --mongo [URL] mongodb connection url
-e, --empty-collection make empty collection before migration
-b, --bulk-copy do bulk copy if no template available
-d, --dry-run do not insert into collection
configuration
config file(json)
- NOTE: no comment allowed here. this file should conforms to strict json format.
- NOTE:
{{
and}}
is just a marker. replace it with yours.
"mysql": "mysql://user:password@host:port/database" "mongo": "mongodb://user:password@host:port/db" "context": "{{custom_attr_key}}": "{{custom_attr_value}}" ... "before": "{{before_script.js}}" ... "after": "{{after_script.js}}" ... "collections": "collection{ "collection": "mongo_collection_name", "template": "path_to_collection_template", "query": "mysql_select_query" }, { "collection": "mongo_collection_name", "template": "path_to_collection_template", "query": [ "mysql_select_query_part1", "mysql_select_query_part2", ... ] }, ... ]}
context
is optional. this could be accessed via$
variable across processing all collections.before
is optional. these scripts are evalulated before processing the first collection.after
is optional. these scripts are evalulated after processing the last collection.template
is optional(default:{{collection name}}.json
).query
is string or array. array will be joined to a query.
mapping template/script
NOTE: comment allowed here. this file is javascript or something. ;)
NOTE:
{{
and}}
is just a marker. replace it with yours.
simple mapping template
// use mongodb ObjectID "_id": // use mysql column values "{{mongo_field_name}}": $mysql_column_name ...
complex mapping template
var id = ;... "_id": id "foo": $bar "sum": $num1 + $num2 "now": ...
- NOTE on the enclosing braces the result. do not use
return
keyword.
async mapping template(EXPERIMENTAL)
var d = qdefer;var id = ;;dpromise;
- NOTE on the last line
d.promise;
. do not usereturn
keyword.
predefined objects
$ROW
- the current mysql row as object.$COLLECTION
- the current mongo collection as object.$MYSQL
- the active mysql connection.$MONGO
- the active mongo connection.$CONTEXT
- a shared object across all mappings.mongo
- mongo modulemysql
- mysql moduleconsole
- console objectutil
- util modulefs
- fs modulepath
- path moduleQ
- q module_
- lodash module.- and so on...
TBD... ;)
That's all folks.
May the SOURCE be with you...