mongo-oplog-reader
Read MongoDB oplogs from a sharded cluster.
- Supports multiple processes for high availability
- Emits event after document has been written to the majority of members of the shard's replica set
- Resumes from the last emitted event of the replica set (in the case of process crash/stop/restart)
- Ability to add/remove shards/hosts without restarting
Notes:
- The order of the oplog events is not guaranteed to be strictly chronological if there are multiple workers per oplog
Requirements
- Redis 3.2+
Install
npm i mongo-oplog-reader redis
Usage
;; const redisClient = redis; const connectionStrings = 'mongodb://shard0-primary/local' 'mongodb://shard0-secondary0/local' 'mongodb://shard0-secondary1/local' 'mongodb://shard1-primary/local' 'mongodb://shard1-secondary0/local' 'mongodb://shard1-secondary1/local'; const reader = redisClient workersPerOplog: 1 // total # of redundant workers per oplog (respected across all processes);reader; // ignore 'update' operationsreader;reader;readerstart;
License
MIT