This document discusses strategies for migrating MongoDB data, including rsync, dump/restore, replica set extension, and Mongo2Toku sync. It provides details on the pros and cons of each method as well as examples. The level of effort and downtime associated with each method is also summarized. Rsync has the fastest transfer speed but highest potential downtime. Dump/restore has the least effort but long import times. Replica set extension takes advantage of built-in replication with almost no downtime.

1. Big Data Movement
(or how to migrate MongoDB data)
MongoDB Days Boston 2014
Andy Woodard
Data Engineer/Migration Manager
@andy_woodard - awoodard@objectrocket.com

2. What is a migration?
The process of moving data from one place to another.
Why would I do it?
● Hardware Upgrades
● Changes to environment:
Providers
Data Centers
Operating Systems
Mongo Distributions

3. Common Migration Strategies
1) Rsync
2) Dump/Restore
3) ReplicaSet Extension
Bonus:
1) Mongo2Toku

4. Rsync
Pros
• Fastest way for large data sets
• Direct copy of the Mongo data files
• Do not have to rebuild indexes
• Low amount of effort
• Padding Factor stays intact
Cons
• Potentially high downtime - need to shutdown Mongo during
transfer
• Data is still fragmented (indexes too)
Use Cases
• Millions of Documents
• Large Indexes
• Avoid time and resource constraints related index builds
Full Examples at: https://github.com/astrowicked/mongo-migation-examples/blob/master/rsync.txt

5. Rsync- Example
Tip: Use with blowfish for more secure transfer:

6. Dump and Restore
Pros
• Full Backup of data
• Least amount of effort
• Reclaim space from fragmentation
Cons
• Potentially High Downtime
• Long import times due to data size and # of docs/indexes
• Lose the padding factor that Mongo ‘learned’ during usage
Use Cases
• Cross data center migration
• Single database migration
• Small overall data size
Full Example at: https://github.com/astrowicked/mongo-migation-examples/blob/master/mongodump_restore.txt

7. Dump and Restore - Example
Full Backup/mongodump:
Single Database:

8. Dump and Restore - Example
Restore:
Single DB Restore:

9. ReplicaSet Extension
Pros
• Takes advantage of built-in replication
• Almost no downtime
• Defragments data files/indexes and reclaims space
Cons
• More moving parts (network components) and small interruptions
(adding nodes and reconfig)
• Lose the padding factor that mongo ‘learned’ from your usage
Use Cases
• Geographical moves (depends on distance)
• Always up, no room for downtime
Full Example at: https://github.com/astrowicked/mongo-migation-examples/blob/master/replset_extension.txt

10. ReplicaSet Extension - Example
Add destination nodes on current Primary:
Ready for cutover, reconfig the set:

11. Mongo2Toku Sync
Pros
• Full Backup/Copy of data
• Least amount of effort
• Reclaim space from fragmentation
Cons
• Lose the padding factor that Mongo ‘learned’ during usage
• Some factors could require the need to restart the sync
Use Cases
• Moving in the same region (or DC)
• Single database migration
• Cannot afford downtime of previous methods
Full Example at: https://github.com/astrowicked/mongo-migration-examples/blob/master/mongo2toku_sync.txt

12. Mongo2Toku -Example

13. Method Level of Effort Downtime
Rsync Low Medium - High
Dump/Restore Low Medium-High
ReplSet
Extension
Medium - High Low
Mongo2Toku
Sync
Low Low

14. Contact
@andy_woodard
@objectrocket
awoodard@objectrocket.com
https://www.objectrocket.com
WE ARE HIRING!
https://www.objectrocket.com/careers/
http://github.com/objectrocket.com/Utils
https://www.surveymonkey.com/r/XSB3ND3
http://www.slideshare.net/andyjwoodard/mongodbmigrationstrategies