Practical Replication June-2011


Published on

My talk for MongoDC on 6/27/11

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Practical Replication June-2011

  1. 1. Practical Replication<br />Chris Westin<br />Software Engineer, 10gen<br />© Copyright 2010 10gen Inc.<br />
  2. 2. What is Replication for?<br />High availability<br />If a node fails, another node can step in<br />Extra copies of data for recovery<br />Scaling reads<br />Applications with high read rates can read from replicas<br />
  3. 3. What Does Replication Look Like?<br />Replica Set<br />A set of mongod servers<br />Minimum of 3<br />Can use “arbiters”<br />Consensus election of a “primary”<br />All writes go to primary<br />“Secondaries” replicate from primary<br />
  4. 4. Configuring a Replica Set<br />Start mongod processes with --replSet <name><br />Then:<br />
  5. 5. Managing a Replica Set<br />--oplogSize <MB><br />mongod option: choose size of replication log<br />--fastsync<br />mongod option: skip full resync of new secondary that was created from a recent copy<br />rs.add(“hostname:<port>”)<br />Shell helper: add a new member<br />rs.remove(“hostname:<port>”)<br />Shell helper: remove a member<br />
  6. 6. Some Administrative Commands<br />replSetGetStatus<br />Reports status of the replica set from one node’s point of view<br />replSetStepDown<br />Request the primary to step down<br />replSetFreeze<br />Prevents any changes to the current replica set configuration (primary/secondary status)<br />Use during backups<br />
  7. 7. How Does it Work?<br />Change operations are written to the oplog<br />The oplog is a capped collection<br />Must have enough space to allow new secondaries to catch up after copying from a primary<br />Must have enough space to cope with any applicable slaveDelay<br />Secondaries query the primary’s oplog and apply what they find<br />
  8. 8. Failover<br />Replica set members monitor each other via heartbeats<br />If the primary can’t be reached, a new one is elected<br />The secondary with the most up-to-date oplog is chosen<br />If, after election, a secondary has changes not on the new primary, those are undone, and moved aside (changes saved to a BSON file)<br />If you require a guarantee, ensure data is written to a majority of the replica set<br />
  9. 9. Priority<br />Optional parameter to replica set member configuration<br />All other things being equal, the highest priority member wins the election for primary<br />Changes in secondaries’ relative lag, i.e., catching up to primary, can trigger an election<br />Zero priority: can never become primary<br />Use for remote DR, delayed slaves, backups, analytics sources<br />
  10. 10. For Applications<br />getLastError( { w : … } )<br />Application waits until changes are written to the specified number of servers<br />Defaults can be set in the replica set’s configuration<br />“Safe mode” for critical writes: setWriteConcern()<br />Another way to force writes to a number of servers<br />Drivers support “slaveOk” for sending queries to a secondary<br />This is for scaling reads<br />
  11. 11. Replication and Sharding<br />Each shard needs its own replica set<br />Drivers use a mongos process to route queries to the appropriate shard(s)<br />Configuration servers maintain the shard key range metadata<br />
  12. 12. Replication and Sharding<br />
  13. 13. Data Center Awareness<br />Tag nodes in replica set configuration<br />Apply hierarchical labels to replica set members<br />Define getLastError modes<br />Require number of nodes writes must go to<br />Require locations of nodes writes must go to<br />Combinations<br />Available in 1.9.1<br />
  14. 14. Tagging Example<br />
  15. 15. Documentation<br /><br />Index of documents on replication topics<br />