2013 london advanced-replication

253 views
221 views

Published on

In this session covered wide area replica sets and using tags for backup.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
253
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

2013 london advanced-replication

  1. 1. #MongoDBDays - @m4rcschAdvanced ReplicationMarc SchweringSolutions Architect, 10gen
  2. 2. Notes to the presenterThemes for this presentation:•  Prepare your console use properly
  3. 3. Roles & Configuration
  4. 4. Node 1 Node 2 Secondary Arbiter Heartbeat Re pli cat on i Node 3 PrimaryReplica Set Roles
  5. 5. Configuration Options> conf = { _id : "mySet", members : [ {_id : 0, host : "A"}, {_id : 1, host : "B"}, {_id : 2, host : "C", "arbiter" : true} ]}> rs.initiate(conf)
  6. 6. Simple Setup Demo
  7. 7. Behind the Curtain
  8. 8. Implementation details•  Heartbeat every 2 seconds –  Times out in 10 seconds•  Local DB (not replicated) –  system.replset –  oplog.rs •  Capped collection •  Idempotent version of operation stored
  9. 9. Op(erations) Log
  10. 10. Op(erations) Log is idempotent> db.replsettest.insert({_id:1,value:1}){ "ts" : Timestamp(1350539727000, 1), "h" :NumberLong("6375186941486301201"), "op" : "i", "ns" :"test.replsettest", "o" : { "_id" : 1, "value" : 1 } }> db.replsettest.update({_id:1},{$inc:{value:10}}){ "ts" : Timestamp(1350539786000, 1), "h" :NumberLong("5484673652472424968"), "op" : "u", "ns" :"test.replsettest", "o2" : { "_id" : 1 }, "o" : { "$set" : { "value" : 11 } } }
  11. 11. oplog and multi-updates
  12. 12. Single operation can have manyentries> db.replsettest.update({},{$set:{name : ”foo”}, false,true}){ "ts" : Timestamp(1350540395000, 1), "h" :NumberLong("-4727576249368135876"), "op" : "u", "ns" :"test.replsettest", "o2" : { "_id" : 2 }, "o" :{ "$set" : { "name" : "foo" } } }{ "ts" : Timestamp(1350540395000, 2), "h" :NumberLong("-7292949613259260138"), "op" : "u", "ns" :"test.replsettest", "o2" : { "_id" : 3 }, "o" :{ "$set" : { "name" : "foo" } } }{ "ts" : Timestamp(1350540395000, 3), "h" :NumberLong("-1888768148831990635"), "op" : "u", "ns" :"test.replsettest", "o2" : { "_id" : 1 }, "o" :{ "$set" : { "name" : "foo" } } }
  13. 13. Operations
  14. 14. Maintenance and Upgrade•  No downtime•  Rolling upgrade/maintenance –  Start with Secondary –  Primary last –  Commands: •  rs.stepDown(<secs>) •  db.version() •  db.serverBuildInfo()
  15. 15. Upgrade Demo
  16. 16. Replica Set – 1 Data Center •  Single datacenter Datacenter •  Single switch & power Member 1 •  Points of failure: –  Power Member 2 –  Network –  Data center Datacenter 2 Member 3 –  Two node failure •  Automatic recovery of single node crash
  17. 17. Replica Set – 2 Data Centers •  Multi data center Datacenter 1 •  DR node for safety Member 1 •  Can’t do multi data Member 2 center durable write safely since only 1 node in distant DC Datacenter 2 Member 3
  18. 18. Replica Set – 2 Data Centers •  Analytics •  Disaster Recovery •  Batch Jobs •  Options –  low or zero priority –  hidden –  slaveDelay
  19. 19. Replica Set – 3 Data Centers Datacenter 1 •  Three data centers Member 1 Member 2 •  Can survive full data center loss Datacenter 2 Member 3 •  Can do w= { dc : 2 } to Member 4 guarantee write in 2 data centers (with tags) Datacenter 3 Member 5
  20. 20. Replica Set – 3+ Data Centers Secondary Primary Secondary Secondary Secondary delayed Secondary Secondary
  21. 21. Commands•  Managing –  rs.conf() –  rs.initiate(<conf>) & rs.reconfig(<conf>) –  rs.add(host:<port>) & rs.addArb(host:<port>) –  rs.status() –  rs.stepDown(<secs>)•  Minority reconfig –  rs.reconfig( cfg, { force : true} )
  22. 22. Options•  Priorities•  Hidden•  Slave Delay•  Disable indexes (on secondaries)•  Default write concerns
  23. 23. Developing with Replica Sets
  24. 24. Client Application Driver Write Read Primary Secondary SecondaryStrong Consistency
  25. 25. Client Application Driver Write ad Re Re ad Primary Secondary SecondaryDelayed Consistency
  26. 26. Write Concern•  Network acknowledgement•  Wait for error•  Wait for journal sync•  Wait for replication –  number –  majority –  Tags
  27. 27. Write Concern Demo
  28. 28. Datacenter awareness (Tagging)•  Control where data is written to, and read from•  Each member can have one or more tags –  tags: {dc: "ny"} –  tags: {dc: "ny", subnet: "192.168", rack: "row3rk7"}•  Replica set defines rules for write concerns•  Rules can change without changing app code
  29. 29. Tagging Example{ _id : "mySet", members : [ {_id : 0, host : "A", tags : {"dc": "ny"}}, {_id : 1, host : "B", tags : {"dc": "ny"}}, {_id : 2, host : "C", tags : {"dc": "sf"}}, {_id : 3, host : "D", tags : {"dc": "sf"}}, {_id : 4, host : "E", tags : {"dc": "cloud"}}], settings : { getLastErrorModes : { allDCs : {"dc" : 3}, someDCs : {"dc" : 2}} }}> db.blogs.insert({...})> db.runCommand({getLastError : 1, w : "someDCs"})> db.getLastErrorObj({"someDCs"})
  30. 30. Driver getLastError W:allDCs write Primary (SF) replicate apply in memory Secondary (NY) replicate Secondary (Cloud)Wait for Replication
  31. 31. Write Concern with timeoutsettings : { getLastErrorModes : { allDCs : {"dc" : 3}, someDCs : {"dc" : 2}} }}> db.getLastErrorObj({"allDCs"},100);> db.getLastErrorObj({”someDCs"},500);> db.getLastErrorObj(1,500);
  32. 32. Read Preference Modes•  5 modes –  primary (only) - Default –  primaryPreferred –  secondary –  secondaryPreferred –  Nearest When more than one node is possible, closest node is used for reads (all modes but primary)
  33. 33. Tagged Read Preference•  Custom read preferences•  Control where you read from by (node) tags –  E.g. { "disk": "ssd", "use": "reporting" }•  Use in conjunction with standard read preferences –  Except primary
  34. 34. Tags{"dc.va": "rack1", disk:"ssd", ssd: "installed" }{"dc.va": "rack2", disk:"raid"}{"dc.gto": "rack1", disk:"ssd", ssd: "installed" }{"dc.gto": "rack2", disk:"raid”}> conf.settings = { getLastErrorModes: { MultipleDC :{ "dc.va": 1, "dc.gto": 1}}> conf.settings = { "getLastErrorModes" : { "ssd" : { "ssd" : 1 },...
  35. 35. Tagged Read Preference{ disk: "ssd" }JAVA:ReadPreference tagged_pref = ReadPreference.secondaryPreferred( new BasicDBObject("disk", "ssd") );DBObject result = coll.findOne(query, null, tagged_pref);
  36. 36. Tagged Read Preference•  Grouping / Failover {dc : "LON", loc : "EU"} {dc : "FRA", loc : "EU"} {dc : "NY", loc : "US”} DBObject t1 = new BasicDBObject("dc", "LON"); DBObject t2 = new BasicDBObject("loc", "EU"); ReadPreference pref = ReadPreference.primaryPreferred(t1, t2);
  37. 37. Tagging Demo
  38. 38. Conclusion
  39. 39. Best practices and tips•  Odd number of set members•  Read from the primary except for –  Geographically distribution –  Analytics (separate workload)•  Use logical names not IP Addresses in configs•  Set WriteConcern appropriately for what you are doing•  Monitor secondaries for lag (Alerts in MMS)
  40. 40. #MongoDBDays - @m4rcschThank YouMarc SchweringSolutions Architect, 10gen

×