Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Running MongoDB for High Availability

2,174 views

Published on

My talk at OSCON 2012.

Published in: Technology
  • Be the first to comment

Running MongoDB for High Availability

  1. 1. Running MongoDB for High Availability Greg Brockman Stripe twitter.com/thegdb gdb@stripe.com
  2. 2. Overview• Intro• Tips & Tricks• Best Practices• Questions
  3. 3. Traditional DB Availability
  4. 4. When the master fails...
  5. 5. Replica Set Crash Course PRIMARY SECONDARYWRITEREAD SECONDARY
  6. 6. Replica Set Crash Course DEAD SECONDARYWRITE !!READ SECONDARY
  7. 7. Replica Set Crash Course DEAD SECONDARYWRITEREAD PRIMARY
  8. 8. If at first you don’t succeed...
  9. 9. How it works in practice
  10. 10. Without retries zz
  11. 11. With retries
  12. 12. All the (hard) problems are solved
  13. 13. Tips & Tricks
  14. 14. Force initial syncs from secondariesnewnode> iptables -I OUTPUT -j DROP -d master# new in 2.2newnode> rs.syncFrom(...)
  15. 15. Rebuild whole clusters at a time Old New
  16. 16. Rebuild whole clusters at a time Old New
  17. 17. Rebuild whole clusters at a time > rs.stepDown() > rs.freeze(60)
  18. 18. Rebuild whole clusters at a time
  19. 19. Rebuild whole clusters at a time
  20. 20. Rebuild whole clusters at a time
  21. 21. Rebuild whole clusters at a time > rs.stepDown()> rs.freeze(60) > rs.freeze(60)> rs.freeze(60) > rs.freeze(60)
  22. 22. Rebuild whole clusters at a time
  23. 23. Rebuild whole clusters at a time
  24. 24. Consider using IPs vs hostnames
  25. 25. Do zero-downtime migrationsUse object Avoid multi-versioning updates!
  26. 26. Create indexes online• Build in background• Build one node at a time $ mongod --port 8000
  27. 27. Best Practices
  28. 28. Always rehearse your exact procedure
  29. 29. Be skeptical
  30. 30. Read the issues
  31. 31. Understand the guarantees
  32. 32. Keep a record of how you’ve been burned
  33. 33. Greg Brockman Stripetwitter.com/thegdb gdb@stripe.com

×