High Availabiltity & Replica Sets with mongoDB


Published on

A short presentation with some discussion about mongoDB, why I use it and how to set up Replica Sets for a fully HA/failover enabled system.

Published in: Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

High Availabiltity & Replica Sets with mongoDB

  1. 1. High Availability & Replica Sets with Gareth Davies @ShaolinTiger www.shaolintiger.com
  2. 2. Who am I?- Blogger (shaolintiger.com/darknet.org.uk)- Community Starter (security-forums.com/shutterasia.com)- Geek/Sys-admin- WordPress/Web Publishing Scaling Expert- Recent MongoDB user- Currently working at Mindvalley
  3. 3. Why I <3 MongoDB- Its FAST- Its relatively easy to setup- Its a LOT easier to scale than say..MySQL- Does anyone know about scaling MySQL?
  4. 4. Scaling MySQL Is Like...
  5. 5. Basic Concepts – Master Slave
  6. 6. The Next Level – Replica Set
  7. 7. Master Slave vs Replica Set
  8. 8. Replica Sets – Things to Grok- The primary AKA Master is auto-elected- Drivers and mongos can detect the primary- Replica sets provide you: - Data Redundancy - Automated Failover AKA High-availaiblity - Distributed Read Load/Read Scaling - Disaster Recovery
  9. 9. Replica Sets - Caveats- You must have an odd number of mongos at all times to avoid vote locking & primary becoming read only- You can have a maximum of 12 nodes in a set, if you need more read capacity – look to sharding- If you are reading from the secondary servers you need to understand Eventual Consistency
  10. 10. Getting Started- I <3 Linode!- Easy scaling/Nodebalancers/Cloning/Fast Roll- up/Fastest IO in the industry/Ubuntu 12.04LTS etc– Examples are done on Linode
  11. 11. Add 3 new Nodes- Chose your location, put 2 in your primary DC and 1 in a different geographical DC- In my case this would be 2 servers in Atlanta, GA and 1 in Dallas TX– This gives you a replica set that works if a whole datacenter goes down
  12. 12. Select The OS- Ubuntu 12.04LTS 64-bit – this gives you package support for the next 5 years- Also gives you the ability to grow your MongoDB instance above 2GB safely
  13. 13. Do your basic shizzles- Set the hostname/local IP address etc- Disable Swap- Change SSH port- Remove password based login- Block root SSH acess- aptitude update; aptitude safe-upgrade- Install base packages (munin/iotop/sysstat etc)- Configure unattended security updates
  14. 14. Install MongoDB- I dont recommend installing from the regular repo as it will be out of date after some time- Install direct from the 10gen reposudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10sudo nano /etc/apt/sources.list – add this:deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10genaptitude update; aptitude install mongodb-10gen- Thats it – its installed!
  15. 15. Clone that bad boy!- Bear in mind you only have to do all of that stuff once! When its done – just clone it over to your two new nodes.- Remember to delete the config & disk images from your target first & power down the initial machine.* Do note when copying to the remote DC it will take quite a long time
  16. 16. Get the Replica Set StartedDo:sudo nano /etc/mongodb.confFind the line like so:# in replica set configuration, specify the name of the replica set# replSet = setnameChange it to:# in replica set configuration, specify the name of the replica setreplSet = yoursetnameDo this on all your Mongos and then restart themsudo service mongodb restart
  17. 17. Configure the Replica Set- After restarting if you check the logs youll seesomething like this:- This basically means the Replica set is running,but its not yet aware of the other nodes
  18. 18. Add the member nodes- Youll need to run mongo to get the mongo shell then:rs.initiate({_id: yoursetname, members: [{_id: 0, host:},{_id: 1, host:},{_id: 2, host:}]})- This will spin up the set
  19. 19. Check that it worked- I suggest running tail -f on the logs on one of the other nodes, youll see a bunch of messages about replSet & rsStart (hopefully)- If you see all that in /var/log/mongodb/mongodb.log – youre good!
  20. 20. Thats It!- Yah I know, too easy right?- Thats how hard it is to set up a fully scalable, high availability database cluster with MongoDB
  21. 21. Things to Consider- ALWAYS monitor, make decisions made on statistics and numbers not on assumptions- I like (and very actively use) munin- munin works well with MongoDB and a myriad of other software
  22. 22. Further Learnings- Think about security (Bind address/IPTables/Authentication/Cluster Keys)- If you have a write heavy application and you need to scale writes – look to sharding- Sharding and replica sets work well together (but each shard needs a replica set)- Try and give your MongoDB instance enough RAM to keep the hot index in memory
  23. 23. THE END! Questions?This presentation will be available at http://slideshare.net/shaolintiger