Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

High Availabiltity & Replica Sets with mongoDB

A short presentation with some discussion about mongoDB, why I use it and how to set up Replica Sets for a fully HA/failover enabled system.

High Availabiltity & Replica Sets with mongoDB

  1. 1. High Availability & Replica Sets with Gareth Davies @ShaolinTiger www.shaolintiger.com
  2. 2. Who am I?- Blogger (shaolintiger.com/darknet.org.uk)- Community Starter (security-forums.com/shutterasia.com)- Geek/Sys-admin- WordPress/Web Publishing Scaling Expert- Recent MongoDB user- Currently working at Mindvalley
  3. 3. Why I <3 MongoDB- Its FAST- Its relatively easy to setup- Its a LOT easier to scale than say..MySQL- Does anyone know about scaling MySQL?
  4. 4. Scaling MySQL Is Like...
  5. 5. Basic Concepts – Master Slave
  6. 6. The Next Level – Replica Set
  7. 7. Master Slave vs Replica Set
  8. 8. Replica Sets – Things to Grok- The primary AKA Master is auto-elected- Drivers and mongos can detect the primary- Replica sets provide you: - Data Redundancy - Automated Failover AKA High-availaiblity - Distributed Read Load/Read Scaling - Disaster Recovery
  9. 9. Replica Sets - Caveats- You must have an odd number of mongos at all times to avoid vote locking & primary becoming read only- You can have a maximum of 12 nodes in a set, if you need more read capacity – look to sharding- If you are reading from the secondary servers you need to understand Eventual Consistency
  10. 10. Getting Started- I <3 Linode!- Easy scaling/Nodebalancers/Cloning/Fast Roll- up/Fastest IO in the industry/Ubuntu 12.04LTS etc– Examples are done on Linode
  11. 11. Add 3 new Nodes- Chose your location, put 2 in your primary DC and 1 in a different geographical DC- In my case this would be 2 servers in Atlanta, GA and 1 in Dallas TX– This gives you a replica set that works if a whole datacenter goes down
  12. 12. Select The OS- Ubuntu 12.04LTS 64-bit – this gives you package support for the next 5 years- Also gives you the ability to grow your MongoDB instance above 2GB safely
  13. 13. Do your basic shizzles- Set the hostname/local IP address etc- Disable Swap- Change SSH port- Remove password based login- Block root SSH acess- aptitude update; aptitude safe-upgrade- Install base packages (munin/iotop/sysstat etc)- Configure unattended security updates
  14. 14. Install MongoDB- I dont recommend installing from the regular repo as it will be out of date after some time- Install direct from the 10gen reposudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10sudo nano /etc/apt/sources.list – add this:deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10genaptitude update; aptitude install mongodb-10gen- Thats it – its installed!
  15. 15. Clone that bad boy!- Bear in mind you only have to do all of that stuff once! When its done – just clone it over to your two new nodes.- Remember to delete the config & disk images from your target first & power down the initial machine.* Do note when copying to the remote DC it will take quite a long time
  16. 16. Get the Replica Set StartedDo:sudo nano /etc/mongodb.confFind the line like so:# in replica set configuration, specify the name of the replica set# replSet = setnameChange it to:# in replica set configuration, specify the name of the replica setreplSet = yoursetnameDo this on all your Mongos and then restart themsudo service mongodb restart
  17. 17. Configure the Replica Set- After restarting if you check the logs youll seesomething like this:- This basically means the Replica set is running,but its not yet aware of the other nodes
  18. 18. Add the member nodes- Youll need to run mongo to get the mongo shell then:rs.initiate({_id: yoursetname, members: [{_id: 0, host: 192.168.1.1:27017},{_id: 1, host: 192.168.1.2:27017},{_id: 2, host: 192.168.1.3:27017}]})- This will spin up the set
  19. 19. Check that it worked- I suggest running tail -f on the logs on one of the other nodes, youll see a bunch of messages about replSet & rsStart (hopefully)- If you see all that in /var/log/mongodb/mongodb.log – youre good!
  20. 20. Thats It!- Yah I know, too easy right?- Thats how hard it is to set up a fully scalable, high availability database cluster with MongoDB
  21. 21. Things to Consider- ALWAYS monitor, make decisions made on statistics and numbers not on assumptions- I like (and very actively use) munin- munin works well with MongoDB and a myriad of other software
  22. 22. Further Learnings- Think about security (Bind address/IPTables/Authentication/Cluster Keys)- If you have a write heavy application and you need to scale writes – look to sharding- Sharding and replica sets work well together (but each shard needs a replica set)- Try and give your MongoDB instance enough RAM to keep the hot index in memory
  23. 23. THE END! Questions?This presentation will be available at http://slideshare.net/shaolintiger

×