High Availabiltity & Replica Sets with mongoDB

  • 16,007 views
Uploaded on

A short presentation with some discussion about mongoDB, why I use it and how to set up Replica Sets for a fully HA/failover enabled system.

A short presentation with some discussion about mongoDB, why I use it and how to set up Replica Sets for a fully HA/failover enabled system.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
16,007
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
98
Comments
1
Likes
13

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. High Availability & Replica Sets with Gareth Davies @ShaolinTiger www.shaolintiger.com
  • 2. Who am I?- Blogger (shaolintiger.com/darknet.org.uk)- Community Starter (security-forums.com/shutterasia.com)- Geek/Sys-admin- WordPress/Web Publishing Scaling Expert- Recent MongoDB user- Currently working at Mindvalley
  • 3. Why I <3 MongoDB- Its FAST- Its relatively easy to setup- Its a LOT easier to scale than say..MySQL- Does anyone know about scaling MySQL?
  • 4. Scaling MySQL Is Like...
  • 5. Basic Concepts – Master Slave
  • 6. The Next Level – Replica Set
  • 7. Master Slave vs Replica Set
  • 8. Replica Sets – Things to Grok- The primary AKA Master is auto-elected- Drivers and mongos can detect the primary- Replica sets provide you: - Data Redundancy - Automated Failover AKA High-availaiblity - Distributed Read Load/Read Scaling - Disaster Recovery
  • 9. Replica Sets - Caveats- You must have an odd number of mongos at all times to avoid vote locking & primary becoming read only- You can have a maximum of 12 nodes in a set, if you need more read capacity – look to sharding- If you are reading from the secondary servers you need to understand Eventual Consistency
  • 10. Getting Started- I <3 Linode!- Easy scaling/Nodebalancers/Cloning/Fast Roll- up/Fastest IO in the industry/Ubuntu 12.04LTS etc– Examples are done on Linode
  • 11. Add 3 new Nodes- Chose your location, put 2 in your primary DC and 1 in a different geographical DC- In my case this would be 2 servers in Atlanta, GA and 1 in Dallas TX– This gives you a replica set that works if a whole datacenter goes down
  • 12. Select The OS- Ubuntu 12.04LTS 64-bit – this gives you package support for the next 5 years- Also gives you the ability to grow your MongoDB instance above 2GB safely
  • 13. Do your basic shizzles- Set the hostname/local IP address etc- Disable Swap- Change SSH port- Remove password based login- Block root SSH acess- aptitude update; aptitude safe-upgrade- Install base packages (munin/iotop/sysstat etc)- Configure unattended security updates
  • 14. Install MongoDB- I dont recommend installing from the regular repo as it will be out of date after some time- Install direct from the 10gen reposudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10sudo nano /etc/apt/sources.list – add this:deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10genaptitude update; aptitude install mongodb-10gen- Thats it – its installed!
  • 15. Clone that bad boy!- Bear in mind you only have to do all of that stuff once! When its done – just clone it over to your two new nodes.- Remember to delete the config & disk images from your target first & power down the initial machine.* Do note when copying to the remote DC it will take quite a long time
  • 16. Get the Replica Set StartedDo:sudo nano /etc/mongodb.confFind the line like so:# in replica set configuration, specify the name of the replica set# replSet = setnameChange it to:# in replica set configuration, specify the name of the replica setreplSet = yoursetnameDo this on all your Mongos and then restart themsudo service mongodb restart
  • 17. Configure the Replica Set- After restarting if you check the logs youll seesomething like this:- This basically means the Replica set is running,but its not yet aware of the other nodes
  • 18. Add the member nodes- Youll need to run mongo to get the mongo shell then:rs.initiate({_id: yoursetname, members: [{_id: 0, host: 192.168.1.1:27017},{_id: 1, host: 192.168.1.2:27017},{_id: 2, host: 192.168.1.3:27017}]})- This will spin up the set
  • 19. Check that it worked- I suggest running tail -f on the logs on one of the other nodes, youll see a bunch of messages about replSet & rsStart (hopefully)- If you see all that in /var/log/mongodb/mongodb.log – youre good!
  • 20. Thats It!- Yah I know, too easy right?- Thats how hard it is to set up a fully scalable, high availability database cluster with MongoDB
  • 21. Things to Consider- ALWAYS monitor, make decisions made on statistics and numbers not on assumptions- I like (and very actively use) munin- munin works well with MongoDB and a myriad of other software
  • 22. Further Learnings- Think about security (Bind address/IPTables/Authentication/Cluster Keys)- If you have a write heavy application and you need to scale writes – look to sharding- Sharding and replica sets work well together (but each shard needs a replica set)- Try and give your MongoDB instance enough RAM to keep the hot index in memory
  • 23. THE END! Questions?This presentation will be available at http://slideshare.net/shaolintiger