Getting Started with MongoDB  and Amazon Web Services            Sandeep Parikh      Technical Product Marketing      sand...
What We’ll Cover•   MongoDB components » AWS instances•   Deployment configurations•   Storage configuration•   Production...
MongoDB Componentsmongod• 64-bit instances• Where your data lives• Needs RAM and Disk I/Omongos• Stateless request router•...
Amazon EC2 Components         • mongod           – Standard: Large or Extra Large             (typical)            – Hi-Me...
DEPLOYMENT CONFIGURATIONS
Single Node• mongod  – 64-bit EC2 instance          mongod• Storage  – Multiple EBS volumes  – RAID 10  – Configured using...
Replica Setmongod       mongod      mongodprimary     secondary   secondaryRAID 10      RAID 10     RAID 10
Replica Set – Using Zonesmongod       mongod      mongodprimary     secondary   secondaryRAID 10      RAID 10     RAID 10Z...
Replica Set – Multiple Regions                      appmongod            mongod        mongodprimary          secondary   ...
Replica Set – Security Groups              app       “application”mongod       mongod       mongodprimary     secondary   ...
Sharded Deployment                                           configapp server     app server     app server   config mongo...
CONFIGURING STORAGE
Recommendations• EBS-backed storage vs.  instance-based   – Persistent vs. ephemeral• RAID 10: “striped  mirrors”• 4-8 EBS...
DEPLOYMENT NOTES
Some Tips• Know your deployment   – Security   – Instances   – Storage• Configure filesystem as Ext4 or XFS• Reduce I/O ov...
More Tips• EBS snapshots are an easy way to back up  data• Deploy in a trusted environment  – Consider authentication
REPLICA SET DEPLOYMENT STEPS
Security Groups• Create groups for the following  – app servers  – MongoDB components• Ports to remember  –   mongod: 2701...
Single Node                • We’ll use this as a                  starting pointmongod                • Single EC2 instanc...
Creating the ComponentsCreate an instance                $ ec2-run-instances ami-41814f28 -n 1 -g database -k cluster-keyp...
Storage ConfigurationAttach storage (4x)$ ec2-attach-volume vol-e796108a -i i-11eee072 -d /dev/sdh1                       ...
Storage ConfigurationPartition device and make filesystem$ sudo fdisk /dev/md0$ sudo mkfs.ext4 /dev/md0p1Create mount poin...
Install MongoDBUpdate local repo settings$ echo "[10gen]name=10gen Repositorybaseurl=http://downloads-distro.mongodb.org/r...
MongoDB ConfigurationSet the dbpath$ sudo nano /etc/mongod.conf...dbpath=/data...Set to autostart at boot and start the se...
Adding Additional Nodes• For each node, first  repeat steps for   – Creating instances and     volumes   – Configuring sto...
MongoDB ConfigurationSet the dbpath$ sudo nano /etc/mongod.conf...dbpath=/datareplSet=replicaSetName            replica se...
Initialize Replica SetConnect to MongoDB$ mongoMongoDB shell version: 2.0.2connecting to: test>Initialize the replica set>...
Initialize Replica SetThe mongo prompt should go from this>To thisPRIMARY>Or thisSECONDARY>
BACKUP/RESTORE
Backing Up Your DataLock the database$ mongoSECONDARY> use adminSECONDARY> db.runCommand({fsync:1,lock:1});{  "info" : "no...
Backing Up Your DataUnlock the database$ mongoSECONDARY> db.$cmd.sys.unlock.findOne();{ "ok" : 1, "info" : "unlock request...
Restoring Your DataCreate and attach a volume from each snapshot$ ec2-create-volume --availability-zone us-east-1a --snaps...
Restoring Your DataUpdate the filesystem table and mount$ sudo echo „/dev/md0p1 /data2 auto noatime,noexec,nodiratime 0 0‟...
Recommended Backup Methods• Create backups on secondary servers when  using replication  – Doesn’t block on the primary  –...
QUESTIONS?
Learn More• MongoDB on Amazon Web Services white paper:  http://www.10gen.com/white-papers• Amazon EC2 info at MongoDB wik...
Contact Info• MongoDB  – http://www.mongodb.org  – Downloads, Docs, Forums, etc.• 10gen  – http://www.10gen.com/contact  –...
Upcoming SlideShare
Loading in …5
×

Getting Started with MongoDB on Amazon Web Services

10,345
-1

Published on

Published in: Technology
1 Comment
19 Likes
Statistics
Notes
  • At MongoDirector (www.mongodirector.com) we automate the entire process of deploying and managing Mongo replica sets and shards using a simple two step wizard. You can pick the number of replicas and shards and the regions in which you want to place them. Provisioned IOPS and RAID can be used for optimal performance. We also use LVM snapshots for backup so that your backups take the same amount of time irrespective of the size of data.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
10,345
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
137
Comments
1
Likes
19
Embeds 0
No embeds

No notes for slide

Getting Started with MongoDB on Amazon Web Services

  1. 1. Getting Started with MongoDB and Amazon Web Services Sandeep Parikh Technical Product Marketing sandeep.parikh@10gen.com
  2. 2. What We’ll Cover• MongoDB components » AWS instances• Deployment configurations• Storage configuration• Production tips• Setting up a three-node replica set
  3. 3. MongoDB Componentsmongod• 64-bit instances• Where your data lives• Needs RAM and Disk I/Omongos• Stateless request router• Lives on your app server• Minimal data needsconfig• Metadata server for sharded configurations• Minimal data needs• 64-bit instances
  4. 4. Amazon EC2 Components • mongod – Standard: Large or Extra Large (typical) – Hi-Memory: XL, XXL or 4XL (large data sets) – Cluster instances provide increased capacity and bandwith • mongos – Deploy onto your app server (32 or 64-bit) • config – Standard instances – Micro instance is sufficient
  5. 5. DEPLOYMENT CONFIGURATIONS
  6. 6. Single Node• mongod – 64-bit EC2 instance mongod• Storage – Multiple EBS volumes – RAID 10 – Configured using mdadm RAID 10
  7. 7. Replica Setmongod mongod mongodprimary secondary secondaryRAID 10 RAID 10 RAID 10
  8. 8. Replica Set – Using Zonesmongod mongod mongodprimary secondary secondaryRAID 10 RAID 10 RAID 10Zone 1 Zone 2 Zone 3
  9. 9. Replica Set – Multiple Regions appmongod mongod mongodprimary secondary secondaryRAID 10 RAID 10 RAID 10 Region 1 Region 2
  10. 10. Replica Set – Security Groups app “application”mongod mongod mongodprimary secondary secondaryRAID 10 RAID 10 RAID 10 “database”
  11. 11. Sharded Deployment configapp server app server app server config mongos mongos mongos config Shard 1 Shard 2 Shard 3
  12. 12. CONFIGURING STORAGE
  13. 13. Recommendations• EBS-backed storage vs. instance-based – Persistent vs. ephemeral• RAID 10: “striped mirrors”• 4-8 EBS volumes for best performance http://en.wikipedia.org/wiki/Nested_RAID_levels#RAID_1_.2B_0
  14. 14. DEPLOYMENT NOTES
  15. 15. Some Tips• Know your deployment – Security – Instances – Storage• Configure filesystem as Ext4 or XFS• Reduce I/O overhead – noatime, nodiratime• Raise file descriptor limits
  16. 16. More Tips• EBS snapshots are an easy way to back up data• Deploy in a trusted environment – Consider authentication
  17. 17. REPLICA SET DEPLOYMENT STEPS
  18. 18. Security Groups• Create groups for the following – app servers – MongoDB components• Ports to remember – mongod: 27017, 28017 (web-based status) – mongod shard server: 27018 – mongos: 27017 – config: 27019• Security groups act as ACLs• Instances are accessible via SSH keys
  19. 19. Single Node • We’ll use this as a starting pointmongod • Single EC2 instance • 4 EBS volumes • RAID10 – Example:RAID 10 • 100 GiB total • 50 GiB usable
  20. 20. Creating the ComponentsCreate an instance $ ec2-run-instances ami-41814f28 -n 1 -g database -k cluster-keypair -t m1.large -z us-east-1a AMI group size count keypair zoneCreate storage volumes (4x)$ ec2-create-volume –s 25 -z us-east-1a size zone
  21. 21. Storage ConfigurationAttach storage (4x)$ ec2-attach-volume vol-e796108a -i i-11eee072 -d /dev/sdh1 volume device instanceStart the RAID$ sudo mdadm --create -l10 -n4 /dev/md0 /dev/sdh* type devices new device
  22. 22. Storage ConfigurationPartition device and make filesystem$ sudo fdisk /dev/md0$ sudo mkfs.ext4 /dev/md0p1Create mount point and set ownership$ sudo mkdir /data$ sudo chown `id -u` /dataUpdate the filesystem table and mount$ sudo echo „/dev/md0p1 /data auto noatime,noexec,nodiratime 0 0‟ >> /etc/fstab$ sudo mount –a /dev/md0p1 /data
  23. 23. Install MongoDBUpdate local repo settings$ echo "[10gen]name=10gen Repositorybaseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64gpgcheck=0" | sudo tee -a /etc/yum.repos.d/10gen.repoInstall MongoDB and tools$ sudo yum -y install mongo-10gen-server$ sudo yum -y install sysstatChange auto-start settings$ sudo chkconfig --level 3 mongod off$ sudo chkconfig --level 5 mongod off
  24. 24. MongoDB ConfigurationSet the dbpath$ sudo nano /etc/mongod.conf...dbpath=/data...Set to autostart at boot and start the server now$ sudo chkconfig mongod on$ sudo /etc/init.d/mongod startConnect to MongoDB$ mongoMongoDB shell version: 2.0.2connecting to: test>
  25. 25. Adding Additional Nodes• For each node, first repeat steps for – Creating instances and volumes – Configuring storage• Include replica set parameter in MongoDB configuration• Start mongod
  26. 26. MongoDB ConfigurationSet the dbpath$ sudo nano /etc/mongod.conf...dbpath=/datareplSet=replicaSetName replica set name...Set to autostart at boot and start the server now$ sudo chkconfig mongod on$ sudo /etc/init.d/mongod start
  27. 27. Initialize Replica SetConnect to MongoDB$ mongoMongoDB shell version: 2.0.2connecting to: test>Initialize the replica set> rs.initiate(){ "info2" : "no configuration explicitly specified -- making one", "me" : "ip-10-127-127-91:27017", "info" : "Config now saved locally. Should come online in about a minute.", "ok" : 1}Add each replica set member> rs.add(“ec2-abc.def.amazonaws.com”) external DNS name{ “ok” : 1 }
  28. 28. Initialize Replica SetThe mongo prompt should go from this>To thisPRIMARY>Or thisSECONDARY>
  29. 29. BACKUP/RESTORE
  30. 30. Backing Up Your DataLock the database$ mongoSECONDARY> use adminSECONDARY> db.runCommand({fsync:1,lock:1});{ "info" : "now locked against writes, use db.$cmd.sys.unlock.findOne() to unlock", "ok" : 1}Run mdadm and note the UUID$ mdadm --detail /dev/md0p1Create a snapshot for each attached volume$ ec2-create-snapshot vol-1234abcd --description "MongoDB RAID backup1" volume description
  31. 31. Backing Up Your DataUnlock the database$ mongoSECONDARY> db.$cmd.sys.unlock.findOne();{ "ok" : 1, "info" : "unlock requested" }
  32. 32. Restoring Your DataCreate and attach a volume from each snapshot$ ec2-create-volume --availability-zone us-east-1a --snapshot vol-1234abcd$ ec2-attach-volume vol-1234abcd -i i-aa3bc4c8 -d /dev/sdh1Reassemble the RAID$ sudo mdadm --assemble --auto-update-homehost -u 07552c4d…a9c2f2fc --no-degraded /dev/md0mdadm: /dev/md0 has been started with 4 drives. UUID from mdadmCreate mount point and set ownership$ sudo mkdir /data2$ sudo chown `id -u` /data2
  33. 33. Restoring Your DataUpdate the filesystem table and mount$ sudo echo „/dev/md0p1 /data2 auto noatime,noexec,nodiratime 0 0‟ >> /etc/fstab$ sudo mount –a /dev/md0p1 /data2Verify the restore (optional, requires time and disk space)$ mongod --dbpath /data2$ mongoMongoDB shell version: 2.0.2connecting to: test> use yourDb> db.yourCollection.validate({full:true})Set /data2 to /data$ sudo cp /data2 /data
  34. 34. Recommended Backup Methods• Create backups on secondary servers when using replication – Doesn’t block on the primary – Secondary updates once unlocked• Use mongodump to backup live data – Collections, databases, everything – Restore with mongorestore
  35. 35. QUESTIONS?
  36. 36. Learn More• MongoDB on Amazon Web Services white paper: http://www.10gen.com/white-papers• Amazon EC2 info at MongoDB wiki: http://www.mongodb.org/display/DOCS/Amazon+EC2• Scripts for working with EBS (creating, attaching, etc.): https://gist.github.com/1482182
  37. 37. Contact Info• MongoDB – http://www.mongodb.org – Downloads, Docs, Forums, etc.• 10gen – http://www.10gen.com/contact – info@10gen.com – Consulting, Support, etc.• Sandeep Parikh – sandeep.parikh@10gen.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×