MongoDB on EC2 and EBS
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

MongoDB on EC2 and EBS

on

  • 55,026 views

Best practices and recommendations for deploying MongoDB on Amazon EC2 and EBS.

Best practices and recommendations for deploying MongoDB on Amazon EC2 and EBS.

Statistics

Views

Total Views
55,026
Views on SlideShare
30,682
Embed Views
24,344

Actions

Likes
114
Downloads
811
Comments
3

80 Embeds 24,344

http://nosql.mypopescu.com 10058
http://www.10gen.com 6272
http://todobi.blogspot.com 2683
http://www.newvem.com 996
http://blog.nosqlfan.com 924
http://todobi.blogspot.com.es 391
http://www.mongodb.com 316
http://architects.dzone.com 314
http://www.dataprix.com 272
http://www.pescatoreluca.com 229
http://www.scoop.it 198
http://journal.sooey.com 192
http://www.nosqldatabases.com 159
http://todobi.blogspot.mx 141
http://data.story.lu 115
http://newvem.staging.wpengine.com 105
http://cloud.csdn.net 104
http://www.techgig.com 103
http://www.csdn.net 95
https://www.10gen.com 82
http://todobi.blogspot.com.ar 77
http://java.dzone.com 67
http://rg443blog.wordpress.com 52
http://feeds.feedburner.com 37
http://paper.li 32
http://www.todobi.blogspot.com 29
http://archive.10gen.com 27
http://web.vabx.com 26
http://j-reference.blogspot.com 19
http://linyo.ws 17
http://cloud.dzone.com 17
http://fe.com 17
http://mickeeeeey.blogspot.com 17
http://localhost 14
http://webcache.googleusercontent.com 13
http://abjkk.posterous.com 12
http://www.hanrss.com 9
http://static.slidesharecdn.com 9
http://twitter.com 8
http://www.linkedin.com 7
url_unknown 6
https://twitter.com 6
http://mickeeeeey.blogspot.jp 6
https://www.mongodb.com 6
http://translate.googleusercontent.com 5
http://todobi.blogspot.com.br 5
http://saoqiao.com 4
http://a0.twimg.com 4
http://todobi.blogspot.it 4
http://drupal1.10gen.cc 3
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • At MongoDirector (www.mongodirector.com) we automate the entire process of deploying and managing Mongo replica sets and shards. Provisioned IOPS are used for optimal performance. We also use LVM snapshots for backup so that your backups take the same amount of time irrespective of the size of data.
    Are you sure you want to
    Your message goes here
    Processing…
  • The suggestions given here were really helpful and thanks a lot for that. We are currently moving our EBS volumes to RAID 10 structure, and I have a question related to that.

    If we use mdadm *only*, it looks like there is no way to increase the size of storage after the initial set-up.

    If I use LVM on top of mdadm, then we can extend the storage online by adding one or more RAID 10 devices to LVM.
    However adding LVM means one more piece and more complexity. It would be great if we could just do it using mdadm, but I can't seem to find a solution. Any tips here would be great. Thanks.
    Are you sure you want to
    Your message goes here
    Processing…
  • Well, Good luck with it...
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

MongoDB on EC2 and EBS Presentation Transcript

  • 1. MongoDB on Amazon Web Services
    Webinar 3-14-2011
  • 2. Agenda
    MongoDB deployment components
    Amazon EC2 infrastructure
    Deployment scenarios
  • 3. Mongodb components
  • 4. MongoDB Components
    • Actual data
    • 5. Needs RAM + Disk IO
    • 6. Can run as Arbiter
    • 7. No data
    • 8. Just votes to elect primary
    • 9. Stateless router
    • 10. Typically run on App Servers
    • 11. Stores sharding configuration
    • 12. Stores small amounts of data
    • 13. Infrequently queried/updated by MongoS
  • Replica Sets
    Primary
    Primary
    Primary
    Secondary
    Secondary
    Secondary
    Secondary
    Arbiter
    Secondary
    Secondary
    Secondary
  • 14. Sharding
    MongoD
    MongoD
    MongoD
    MongoD
    MongoS
    MongoS
    MongoS
    ConfigD
    MongoD
    MongoD
    MongoD
    MongoD
    ConfigD
    ConfigD
    MongoD
    MongoD
    MongoD
    MongoD
  • 15. Amazon EC2
  • 16. Amazon EC2 Instance Types
    32-bit = Don’t Use
    Typical MongoD
    ConfigD/ Arbiter
    Big MongoD
    32-bit = Don’t Use
    High CPU not necessary
  • 17. OS
    Turn off atime
    Raise file descriptor limits
    cat >> /etc/security/limits.conf << EOF
    * hard nofile 65536
    * soft nofile 65536
    EOF
    DO NOT use large VM pages
    Use ext4, xfs
    Use RAID
    RAID10 on MongoD
    RAID1 on ConfigbDB
    Warning! Known problems with Ubuntu 10.04 & EBS
    https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853
    https://bugzilla.kernel.org/show_bug.cgi?id=16991
  • 18. Mongodb components on ec2
  • 19. MongoD on EC2
    MongoD
    64-Bit EC2 Instance
    LVM or
    MDADM
    RAID 10
    EBS Volumes
  • 20. MongoS on EC2
    Language Specific Driver
    Your App
    32- or 64-Bit EC2 Instance
    (Typically your app server)
    MongoS
  • 21. Arbiter on EC2
    64-Bit EC2 Instance
    (Micro Instance is Fine)
    Arbiter
    Local Instance Storage
  • 22. ConfigDB on EC2
    64-Bit EC2 Instance
    (Micro Instance is Fine)
    ConfigDB
    LVM or MDADM
    RAID 1
    EBS Volumes
  • 23. Deployment Scenarios
  • 24. 3-Node Replica Set
    Your App
    Large
    Large
    Large
    MongoD
    (Primary)
    MongoD
    (Secondary)
    MongoD
    (Secondary)
    RAID 10
    RAID 10
    RAID 10
  • 25. 2-Nodes + Arbiter
    Your App
    Large
    Large
    Micro
    MongoD
    (Primary)
    MongoD
    (Secondary)
    Arbiter
    RAID 10
    RAID 10
  • 26. 3-Node Replica Set
    Your App
    US West
    US East
    Large
    Large
    Large
    MongoD
    (Primary)
    MongoD
    (Secondary)
    MongoD
    (Secondary)
    Priority = 0
    RAID 10
    RAID 10
    RAID 10
  • 27. Config DB
    Sharding
    Amazon ELB
    App Tier
    Shard 2
    Shard 1
    Shard 3
  • 28. Running in ec2
  • 29. Operational Considerations
    Automating Deployment
    Puppet
    Chef
    Amazon CloudFormation
  • 30. Backup / Restore
    EBS Snapshots
    Also for rapid replica provisioning
    LVM Snapshots
    Use secondaries for zero downtime backups
    http://www.mongodb.org/pages/viewpage.action?pageId=19562846
  • 31. Security Groups
    • All nodes listen on all interfaces unless –bind_ip is specified
    • 32. Specify alternate port with –port
    • 33. Don’t mix names & IP’s. Names preferred
    • 34. Use security groups as access control lists
    • 35. (e.g. allow 27017 from app servers, but not everybody)
  • Final thoughts
    EC2 is a great platform for Mongo
    Lots of production use
    Try it out! Test a big cluster for cheap!
    We’re here to help!
  • 36. Questions?
    I am
    Jared Rosoff (jsr@10gen.com) @forjared
    Try it out!
    http://www.mongodb.org/downloads
    http://www.mongodb.org/display/DOCS/Amazon+EC2