MongoDB on EC2 and EBS

  • 54,062 views
Uploaded on

Best practices and recommendations for deploying MongoDB on Amazon EC2 and EBS.

Best practices and recommendations for deploying MongoDB on Amazon EC2 and EBS.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • At MongoDirector (www.mongodirector.com) we automate the entire process of deploying and managing Mongo replica sets and shards. Provisioned IOPS are used for optimal performance. We also use LVM snapshots for backup so that your backups take the same amount of time irrespective of the size of data.
    Are you sure you want to
    Your message goes here
  • The suggestions given here were really helpful and thanks a lot for that. We are currently moving our EBS volumes to RAID 10 structure, and I have a question related to that.

    If we use mdadm *only*, it looks like there is no way to increase the size of storage after the initial set-up.

    If I use LVM on top of mdadm, then we can extend the storage online by adding one or more RAID 10 devices to LVM.
    However adding LVM means one more piece and more complexity. It would be great if we could just do it using mdadm, but I can't seem to find a solution. Any tips here would be great. Thanks.
    Are you sure you want to
    Your message goes here
  • Well, Good luck with it...
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
54,062
On Slideshare
0
From Embeds
0
Number of Embeds
41

Actions

Shares
Downloads
818
Comments
3
Likes
114

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. MongoDB on Amazon Web Services
    Webinar 3-14-2011
  • 2. Agenda
    MongoDB deployment components
    Amazon EC2 infrastructure
    Deployment scenarios
  • 3. Mongodb components
  • 4. MongoDB Components
    • Actual data
    • 5. Needs RAM + Disk IO
    • 6. Can run as Arbiter
    • 7. No data
    • 8. Just votes to elect primary
    • 9. Stateless router
    • 10. Typically run on App Servers
    • 11. Stores sharding configuration
    • 12. Stores small amounts of data
    • 13. Infrequently queried/updated by MongoS
  • Replica Sets
    Primary
    Primary
    Primary
    Secondary
    Secondary
    Secondary
    Secondary
    Arbiter
    Secondary
    Secondary
    Secondary
  • 14. Sharding
    MongoD
    MongoD
    MongoD
    MongoD
    MongoS
    MongoS
    MongoS
    ConfigD
    MongoD
    MongoD
    MongoD
    MongoD
    ConfigD
    ConfigD
    MongoD
    MongoD
    MongoD
    MongoD
  • 15. Amazon EC2
  • 16. Amazon EC2 Instance Types
    32-bit = Don’t Use
    Typical MongoD
    ConfigD/ Arbiter
    Big MongoD
    32-bit = Don’t Use
    High CPU not necessary
  • 17. OS
    Turn off atime
    Raise file descriptor limits
    cat >> /etc/security/limits.conf << EOF
    * hard nofile 65536
    * soft nofile 65536
    EOF
    DO NOT use large VM pages
    Use ext4, xfs
    Use RAID
    RAID10 on MongoD
    RAID1 on ConfigbDB
    Warning! Known problems with Ubuntu 10.04 & EBS
    https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853
    https://bugzilla.kernel.org/show_bug.cgi?id=16991
  • 18. Mongodb components on ec2
  • 19. MongoD on EC2
    MongoD
    64-Bit EC2 Instance
    LVM or
    MDADM
    RAID 10
    EBS Volumes
  • 20. MongoS on EC2
    Language Specific Driver
    Your App
    32- or 64-Bit EC2 Instance
    (Typically your app server)
    MongoS
  • 21. Arbiter on EC2
    64-Bit EC2 Instance
    (Micro Instance is Fine)
    Arbiter
    Local Instance Storage
  • 22. ConfigDB on EC2
    64-Bit EC2 Instance
    (Micro Instance is Fine)
    ConfigDB
    LVM or MDADM
    RAID 1
    EBS Volumes
  • 23. Deployment Scenarios
  • 24. 3-Node Replica Set
    Your App
    Large
    Large
    Large
    MongoD
    (Primary)
    MongoD
    (Secondary)
    MongoD
    (Secondary)
    RAID 10
    RAID 10
    RAID 10
  • 25. 2-Nodes + Arbiter
    Your App
    Large
    Large
    Micro
    MongoD
    (Primary)
    MongoD
    (Secondary)
    Arbiter
    RAID 10
    RAID 10
  • 26. 3-Node Replica Set
    Your App
    US West
    US East
    Large
    Large
    Large
    MongoD
    (Primary)
    MongoD
    (Secondary)
    MongoD
    (Secondary)
    Priority = 0
    RAID 10
    RAID 10
    RAID 10
  • 27. Config DB
    Sharding
    Amazon ELB
    App Tier
    Shard 2
    Shard 1
    Shard 3
  • 28. Running in ec2
  • 29. Operational Considerations
    Automating Deployment
    Puppet
    Chef
    Amazon CloudFormation
  • 30. Backup / Restore
    EBS Snapshots
    Also for rapid replica provisioning
    LVM Snapshots
    Use secondaries for zero downtime backups
    http://www.mongodb.org/pages/viewpage.action?pageId=19562846
  • 31. Security Groups
    • All nodes listen on all interfaces unless –bind_ip is specified
    • 32. Specify alternate port with –port
    • 33. Don’t mix names & IP’s. Names preferred
    • 34. Use security groups as access control lists
    • 35. (e.g. allow 27017 from app servers, but not everybody)
  • Final thoughts
    EC2 is a great platform for Mongo
    Lots of production use
    Try it out! Test a big cluster for cheap!
    We’re here to help!
  • 36. Questions?
    I am
    Jared Rosoff (jsr@10gen.com) @forjared
    Try it out!
    http://www.mongodb.org/downloads
    http://www.mongodb.org/display/DOCS/Amazon+EC2