MongoDB in the Cloud -- Mongo Boulder
•Download as KEY, PDF•
3 likes•862 views
This document summarizes some of the challenges of hosting MongoDB in the cloud on platforms like Amazon AWS, Rackspace, and Terremark. It discusses issues like choosing optimal instance types and disk configurations, handling growth and expansion limitations, implementing effective disaster recovery strategies, and the benefits of using a managed service like Mongo Machine to avoid these headaches. Key points covered include the tradeoffs of different instance properties, using LVM and RAID configurations to allow for resizing while maintaining performance, and practicing recovery scenarios to prepare for failures.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to MongoDB in the Cloud -- Mongo Boulder
Similar to MongoDB in the Cloud -- Mongo Boulder (20)
More from Justin Smestad
Recently uploaded
Recently uploaded (20)
MongoDB in the Cloud -- Mongo Boulder
- 1. MongoDB in the Cloud Justin Smestad @jsmestad justin@mongomachine.com Mongo Boulder Jan 21, 2011
- 3. Why host in the “Cloud”?
- 4. It makes MongoDB a commodity
- 5. Make scaling & deployment easy
- 7. Platforms: Amazon AWS The Good Large RAM Sizes Community Paired Services (SQS, S3, etc) Reserved Instances
- 8. Platforms: Amazon AWS The Good Large RAM Sizes Community Paired Services (SQS, S3, etc) Reserved Instances
- 9. Platforms: Amazon AWS The Good Large RAM Sizes Community Paired Services (SQS, S3, etc) Reserved Instances
- 10. Platforms: Amazon AWS The Good Large RAM Sizes Community Paired Services (SQS, S3, etc) Reserved Instances
- 11. Platforms: Amazon AWS The Not So Good Lack of small 64-bit instances IO Bottlenecks EBS Volumes
- 12. Platforms: Amazon AWS The Not So Good Lack of small 64-bit instances IO Bottlenecks EBS Volumes
- 13. Platforms: Amazon AWS The Not So Good Lack of small 64-bit instances IO Bottlenecks EBS Volumes
- 14. Platforms: Rackspace The Good Internal Disk Raw Performance (CPU vs. ECPU) 64-bit instances
- 15. Platforms: Rackspace The Good Internal Disk Raw Performance (CPU vs. ECPU) 64-bit instances
- 16. Platforms: Rackspace The Good Internal Disk Raw Performance (CPU vs. ECPU) 64-bit instances
- 17. Platforms: Rackspace The Not So Good Lack of large RAM instances * Community Lack of paired services
- 18. Platforms: Rackspace The Not So Good Lack of large RAM instances * Community Lack of paired services
- 19. Platforms: Rackspace The Not So Good Lack of large RAM instances * Community Lack of paired services
- 20. Platforms: Terremark The Good VMWare-based Fastest Cloud Option Fiber connected drives
- 21. Platforms: Terremark The Good VMWare-based Fastest Cloud Option Fiber connected drives
- 22. Platforms: Terremark The Good VMWare-based Fastest Cloud Option Fiber connected drives
- 23. Platforms: Terremark The Not So Good Lack of large RAM instances * Community Lack of paired services
- 24. Platforms: Terremark The Not So Good Lack of large RAM instances * Community Lack of paired services
- 25. Platforms: Terremark The Not So Good Lack of large RAM instances * Community Lack of paired services
- 26. but don’t forget... it’s not all sunshine and rainbows
- 27. Headaches
- 29. Headaches: Instance Selection 32-bit versus 64-bit High Memory versus High CPU Instance Location (even inside availability zones)
- 30. Headaches: Instance Selection 32-bit versus 64-bit High Memory versus High CPU Instance Location (even inside availability zones)
- 31. Headaches: Instance Selection 32-bit versus 64-bit High Memory versus High CPU Instance Location (even inside availability zones)
- 32. Our Suggestion Largest instance you can afford to buy in pairs.* * - that will meet your dataset requirements.
- 33. Disk Performance
- 34. Headaches: Disk Performance Ephemeral versus Elastic Block Store (EBS) EBS Performance Choosing a RAID type (0, 1, or 5) Instance or EBS based AMI (machine images)
- 35. Headaches: Disk Performance Ephemeral versus Elastic Block Store (EBS) EBS Performance Choosing a RAID type (0, 1, or 5) Instance or EBS based AMI (machine images)
- 36. Headaches: Disk Performance Ephemeral versus Elastic Block Store (EBS) EBS Performance Choosing a RAID type (0, 1, or 5) Instance or EBS based AMI (machine images)
- 37. Headaches: Disk Performance Ephemeral versus Elastic Block Store (EBS) EBS Performance Choosing a RAID type (0, 1, or 5) Instance or EBS based AMI (machine images)
- 38. Our Suggestion Use RAID-0 (if acceptable) and an EBS-based AMI. Do not rely on EBS performance to be constant.
- 40. Headaches: Growth/Expansion EBS Resizing (they must be detached) RAID 0 & RAID 1 can’t be resized LVM and XFS provide answers, but also add complexity {fsync: 1, lock: 1} is important. Also don’t close the session!
- 41. Headaches: Growth/Expansion EBS Resizing (they must be detached) RAID 0 & RAID 1 can’t be resized LVM and XFS provide answers, but also add complexity {fsync: 1, lock: 1} is important. Also don’t close the session!
- 42. Headaches: Growth/Expansion EBS Resizing (they must be detached) RAID 0 & RAID 1 can’t be resized LVM and XFS provide answers, but also add complexity {fsync: 1, lock: 1} is important. Also don’t close the session!
- 43. Headaches: Growth/Expansion EBS Resizing (they must be detached) RAID 0 & RAID 1 can’t be resized LVM and XFS provide answers, but also add complexity {fsync: 1, lock: 1} is important. Also don’t close the session!
- 44. Our Suggestion Use mdadm with LVM for RAID to enhance IO performance while keeping the ability to expand. This makes snapshotting volumes very hard.* * - alternative option is using RAID-5 but you will lose some performance.
- 45. Disaster Recovery & Failover
- 46. Headaches: Disaster Recovery Can feel a bit like “Yak Shaving” To be good at it you will have to practice. Knowing what can fail (ex: entire availability zones) ... this is going to take a few slides ...
- 47. Headaches: Disaster Recovery Can feel a bit like “Yak Shaving” To be good at it you will have to practice. Knowing what can fail (ex: entire availability zones) ... this is going to take a few slides ...
- 48. Headaches: Disaster Recovery Can feel a bit like “Yak Shaving” To be good at it you will have to practice. Knowing what can fail (ex: entire availability zones) ... this is going to take a few slides ...
- 49. Headaches: Disaster Recovery Can feel a bit like “Yak Shaving” To be good at it you will have to practice. Knowing what can fail (ex: entire availability zones) ... this is going to take a few slides ...
- 50. Headaches: Disaster Recovery EBS Reliability (better than physical, but not by much) Snapshotting to S3 (expensive) Replication Scenarios Connection Strings
- 51. Headaches: Disaster Recovery EBS Reliability (better than physical, but not by much) Snapshotting to S3 (expensive) Replication Scenarios Connection Strings
- 52. Headaches: Disaster Recovery EBS Reliability (better than physical, but not by much) Snapshotting to S3 (expensive) Replication Scenarios Connection Strings
- 53. Headaches: Disaster Recovery EBS Reliability (better than physical, but not by much) Snapshotting to S3 (expensive) Replication Scenarios Connection Strings
- 54. Our Suggestion - Practice recovery scenarios even the unlikely ones. - Replication only gets you so far unless you go across availability zones. - Calculate your “acceptable downtime” and base your infrastructure around that. If you want high availability & recovery, your better off paying someone else.
- 55. Why Mongo Machine? Moving On.
- 56. mongo hosted MongoDB + data management www.mongomachine.com
- 57. Our Product
- 58. Managed SaaS Wherever you need it.
- 59. mongo machine hosted MongoDB + data management Managed MongoDB Infrastructure Our infrastructure on AWS or Rackspace Management & Deployment to your own infrastructure Management Console Instantly create new databases Track database trends Scale up & down on-demand
- 60. Management Console Analytics data about what your DB is doing.
- 61. Management Console Analytics data about what your DB is doing.
- 62. mongo machine hosted MongoDB + data management Our Goals Increase developer efficiency Be transparent with customers Educate our clients
Editor's Notes
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n