#mongodb

MongoDB and AWS
Part 1: Deployment and Availability

Sandeep Parikh
Senior Solutions Architect, MongoDB Inc.
Agenda
• MongoDB Basics
• Deployment Configurations
• AWS EC2 Instances

• Configuring Instances
• Next Steps
MongoDB Basics
MongoDB Basics
• Open source
• Document database
• High performance

• Horizontally scalable
• Full featured
• Built to ma...
MongoDB Features
• Flexible document data model
• Rich ad-hoc queries and in-place updates
• Real-time aggregation

• Geos...
Accessing MongoDB
Drivers
Drivers for most popular
programming languages and
frameworks

Java

Ruby

JavaScript

Python

S...
Deployment
Configurations
Deploying MongoDB
• Single node
– Development: prototyping, testing
• Replica Set
– Production: high availability, disaste...
MongoDB: Single Node

App

MongoDB
MongoDB: Replica Sets
App

MongoDB
Primary

MongoDB
Secondary

MongoDB
Secondary
MongoDB: Sharding
App
config

App

App

mongos

mongos

mongos

MongoDB
Primary

MongoDB
Primary

MongoDB
Primary

MongoDB...
Amazon Web Services
Amazon Web Services
• Complete cloud infrastructure
– Compute
– Storage
– Database
– Analytics
– Deployment
• Multitude of...
EC2 Instance Types
• General Purpose
• Compute-optimized
• GPU

• Memory-optimized
• Storage-optimized
• Micro
EC2 Instance Types
• General Purpose
• Compute-optimized
• GPU (compute resources not needed)

• Memory-optimized
• Storag...
EC2 Instance Types
• General Purpose
– M1, M3
• Compute-optimized
– C1, CC2, C3
• Memory-optimized
– M2, CR1
• Storage-opt...
Instance Details
• Differences in
– CPU, Memory, Storage, Networking
• Networking
– EBS-optimized, Enhanced, Cluster
• Ava...
Zones, Regions, Security
• Regions
– Data center
• Availability Zones
– Rack in a data center
• Security
– Security Groups...
Components and Sizing
mongod

config

Core database
process

Sharding
metadata

High
performance

Smaller

Memory,
Storage...
Replica Sets: Security Groups
App

“application”

MongoDB
Primary

MongoDB
Secondary

MongoDB
Secondary
“database”
Replica Sets: Availability Zones
App

MongoDB
Primary

MongoDB
Secondary
Zone 1

MongoDB
Secondary
Zone 2

Zone 3
Replica Sets: Regions
App

MongoDB
Primary

MongoDB
Secondary
Region 1

MongoDB
Secondary
Region 2
Replica Sets: Regions and Zones
App

MongoDB
Primary

MongoDB
Secondary
Region 1

MongoDB
Secondary
Region 2
Sharding: Regions
App
config

App

App

mongos

mongos

mongos

config
config

Region 1

MongoDB
Primary

MongoDB
Primary
...
Sharding: Regions
App

App

mongos

mongos

mongos

MongoDB
Primary

MongoDB
Primary

MongoDB
Primary

MongoDB
Secondary

...
High Availability
• Use Replica Sets
– Deploy in odd numbers
– Maintain majority

MongoDB
Primary

• Withstand the loss of...
Deployment Considerations
• How much availability do you need?
• Can you withstand loss of zone, region?
• Where is your a...
Best Practices
Instance Configuration Best
Practices
• Amazon Linux
• Install via yum for flexibility
• Use PIOPS EBS

• Update system se...
Instance Configuration Best
Practices
• Launch as EBS-optimized
• Use separate PIOPS volumes for data, log, journal
• Use ...
Sensible Instance Defaults
• Best practices are meant to be a sensible starting

point
• Amazon Linux optimized for EC2
• ...
Sensible Instance Defaults
• Individual volumes to reduce IO contention
• Default read ahead is too high
• Default ulimits...
AWS Marketplace
AWS Marketplace
• The AWS Marketplace is an online store that helps

users find, buy, and deploy software.
• Launch pre-co...
MongoDB on AWS Marketplace
MongoDB on AWS Marketplace
MongoDB Configurations
• Follows MongoDB best practices
– Amazon Linux, MongoDB installed via yum
– EBS PIOPS volumes per ...
MongoDB Management Service
MongoDB Management Service
• MongoDB Management Service (MMS) is a suite of services for managing

MongoDB deployments.
• ...
MMS Monitoring
MMS Monitoring
• MongoDB-specific metrics
• Performance visualization
• Metrics dashboards

• Custom alerts and notificati...
MMS Monitoring
MMS Monitoring
• Agent-based deployment monitoring
• Opcounters, memory usage, index usage, lock %,

connections, backgrou...
Coming Up Next
• Part 2 in late March
– Storage Configurations
– Storage Tradeoffs
– Disaster Recovery
• Part 3 in late Ap...
Questions?
Resources
• MongoDB on AWS best practices:
– http://docs.mongodb.org/ecosystem/platforms/amazonec2/#deploy-mongodb-on-ec2
...
#MongoDBWorld

MongoDB World
New York City, June 23-25
See what’s next in MongoDB including
• MongoDB 2.6
• Sharding
• Rep...
Upcoming SlideShare
Loading in...5
×

MongoDB and Amazon Web Services: Deploying for High Availability

2,251

Published on

Published in: Technology
0 Comments
13 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,251
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
71
Comments
0
Likes
13
Embeds 0
No embeds

No notes for slide

MongoDB and Amazon Web Services: Deploying for High Availability

  1. 1. #mongodb MongoDB and AWS Part 1: Deployment and Availability Sandeep Parikh Senior Solutions Architect, MongoDB Inc.
  2. 2. Agenda • MongoDB Basics • Deployment Configurations • AWS EC2 Instances • Configuring Instances • Next Steps
  3. 3. MongoDB Basics
  4. 4. MongoDB Basics • Open source • Document database • High performance • Horizontally scalable • Full featured • Built to match agile development and deployment
  5. 5. MongoDB Features • Flexible document data model • Rich ad-hoc queries and in-place updates • Real-time aggregation • Geospatial support • Text search • Built-in support for – Redundancy, failover, auto-partitioning
  6. 6. Accessing MongoDB Drivers Drivers for most popular programming languages and frameworks Java Ruby JavaScript Python Shell Command-line shell for interacting directly with database Perl Haskell > db.collection.insert({product:“MongoDB”, type:“Document Database”}) > > db.collection.findOne() { “_id” : ObjectId(“5106c1c2fc629bfe52792e86”), “product” : “MongoDB” “type” : “Document Database” }
  7. 7. Deployment Configurations
  8. 8. Deploying MongoDB • Single node – Development: prototyping, testing • Replica Set – Production: high availability, disaster recovery • Sharding – Production: auto-paritioning, linear read/write scale
  9. 9. MongoDB: Single Node App MongoDB
  10. 10. MongoDB: Replica Sets App MongoDB Primary MongoDB Secondary MongoDB Secondary
  11. 11. MongoDB: Sharding App config App App mongos mongos mongos MongoDB Primary MongoDB Primary MongoDB Primary MongoDB Secondary MongoDB Secondary MongoDB Secondary MongoDB Secondary MongoDB Secondary MongoDB Secondary Shard Shard Shard config config
  12. 12. Amazon Web Services
  13. 13. Amazon Web Services • Complete cloud infrastructure – Compute – Storage – Database – Analytics – Deployment • Multitude of configuration options • Pricing flexbility – On-demand, Spot, Reserved instances
  14. 14. EC2 Instance Types • General Purpose • Compute-optimized • GPU • Memory-optimized • Storage-optimized • Micro
  15. 15. EC2 Instance Types • General Purpose • Compute-optimized • GPU (compute resources not needed) • Memory-optimized • Storage-optimized • Micro (bursty, no sustained CPU)
  16. 16. EC2 Instance Types • General Purpose – M1, M3 • Compute-optimized – C1, CC2, C3 • Memory-optimized – M2, CR1 • Storage-optimized – HS1, HI1, I2
  17. 17. Instance Details • Differences in – CPU, Memory, Storage, Networking • Networking – EBS-optimized, Enhanced, Cluster • Availability – Not all are available in all regions
  18. 18. Zones, Regions, Security • Regions – Data center • Availability Zones – Rack in a data center • Security – Security Groups, VPC us-east1 uswest-1 uswest-2 1b 1a 2a 1c 1d 2b 1c 2c
  19. 19. Components and Sizing mongod config Core database process Sharding metadata High performance Smaller Memory, Storage, Network m1.small or m1.medium mongos Sharding query router Deploy on app server
  20. 20. Replica Sets: Security Groups App “application” MongoDB Primary MongoDB Secondary MongoDB Secondary “database”
  21. 21. Replica Sets: Availability Zones App MongoDB Primary MongoDB Secondary Zone 1 MongoDB Secondary Zone 2 Zone 3
  22. 22. Replica Sets: Regions App MongoDB Primary MongoDB Secondary Region 1 MongoDB Secondary Region 2
  23. 23. Replica Sets: Regions and Zones App MongoDB Primary MongoDB Secondary Region 1 MongoDB Secondary Region 2
  24. 24. Sharding: Regions App config App App mongos mongos mongos config config Region 1 MongoDB Primary MongoDB Primary MongoDB Primary MongoDB Secondary MongoDB Secondary MongoDB Secondary MongoDB Secondary MongoDB Secondary MongoDB Secondary Shard Shard Shard Region 2
  25. 25. Sharding: Regions App App mongos mongos mongos MongoDB Primary MongoDB Primary MongoDB Primary MongoDB Secondary MongoDB Secondary MongoDB Secondary MongoDB Secondary MongoDB Secondary MongoDB Secondary Shard config App Shard Shard config config Region 1 Region 2
  26. 26. High Availability • Use Replica Sets – Deploy in odd numbers – Maintain majority MongoDB Primary • Withstand the loss of – Any single zone? – Any single region? – Deploy in 3 places MongoDB Secondar y • Scale – Replica Sets for HA – Sharding for scale – Combine for both MongoDB Secondar y 1 2 3
  27. 27. Deployment Considerations • How much availability do you need? • Can you withstand loss of zone, region? • Where is your app and where are your users? • What are the security requirements? – Note: security groups don’t span regions – SSL or VPC
  28. 28. Best Practices
  29. 29. Instance Configuration Best Practices • Amazon Linux • Install via yum for flexibility • Use PIOPS EBS • Update system settings • Launch EBS optimized
  30. 30. Instance Configuration Best Practices • Launch as EBS-optimized • Use separate PIOPS volumes for data, log, journal • Use EXT4 • Set read ahead • Update ulimits • Update TCP KeepAlive
  31. 31. Sensible Instance Defaults • Best practices are meant to be a sensible starting point • Amazon Linux optimized for EC2 • EBS provides persistent storage • EBS-optimized allocates additional NIC for storage • Provisioned IOPS provides consistent EBS performance
  32. 32. Sensible Instance Defaults • Individual volumes to reduce IO contention • Default read ahead is too high • Default ulimits are too low • Default keepalive too low
  33. 33. AWS Marketplace
  34. 34. AWS Marketplace • The AWS Marketplace is an online store that helps users find, buy, and deploy software. • Launch pre-configured software and let AWS handle billing and payments
  35. 35. MongoDB on AWS Marketplace
  36. 36. MongoDB on AWS Marketplace
  37. 37. MongoDB Configurations • Follows MongoDB best practices – Amazon Linux, MongoDB installed via yum – EBS PIOPS volumes per mount (data, log, journal) – Configured: ulimits, read ahead, keep alive Data Config Log Journal Size IOPS Size IOPS Size IOPS 1000 IOPS 200 GB 1000 10 GB 100 25 GB 250 2000 IOPS 200 GB 2000 15 GB 150 25 GB 250 4000 IOPS 400 GB 4000 20 GB 200 25 GB 250
  38. 38. MongoDB Management Service
  39. 39. MongoDB Management Service • MongoDB Management Service (MMS) is a suite of services for managing MongoDB deployments. • Engineered by the team who develops MongoDB, MMS makes it easier to operate MongoDB at any scale. • MMS provides monitoring, backup and recovery, helping users optimize clusters and mitigate operational risk. • MMS users can visualize database performance and set custom alerts that notify when particular metrics are out of normal range. • MMS is also the only continuous backup solution for MongoDB, providing point-in-time recovery for replica sets and cluster-wide snapshots of sharded systems.
  40. 40. MMS Monitoring
  41. 41. MMS Monitoring • MongoDB-specific metrics • Performance visualization • Metrics dashboards • Custom alerts and notifications
  42. 42. MMS Monitoring
  43. 43. MMS Monitoring • Agent-based deployment monitoring • Opcounters, memory usage, index usage, lock %, connections, background flush, queues, page faults, and more • Notifications and alerts on any metric and condition • Agent pre-installed on AWS Marketplace instances
  44. 44. Coming Up Next • Part 2 in late March – Storage Configurations – Storage Tradeoffs – Disaster Recovery • Part 3 in late April – Integrating with AWS Services – CloudFormation – Elastic MapReduce
  45. 45. Questions?
  46. 46. Resources • MongoDB on AWS best practices: – http://docs.mongodb.org/ecosystem/platforms/amazonec2/#deploy-mongodb-on-ec2 • MongoDB on AWS Marketplace: – https://aws.amazon.com/marketplace/sellerprofile/ref=dtl_pcp_sold_by?ie=UTF8&id=c9032c7b-70dd459f-834f-c1e23cf3d092 • MongoDB docs – http://docs.mongodb.org
  47. 47. #MongoDBWorld MongoDB World New York City, June 23-25 See what’s next in MongoDB including • MongoDB 2.6 • Sharding • Replication • Aggregation http://world.mongodb.com Save 25% with discount code 25SandeepParikh
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×