Production deployment
•Download as PPTX, PDF•
7 likes•4,333 views
It's not uncommon for a developer to download MongoDB to a workstation and, in under an hour, use it to solve what had been a show-stopping database problem. But that's not yet a production solution. MongoDB provides a wide variety of capabilities to fit production architectures. "Production" may mean anything from a 16-gram embedded device all the way to a globally distributed hybrid cloud with hundreds of servers. Join us as we apply sharding, replication, the new Ops Manager, and high-availability design to multiple enterprise production environments, addressing requirements such as five nines High Availability and Disaster Recovery.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (16)
Similar to Production deployment
Similar to Production deployment (20)
More from MongoDB
More from MongoDB (20)
Recently uploaded
Recently uploaded (20)
Production deployment
- 3. Agenda MongoDB Introduction Data Model General Production Considerations Durability, Scalability, Availability Deployment Architectures & Operations Demonstration: three servers in two data centers
- 4. MongoDB in the Market of Ideas
- 5. MongoDB, Inc. 400+ employees 2,000+ customers Over $311 million in funding13 offices around the world
- 6. THE LARGEST ECOSYSTEM 9,000,000+ MongoDB Downloads 250,000+ Online Education Registrants 35,000+ MongoDB User Group Members 35,000+ MongoDB Management Service (MMS) Users 750+ Technology and Services Partners 2,000+ Customers Across All Industries
- 8. MongoDB Use Cases Single View Internet of Things Mobile Real-Time Analytics Catalog Personalization Content Management
- 9. Data Model
- 10. Document Data Model Relational MongoDB { first_name: 'Paul', surname: 'Miller', city: 'London', location: [45.123,47.232], cars: [ { model: 'Bentley', year: 1973, value: 100000, … }, { model: 'Rolls Royce', year: 1965, value: 330000, … } ] }
- 11. Documents are Rich Data Structures { first_name: 'Paul', surname: 'Miller', cell: '+447557505611' city: 'London', location: [45.123,47.232], Profession: [banking, finance, trader], cars: [ { model: 'Bentley', year: 1973, value: 100000, … }, { model: 'Rolls Royce', year: 1965, value: 330000, … } ] } Fields can contain an array of sub- documents Fields Typed field values Fields can contain arrays
- 12. Fully Featured Database Queries
- 13. Do More With Your Data MongoDB { first_name: 'Paul', surname: 'Miller', city: 'London', location: [45.123,47.232], cars: [ { model: 'Bentley', year: 1973, value: 100000, … }, { model: 'Rolls Royce', year: 1965, value: 330000, … } } } Rich Queries Find Paul's cars Find everybody in London with a car built between 1970 and 1980 Geospatial Find all of the car owners within 5km of Trafalgar Sq. Text Search Find all the cars described as having leather seats Aggregation Calculate the average value of Paul's car collection Map Reduce What is the ownership pattern of colors by geography over time? (is purple trending up in China?)
- 14. Morphia MEAN Stack Java Python PerlRuby Support for the most popular languages and frameworks Drivers & Ecosystem
- 16. 16 Expectations 5-9’s / High Availability with replication • No scheduled downtime • Zero-downtime maintenance Linear scale-out for read and write • Commodity hardware • Cloud – Public – Private – Hybrid
- 17. 17 Infrastructure Priorities 1. Storage. It’s all about the IOPS! RAID 10 or 0. 2. RAM. Working set (only) in cache for web-scale reads. 3. CPU. Web-scale writes with WiredTiger storage engine. 4. Network. Commodity server or virtual instance, best power/price • Dual-CPU Intel, 128GB+ • Locally mounted block storage – Spinning disk – SSD – Enterprise storage with guaranteed IOPS
- 18. Production in 16 grams
- 20. Standalone
- 21. Replica Sets Replica Set – 2 to 50 copies Self-healing shard Data Center Aware Addresses availability considerations: High Availability Disaster Recovery Maintenance Workload Isolation: operational & analytics
- 22. Write Concern for Durability
- 24. Replicate Data Near Users
- 25. Automatic Sharding Three types: hash-based, range-based, location-aware Increase or decrease capacity as you go Automatic balancing
- 26. Query Routing Multiple query optimization models Each sharding option appropriate for different apps
- 28. Deployment Architectures & Operations
- 29. Development Architecture Laptop Application mongod SSD 127.0.0.1 / wire protocol SATA / MMAPv1 or Wired Tiger Driver
- 30. Single Data Center, 3 Racks Automated failover Tolerates server failures Tolerates rack failures Number of replicas defines failure tolerance
- 31. DMZDMZ Ideal: 3 Full Data Centers App Server Application Driver mongos DC1 Primary Storage DC2 Secondary Storage DC3 Secondary Storage
- 32. DMZDMZ Ideal: 3 Full Data Centers App Server Application Driver mongos DC1 Down Storage DC2 Primary Storage DC3 Secondary Storage✗
- 33. DMZDMZ Hybrid Cloud App Server Application Driver mongos DC1 Primary Storage DC2 Secondary Storage The Cloud Secondary Storage
- 34. 3 Data Centers (or servers, racks…) You can have it all • Durable commits (w: "majority") • Automatic failover and recovery • Lose any server • Lose any data center
- 35. DMZDMZ 2.1 Data Centers App Server Application Driver mongos DC1 Primary [Anywhere] Arbiter DC2 Secondary
- 36. DMZDMZ 2.1 Data Centers App Server Application Driver mongos DC1 Primary [Anywhere] Arbiter DC2 Primary ✗
- 37. Active/Active Data Center Tolerates server, rack, data center failures, network partitions
- 38. DMZDMZ Only 2 Data Centers App Server Application Driver mongos DC1 Primary [Nowhere] Nothing DC2 Secondary
- 39. DMZDMZ Only 2 Data Centers App Server Application Driver mongos DC1 Primary DC2 Secondary
- 40. 2 Data Centers (or 2 servers, racks…) Can’t have it all with two data centers • Durable commits (w:majority) • Automatic failover and recovery • Lose any server (OK so far) • Lose either data center
- 41. DMZDMZ Only 2 Data Centers App Server Application Driver mongos DC1 Primary DC2 Secondary
- 42. DMZDMZ Only 2 Data Centers App Server Application Driver mongos DC1 Primary DC2 Secondary Secondary
- 43. DMZDMZ Only 2 Data Centers App Server Application Driver mongos DC1 Primary DC2 Secondary Secondary ✗
- 44. DMZDMZ Only 2 Data Centers App Server Application Driver mongos DC1 Down DC2 Secondary Primary ✗
- 45. DMZDMZ Only 2 Data Centers App Server Application Driver mongos DC1 Primary DC2 Secondary Secondary✗
- 46. 2 Data Centers Mutually exclusive • Durable commits (w:majority) • Automatic failover and recovery • Lose either data center
- 47. 2 Data Centers Mutually exclusive • Durable commits (w:majority) • Automatic failover and recovery • Lose either data center We need an out-of-band actor
- 48. DMZDMZ Only 2 Data Centers App Server Application Driver mongos DC1 Primary DC2 Secondary priority:0.5 Secondary
- 50. DMZDMZ Only 2 Data Centers App Server Application Driver mongos DC2 Secondary priority:0.5 DC1 Down Primary ✗
- 52. DMZDMZ Ensuring DC1 Servers Stay Down App Server Application Driver mongos DC1 Down DC2 Secondary priority:0.5 Down ✗
- 53. DMZDMZ Remove DC1 votes (3.0 required) App Server Application Driver mongos DC1 votes:0 DC2 Primary votes:0 ✗
- 54. DMZDMZ Remove DC1 votes (3.0 required) App Server Application Driver mongos DC1 votes:0 DC2 Secondary votes:0 ✗
- 55. DMZDMZ This must not happen! App Server Application Driver mongos DC1 Primary DC2 Primary Secondary
- 58. DMZDMZ Recovery Complete App Server Application Driver mongos DC1 Primary DC2 Secondary Secondary
- 59. Single-click provisioning, scaling & upgrades, admin tasks Monitoring, with charts, dashboards and alerts on 100+ metrics Backup and restore, with point-in-time recovery, support for sharded clusters MongoDB Ops Manager The Best Way to Manage MongoDB In Your Data Center Up to 95% Reduction in Operational Overhead
- 60. How MongoDB Ops Manager helps you Scale EasilyMeet SLAs Best Practices, Automated Cut Management Overhead
- 61. How Ops Manager Works Ops Manager mongod mongodmongod Agent Agent Agent NewConfig.
- 63. Install and Configure two DCs brew install mongodb git clone https://github.com/rueckstiess/mtools.git [[ -d ~/data/replset ]] && rm -rf ~/data/replset mlaunch init --nodes 3 --replicaset mongo localhost:27017 #DC1 //rs.init() rs.status() //rs.add("cbiow.local:27018") //DC1 //rs.status() //rs.add("cbiow.local:27019") //simulating DC2 //rs.status() //rs.status()
- 64. Reconfigure and Fail Over in DC1 db.mycoll.insert({a:1},{writeConcern: {w: "majority", wtimeout: 5000}}) db.mycoll.find() r = rs.config() r.members[2].priority = 0.5 rs.reconfig(r) pkill -f 27017 mongo localhost:27018 rs.status() db.mycoll.insert({a:2},{writeConcern: {w: "majority", wtimeout: 5000}}) db.mycoll.find() db.mycoll.count()
- 65. DC1 Down and Recover in DC2 pkill –f 27018 mongo localhost:27018 rs.status() db.mycoll.insert({a:3},{writeConcern: {w: "majority", wtimeout: 5000}}) r = rs.config() r.members[0].votes = 0 r.members[1].votes = 0 rs.reconfig(r, { force: true }) rs.status() db.mycoll.insert({a:4},{writeConcern: {w: "majority", wtimeout: 5000}}) db.mycoll.find()
- 66. DC1 Recovery and Restore mlaunch start 27017 mongo localhost:27017 rs.status() db.mycoll.insert({a:5},{writeConcern: {w: "majority", wtimeout: 5000}}) db.mycoll.find() mlaunch start 27018 mongo localhost:27017 rs.status() r = rs.config() r.members[0].votes = 1 r.members[1].votes = 1 rs.reconfig(r)
- 67. For More Information Resource Location Case Studies mongodb.com/customers Presentations mongodb.com/presentations Free Online Training education.mongodb.com Webinars and Events mongodb.com/events Documentation docs.mongodb.org MongoDB Downloads mongodb.com/download Additional Info info@mongodb.com
Editor's Notes
- Here we have greatly reduced the relational data model for this application to two tables. In reality no database has two tables. It is much more common to have hundreds or thousands of tables. And as a developer where do you begin when you have a complex data model?? If you're building an app you're really thinking about just a hand full of common things, like products, and these can be represented in a document much more easily that a complex relational model where the data is broken up in a way that doesn't really reflect the way you think about the data or write an application.
- Rich queries, text search, geospatial, aggregation, mapreduce are types of things you can build based on the richness of the query model.
- High Availability – Ensure application availability during many types of failures Disaster Recovery – Address the RTO and RPO goals for business continuity Maintenance – Perform upgrades and other maintenance operations with no application downtime Secondaries can be used for a variety of applications – failover, hot backup, rolling upgrades, data locality and privacy and workload isolation
- High Availability – Ensure application availability during many types of failures Disaster Recovery – Address the RTO and RPO goals for business continuity Maintenance – Perform upgrades and other maintenance operations with no application downtime Secondaries can be used for a variety of applications – failover, hot backup, rolling upgrades, data locality and privacy and workload isolation
- High Availability – Ensure application availability during many types of failures Disaster Recovery – Address the RTO and RPO goals for business continuity Maintenance – Perform upgrades and other maintenance operations with no application downtime Secondaries can be used for a variety of applications – failover, hot backup, rolling upgrades, data locality and privacy and workload isolation
- High Availability – Ensure application availability during many types of failures Disaster Recovery – Address the RTO and RPO goals for business continuity Maintenance – Perform upgrades and other maintenance operations with no application downtime Secondaries can be used for a variety of applications – failover, hot backup, rolling upgrades, data locality and privacy and workload isolation
- MongoDB provides horizontal scale-out for databases using a technique called sharding, which is trans- parent to applications. Sharding distributes data across multiple physical partitions called shards. Sharding allows MongoDB deployments to address the hardware limitations of a single server, such as bottlenecks in RAM or disk I/O, without adding complexity to the application. MongoDB supports three types of sharding: • Range-based Sharding. Documents are partitioned across shards according to the shard key value. Documents with shard key values “close” to one another are likely to be co-located on the same shard. This approach is well suited for applications that need to optimize range- based queries. • Hash-based Sharding. Documents are uniformly distributed according to an MD5 hash of the shard key value. Documents with shard key values “close” to one another are unlikely to be co-located on the same shard. This approach guarantees a uniform distribution of writes across shards, but is less optimal for range-based queries. • Tag-aware Sharding. Documents are partitioned according to a user-specified configuration that associates shard key ranges with shards. Users can optimize the physical location of documents for application requirements such as locating data in specific data centers. MongoDB automatically balances the data in the cluster as the data grows or the size of the cluster increases or decreases.
- Sharding is transparent to applications; whether there is one or one hundred shards, the application code for querying MongoDB is the same. Applications issue queries to a query router that dispatches the query to the appropriate shards. For key-value queries that are based on the shard key, the query router will dispatch the query to the shard that manages the document with the requested key. When using range-based sharding, queries that specify ranges on the shard key are only dispatched to shards that contain documents with values within the range. For queries that don’t use the shard key, the query router will dispatch the query to all shards and aggregate and sort the results as appropriate. Multiple query routers can be used with a MongoDB system, and the appropriate number is determined based on performance and availability requirements of the application.
- MongoDB provides horizontal scale-out for databases using a technique called sharding, which is trans- parent to applications. Sharding distributes data across multiple physical partitions called shards. Sharding allows MongoDB deployments to address the hardware limitations of a single server, such as bottlenecks in RAM or disk I/O, without adding complexity to the application. MongoDB supports three types of sharding: • Range-based Sharding. Documents are partitioned across shards according to the shard key value. Documents with shard key values “close” to one another are likely to be co-located on the same shard. This approach is well suited for applications that need to optimize range- based queries. • Hash-based Sharding. Documents are uniformly distributed according to an MD5 hash of the shard key value. Documents with shard key values “close” to one another are unlikely to be co-located on the same shard. This approach guarantees a uniform distribution of writes across shards, but is less optimal for range-based queries. • Tag-aware Sharding. Documents are partitioned according to a user-specified configuration that associates shard key ranges with shards. Users can optimize the physical location of documents for application requirements such as locating data in specific data centers. MongoDB automatically balances the data in the cluster as the data grows or the size of the cluster increases or decreases.
- MMS can do a lot for [ops teams]. Best Practices, Automated. MMS takes best practices for running MongoDB and automates them. So you run ops the way MongoDB engineers would do it. This not only makes it more fool-proof, but it also helps you… Cut Management Overhead. No custom scripting or special setup needed. You can spend less time running and managing manual tasks because MMS takes care of a lot of the work for you, letting you focus on other tasks. Meet SLAs. Automating critical management tasks makes it easier to meet uptime SLAs. This includes managing failover as well as doing rolling upgrades with no downtime. Scale Easily. Provision new nodes and systems with a single click.