Webinar: Best Practices for Upgrading to MongoDB 3.2
•Download as PPTX, PDF•
1 like•4,695 views
MongoDB 3.2 introduces a host of new features and benefits, including encryption at rest, document validation, MongoDB Compass, numerous improvements to queries and the aggregation framework, and more. To take advantage of these features, your team needs an upgrade plan. In this session, we’ll walk you through how to build an upgrade plan. We’ll show you how to validate your existing deployment, build a test environment with a representative workload, and detail how to carry out the upgrade. By the end, you should be prepared to start developing an upgrade plan for your deployment.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (15)
Similar to Webinar: Best Practices for Upgrading to MongoDB 3.2
Similar to Webinar: Best Practices for Upgrading to MongoDB 3.2 (20)
Recently uploaded
Recently uploaded (20)
Webinar: Best Practices for Upgrading to MongoDB 3.2
- 1. Best Practices for Upgrading to MongoDB 3.2 Richard Kreuter MongoDB
- 2. • Upgrading the database itself is pretty easy. • Upgrading all the bits around the database is… less easy. • The effort is worth it. • We can help you.
- 3. What’s cool in 3.2
- 4. More use cases. Pluggable storage engines enables you to use MongoDB in more projects with the same core database. Mission-critical apps. MongoDB delivers major advances in the critical areas of governance, high availability, and disaster recovery. New tools for new users. Now MongoDB is an integral part of the tooling and workflows of Data Analysts, DBAs, and Operations teams. 3.2 Overview
- 5. Document Validation What you get • Implement data governance • Enforce data quality • Use familiar MongoDB queries to specify validation rules More power to the DBAs • The DBA can specify which documents in a collection should be validated – Pre-existing documents can be left unvalidated, optionally. • Validation failure consequences are tunable • Hard error • Just a warning
- 6. Document Validation Example The example on the left adds a rule to the contacts collection that validates: • The year of birth is no later than 1994 • The document contains a phone number and / or an email address • When present, the phone number and email addresses are strings
- 7. Partial Indexes { "_id" : 1234, "archived" : "true" ... } A partial index can ignore all documents where ignored is true.
- 8. $lookup Join What you get • Equivalent to left outer join • Built into aggregation framework $lookup: { localField : "stock", foreignField : "stockQuote", from : "other_collection", as : "joined_stock" }
- 10. Faster Failover for Replica Sets • Enhanced algorithm detects failure and isolation of primary. • Clusters more resilient to overloaded or unreliable networks What you get • <2 seconds failover (tunable with the electionTimeoutMillis) How to upgrade: • Conceptually: – cfg=rs.conf() – cfg.protocolVersion=1 – rs.reconfig(cfg)
- 11. Config Servers as Replica Sets Overview • Config server replica sets can span more than 3 data centers with up to 50 replica set members supported What you get • Simplified sharded deployments – Config servers are deployed as replica sets • Improved metadata consistency • Easily scale to many data centers How to upgrade • New clusters only, or a cluster reboot (for now)
- 12. Storage Engines
- 13. Storage Engine Architecture in 3.2 Content Repo IoT Sensor Backend Ad Service Customer Analytics Archive MongoDB Query Language (MQL) + Native Drivers MongoDB Document Data Model WT MMAP Available for MongoDB 3.2 Management Security In-memory (beta) Encrypted 3rd party
- 14. WiredTiger is the Default What you get • Best general-purpose storage engine • 7-10x better throughput • Up to 80% compression “Out of the box” • No special configuration needed
- 15. 7x-10x Performance, 50%-80% Less Storage How: WiredTiger Storage Engine • Same data model, same query language, same ops • Write performance gains driven by document-level concurrency control • Storage savings driven by native compression • 100% backwards compatible • Non-disruptive upgrade MongoDB 3.0 MongoDB 2.6 Performance
- 16. 50%-80% Less Storage via Compression • Better storage utilization • Higher I/O scalability • Multiple compression options – Snappy – zlib – None • Data and journal compressed on disk • Indexes compressed on disk and in memory
- 18. Encrypted Storage What you get • Encryption of sensitive data for regulated industries • ~ 15% overhead • Better than many 3rd party encryption tools Note • Based on the WiredTiger Storage Engine • Available in MongoDB Enterprise Advanced How to upgrade • Rolling upgrade
- 19. In-Memory Storage • Predictable throughput and latency • In-memory computing without trading away guarantees of disk-based databases like – Rich query flexibility – Real-time analytics – Scalable capacity – Durability • Durability via replica set secondaries (using mmap/wt)
- 20. One Deployment Powering MultipleApps
- 21. New Tools for New Users
- 22. Data Analysts • BI Connector • $lookup • Aggregation Enhancements: Faster & simpler real-time analytics
- 23. DBAs MongoDB Compass • Visualize schema • View Documents • Graphically build queries • Authenticated access
- 24. Operations Integration with standard operational workflow reduces overhead • Start from global view of infrastructure: APM tools integration, e.g. New Relic, AppDynamics. • Then, drill down: Profiler visualization in Ops Manager • Then, deploy: Automated index builds • Next, refine: Partial Indexes improve resource utilization
- 25. Ops Manager & Cloud Manager
- 26. Single-click provisioning, scaling & upgrades, admin tasks Monitoring, with charts, dashboards and alerts on 100+ metrics Backup and restore, with point-in-time recovery, support for sharded clusters MongoDB Ops Manager / Cloud Manager The Best Way to Manage MongoDB In Your Data Center Up to 95% Reduction in Operational Overhead
- 27. How Ops Manager / Cloud Manager Helps You Scale EasilyMeet SLAs Best Practices, Automated Cut Management Overhead
- 28. Query Perf. Visualizations & Optimization Fast and simple query optimization with the new Visual Query Profiler • Query and write latency are consolidated and displayed visually; your ops teams can easily identify slower queries and latency spikes • Visual query profiler analyzes the data it displays and provides recommendations for new indexes that can be created to improve query performance • Ops Manager and Cloud Manager can automate the rollout of new indexes, reducing risk and your team’s operational overhead
- 29. Ops Manager Backup Enhancements 3.2 includes Ops Manager enhancements to improve the productivity of your ops teams and further simplify installation and management • MongoDB backup on standard network-mountable filesystems; integrates with your existing storage infrastructure • Automated database restores; Build clusters from backup in a few clicks • Faster time to first database snapshot • Support for maintenance windows • Centralized UI for installation and config of all application and backup components
- 30. These new capabilities and enhancements are why it's worth upgrading to MongoDB 3.2.
- 31. Upgrading MongoDB in Outline • Manual upgrades – lots of steps, but doable • Automatic – Ops Manager / Cloud Manager
- 32. Manual Upgrade
- 33. Upgrading a cluster manually • Follow the steps in the fine documentation: – http://docs.mongodb.org/manual/release-notes/3.2-upgrade/ – First result if Googling for “MongoDB Upgrade” • It's critical to note that upgrade order matters: – For replica sets, first upgrade the secondaries, then the primaries. – For a sharded cluster, follow the 7 steps in the documentation.
- 34. Automatic
- 35. Upgrading a cluster withAutomation • Ops Manager / Cloud Manager is the easiest, fastest, and best way to upgrade a MongoDB deployment – One-click upgrades. – Rolling upgrades, so zero downtime.
- 36. Upgrades • Select Desired Version from Dropdown
- 37. As you can see, upgrading the database software itself is pretty easy.
- 38. The fine details • Backward compatibility & your application • Backward compatibility & your operations • Database behavior & performance • Enabling 3.2 Features
- 40. Backward Compatibility: App • Comprehensive documentation about incompatible changes – https://docs.mongodb.org/manual/release-notes/3.2-compatibility • Highlights – You may need to upgrade your applications' drivers before upgrading MongoDB! – Some commands/methods deprecated, removed, changed. • Audit your codebase for uses of deprecated functionality.
- 41. Backward Compatibility: Ops • Comprehensive documentation about incompatible changes – https://docs.mongodb.org/manual/release-notes/3.2-compatibility • Highlights – Some settings deprecated, others mmapv1-only. – Stricter replica set configuration validation – mongo* tools' options changed – Legacy (pre-2.6) user account model removed • Audit scripts & other infrastructure you've got around.
- 42. Database Behavior & Performance
- 43. Database Behavior & Performance • Always test your application with the new database version before upgrading production – Apps sometimes rely on unspecified behaviors (e.g. stability of results from unsorted queries) – Query optimizer improvements may affect index selection (check important queries with explain)
- 44. Good practice: Test against Staging • Run all your performance correctness tests against a staging environment for some time before upgrading production. • Best practice: run a real application workload against a 3.2 staging environment, too. – Replicate the workload manually – Use Facebook’s Flashback tool
- 45. In our experience, upgrading everything around the database software is where the work is.
- 46. Recap
- 47. Upgrading Revisited • Plan • Practice • Productionize
- 48. Plan Your Upgrade • Review the compatibility notes – Audit apps, ops for deprecated details • Upgrade drivers, scripts, if necessary – Might require some recoding, in edge cases • Make sure you've got good tests – App behavior, performance, etc. • Write a checklist of upgrade steps for your environments
- 49. Practice Your Upgrade • Upgrade apps & database in staging environment • Test, test, test. – If no problems appear within (say) a week of continuous testing, probably a decent candidate.
- 50. Productionize Your Upgrade • Upgrade apps & database in production environment – Ideally, during minimum load/traffic conditions, just for good measure
- 51. MongoDB's team can help you!
- 52. How we help users upgrade • MongoDB's Professional Services team offers consulting engagements specifically for upgrading: – To ensure upward compatibility for your apps, tools – To assist with performance & behavior testing around database upgrades – To prepare & execute upgrade plans with your team in your environments. Contact us for more info: http://mongodb.com/upgrade
Editor's Notes
- Presentation at 5, so I'll do a quick overview of the features
- Validation is optional, and can be as simple as a single field, all the way to every field, including existence, data types, and regular expressions. Failed validation tests can result in a hard error or just a warning
- Validation is optional, and can be as simple as a single field, all the way to every field, including existence, data types, and regular expressions. Failed validation tests can result in a hard error or just a warning
- Raft like protocol, with election terms Timeout was previously for all Fast Failure - in many scenarios failover will be well under 2 seconds including sub-second, but we want to set reliable expectations The exact failover time is dependent on the system’s configuration (for example the network latency between data centers), but for a typical configuration it would be no more than 2 seconds – the user can tune this using the electionTimeoutMillis parameter.
- Config server setup provides higher levels of availability and lower cross-region latency.
- Now let’s take a deep dive into the Storage Engines and how they can help you use MongoDB for a wide array of new projects. With this release you get a lot more choice with your storage engines. You can mix and match multiple storage engines within a deployment to bring a new level of optimization to your applications. Let’s take a look so you can see which is right for you.
- WiredTiger was available as an alternative to the MMAP Storage Engine, which was the default storage engine in previous to MongoDB 3.0. WiredTiger offers a lot of benefits to MongoDB users. WiredTiger has granular concurrency control and native compression, giving you better storage efficiency and higher performance for a broad range of apps. Now it is the default storage engine in MongoDB 3.2, so no need to do any special configuration to get these benefits.
- Upgrade can be made transparently, the application will not know. Different performance operational profiles. WT: Excellent write performance
- Encrypted Storage engine is the name of the feature but is NOT separate from WT, rather an option -- want to avoid any chance of confusion! KMIP is a popular standard that we plan to support; SafeNet and Vormetric are good examples, as is Amazon’s KMS
- In-memory computing is known to be much faster, and it enables data access and analytics at speeds never before possible. However, in the past, memory costs were prohibitive. Now, that the cost of memory has gone down, and will continue to tumble, you can take advantage of these performance gains.
- As illustrated by the ecommerce example above, user data is managed by the In-Memory engine to provide the throughput and bounded latency essential for great customer experience. However, the product catalog’s data storage requirements exceed server memory capacity, so is provisioned to another MongoDB replica set configured with the disk-based WiredTiger storage engine. In this example, MongoDB’s flexible storage architecture means developers are freed from the complexity of having to use different in-memory and disk-based databases to support the e-commerce application. Administrators are freed from the complexity of having to configure and manage separate data layers. Instead, the application uses the same MongoDB database with each service powered by the storage engine best optimized for the use case.
- BI Connector requires Enterprise Advanced subscription SQL supported is read-only and covers what is required for most BI tools (we can’t simply say SQL92 b/c it only supports reads)
- MongoDB Compass requires Professional of Enterprise Advanced subscription “Determine validator rules”: you can use the tool to figure out what you want to set as validation rules. A future version could integrate with the database to set document validation rules for a collection.
- MongoDB Ops Manager and Cloud Manager is the best way to run MongoDB, reducing tasks such as deployment, scaling, upgrades and backups to just a few clicks or an API call. Operations teams can be 10-20x more productive using the Ops or Cloud Manager platforms. With these enhancements to Ops and Cloud manager, administrators can Integrate MongoDB alongside existing Application Performance Monitoring platforms for global health visibility over the entire IT estate, all from a single pane of glass Drill down into any MongoDB-specific issues using Ops Manager’s monitoring of key database telemetry, including new query profiler visualizations Create a point-in-time backups and consistent snapshots of the database on standard network-mountable filesystems Use Ops Manager automation to initiate zero-downtime maintenance and upgrade activities, such as rolling out new indexes across a sharded cluster Profiler Visualization: Enabling Fast and Simple Query Optimization Automated Index Builds New Indexing Option: Partial Indexes
- Ops Manager can do a lot for [ops teams]. Best Practices, Automated. Ops Manager takes best practices for running MongoDB and automates them. So you run ops the way MongoDB engineers would do it. This not only makes it more fool-proof, but it also helps you… Cut Management Overhead. No custom scripting or special setup needed. You can spend less time running and managing manual tasks because Ops Manager takes care of a lot of the work for you, letting you focus on other tasks. Meet SLAs. Automating critical management tasks makes it easier to meet uptime SLAs. This includes managing failover as well as doing rolling upgrades with no downtime. Scale Easily. Provision new nodes and systems with a single click.
- Order Matters
- xxx
- If upgrading to 3.0, votes User Roles, of the SuperUser/ReadOnly Audit
- Be careful, MongoDB will have some changes in the behavriou or performance Large complicated piece of software. Some things we intend to do, and we document them, and there are behavrious that we don’t follow. From time to time, changes in implementaion. Your application might inadvertenly rely on this.
- Always test your application against the new version. Most of you might way of course. Optimizer, look at the explain plain When you throw a query at MongoDB when you don’t specify sort order, you’ll get one order, and newer version don’t make that promeise. They never made that promise
- Always test your application against the new version. Most of you might way of course. Optimizer, look at the explain plain When you throw a query at MongoDB when you don’t specify sort order, you’ll get one order, and newer version don’t make that promeise. They never made that promise
- enable 3.2.0 features
- A few days, perhaps a week. Rolling upgrade Burn in test
- We’ll construct these plans, we’ll help write test scripts to simulate the workload or maybe even record the workload and then replay it.