Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to upgrade to MongoDB 4.0 - Percona Europe 2018

749 views

Published on

Every new version of MongoDB comes with exciting new features and a lot of improvements and version 4.0 couldn't be an exception to this rule. An upgrade from previous versions will unlock long waiting features like transactions but at the same time without proper planning could be catastrophic for your organization.

This presentation will guide you through the stapes for planning and implementing an upgrade to MongoDB 4.0. We will examine how MongoDB 4.0 affects your organization ecosystem and what changes might be necessary prior to the upgrade. We will demonstrate the upgrade steps with a detailed rollback plan. Finally, we will cover some post-upgrade considerations that will allow you to release the power of MongoDB 4.0.

Published in: Software
  • Login to see the comments

  • Be the first to like this

How to upgrade to MongoDB 4.0 - Percona Europe 2018

  1. 1. Upgrade to MongoDB 4.0 Antonios Giannopoulos DBA @ Rackspace/ObjectRocket linkedin.com/in/antonis/ 1
  2. 2. Introduction www.objectrocket.com 2 Antonios Giannopoulos
  3. 3. Overview • Upgrade Procedure • Application Layer • Middleware • Database Layer • Rollback Procedure • Why 4.0? www.objectrocket.com 3
  4. 4. MongoDB 4.0 www.objectrocket.com 4 MongoDB 4.0 released on June 26,2018 Current minor version is 4.0.3 You can obtain it from: - Mongo Inc Download Center - Percona Server Download Center - Repos like yum, apt-get…
  5. 5. MongoDB 4.0 www.objectrocket.com 5
  6. 6. Upgrade Replica-Set www.objectrocket.com 6 Upgrade the secondary, one at a time o Shut down the mongod instance o Replace the binary with the 4.0 binary o Restart the member Connect a mongo shell to the primary o Issue rs.stepDown() o Ensure a new Primary is elected Upgrade the ex-Primary o Shut down the mongod instance o Replace the binary with the 4.0 binary o Restart the member Connect a mongo shell to the primary Enable backwards-incompatible 4.0 features db.adminCommand( { setFeatureCompatibilityVersion: "4.0" }
  7. 7. Sharded Cluster www.objectrocket.com 7 s1 s2 Stop Balancer o sh.stopBalancer() o sh.getBalancerState() Upgrade config servers o Use the replica-set steps Upgrade the shards o Use the replica-set steps Upgrade the mongos o One at a time o Replace binary and restart Enable backwards-incompatible 4.0 features db.adminCommand( { setFeatureCompatibilityVersion: "4.0" } ) Restart the mongos
  8. 8. Application • Upgrade Driver • Upgrade Language • Inspect the code base www.objectrocket.com 8
  9. 9. Driver www.objectrocket.com 9 Your driver must be compactible with 4.0
  10. 10. Driver www.objectrocket.com 10
  11. 11. Driver www.objectrocket.com 11
  12. 12. So just upgrade the driver? www.objectrocket.com 12 The underling language version must also be compactible with your driver Mongo 4.0 Compatible
  13. 13. Upgrade the driver/language www.objectrocket.com 13 Perform safe upgrades /always have rollback in mind Python supports “virtual environments” Python “virtualenv” creates an isolated environment for Python projects. Each project can have its own dependencies, regardless of what dependencies every other project has. There are no limits to the number of environments ,they’re just directories containing scripts
  14. 14. Upgrade the driver/language www.objectrocket.com 14
  15. 15. Upgrade the driver/language www.objectrocket.com 15 What about other languages: There are equivalents of “virtualenv” like JAVA use classpath Ruby use Ruby Version Manager (RVM)
  16. 16. Upgrade the driver/language www.objectrocket.com 16 Do not forget frameworks and their dependencies o Very popular in PHP o They have their own compatibility matrix Should I always use the latest driver? o Check the change log o Check the open bugs
  17. 17. Driver version? Dunno www.objectrocket.com 17 Inspect the logs Manipulate the logs to export driver name & version Quick hack: less /var/log/messages |grep "driver: { name" | awk '{print $16, $17, $18, $19}' | uniq
  18. 18. Driver version? Dunno www.objectrocket.com 18 Inspect the system.profile Collect a representative sample On a sharded cluster, all shards must examined.
  19. 19. Inspect the codebase www.objectrocket.com 19 Each version Removes & Deprecates operators. Remove: Operator is no longer available Deprecate: Operator will be removed to the next version Take Actions: o Patched the codebase against the removed operators o Plan to replace the deprecated operators
  20. 20. Inspect the codebase www.objectrocket.com 20 Removed Operators: o $isolated operator: If you have an existing partial index that includes the $isolated operator or a view that includes a $isolated operator, recreate the index or view without the operator in the definition before upgrading. Deprecated Operators: o maxScan o geoNear command o copydb and the clone commands
  21. 21. Inspect the codebase www.objectrocket.com 21 Exception handling o Error descriptions may change between versions o Error format may also change o New error codes may introduced o New warning codes may introduced
  22. 22. Middleware • Monitoring • Backups • Deployments • Utilities • OS changes • TLS/SSL www.objectrocket.com 22
  23. 23. Monitoring www.objectrocket.com 23 The monitoring system must be able to connect to MongoDB Vast majority relies on db.serverStatus() A newer version may be necessary to installed prior to upgrade
  24. 24. Deployment & Managment www.objectrocket.com 24 Automatic deployment scripts also connect to MongoDB For example: o Deploy a replica-set (rs.add(), rs.status()) o Add a shard (sh.addShard()) A change to error code formats may affect deployment scripts User-roles may also change Management tools may also affected
  25. 25. Backup www.objectrocket.com 25 - Filesystems Snapshots - Copy files - Hot Backup (Percona Server) - Mongodump They should all work but testing is recommended. In the case of restore, a downgrade may be required.
  26. 26. OS www.objectrocket.com 26 Removes support for: o SLES 11 o Ubuntu 12.04 o Debian 7 Deprecates: o Windows 7/2008R2 o Windows 8/2012 o Windows 8.1/2012R2 o Ubuntu 14.04 Compatibility matrix: https://bit.ly/2ys8Crs
  27. 27. TLS www.objectrocket.com 27 Removes support for TLS 1.0 o On systems where TLS 1.1+ is available Latest PCI compliance standards require the use of TLS 1.1+ If you need to support TLS 1.0 (hopefully temporarily): o Set to none either net.ssl.disabledProtocols or --sslDisabledProtocols Test: openssl s_client -connect <host>:<port> -tls1 (tls1_1 and tls1_2)
  28. 28. Database Layer • Configuration files • Prerequisites • Deprecated items • Storage engine • Miscellaneous • FCV www.objectrocket.com 28
  29. 29. mongo.conf www.objectrocket.com 29 net.transportLayer cant be set to legacy anymore net.host: Resolves localhost IP address as configured instead of assuming 127.0.0.1 Can’t combine storage.journal.enabled: false with WiredTiger storage engine Can’t combine storage.indexBuildRetry with replication.replSetName When ssl.allowInvalidCertificates: true with x.509 authentication, an invalid certificate is only sufficient to establish a TLS/SSL connection but is insufficient for authentication.
  30. 30. Prerequisites www.objectrocket.com 30 In order to upgrade featureCompatibilityVersion must be set to 3.6 How to check: o db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } ) How to set FCV to 3.6: o db.adminCommand( { setFeatureCompatibilityVersion: <version> } ) ,where <version> ”3.6” It automatically enables 3.6 new features
  31. 31. Prerequisites www.objectrocket.com 31 3.6 new features o UUID for collections o $jsonSchema document validation o Change Streams o Chunk aware secondaries o View definitions, document validators, and partial index filters that use 3.6 query features o Sessions and retryable writes o Users and roles with authenticationRestrictions Sessions: o Creates a system.session collection o On sharded clusters is sharded on {_id:1} o Has a 30 minute TTL index o Pre-3.6.7 bug, wasn’t creating TTL and sharding o May trigger a performance overhead (updates/deletes)
  32. 32. Prerequisites www.objectrocket.com 32 In order to upgrade pv must be set to pv1 How to check: o pv=rs.conf().protocolVersion How to set pv1: o Login to every replica-set and execute cfg = rs.conf(); cfg.protocolVersion=1; rs.reconfig(cfg); Enjoy the benefits of pv1
  33. 33. Prerequisites www.objectrocket.com 33 Master-slave replication, no longer supported You must upgrade to a replica set Prior the upgrade Conversion involves downtime & affects high availability o Stop mongo o Start Master with --replSet <setname> o Initialize the replica set rs.initiate( { _id: "<setname>", members: [ { _id: 0, host: "<host:port>" } ] } ) o Add nodes, rs.add("<host:port>" )
  34. 34. Prerequisites www.objectrocket.com 34 Removes MongoDB Challenge-Response (MONGODB-CR) auth mechanism Deprecated since 3.0 version. Only possible if you upgraded from 2.6 How to check: db.getSiblingDB('admin').system.users.find({"credentials.MONGODB-CR":{$exists:true}}) How to upgrade to SCRAM: db.adminCommand({authSchemaUpgrade: 1}); Considerations o authMechanism on connection string o Local users on sharded clusters
  35. 35. Storage engine www.objectrocket.com 35 MongoDB 4.0 deprecates MMAPv1 It’s likely the next version to support only WiredTiger You should consider a switch to WiredTiger Change to WiredTiger may be challenging: o In-place Updates o Range Queries o Different HW specs o Different shard keys
  36. 36. Storage engine www.objectrocket.com 36 storage: dbPath: <data dir> engine: mmapv1 mmapv1: <mmap configuration> storage: dbPath: <data dir> engine: wiredTiger mmapv1: <mmap configuration> wiredTiger: collectionConfig: blockCompressor: <value> engineConfig: cacheSizeGB: <value> directoryForIndexes: true journalCompressor: <value> indexConfig: prefixCompression: <value> Not mandatory Better control
  37. 37. Storage engine www.objectrocket.com 37 Switch to WT (without secondary reads) WT 1)Convert one secondary - Alter the configuration - Stop mongo, wipe data dir,start mongodb - Initial sync converts the secondary to WT 2)Promote the secondary to become Primary - rs.freeze(300) the MMAPv1 secondary - rs.stepdown(300) WT 3)Burn Period - Convert the remaining nodes - Rollback to MMAPv1 Freeze
  38. 38. Storage engine www.objectrocket.com 38 Switch to WT (secondary reads) WT 1)Convert one secondary 3)Promote the secondary to become Primary WT 2)Burn Period (Secondary) 4)Burn Period (Primary) Important: Make sure less secondary can serve your workload WT
  39. 39. Storage engine www.objectrocket.com 39 What can I do in advance? o Benchmark with a real workload o Prepare to “upgrade” your hardware o Divide and conquer databases/collections o Optimize range scans, if possible o Replace updates with inserts, if possible o Change shard keys, if possible Workloads that MMAPv1 may perform better o Read-Only o Updates that change a small fraction of the document (counter)
  40. 40. Miscellaneous www.objectrocket.com 40 o Removes the limit on the amount of data that can be rolled back, was 300MiB o The rollback time limit defaults to 1 day, was 30 minutes o Configurable via rollbackTimeLimitSecs, wasn’t configurable o If you don’t have enough oplog, rollback will fail o Make sure oplog duration > rollbackTimeLimitSecs o Disable Rollbacks createRollbackDataFiles o The oplog can grow past its configured size limit
  41. 41. Miscellaneous www.objectrocket.com 41 taskExecutorPoolSize: Number of Task Executor connection pools o New default is 1, was 0. o Value of 0 means - number of cores < 4, the number of pools is 4. - 4 <= number of cores =< 64, the number of pools is the number of cores. - number of cores> 64, the number of pools is 64. AsyncRequestsSenderUseBaton: Default true Enables performance optimization on Linux for scatter/gather operations on mongos when using a single Task Executor connection pool. Under investigation!
  42. 42. FCV www.objectrocket.com 42 o To enable 4.0 new features you must execute: db.adminCommand({setFeatureCompatibilityVersion: ”4.0"}) o Its advisable to wait for a brief period of time o During this period none of the 4.0 features will be available o Rollback to 3.6 would be more difficult after raise the FCV - We will examine rollback in the next chapter
  43. 43. FCV – What does it do? www.objectrocket.com 43 Enables: o SCRAM-SHA-256 o New type conversion operators and enhancements (typical for major version) o Multi-document transactions o $dateToString option changes (onNull) o New change stream methods o Change stream resume token data type change (hex-encoded string)
  44. 44. Confidence level www.objectrocket.com 44 Major Version Replica-set Sharded Cluster Reason 2.2 2.2.3 2.4 2.4.3 2.4.6 Chunk Migration 2.6 2.6.6 2.6.6 Optimizer 3.0 3.0.5 3.0.8 Data loss 3.2 3.2.6 3.2.11 ASIO bug 3.4 3.4.4 3.4.7 Minor bugs 3.6 3.6.3 3.6.8 sessions
  45. 45. Downgrade • Downgrade FCV • Rollback a replica set • Rollback a sharded cluster www.objectrocket.com 45
  46. 46. Downgrade to 3.6 www.objectrocket.com 46 Downgrade FCV to 3.6 o db.adminCommand({setFeatureCompatibilityVersion: "3.6"}) Remove incompatible features: o Drop views, document validators, and partial index filters that use 4.0 query features o Downgrade the authentication mechanism from SCRAM-SHA-2 to "SCRAM-SHA-1" db.updateUser( ”username", { mechanisms: [ "SCRAM-SHA-1" ], pwd: <newpwd> } ) o Take a backup o Verify that all replica set members are in sync o Use latest 3.6 if possible
  47. 47. Downgrade Replica-Set www.objectrocket.com 47 Downgrade the secondary, one at a time o Shut down the mongod instance o Replace the binary with the 3.6 binary o Restart the member Connect a mongo shell to the primary o Issue rs.stepDown() o Ensure a new Primary is elected Upgrade the ex-Primary o Shut down the mongod instance o Replace the binary with the 3.6 binary o Restart the member
  48. 48. Sharded Cluster www.objectrocket.com s1 s2 Stop Balancer o sh.stopBalancer() o sh.getBalancerState() Downgrade config servers o Use the replica-set steps Downgrade the shards o Use the replica-set steps Downgrade the mongos o One at a time o Replace binary and restart Start Balancer o sh.startBalancer() o sh.getBalancerState()
  49. 49. Why 4.0? • Transactions • Secure authentication • Change streams • Non blocking Secondary reads • Performance • Miscellaneous improvements www.objectrocket.com 49
  50. 50. Transactions www.objectrocket.com 50 MongoDB 4.0 supports multi-document transactions o Only available on replica sets o Requires FCV = “4.0” o Requires 4.0 compatible drivers o Requires WiredTiger storage engine o Transactions buffer lives in cache o 16MB document size limit due to the oplog
  51. 51. Transactions www.objectrocket.com 51
  52. 52. SCRAM-SHA-256 www.objectrocket.com 52 o SHA1 has been deprecated due to its security vulnerabilities o SHA2 is the successor of SHA1 and SHA-256 its one of its variants o SHA-256 produces a 256 bits hash (vs 160 on SHA1) o SHA2 is less vulnerable to collision attacks o SHA2 is vulnerable to collision attacks length extension attack
  53. 53. Change Streams www.objectrocket.com 53 o Open a change stream cursor for a single database* o Open a change stream cursor for a deployment** o Adds the startAtOperationTime option o Resume token is now a hex-encoded string. Allows comparison and sort. *excluding admin, local, and config database ** In 3.6 the type is BinData
  54. 54. Secondary Reads www.objectrocket.com 54 Before 4.0 o Writes are applied as batches. o Block reads during batches to avoid a "wrong" order. o Periodically the readers have to wait for replication batches to be applied o Batches needs a lock that requires all reads to complete before it can be taken o Write heavy workloads increase latency In 4.0 o Timestamps in the storage engine o Using transactions to get a consistent view of data at a specific "cluster time". o Secondary reads takes advantage of snapshots isolation o Relax the replication lock/allows reads while writes are happening. o Index format changed to avoid collisions (unique indexes)
  55. 55. WiredTiger www.objectrocket.com 55 o WiredTiger 3.1.1: (July 12, 2018) vs WiredTiger 2.9.2: (December 23, 2016) o Improved Eviction (WT-3683, WT-3437, WT-4141) o Improved Checkpoints (WT-4111) o Reduces the number of internal writes by 50% o WiredTiger timestamps (uses MongoDB timestamps)
  56. 56. Miscellaneous www.objectrocket.com 56 $near and $nearSphere supports querying on sharded collections Adds key option for the $geoNear aggregation operator and geoNear command Mongos can log slow statements: Command line options: slowms Configuration file options: operationProfiling.slowOpThresholdMs Mongos can rate limit slow statements: Command line options: --slowOpSampleRate Configuration file options: operationProfiling.slowOpSampleRate Mongo shell method supports convertShardKeyToHashed
  57. 57. Miscellaneous www.objectrocket.com 57 Aggregation framework supports $convert Converts a value to specified type:
  58. 58. Questions? www.objectrocket.com 58
  59. 59. Feedback www.objectrocket.com 59 antonios.giannopoulos@rackspace.co.uk @iamantonios linkedin.com/in/antonis/ Please share: Your feedback regarding this presentation What are you most interest to hear from us in the future
  60. 60. www.objectrocket.com 60 We’re Hiring! Looking to join a dynamic & innovative team? https://www.objectrocket.com/careers/ or email careers@objectrocket.com
  61. 61. Thank you! Address: 401 Congress Ave Suite 1950 Austin, TX 78701 Support: 1-800-961-4454 Sales: 1-888-440-3242 www.objectrocket.com 61

×