Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Percona Backup for
MongoDB
Solution for consistent backups of MongoDB sharded cluster
The Main
Problem
Consistency
If there are write operations during the
dump operation, the dump will not reflect a
single m...
Single-Shard
Backups
Use “mongodump --oplog” option
Creates a oplog file as part of the
mongodump output. Which contains o...
Single-Shard
Backups
4
Start
Consistencyattheend
mongodump
oplog
Multi-Shard
Backups
Backups created with mongodump do not
maintain the atomicity guarantees of
transactions across shards....
Multi-Shard
Backups
6
Start
NOConsistency
ShardB mongodump
ShardB oplog
ShardC mongodump
ShardA oplog
ShardC oplog
ShardA ...
Why?
With mongodump it is basically impossible
to stop oplog exactly on the same
transaction on all shards simultaneously
...
Multi-Shard
Backups
8
Start
Consistencyattheend
ShardB mongodump
ShardB oplog
ShardC mongodump
ShardA oplog
ShardC oplog
S...
Percona Backup for
MongoDB
Solution for consistent backups of MongoDB sharded cluster
Agent
● Agent must be attached on localhost of each
mongod instance (including secondaries and config
server instances).
●...
CLI
● Administrators observe and control the backups or
restores with a pbm CLI command that they can run
from any host wi...
How to start
using it?
● Create user If authentication is enabled
● Run agent locally for each mongod instance
● Configure...
Current state and
Roadmap
Current state
● MongoDB 3.6, 4.0, 4.2 versions are supported.
● Backup and restore via S3-protocol.
Officially supported o...
Roadmap
● Recovery to any point-in-time (PITR)
● Centralize logs
● Encryption support
● Enhanced selection of backup sourc...
We are Hiring!
Speak to me ;)
© 2019 Percona16
Q & A
Mykola Marzhan
Director of Server Engineering
Upcoming SlideShare
Loading in …5
×

DevOps Fest 2020. Николай Маржан. Consistent backups of multi-shard MongoDB

It only sounds like a simple thing to make a consistent backup of the database which builds around eventual consistency ideas.
Let's review the current types of backups and their limitations when it comes to sharding. From there we will move into why you can't be consistent with a single node, and how you can take sharded or unsharded consistent backups.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

DevOps Fest 2020. Николай Маржан. Consistent backups of multi-shard MongoDB

  1. 1. Percona Backup for MongoDB Solution for consistent backups of MongoDB sharded cluster
  2. 2. The Main Problem Consistency If there are write operations during the dump operation, the dump will not reflect a single moment in time. Changes made to the database during the backup process can affect the output of the backup. Source: https://docs.mongodb.com/manual/reference/program/mongodump/ © 2019 Percona 2
  3. 3. Single-Shard Backups Use “mongodump --oplog” option Creates a oplog file as part of the mongodump output. Which contains oplog entries that occur during the mongodump operation. This file provides an consistent snapshot of the state of a mongod instance. Source: https://docs.mongodb.com/manual/reference/program/mongodump/ © 2019 Percona 3
  4. 4. Single-Shard Backups 4 Start Consistencyattheend mongodump oplog
  5. 5. Multi-Shard Backups Backups created with mongodump do not maintain the atomicity guarantees of transactions across shards. Source: https://docs.mongodb.com/manual/reference/program/mongodump/ © 2019 Percona 5
  6. 6. Multi-Shard Backups 6 Start NOConsistency ShardB mongodump ShardB oplog ShardC mongodump ShardA oplog ShardC oplog ShardA mongodump
  7. 7. Why? With mongodump it is basically impossible to stop oplog exactly on the same transaction on all shards simultaneously without stop writes. © 2019 Percona 7
  8. 8. Multi-Shard Backups 8 Start Consistencyattheend ShardB mongodump ShardB oplog ShardC mongodump ShardA oplog ShardC oplog ShardA mongodump
  9. 9. Percona Backup for MongoDB Solution for consistent backups of MongoDB sharded cluster
  10. 10. Agent ● Agent must be attached on localhost of each mongod instance (including secondaries and config server instances). ● Agent connects and works only with/via MongoDB. No need to handle additional security aspects: open ports, TLS/SSL, authorization etc. ● Agent detects if it is a good candidate to do the backup or restore operations and coordinates with the other instances to accomplish the requested actions. © 2019 Percona10
  11. 11. CLI ● Administrators observe and control the backups or restores with a pbm CLI command that they can run from any host with the access to the MongoDB cluster. ● CLI connects and works only with/via MongoDB. Not needed to handle agent addresses, TLS/SSL, authorization etc. © 2019 Percona11
  12. 12. How to start using it? ● Create user If authentication is enabled ● Run agent locally for each mongod instance ● Configure location of remote storage ● Run “pbm backup” command © 2019 Percona12
  13. 13. Current state and Roadmap
  14. 14. Current state ● MongoDB 3.6, 4.0, 4.2 versions are supported. ● Backup and restore via S3-protocol. Officially supported object stores: AWS S3, GCP GCS (interoperability), MinIO. ● Backup and restore from locally mounted NAS/SAN. ● Take backup from Secondary or hidden nodes if possible. ● Klaus Post S2 compression (extension of Snappy) enabled by default for oplogs and logical backups. Gzip and LZ4 are supported also. © 2019 Percona14 Percona Solutions
  15. 15. Roadmap ● Recovery to any point-in-time (PITR) ● Centralize logs ● Encryption support ● Enhanced selection of backup source © 2019 Percona15 Percona Solutions
  16. 16. We are Hiring! Speak to me ;) © 2019 Percona16
  17. 17. Q & A Mykola Marzhan Director of Server Engineering

×