Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Percona Xtrabackup - Highly Efficient Backups

2,281 views

Published on

This Presentation is on Percona Xtrabackup and its various use cases in Backup and Recovery.

Published in: Engineering
  • @Suraj Deshmukh XtraBackup do not work with Amazon RDS, you have to use Snapshots to create new instance and logical backup ( mysqldump / mysqldumper ) to move it out of EC2.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • On slide#14 you say that > READ, WRITE & EXECUTE permissions are needed at a filesystem level in server's datadir. > This makes Xtrabackup incompatible with RDS So this confirms that there is no way to use percona Xtrabackup with RDS. What would you suggest to do to create a MySQL read Replica on prem of a RDS Master? I have tried a solution where you first create Read Replica on RDS once it's done replicating do mysqldump of this RDS Read Replica locally and then feed it to local MySQL Read Replica and ask it to follow the RDS Master. This is very manual process, is there an automated way to do it?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Percona Xtrabackup - Highly Efficient Backups

  1. 1. Percona XtraBackup Highly Efficient Backups Presenter Harish Kumar R Mydbops www.mydbops.com info@mydbops.com
  2. 2. Mydbops Mydbops is into MySQL/MongoDB Support and Consulting. It is founded by experts who have scaled database at Yahoo! ,Percona and Datavail. We are providing an expert level support and 24*7 monitoring for MySQL databases and its related technologies like MariaDB , Percona ( also clustering ) . We support modern database technologies in MySQL which includes Galera ( Clustering ), Group Replication , SQL aware Load balancers like Maxscale / ProxySQL.
  3. 3. Some of Our Happy Clients www.mydbops.com info@mydbops.com
  4. 4. Table of Contents 1. Introduction 2. Features 3. Compatibility 4. Installation 5. Working of Xtrabackup 6. Backup Scenarios
  5. 5. Introduction ➢ The most important and daily tasks for DBAs is performing backup and restore operations, not only for adding new replication slave, but also to implement disaster recovery procedures or when we want to implement testing in the server. ➢ Even when making any changes to the database schema, it's recommended to have fresh backup copy before making any live changes to avoid facing troubles in production server. ➢ If we're going to talk about backup and restore operations, then Percona Xtrabackup tool will be strongly appeared.
  6. 6. Introduction ➢ Percona Xtrabackup provides a fully open-source, high-performance, non-blocking backup system for InnoDB or XtraDB tables ➢ It can also backup MyISAM, Archive, Merge and other SQL-level objects ➢ It is a reliable, widely-used alternative to Oracle's MySQL Enterprise Backup
  7. 7. Introduction With Percona XtraBackup, you can achieve the following benefits: ➢ Backups that complete quickly and reliably ➢ Uninterrupted transaction processing during backups ➢ Savings on disk space and network bandwidth ➢ Automatic backup verification ➢ Higher uptime due to faster restore time
  8. 8. Table of Contents 1. Introduction 2. Features 3. Compatibility 4. Installation 5. Working of Xtrabackup 6. Backup Scenarios
  9. 9. Features For more details : https://www.percona.com/doc/percona-xtrabackup/2.4/intro.html ➢ Open Source (GPL) ➢ Price : Free ➢ Streaming backups ➢ Encrypted backups ➢ Supported all MySQL flavors ➢ Non-blocking InnoDB backups ➢ Incremental backups ➢ Full/Incremental compress backup ➢ Fast incremental backups ➢ Incremental backups with archived logs feature in Percona Server ➢ Backup locks ➢ Parallel local backups ➢ Parallel compression ➢ Parallel encryption ➢ Parallel apply-log ➢ Partial backups ➢ Partial backups of individual partitions ➢ Throttling ➢ Point-in-time recovery support ➢ Safe slave backups ➢ Compact backups ➢ Buffer pool state backups ➢ Individual tables export ➢ Individual partitions export ➢ Restoring tables to a different server ➢ Data & index file statistics ➢ InnoDB secondary indexes defragmentation ➢ rsync support to minimize lock time ➢ Improved FTWRL handling ➢ Backup history table
  10. 10. Table of Contents 1. Introduction 2. Features 3. Compatibility 4. Installation 5. Working of Xtrabackup 6. Backup Scenarios
  11. 11. Compatibility ➢ Versions <= 2.0 are compatible with : Oracle MySQL, 5.1, 5.5 and 5.6, equivalent versions of Percona Server and MariaDB. ➢ Version 2.1 and 2.2 are compatible with: Oracle MySQL 5.1 (plugin version), 5.5, 5.6, corresponding versions of Percona Server, Percona XtraDB Cluster and MariaDB, including MariaDB 10.0.x. ➢ Version 2.3 is compatible with : Oracle MySQL 5.1, 5.5, 5.6, and corresponding versions of Percona Server, Percona XtraDB Cluster and also MariaDB 10.1.x (except encryption). ➢ Version 2.4 is compatible with : Oracle MySQL, 5.1, 5.5, 5.6 and 5.7, also the corresponding versions of Percona Server, Percona XtraDB Cluster and also MariaDB 10.2.x ( Beyond the compression and TDE ) [P.S - Thank you for your corrections, Mr.Marko Mäkelä and Mr.Aleksandr Kuzminsky]
  12. 12. Table of Contents 1. Introduction 2. Features 3. Compatibility 4. Installation a. Installation Process b. Requirements and Limitations 5. Working of Xtrabackup 6. Backup Scenarios
  13. 13. Installation Process To install Percona XtraBackup, the available options are: ➢ Installing Percona XtraBackup from Repositories (recommended) [RHEL and CentOS] [Debian and Ubuntu] ➢ Installing Percona XtraBackup from Downloaded rpm or apt packages ➢ Compiling and Installing from Source Code [GitHub]
  14. 14. Requirements and Limitations • Local access to the MySQL datadir is required • READ, WRITE and EXECUTE permissions are needed at a filesystem level in the server’s datadir. • This makes Xtrabackup incompatible with RDS or some web hosting providers. • The Database user needs the RELOAD, LOCK TABLES, PROCESS, REPLICATION CLIENT privileges on the tables/databases to be backed up.
  15. 15. Table of Contents 1. Introduction 2. Features 3. Compatibility 4. Installation 5. Working of Xtrabackup a. Backup b. Preparation phase c. Restore 6. Backup Scenarios
  16. 16. How Percona XtraBackup Works ➢ Percona XtraBackup is based on InnoDB’s crash-recovery functionality. ➢ It copies your InnoDB data files, which results in data that is internally inconsistent. ➢ Then it performs crash recovery on the files to make them a consistent, usable database again.
  17. 17. How Percona XtraBackup Works - Backup InnoDB maintains a redo log (transaction log) that contains a record of every change to InnoDB data. When InnoDB starts, it inspects the data files and the transaction log, and performs two steps : • Applies committed transaction log entries to the data files • Performs an undo operation on any transactions that modified data but did not commit.
  18. 18. How Percona XtraBackup Works - Backup ➢ Percona XtraBackup will use Backup locks where available as a lightweight alternative to FLUSH TABLES WITH READ LOCK (FTWRL). This feature is available in Percona Server 5.6+. ➢ After XtraBackup is finished backing up all InnoDB/XtraDB data and logs, it will run the LOCK TABLES FOR BACKUP. Locking is done only for MyISAM and other non-InnoDB tables. ➢ After that XtraBackup will use LOCK BINLOG FOR BACKUP to block all operations that might change either binary log position or Exec_Master_Log_Pos or Exec_Gtid_Set. Xtrabackup will then finish copying the REDO log files and fetch the binary log coordinates. After this is completed xtrabackup will unlock the binary log and tables.
  19. 19. How Percona XtraBackup Works - Backup ➢ Finally, the binary log position will be printed and Xtrabackup will exit returning 0 if all went OK. Backup command: Final Output : The log sequence number (lsn) will be different, that depends on your server.
  20. 20. How Percona XtraBackup Works - Backup Files in the Target directory (Backup Directory) : ➢ Hierarchy of files and directories mirroring the original database structure. ➢ backup-my.cnf : Not a backup of the server configuration. It only contains the minimal InnoDB settings at the time the backup to execute the --apply-log phase ➢ xtrabackup_info: Contains all information about xtrabackup binary, server version, start and end time of backup, binlog position etc. ➢ xtrabackup_checkpoints: Metadata about the backup (type of backup, lsn, etc.) ➢ xtrabackup_logfile: Data needed for the –apply-log phase.
  21. 21. How Percona XtraBackup Works - Prepare ➢ Before the backup directory can be used, we must: 1. Make sure that the InnoDB tablespaces have consistent, non-corrupt data 2. Create new transaction logs ➢ Only Xtrabackup and the backup directory are needed. ➢ It can be done on different machine than the original server (we have the xtrabackup binary and the innodb configuration parameters) ➢ Generally, this is done just before restore.
  22. 22. How Percona XtraBackup Works - Restore
  23. 23. How Percona XtraBackup Works - Restore ➢ During restoration Xtrabackup will read the variables datadir, innodb_data_home_dir, innodb_data_file_path, innodb_log_group_home_dir from the my.cnf and check that the directories exist. ➢ Then it will copy the MyISAM tables, indexes, etc. (.frm, .MRG, .MYD, .MYI, .TRG, .TRN, .ARM, .ARZ, .CSV, .CSM, par and .opt files) first, InnoDB tables and indexes next and the log files at last. ➢ It will preserve file’s attributes when copying them, you may have to change the files’ ownership to mysql before starting the database server, as they will be owned by the user who created the backup. (#chown -R mysql:mysql /var/lib/mysql)
  24. 24. How Percona XtraBackup Works - Restore Restore Options: The Prepared backup data will be equivalent to the datadir with some additional xtrabackup files. It can restored by either of the following option. --copy-back : used to copy the data from the backup files to the server --move-back : used to move the data from the backup files to the server, this is used where there is a shortage of memory to maintain both backup files and the server data files.
  25. 25. 1. Introduction 2. Features 3. Compatibility 4. Installation 5. Working of Xtrabackup 6. Backup Scenarios a. Full Backup b. Partial Backup c. Compressed Backup d. Compact Backup e. Encrypted Backup f. Streaming Backup g. Incremental Backup Table of Contents
  26. 26. Backup Scenarios – Full Backup ➢ ➢ ➢ As of now, innobackupex program is deprecated Or #xtrabackup --backup --target-dir=/path/to/backup
  27. 27. Backup Scenarios – Full Backup Preparation Phase: xtrabackup --prepare --target-dir=/path/to/backup ➢ The prepare step uses this “embedded InnoDB” to perform crash recovery on the copied data files, using the copied log file. ➢ When this finishes, you will see an InnoDB shutdown a message containing the value of LSN depends on your system. ➢ It will not change the already prepared data files, you’ll see that output says:
  28. 28. Backup Scenarios – Full Backup Restoration: Restoration should be done only after preparing the backup, in order to maintain the consistency of the backup data. ➢ Restoration options in XtraBackup: # xtrabackup --copy-back --target-dir=/path/to/backup # xtrabackup --move-back --target-dir=/path/to/backup ➢ Alternative Methods : You can use rsync or cp to restore the files to the datadir. Before Restoring, • Ensure that MySQL server is shut down and the datadir is empty before restoring the backup. You can’t restore to a datadir of a running mysqld instance • You will need to change the files’ ownership to mysql before starting the database server, as they will be owned by the user who created the backup.
  29. 29. Backup Scenarios – Partial Backup ➢ Percona XtraBackup features partial backups, which means that you may backup only some specific tables or databases. ➢ The tables you back up must be in separate tablespaces, as a result of being created or altered after you enabled the innodb_file_per_table option on the server. Backup Options: --include , --tables-file, --databases Preparing Partial Backups: # xtrabackup --prepare --apply-log --export --target-dir=/path/to/backup
  30. 30. Backup Scenarios – Partial Backup Restore: ➢ Restoring should be done by restoring individual tables in the partial backup to the server. ➢ As we used export option while preparing the backup, .exp files will be created for each table. ➢ To import the table using these files, we have to create new tables with the same structure and then discard its tablespace ➢ After this, copy tablename.ibd and tablename.exp ( or tablename.cfg if importing to MySQL 5.6) files to database’s home, and import its tablespace.
  31. 31. Backup Scenarios – Compressed Backup ➢ Percona XtraBackup has implemented support for compressed backups. It can be used to compress/decompress local or streaming backup with xbstream. ➢ To make a compressed backup xtrabackup --compress option is used. ➢ To make the compression process faster, you can use parallel compression by enabling xtrabackup --compress-threads option. ➢ Compressed Backups should be decompressed before preparing it. Percona XtraBackup has implemented xtrabackup --decompress option that can be used to decompress the backup.
  32. 32. Backup Scenarios – Compressed Backup Prepare: ➢ To use xtrabackup --decompress you’ll need qpress file archiver ➢ Xtrabackup will not remove the compressed files automatically, xtrabackup --remove-original option should be used to remove the compressed files (.qp files). ➢ After decompress it can be prepared just the same as a full backup. Restore: Restoration can be done the same way for the full backup. Even if the compressed files is not removed it won’t be copied to the datadir when restoring using Xtrabackup options.
  33. 33. Backup Scenarios – Compact Backup When doing the backup of InnoDB tables it’s possible to omit the secondary index pages to make the backups more compact, and it will take less space on disk. The main disadvantage is that the backup prepare process takes longer as those secondary indexes need to be recreated. Backup: #xtrabackup --backup --compact --target-dir=/path/to/backup Prepare: #xtrabackup --prepare --apply-log --rebuild-indexes --target-dir=/path/to/backup Restore: #xtrabackup --copy-back --target-dir=/path/to/directory
  34. 34. Backup Scenarios – Encrypted Backup Percona XtraBackup can be used to encrypt/decrypt local or streaming backup with xbstream option (streaming tar backups are not supported) in order to add another layer of protection to the backups. Encryption is done with the libgcrypt library. To make an encrypted backup following options need to be specified : ➢ ➢ ➢
  35. 35. Backup Scenarios – Encrypted Backup The options xtrabackup --encrypt-key and xtrabackup --encrypt-key-file are mutually exclusive, i.e., just one of them needs to be provided. Encryption key can be generated with this command : This value then can be used as the encryption key Encryption and Backup:
  36. 36. Backup Scenarios – Encrypted Backup Optimizing Encryption: The following options are used to speed up the encryption process, ➢ xtrabackup --encrypt-threads: to specify multiple threads for parallel encryption ➢ xtrabackup --encrypt-chunk-size: used to specify the size (in bytes) of the working encryption buffer for each encryption thread (default is 64K). Decryption:
  37. 37. Backup Scenarios – Encrypted Backup ➢ Percona XtraBackup doesn’t automatically remove the encrypted files. ➢ Users have to remove the *.xbcrypt file from backup directory. ➢ In Percona XtraBackup 2.4.6 xtrabackup --remove-original option has been implemented that you can use to remove the encrypted files once they’ve been decrypted. ➢ xtrabackup --parallel can be used with xtrabackup --decrypt option to decrypt multiple files simultaneously. ➢ After decryption, the backup can be prepared for restoration. ➢ Preparation and Restoration are same as that as Full Backup.
  38. 38. Backup Scenarios – Streaming Backup , socat socat is recommended because it allows for socket options like transfer buffer sizes
  39. 39. Backup Scenarios – Streaming Backup To extract the resulting tar file, you must use the -i option, such as tar -ixvf backup.tar
  40. 40. Backup Scenarios – Streaming Backup
  41. 41. Backup Scenarios – Incremental Backup ➢ Percona Xtrabackup supports incremental backups, which means that they can copy only the data that has changed since the last backup. ➢ You can perform many incremental backups between each full backup. ➢ Incremental backups work because each InnoDB page contains a log sequence number, or LSN. ➢ Each page’s LSN shows how recently it was changed. ➢ There are two algorithms used to find the changed LSN pages. • The first one is to check the page LSN directly by reading all the data pages.(All Servers) • The second one is to enable the changed page tracking feature on the server, which will note the pages as they are being changed. (Percona) (This information is written out in a separate compact file, the bitmap file)
  42. 42. Backup Scenarios – Incremental Backup To make an incremental backup, begin with a full backup as usual. The xtrabackup binary writes a file called xtrabackup_checkpoints that contains a line showing the to_lsn into the backup’s target directory. To start with a full backup, Now, you can take incremental backup based on the full backup, You can take as many incremental backup as you can, with the previously taken backup (full/incremental) as its base backup directory.
  43. 43. Backup Scenarios – Incremental Backup Prepare: ➢ The xtrabackup --prepare step for incremental backups is not the same as for full backups. ➢ The rollback of uncommitted transactions must be skipped. ➢ xtrabackup --apply-log-only option to prevent the rollback phase. ➢ If you do not use the xtrabackup --apply-log-only option to prevent the rollback phase, then your incremental backups will be useless. ➢ ➢ ➢ After preparing the data, it can be restored as same as the full backup restore.
  44. 44. Resources Percona XtraBackup 2.4 - Online Documentation Highly Efficient Backups with Percona Xtrabackup PLMCE-2015 (MySQL 101) Xtrabackup in a Nutshell Fast Incremental Backups with Percona Server and Percona XtraBackup
  45. 45. Contact US: Write to me at harishkumar@mydbops.com DB Consulting Contact : info@mydbops.com Phone : 08048505683 www.mydbops.com info@mydbops.com

×