Zararfa SummerCamp 2012 - Performing fast backups in large scale environments using open source backup tools
MySQL Backup solutions Liz van Dijk - @lizztheblizz Zarafa Summer Camp - June 2012
PerconaMySQL/LAMP Consulting ● MySQL Support ● (co-)Developers of ○ Percona Server (XtraDB) ○ Percona XtraBackup ○ Percona Toolkit (Maatkit, Aspersa) ○ Percona XtraDB Clusterhttp://www.percona.comhttp://www.mysqlperformanceblog.com
Topics● Why MySQL Backup is hard to get right● Whats available on the market?● Perconas Solution: XtraBackup● Demo
MySQL Backup Challenges● MySQL is multi-engine ○ Different storage engines call for different strategies ■ Are we using filecopy or dumps? ■ MyISAM tables handle filecopy quite well ■ InnoDB files cannot simply be copied elsewhere● MySQL can be quite busy ○ Do we need to shut it down? ○ If not, how much load is generated? ○ How quickly can we restore?● MySQL is more than the sum of its datafiles ○ How about binary logging, replication? ○ Does the tool impact MySQL in other ways?
Available backup solutions (InnoDB) Backup Restore InnoDB Impact Warmth Time TimeCold Backup very high cold very fast fastmysqldump medium hot medium slowsnapshotting high/medium hot/warm fast fast MySQL EB low hot fast fast(Enterprise Backup)XtraBackup low hot fast fast
Available backup solutions (mixed) InnoDB/ Backup Restore Impact Warmth MyISAM Time TimeCold Backup very high cold very fast fastmysqldump high warm medium slowsnapshotting high/medium hot/warm fast fast MySQL EB low/medium warm fast fast(Enterprise Backup)XtraBackup low/medium warm fast fast
XtraBackup Features ● Shared with MySQL Enterprise Backup ○ Non-blocking ○ Support for MyISAM ○ Compression ○ Partial Backups ○ Throttling ○ Incremental Backups ● Unique to XtraBackup ○ Exporting/Importing individual tables ○ Streaming (including incremental streaming since 2.0!) ○ Parallel copying (with use of innodb_file_per_table)http://www.percona.com/doc/percona-xtrabackup/intro.html
XtraBackup/innobackupex OperationPerforming and restoring a mixed full backup:Backing up (1): 1. Copy datafiles while recording transaction log changes 2. FLUSH TABLES WITH READ LOCK; 3. Get binlog position 4. Copy all .MYD, .MYI, .TRG, .TRN, ... files 5. Stop recording transaction log changes 6. UNLOCK TABLES;Preparing/Restoring: 7. Prepare backup by applying recorded tlog changes (2) 8. Restoring files to original location (3)
Getting down to business...innobackupex vs. xtrabackup? xtrabackup - Compiled C application, focuses on InnoDBonly innobackupex - Perl script, automates surrounding tasks3 separate "manual" steps are required: (1) - Backing up (2) - Preparing InnoDB files for recovery (3) - Restoring the filesAll of these are easily scriptable!
Some recipesSimple full backup and restore:(1) # innobackupex /path/to/your/backups(2) # innobackupex --apply-log /path/to/your/backups/2012-06-21_13-45-00/(3) # innobackupex --copy-back /path/to/your/backups/2012-06-21_13-45-00/ # chown -R mysql:mysql /var/lib/mysql
Options ● Specified as --variable-name=value ● innobackupex/xtrabackup will read options from your config ○ [client] or [mysql] for credentials ○ [mysqld] for location of datadir, InnoDB startup options ○ Lastly, [xtrabackup] to override any of the previous and specify xtrabackup-specific options** percona.com/doc/percona-xtrabackup/xtrabackup_bin/xbk_option_reference.html* percona.com/doc/percona-xtrabackup/innobackupex/innobackupex_option_reference.html
Some recipesIncremental backup: ● Repeat (1) to create a full backup at location $fullbackup ● # innobackupex --incremental $incbackup --incremental- basedir=$fullbackup ● # innobackupex --apply-log --redo-only $fullbackup ■ Preparing the base backup FIRST ● # innobackupex --apply-log --redo-only $fullbackup --incremental- basedir=$incbackup ■ Applying each incremental backup to the base ● # innobackupex --apply-log $fullbackup ■ Finish preparing the now completed base backup
Some recipesStreaming:# innobackupex --stream=tar ./ | ssh user@desthost "cat - > /data/backups/backup.tar"Note: Extract with tar -i!Filtering tablespaces: --tables, --databases, --tables-fileThrottling: ● Limit to 100 read/writes IOs: --throttle=100 ● In streaming backup, use pv to limit to 10MB/s ○ # innobackupex --stream=tar ./ | pv -q -L10m | ssh user@desthost "cat - > /data/backups/backup.tar"Parallel copying of tablespaces: --parallel 2 #threads
A note about versions ● xtrabackup 1.6 ● xtrabackup 2.0 (GA since 4 April 2012) ○ Backs up galera cluster information ○ Supports parallel compression ○ Streaming incremental backups ○ Backs up LRU dumpfileNote: When using the xtrabackup binary directly, make sure touse the correct binary for your MySQL version!