Your SlideShare is downloading. ×
0
MySQL
Backup & Recovery
Essentials
Ronald Bradford
http://ronaldbradford.com
Buenos Aires, Argentina
2013-10
EffectiveMySQ...
"No one cares if you can
backup, only that you
can restore."
Adapted from W. Curtis Preston - Backup & Recovery (O'Reilly ...
Agenda
Backup Options
Tools available
Restore Options
Necessary Nomenclature
Business Requirements
Slides at http://j.mp/E...
ABOUT AUTHOR
Ronald Bradford
All time top MySQL blogger
Published Author (4++ books)
Oracle ACE Director
MySQL community m...
Four Step
Overview

EffectiveMySQL.com - Performance, Scalability & Business Continuity
Backup Basics
1

Static Consistent Backup
+
Master Binary Logs

EffectiveMySQL.com - Performance, Scalability & Business C...
Recovery Basics
2

Static Restore
+
Point in Time Recovery

EffectiveMySQL.com - Performance, Scalability & Business Conti...
Verification
Necessary at EVERY step

3

Commands complete without error
No errors in logs
Results match expectations
Appr...
Testing
4

“Testing is about trying to
break your software, not
checking that it works!”
Ronald Bradford, circa 2006

Effe...
"MySQL has no single
unbreakable backup
solution [yet]."

EffectiveMySQL.com - Performance, Scalability & Business Continu...
B&R Options

EffectiveMySQL.com - Performance, Scalability & Business Continuity
B&R Options
Included

Open Source

mysqldump

XtraBackup

OS filecopy

mydumper

OS Specific

Commercial

filesystem snapshot...
EXAMPLE DB
SELECT

FROM

SUM(data_length+index_length)/1024/1024 AS total_mb,
SUM(data_length)/1024/1024 AS data_mb,
SUM(i...
Backup Options
mysqldump

Pros
Included with MySQL server
ASCII output
SQL statement
Remote capabilities

EffectiveMySQL.c...
Backup Options
mysqldump

Cons
Single threaded
Locking by default (*)
Slow restore for large DBs
Single threaded

Effectiv...
Backup Options
mysqldump

Recommendations
Great for 5-10GB
No locking with InnoDB (*)
Essential for recording schema
objec...
Backup Options
mysqldump USAGE
$ time mysqldump --all-databases > /mysql/backup/dump1.sql
real
1m31.631s
user
1m12.533s
sy...
Restore Options
mysqldump USAGE
$ time mysql –u[user] -p -f < dump1.sql > dump1.out 2>&1
real
14m13.817s
user
1m6.960s
sys...
Backup Options
mysqldump USAGE
$ mysqldump --all-databases --no-data 
> /mysql/backup/schema.sql
$ mysqldump --all-databas...
Backup Options
VERIFICATION
$ mysqldump --all-databases --no-data --no-create-info 
--events --routines > /mysql/backup/ob...
Backup Options
File Copy

Pros
No software needed
Consistent

EffectiveMySQL.com - Performance, Scalability & Business Con...
Backup Options
File Copy

Cons
MySQL unavailable
Not consistent (**)
Time to warm server caches
Restore must match configur...
Backup Options
File Copy

Recommendations
Great for slaves
When access disabled

EffectiveMySQL.com - Performance, Scalabi...
Backup Options
File Copy USAGE
$ sudo service mysql stop
$ sudo /etc/init.d/mysqld stop
$
$
$
$
#

# Ubuntu
# RHEL

mkdir ...
Backup Options
SNAPSHOT

Pros
Fastest
Database agnostic (*)

EffectiveMySQL.com - Performance, Scalability & Business Cont...
Backup Options
SNAPSHOT

Cons
Must be pre-configured (LVM)
Extra Disk I/O
Inconsistent (*)
FLUSH TABLES duration

Effective...
Backup Options
SNAPSHOT USAGE
$
$
$
$
$
$

sudo su sync ; lvcreate -L1G -s -n dbsnapshot /dev/db/p0
mkdir -p /mnt/dbsnapsh...
Backup Options
XtraBACKUP

Pros
Non-blocking (InnoDB)
Open Source
Supports incremental, compression etc

EffectiveMySQL.co...
Backup Options
XtraBACKUP

Cons
Blocking for non-InnoDB tables

EffectiveMySQL.com - Performance, Scalability & Business C...
Backup Options
XtraBACKUP USAGE
$ time innobackupex --defaults-file=/etc/mysql/my.cnf 
--user=root --password=passwd --no-...
restore Options
XtraBACKUP USAGE
$
$
$
$
$

sudo
sudo
sudo
sudo
sudo

su – mysql
service mysql stop
# Ubuntu
rm -rf /var/l...
Backup Options
MySQL ENTERPRISE BACKUP (MEB)

Pros
Non Blocking (InnoDB)
Commercial Support
Supports incremental, compress...
Backup Options
MySQL ENTERPRISE BACKUP (MEB)

Cons
Warm for non-InnoDB
Cost

EffectiveMySQL.com - Performance, Scalability...
Backup Options
MEB USAGE
$ sudo su - mysql
$ time /opt/meb/bin/mysqlbackup --user=root 
--password=passwd 
--backup-dir=/m...
RESTORE Options
MEB USAGE
$
$
$
$
$

sudo
sudo
sudo
sudo
sudo

su – mysql
service mysql stop
# Ubuntu
rm -rf /var/lib/mysq...
Point in Time

EffectiveMySQL.com - Performance, Scalability & Business Continuity
Binary Logs
Possible with Binary Logs
#my.cnf
[mysqld]
log-bin=mysql-bin
expire-logs-days=5

WARNING: If you care
about yo...
Backup Options
cp
rsync
Slave --log-slave-updates
DRBD - Disk Replicated Block Device
mysqlbinlog --read-from-remote-serve...
Backup needs
Static Backup
Useless without Binary Log position
mysql> SHOW MASTER STATUSG
File: mysql-bin.020616
Position:...
Backup Needs
$ mysqldump --master-data
(or --dump-slave)
CHANGE MASTER TO MASTER_HOST='10.0.0.1', MASTER_USER
='repl',MAST...
Restore Options
mysqlbinlog
$ mysqlbinlog /path/to/mysql-bin.000146 
--start-position=810715371 | mysql -uroot -p
$ mysqlb...
Advanced
Features

EffectiveMySQL.com - Performance, Scalability & Business Continuity
Advanced
Compression
Incremental
Remote
Parallel
Partial
For another presentation

EffectiveMySQL.com - Performance, Scala...
Compression
Utility

Comp (s)

Dec (s)

Saving

lzo (-3)
pigz (-1)
pigz [-6]
gzip [-6]
bzip2
lzo (-9)
lzma
xz

21
43
105
2...
Terminology
MTTD - Mean Time To Detect
MTTR - Mean Time to Recover
RPO - Recovery Point Objective
RDO - Recovery Data Obje...
Conclusion

EffectiveMySQL.com - Performance, Scalability & Business Continuity
CONCLUsiON
This is an introduction
Advance features are important
Best option depends
Replication is important in your str...
PRESENTATIONS

More presentations at

http://ronaldbradford.com/mysql-presentations/
http://effectivemysql.com/presentatio...
220 pages dedicated to B&R

http://j.mp/EM-book2
Ronald Bradford
ronald@effectivemysql.com
Upcoming SlideShare
Loading in...5
×

MySQL Backup and Recovery Essentials

3,840

Published on

"Disaster is inevitable" and "To move forward you must first backup" should be known to all software developers. This presentation will discuss all the options for your valuable data assets in MySQL, and highlight how to maintain site reliability of your data

Published in: Technology, Business
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,840
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
78
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Transcript of "MySQL Backup and Recovery Essentials"

  1. 1. MySQL Backup & Recovery Essentials Ronald Bradford http://ronaldbradford.com Buenos Aires, Argentina 2013-10 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  2. 2. "No one cares if you can backup, only that you can restore." Adapted from W. Curtis Preston - Backup & Recovery (O'Reilly 2009) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  3. 3. Agenda Backup Options Tools available Restore Options Necessary Nomenclature Business Requirements Slides at http://j.mp/EM-BandR EffectiveMySQL.com - Performance, Scalability & Business Continuity
  4. 4. ABOUT AUTHOR Ronald Bradford All time top MySQL blogger Published Author (4++ books) Oracle ACE Director MySQL community member of the year (2009 & 2013) 24 years of RDBMS experience,14 years with MySQL MySQL Inc (2006-2008) Oracle Corporation (1996-1999) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  5. 5. Four Step Overview EffectiveMySQL.com - Performance, Scalability & Business Continuity
  6. 6. Backup Basics 1 Static Consistent Backup + Master Binary Logs EffectiveMySQL.com - Performance, Scalability & Business Continuity
  7. 7. Recovery Basics 2 Static Restore + Point in Time Recovery EffectiveMySQL.com - Performance, Scalability & Business Continuity
  8. 8. Verification Necessary at EVERY step 3 Commands complete without error No errors in logs Results match expectations Approximate Precise EffectiveMySQL.com - Performance, Scalability & Business Continuity
  9. 9. Testing 4 “Testing is about trying to break your software, not checking that it works!” Ronald Bradford, circa 2006 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  10. 10. "MySQL has no single unbreakable backup solution [yet]." EffectiveMySQL.com - Performance, Scalability & Business Continuity
  11. 11. B&R Options EffectiveMySQL.com - Performance, Scalability & Business Continuity
  12. 12. B&R Options Included Open Source mysqldump XtraBackup OS filecopy mydumper OS Specific Commercial filesystem snapshot MEB EffectiveMySQL.com - Performance, Scalability & Business Continuity
  13. 13. EXAMPLE DB SELECT FROM SUM(data_length+index_length)/1024/1024 AS total_mb, SUM(data_length)/1024/1024 AS data_mb, SUM(index_length)/1024/1024 AS index_mb, COUNT(DISTINCT table_schema) AS schema_cnt, COUNT(*) AS tables, CURDATE() AS today, VERSION() information_schema.tablesG *************************** 1. row *************************** total_mb: 5344.63 data_mb: 4545.49 index_mb: 799.13 TIP: Your daily verification schema_cnt: 7 step should include this tables: 103 today: 2012-04-03 VERSION(): 5.1.61-0ubuntu0.11.10.1-log Available on GitHub with Effective MySQL: Backup and Recovery Book EffectiveMySQL.com - Performance, Scalability & Business Continuity
  14. 14. Backup Options mysqldump Pros Included with MySQL server ASCII output SQL statement Remote capabilities EffectiveMySQL.com - Performance, Scalability & Business Continuity
  15. 15. Backup Options mysqldump Cons Single threaded Locking by default (*) Slow restore for large DBs Single threaded EffectiveMySQL.com - Performance, Scalability & Business Continuity
  16. 16. Backup Options mysqldump Recommendations Great for 5-10GB No locking with InnoDB (*) Essential for recording schema objects EffectiveMySQL.com - Performance, Scalability & Business Continuity
  17. 17. Backup Options mysqldump USAGE $ time mysqldump --all-databases > /mysql/backup/dump1.sql real 1m31.631s user 1m12.533s sys 0m10.893s $ echo $? 0 $ ls -lh /mysql/backup/dump1.sql -rw-rw-r-- 1 uid gid 2.9G 2012-04-03 03:04 /mysql/ backup/dump1.sql TIP: Always time and record long running processes for verification EffectiveMySQL.com - Performance, Scalability & Business Continuity
  18. 18. Restore Options mysqldump USAGE $ time mysql –u[user] -p -f < dump1.sql > dump1.out 2>&1 real 14m13.817s user 1m6.960s sys 0m1.516s $ echo $? 0 $ ls -l dump1.out -rw-rw-r-- 1 uid gid 0 2012-04-08 04:07 dump1.out EffectiveMySQL.com - Performance, Scalability & Business Continuity
  19. 19. Backup Options mysqldump USAGE $ mysqldump --all-databases --no-data > /mysql/backup/schema.sql $ mysqldump --all-databases --no-data --no-create-info --events --routines > /mysql/backup/objects.sql TIP: Include daily dumps of database objects EffectiveMySQL.com - Performance, Scalability & Business Continuity
  20. 20. Backup Options VERIFICATION $ mysqldump --all-databases --no-data --no-create-info --events --routines > /mysql/backup/objects.sql mysqldump: Got error: 1142: SELECT,LOCK TABL command denied to user 'root'@'localhost' for table 'cond_instances' when using LOCK TABLES $ echo $? 2 TIP: Error checking is essential and easy to implement EffectiveMySQL.com - Performance, Scalability & Business Continuity
  21. 21. Backup Options File Copy Pros No software needed Consistent EffectiveMySQL.com - Performance, Scalability & Business Continuity
  22. 22. Backup Options File Copy Cons MySQL unavailable Not consistent (**) Time to warm server caches Restore must match configuration Must backup right files EffectiveMySQL.com - Performance, Scalability & Business Continuity
  23. 23. Backup Options File Copy Recommendations Great for slaves When access disabled EffectiveMySQL.com - Performance, Scalability & Business Continuity
  24. 24. Backup Options File Copy USAGE $ sudo service mysql stop $ sudo /etc/init.d/mysqld stop $ $ $ $ # # Ubuntu # RHEL mkdir /mysql/backup/cp1 cp -r /var/lib/mysql /mysql/backup/cp1 echo $? cp /etc/mysql/my.cnf /mysql/backup/cp1 Other directories? $ sudo service mysql start EffectiveMySQL.com - Performance, Scalability & Business Continuity
  25. 25. Backup Options SNAPSHOT Pros Fastest Database agnostic (*) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  26. 26. Backup Options SNAPSHOT Cons Must be pre-configured (LVM) Extra Disk I/O Inconsistent (*) FLUSH TABLES duration EffectiveMySQL.com - Performance, Scalability & Business Continuity
  27. 27. Backup Options SNAPSHOT USAGE $ $ $ $ $ $ sudo su sync ; lvcreate -L1G -s -n dbsnapshot /dev/db/p0 mkdir -p /mnt/dbsnapshot mount -o ro /dev/db/dbsnapshot /mnt/dbsnapshot du -sh /mnt/dbsnapshot ls -al /mnt/dbsnapshot $ mkdir /mysql/backup/snapshot1 $ cp -r /mnt/dbsnapshot/* /mysql/backup/snapshot1 $ sudo su $ mylvmbackup TIP: mylvmbackup does all the hard work http://effectiveMySQL.com/article/configuring-a-new-hard-drive-for-lvm http://effectiveMySQL.com/article/using-mysql-with-lvm http://www.lenzg.net/mylvmbackup/ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  28. 28. Backup Options XtraBACKUP Pros Non-blocking (InnoDB) Open Source Supports incremental, compression etc EffectiveMySQL.com - Performance, Scalability & Business Continuity
  29. 29. Backup Options XtraBACKUP Cons Blocking for non-InnoDB tables EffectiveMySQL.com - Performance, Scalability & Business Continuity
  30. 30. Backup Options XtraBACKUP USAGE $ time innobackupex --defaults-file=/etc/mysql/my.cnf --user=root --password=passwd --no-timestamp /mysql/backup/xtrabackup/first real 2m30.667s user 0m21.933s sys 0m14.713s $ echo $? 0 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  31. 31. restore Options XtraBACKUP USAGE $ $ $ $ $ sudo sudo sudo sudo sudo su – mysql service mysql stop # Ubuntu rm -rf /var/lib/mysql # data directory mkdir -m /var/lib/mysql chown mysql:mysql /var/lib/mysql $ time innobackupex --copy-back /mysql/backup/xtrabackup/first/ $ echo $? EffectiveMySQL.com - Performance, Scalability & Business Continuity
  32. 32. Backup Options MySQL ENTERPRISE BACKUP (MEB) Pros Non Blocking (InnoDB) Commercial Support Supports incremental, compression etc media management software (MMS) Oracle Secure Backup (OSB) - SBT EffectiveMySQL.com - Performance, Scalability & Business Continuity
  33. 33. Backup Options MySQL ENTERPRISE BACKUP (MEB) Cons Warm for non-InnoDB Cost EffectiveMySQL.com - Performance, Scalability & Business Continuity
  34. 34. Backup Options MEB USAGE $ sudo su - mysql $ time /opt/meb/bin/mysqlbackup --user=root --password=passwd --backup-dir=/mysql/backup/meb/first backup-and-apply-log real 3m30.879s user 0m17.081s sys 0m14.565s $ echo $? 0 $ du -sh /mysql/backup/meb/first 5.6G /mysql/backup/meb/first $ ls -lh /mysql/backup/meb/first/datadir/ibd* -rw-rw-r-- 1 uid gid 5.4G 2012-04-03 03:25 /mysql/ backup/meb/first/datadir/ibdata1 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  35. 35. RESTORE Options MEB USAGE $ $ $ $ $ sudo sudo sudo sudo sudo su – mysql service mysql stop # Ubuntu rm -rf /var/lib/mysql # data directory mkdir -m /var/lib/mysql chown mysql:mysql /var/lib/mysql $ time /opt/meb/bin/mysqlbackup --defaults-file=/etc/mysql/my.cnf --backup-dir=/mysql/backup/meb/first --innodb-log-files-in-group=2 copy-back $ echo $? EffectiveMySQL.com - Performance, Scalability & Business Continuity
  36. 36. Point in Time EffectiveMySQL.com - Performance, Scalability & Business Continuity
  37. 37. Binary Logs Possible with Binary Logs #my.cnf [mysqld] log-bin=mysql-bin expire-logs-days=5 WARNING: If you care about your data, enable binary logging mysql> SHOW BINARY LOGS; +------------------+-----------+ | Log_name | File_size | +------------------+-----------+ | mysql-bin.019662 | 104857736 | | mysql-bin.019663 | 104857699 | | mysql-bin.019664 | 104857850 | $ ls -ltr /var/log/mysql | tail -rw-rw---- 1 mysql adm 104857736 2011-09-04 22:00 mysql-bin.019662 -rw-rw---- 1 mysql adm 104857699 2011-09-04 22:08 mysql-bin.019663 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  38. 38. Backup Options cp rsync Slave --log-slave-updates DRBD - Disk Replicated Block Device mysqlbinlog --read-from-remote-server (New in 5.6) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  39. 39. Backup needs Static Backup Useless without Binary Log position mysql> SHOW MASTER STATUSG File: mysql-bin.020616 Position: 63395562 Binlog_Do_DB: Binlog_Ignore_DB: WARNING: Can work on slave and provide the wrong information EffectiveMySQL.com - Performance, Scalability & Business Continuity
  40. 40. Backup Needs $ mysqldump --master-data (or --dump-slave) CHANGE MASTER TO MASTER_HOST='10.0.0.1', MASTER_USER ='repl',MASTER_PASSWORD='******', MASTER_LOG_FILE= 'mysql-bin.000146', MASTER_LOG_POS=810715371; Xtrabackup $ cat xtrabackup_binlog_info mysql-bin.000001 37522 MEB $ grep binlog meta/backup_variables.txt binlog_position=mysql-bin.000017:5555 mydumper $ cat export-20120407-230027/metadata Log: mysql-bin.000017 Pos: 8328 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  41. 41. Restore Options mysqlbinlog $ mysqlbinlog /path/to/mysql-bin.000146 --start-position=810715371 | mysql -uroot -p $ mysqlbinlog /path/to/mysql-bin.000147 /path/to/mysql-bin.00148 ... etc | mysql -uroot -p MySQL replication EffectiveMySQL.com - Performance, Scalability & Business Continuity
  42. 42. Advanced Features EffectiveMySQL.com - Performance, Scalability & Business Continuity
  43. 43. Advanced Compression Incremental Remote Parallel Partial For another presentation EffectiveMySQL.com - Performance, Scalability & Business Continuity
  44. 44. Compression Utility Comp (s) Dec (s) Saving lzo (-3) pigz (-1) pigz [-6] gzip [-6] bzip2 lzo (-9) lzma xz 21 43 105 232 540 20m 58m 59m 34 33 25 78 175 82 180 160 48% 64% 69% 69% 74% 58% 78% 78% Depends greatly on data types EffectiveMySQL.com - Performance, Scalability & Business Continuity
  45. 45. Terminology MTTD - Mean Time To Detect MTTR - Mean Time to Recover RPO - Recovery Point Objective RDO - Recovery Data Objective SLA - Service Level Agreement Determining business priorities is important for any strategy EffectiveMySQL.com - Performance, Scalability & Business Continuity
  46. 46. Conclusion EffectiveMySQL.com - Performance, Scalability & Business Continuity
  47. 47. CONCLUsiON This is an introduction Advance features are important Best option depends Replication is important in your strategy Test, Test, Test. - “Chaos Monkey” Slides at http://j.mp/EM-BandR EffectiveMySQL.com - Performance, Scalability & Business Continuity
  48. 48. PRESENTATIONS More presentations at http://ronaldbradford.com/mysql-presentations/ http://effectivemysql.com/presentation/ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  49. 49. 220 pages dedicated to B&R http://j.mp/EM-book2
  50. 50. Ronald Bradford ronald@effectivemysql.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×