MYSQLDUMP & ZRM COMMUNITY (EN)

Meetup Viadeo & LeMUG / Paris, November 16, 2011 MYSQLDUMP & ZRM COMMUNITY Scheduling and Monitoring Cédric PEINTRE www.mysqlplus.net

2 Old but still valid poll mysqldump is not dead !• Logical backup• Flexibility (All or only some databases, filters, no data…)• Portable SQL statements• All storage engine supported• Ready for point in time recovery (binary logs information inside !)• …>> http://dev.mysql.com/doc/refman/5.5/en/mysqldump.html

35 reasons to use ZRM community• Open source perl scripts• Schedule or start immediate backup• Monitor and browse your backups• Compression and encryption on the go• Email notification• Retention policies• Dump file accessible with or without backup tool• Yes, I said 5 reasons…>> http://www.zmanda.com/backup-mysql.html>> http://wiki.zmanda.com/index.php/MySQL_Backup_and_Recovery

4Prerequisites and installation• Last release : 2.2.0 • >> http://www.zmanda.com/download-zrm.php• MySQL user : • GRANT LOCK TABLES, SELECT, FILE, RELOAD, SUPER ON *.* TO dba_backup@localhost IDENTIFIED BY <PASSWORD>;• Perl libraries : • perl-DBI, MySQL-perl-DBD, perl-XML-Parser• Installation via RPM, DEB or tarball• Check the ZRM version : • /etc/mysql-zrm/mysql-zrm-release

5 Global configuration file backup set conceptOne DB All DB No Data … [ mysqldump –uroot –ppass --single-transaction … ] dump files

6Configuration tree• /etc/mysql-zrm • /alldatabases • mysql-zrm.conf • /nodata • mysql-zrm.conf backup set • /mysqldatabase • mysql-zrm.conf • /onedatabase • mysql-zrm.conf • mysql-zrm.conf • mysql-zrm-release Global configuration • mysql-zrm-reporter.conf • RSS.header

7 Main parametersGlobal configuration• Dump parameters : • Destination • Retention • Compression / Encryption• Database access parameters : • User / Pass • Paths• ZRM Parameters • Notifications • Reports

8Main parametersBackup set• Don’t forget the comment for your backup set• Backup type : • Level : Full or incremental (binlog) • Mode : Logical (mysqldump) or physical (snapshot)• Set the databases to backup• Exclude one or more databases• Add mysqldump options• Some parameters of the global configuration file (optional)

9Compression & Encryption• Manage with basis system tools (gzip, GPG…) • An example plugin is provided for encryption (encrypt.pl)• Compression parameters : • compress : Activate / Desactivate • compress-plugin : Compression tool (default : gzip) • compress-mysqldump-onthefly : Compression on the fly• Encryption parameters : • Use a SSL connexion ! • encrypt : Activate / Desactivate • encrypt-plugin : Encryption plugin (ex : /usr/share/mysql- zrm/plugins/encrypt.pl) • decrypt-option : Command used to decrypt • passfile : Passphrase used to encrypt (/etc/mysql-zrm/.passphrase)

10Scheduling backup• mysql-zrm --action check --backup-set AllDB • Be sure to specify an existing backup set !• Using ZRM : • mysql-zrm-scheduler --now --backup-set alldb • mysql-zrm-scheduler --add --interval daily –start 23:30 --backup-set alldb • mysql-zrm-scheduler --query • mysql-zrm-scheduler --delete --interval daily –start 23:30• Using external tools (cron, shell, perl, php…) : • mysql-zrm-scheduler --now --backup-set alldb • mysql-zrm --action backup --backup-set alldb --databases test• How to purge ? : • mysql-zrm --action purge --destination <rep des dump>• The command line parameters are stronger than others !• ZRM automatically adds a purge at 04am

11Backup files• /<bckp_dir>/<bckp_set>/<Date> • Ex : /database/backup/zrm/alldatabases/20111112231001 • backup-data : Binary file, don’t care about that • backup-sql : Backup file (eventually compressed) • index : All about the backup ! • zrm_checksum : Checksum of the dump file (md5sum)• These files are used by ZRM tools (Dump, restore, report…)• You can use it to find an old backup or other stuff

12Index file example• backup-set=alldatabases• backup-date=20111112231001• mysql-server-os=Linux/Unix• backup-type=regular• host=localhost• backup-date-epoch=1321135801• retention-policy=1D• mysql-zrm-version=ZRM for MySQL Community Edition - version 2.2.0• mysql-version=5.1.59• backup-directory=/database/backup/zrm/alldatabases/20111112231001• comment=Backup FULL / All databases except mysql• backup-level=0• logical-databases=DB1 DB2 DB3 DB4• next-binlog=mysql-bin.000531• last-backup=/database/backup/zrm/alldatabases/20111111231001• backup-size=70.10 GB• compress=/bin/gzip• backup-size-compressed=70.10 GB• read-locks-time=05:40:36• flush-logs-time=00:00:00• compress-encrypt-time=01:09:27:16• backup-time=05:40:37• backup-status=Backup succeeded

13Monitoring and log files• mysql-zrm-reporter --where backup-set=AllDB --show backup-status-info• mysql-zrm-reporter --fields backup-directory,backup- date,backup-level,backup-status,backup-size,backup- size-compressed,comment

14 Monitoring and log filesmysql-zrm-reporter --fields backup-directory,backup-date,backup-status,backup-size,comment

15Monitoring and log files• The ZRM log file provides some usefull informations about the backup progress and status• The full mysqldump command is also available in this file • /var/log/mysql-zrm • mysql-zrm-scheduler [options] >/logs/zrm_dump_<date>.log 2>&1• Use a log for mysqldump to get real status of your ZRM backup : • extra-mysqldump-options="--log- error=/tmp/err_mysqldump_`date +"%Y%m%d_%H%M%S"`.log"• ZRM backup status does not guarantee the mysqldump status !

16Global backup monitoring• The problem is to obtain information about backups that are no longer available on the database server (if the dump is copied to a tape for example)• A global backup monitoring is needed in order to quickly and easily find an old backup• What do you need for the global backup monitoring ? : • Creating a new table in another database (on your monitoring server) • Scripting a launcher for ZRM and insert all needed information in this table : • Server name • Start/End date • Duration • Backup set name • Database name • Backup status • ZRM log file path • mysqldump log file path • Backup directory • … • Creating a global backup alert system (by email, html…)

17Restore files• Using ZRM : • To extract a compressed dump : • mysql-zrm --action extract-backup --source-directory /var/lib/mysql- zrm/20061012232713 • To load the entire file : • mysql-zrm --action restore --backup-set backupSet1 --source-directory /var/lib/mysql-zrm/backupSet1/20060829140710• Using system tools : • cat /var/lib/mysql-zrm/20061012232713/backup-sql | gunzip > backup.sql • mysql –uroot –pxxxx –f < backup.sql

18Want more…• Backup remote databases• Before/after (backup) plugins• « Smart » backup of the binary logs• Raw mode backup (snapshots)• Xtrabackup integration•…

20select * from me;http://www.mysqlplus.netcedric@mysqlplus.fr@cpeintre

http://www.mysqlplus.net	2063
http://planetmariadb.org	110
http://feeds.feedburner.com	40
http://translate.googleusercontent.com	3
http://webcache.googleusercontent.com	2

MYSQLDUMP & ZRM COMMUNITY (EN)

by Cédric PEINTRE on Nov 22, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

5 Embeds 2,218

Statistics

MYSQLDUMP & ZRM COMMUNITY (EN) — Presentation Transcript