Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Webscale backups for MySQL 
Alexey.Kopytov@percona.com 
October 29, 2014
#BEAMERTHEME: Madrid
Êòî ÿ? 
Àëåêñåé Êîïûòîâ <alexey.kopytov@percona.com> 
I Percona Server 
I Percona XtraBackup (ðóêîâîäèòåëü ïðîåêòà) 
I Per...
Òèïû ðåçåðâíûõ êîïèé 
Ëîãè÷åñêèå 
I äàííûå â âèäå SQL, ïîëó÷åííûå 
îò ñåðâåðà 
I òðåáóþò èìïîðò äëÿ 
âîññòàíîâëåíèÿ (ìåäëå...
Óòèëèòû äëÿ áýêàïîâ MySQL 
I mysqldump 
I mydumper 
I mylvmbackup 
I TokuDB Hot Backup 
I MySQL Enterprise Backup 
I Perco...
mysqldump 
I ëîãè÷åñêèé áýêàï 
I + âõîäèò â ïàêåòû MySQL/MariaDB/Percona Server 
I  íåò èíêðåìåíòàëüíûõ áýêàïîâ 
I  îäíîïî...
mydumper 
“How MySQL DBA  support engineer would imagine mysqldump ” 
I ëîãè÷åñêèé áýêàï 
I + ìíîãîïîòî÷íàÿ âåðñèÿ mysqldu...
mylvmbackup 
I ôèçè÷åñêèé áýêàï 
I + ëþáûå òèïû òàáëèö (InnoDB, TokuDB, etc.) 
I  íåò èíêðåìåíòàëüíûõ áýêàïîâ 
I  òîëüêî ë...
TokuDB Hot Backup 
I áèáëèîòåêà, ïåðåõâàòûâàþùàÿ I/O âûçîâû 
I ôàêòè÷åñêè LVM snapshot â userspace
TokuDB Hot Backup 
I ôèçè÷åñêèé áýêàï 
I + ëþáûå òèïû òàáëèö (òåîðåòè÷åñêè) 
I  íåò èíêðåìåíòàëüíûõ áýêàïîâ 
I  òîëüêî ëîê...
MySQL Enterprise Backup 
I ôèçè÷åñêèé áýêàï 
I ïðîïðèåòàðíîå ÏÎ, äîñòóïíî ïî ïëàòíîé ïîäïèñêå 
I + ¾ãîðÿ÷èé¿ áýêàï äëÿ Inn...
Percona XtraBackup 
I ôèçè÷åñêèé áýêàï 
I ñâîáîäíàÿ, áåñïëàòíàÿ àëüòåðíàòèâà MySQL Enterprise Backup 
I ïðàêòè÷åñêè 100% ñ...
×òî âûáðàòü? 
mydumper + Percona XtraBackup ïîêðûâàþò áîëüøèíñòâî ñëó÷àåâ 
I ñòàäàðò äå-ôàêòî äëÿ Percona Managed Services...
Áîëüøèå îáú¼ìû äàííûõ
Áîëüøèå îáú¼ìû äàííûõ: mysqldump 
I mysqldump --innodb-optimize-keys â ïàêåòàõ Percona Server 
5.1/5.5/5.6 
I óáèðàåò âòîð...
Áîëüøèå îáú¼ìû äàííûõ: PXB/MEB 
¾Êîìïàêòíûå¿ áýêàïû â Percona XtraBackup 
innobackupex --compact /backup/dir/ 
¾Skip unuse...
Áëîêèðîâêè 
Âñå óòèëèòû îáåùàþò ¾ãîðÿ÷èé¿ áýêàï. 
Íà ñàìîì äåëå èñïîëüçóþò áëîêèðîâêè äëÿ: 
I ñîãëàñîâàííîñòè ñ íåòðàíçàêö...
Áëîêèðîâêè: FLUSH TABLES WITH READ LOCK 
 ÷¼ì ïðîáëåìà íà âûñîêîíàãðóæåííûõ ñåðâåðàõ? 
1. Ñáðîñèòü query cache 
2. Çàáëîê...
Áëîêèðîâêè: Backup locks â Percona Server 5.6 
LOCK TABLES FOR BACKUP: 
I áëîêèðóåò DDL è çàïèñè â íåòðàíçàêöèîííûå òàáëèö...
Áëîêèðîâêè: FTWRL vs. Backup locks 
Percona XtraBackup: 
FLUSH TABLES WITH READ LOCK 
1. ñêîïèðîâàòü InnoDB òàáëèöû 
2. FL...
Áëîêèðîâêè: FTWRL vs. Backup locks 
mysqldump --single-transaction 
I íå áëîêèðóåò ñåðâåð 
I íå ðàáîòàåò, åñëè åñòü MyISAM...
×àñòè÷íûå êîïèè
×àñòè÷íîå âîññòàíîâëåíèå
Èíêðåìåíòàëüíûå áýêàïû 
Ðåàëèçàöèÿ èíêðåìåíòàëüíûõ áýêàïîâ â Percona XtraBackup è 
MySQL Enterprise Backup: 
I ñêîïèðîâàòü...
Èíêðåìåíòàëüíûå áýêàïû 
Percona Server: innodb_track_changed_pages=1 
I ñîõðàíÿåò bitmap èçìåí¼ííûõ ñòðàíèö â îòäåëüíûõ ôà...
Èíêðåìåíòàëüíûå áýêàïû 
Percona Server: innodb_log_archive=1 
I äóáëèðóåò REDO ëîã â îòäåëüíûõ ôàéëàõ 
Percona XtraBackup:...
Ïðîâåðêà öåëîñòíîñòè 
Ïðîáëåìà: êàê ïðîâåðèòü, ÷òî ñ áýêàïîì ¾âñ¼ â ïîðÿäêå¿? 
I ïðîâåðêà êîíòðîëüíûõ ñóìì, ñïèñêà ôàéëîâ,...
Ïðîâåðêà öåëîñòíîñòè: ðåøåíèå 
Ïðåäëîæåíî ïîëüçîâàòåëåì â ñïèñêå ðàññûëêè: 
1. Ñîçäàòü slave èç áýêàïà 
1. pt-table-checks...
Îáëàêà 
Êòî ïîääåðæèâàåò cloud backups? 
I mysqldump/mydumper 
+ ñòîðîííèå óòèëèòû (s3cmd) 
I MySQL Enterprise Backup ñ âå...
Îáëàêà 
MySQL Enterprise Backup 
I ñåé÷àñ: Amazon S3 
I â ïëàíàõ: ? 
Percona XtraBackup: 
I ñåé÷àñ: OpenStack 
I â ïëàíàõ:...
Îáëàêà: Percona XtraBackup 
Ñîçäàíèå áýêàïà: 
innobackupex --stream=xbstream /tmp | 
xbcloud put --storage=swift 
--swift-...
Îáëàêà: Percona XtraBackup 
Âîññòàíîâëåíèå èç áýêàïà: 
xbcloud get --storage=swift 
--swift-container=testbackup 
--swift-...
Âîïðîñû!
Upcoming SlideShare
Loading in …5
×

Web scale backups for MySQL, Алексей Копытов (Percona)

964 views

Published on

Доклад Алексея Копытова на HighLoad++ 2014.

Published in: Internet
  • Be the first to comment

Web scale backups for MySQL, Алексей Копытов (Percona)

  1. 1. Webscale backups for MySQL Alexey.Kopytov@percona.com October 29, 2014
  2. 2. #BEAMERTHEME: Madrid
  3. 3. Êòî ÿ? Àëåêñåé Êîïûòîâ <alexey.kopytov@percona.com> I Percona Server I Percona XtraBackup (ðóêîâîäèòåëü ïðîåêòà) I Percona XtraDB Cluster
  4. 4. Òèïû ðåçåðâíûõ êîïèé Ëîãè÷åñêèå I äàííûå â âèäå SQL, ïîëó÷åííûå îò ñåðâåðà I òðåáóþò èìïîðò äëÿ âîññòàíîâëåíèÿ (ìåäëåííî!) I âîññòàíîâëåíèå ïîâðåæä¼ííûõ äàííûõ I upgrade/downgrade Ôèçè÷åñêèå I êîïèÿ ôàéëîâ äàííûõ ñ äèñêà I êëîíèðîâàíèå I âîññòàíîâëåíèå âñåé áàçû öåëèêîì Îáû÷íî èñïîëüçóþòñÿ îäíîâðåìåííî
  5. 5. Óòèëèòû äëÿ áýêàïîâ MySQL I mysqldump I mydumper I mylvmbackup I TokuDB Hot Backup I MySQL Enterprise Backup I Percona XtraBackup I íå ñ÷èòàÿ ¾íàäñòðîåê¿
  6. 6. mysqldump I ëîãè÷åñêèé áýêàï I + âõîäèò â ïàêåòû MySQL/MariaDB/Percona Server I íåò èíêðåìåíòàëüíûõ áýêàïîâ I îäíîïîòî÷íûé I Î×ÅÍÜ ìåäëåííîå âîññòàíîâëåíèå
  7. 7. mydumper “How MySQL DBA support engineer would imagine mysqldump ” I ëîãè÷åñêèé áýêàï I + ìíîãîïîòî÷íàÿ âåðñèÿ mysqldump I + îäíà òàáëèöà – îäèí ôàéë I íåò èíêðåìåíòàëüíûõ áýêàïîâ I òîëüêî ëîêàëüíûå áýêàïû (íåëüçÿ ñäåëàòü mydumper | ssh remotehost) I ìåäëåííîå âîññòàíîâëåíèå
  8. 8. mylvmbackup I ôèçè÷åñêèé áýêàï I + ëþáûå òèïû òàáëèö (InnoDB, TokuDB, etc.) I íåò èíêðåìåíòàëüíûõ áýêàïîâ I òîëüêî ëîêàëüíûå áýêàïû
  9. 9. TokuDB Hot Backup I áèáëèîòåêà, ïåðåõâàòûâàþùàÿ I/O âûçîâû I ôàêòè÷åñêè LVM snapshot â userspace
  10. 10. TokuDB Hot Backup I ôèçè÷åñêèé áýêàï I + ëþáûå òèïû òàáëèö (òåîðåòè÷åñêè) I íåò èíêðåìåíòàëüíûõ áýêàïîâ I òîëüêî ëîêàëüíûå áýêàïû I – âñÿ çàïèñü â áàçó äóáëèðóåòñÿ â ïðîöåññå áýêàïà
  11. 11. MySQL Enterprise Backup I ôèçè÷åñêèé áýêàï I ïðîïðèåòàðíîå ÏÎ, äîñòóïíî ïî ïëàòíîé ïîäïèñêå I + ¾ãîðÿ÷èé¿ áýêàï äëÿ InnoDB I + èíêðåìåíòàëüíûå áýêàïû I + áýêàïû íà óäàë¼ííûå õîñò I + áûñòðîå âîññòàíîâëåíèå âñåé áàçû I - áîëüøîé ðàçìåð íà äèñêå I - íå ïîääåðæèâàåò íåêîòîðûå ðàñøèðåíèÿ XtraDB (Percona Server, MariaDB) I - íå ïîääåðæèâàåò TokuDB
  12. 12. Percona XtraBackup I ôèçè÷åñêèé áýêàï I ñâîáîäíàÿ, áåñïëàòíàÿ àëüòåðíàòèâà MySQL Enterprise Backup I ïðàêòè÷åñêè 100% ñîâìåñòèìîñòü â ôóíêöèîíàëüíîñòè (+/- íþàíñû ðåàëèçàöèè) I ïîääåðæèâàåò MySQL, MariaDB, Percona Server, WebscaleSQL, . . .
  13. 13. ×òî âûáðàòü? mydumper + Percona XtraBackup ïîêðûâàþò áîëüøèíñòâî ñëó÷àåâ I ñòàäàðò äå-ôàêòî äëÿ Percona Managed Services TokuDB? Òîãäà mylvmbackup ! I èñïîëüçóåòñÿ â Percona Cloud Tools Åñòü ïîäïèñêà íà MySQL Enterprise? I MySQL Enterprise Backup òîæå âàðèàíò! :)
  14. 14. Áîëüøèå îáú¼ìû äàííûõ
  15. 15. Áîëüøèå îáú¼ìû äàííûõ: mysqldump I mysqldump --innodb-optimize-keys â ïàêåòàõ Percona Server 5.1/5.5/5.6 I óáèðàåò âòîðè÷íûå èíäåêñû èç InnoDB òàáëèö I ïåðåñòðàèâàåò ïîñëå èìïîðòà äàííûõ “áûñòðûì” ìåòîäîì I çíà÷èòåëüíî óñêîðÿåò èìïîðò I ïîêà íå ðåàëèçîâàíî â MySQL (Bug #49120) I ïîêà íå ðåàëèçîâàíî â mydumper
  16. 16. Áîëüøèå îáú¼ìû äàííûõ: PXB/MEB ¾Êîìïàêòíûå¿ áýêàïû â Percona XtraBackup innobackupex --compact /backup/dir/ ¾Skip unused pages¿ â MySQL Enterprise Backup mysqlbackup --skip-unused-pages /backupdir Ìåíüøå ðàçìåð íà äèñêå, íî áîëüøå âðåìåíè íà âîññòàíîâëåíèå!
  17. 17. Áëîêèðîâêè Âñå óòèëèòû îáåùàþò ¾ãîðÿ÷èé¿ áýêàï. Íà ñàìîì äåëå èñïîëüçóþò áëîêèðîâêè äëÿ: I ñîãëàñîâàííîñòè ñ íåòðàíçàêöèîííûìè òàáëèöàìè I ñîãëàñîâàííîñòè .frm ñ äàííûìè I ñîãëàñîâàííîñòè master/slave êîîðäèíàò ñ êîïèåé äàííûõ
  18. 18. Áëîêèðîâêè: FLUSH TABLES WITH READ LOCK  ÷¼ì ïðîáëåìà íà âûñîêîíàãðóæåííûõ ñåðâåðàõ? 1. Ñáðîñèòü query cache 2. Çàáëîêèðîâàòü âñå íîâûå çàïèñè 3. Äîæäàòüñÿ çàâåðøåíèÿ âñåõ âûïîëíÿþùèõñÿ çàïèñåé 4. Äîæäàòüñÿ çàâåðøåíèÿ âñåõ SELECT 5. Çàáëîêèðîâàòü COMMIT
  19. 19. Áëîêèðîâêè: Backup locks â Percona Server 5.6 LOCK TABLES FOR BACKUP: I áëîêèðóåò DDL è çàïèñè â íåòðàíçàêöèîííûå òàáëèöû LOCK BINLOG FOR BACKUP: I áëîêèðóåò COMMIT
  20. 20. Áëîêèðîâêè: FTWRL vs. Backup locks Percona XtraBackup: FLUSH TABLES WITH READ LOCK 1. ñêîïèðîâàòü InnoDB òàáëèöû 2. FLUSH TABLES WITH READ LOCK; 3. ñêîïèðîâàòü .frm, MyISAM, è ò.ä. 4. ïîëó÷èòü master/slave êîîðäèíàòû 5. çàâåðøèòü êîïèðîâàíèå REDO log 6. UNLOCK TABLES; Backup locks 1. ñêîïèðîâàòü InnoDB òàáëèöû 2. LOCK TABLES FOR BACKUP; 3. ñêîïèðîâàòü .frm, MyISAM, è ò.ä. 4. LOCK BINLOG FOR BACKUP; 5. çàâåðøèòü êîïèðîâàíèå REDO log 6. UNLOCK TABLES; 7. ïîëó÷èòü master/slave êîîðäèíàòû 8. UNLOCK BINLOG;
  21. 21. Áëîêèðîâêè: FTWRL vs. Backup locks mysqldump --single-transaction I íå áëîêèðóåò ñåðâåð I íå ðàáîòàåò, åñëè åñòü MyISAM è/èëè DDL mysqldump --lock-all-tables I áëîêèðóåò ñåðâåð I ðàáîòàåò ñ MyISAM è DDL Percona Server: mysqldump -single-transaction --lock-for-backup: I íå áëîêèðóåò ñåðâåð I ðàáîòàåò ñ MyISAM è DDL
  22. 22. ×àñòè÷íûå êîïèè
  23. 23. ×àñòè÷íîå âîññòàíîâëåíèå
  24. 24. Èíêðåìåíòàëüíûå áýêàïû Ðåàëèçàöèÿ èíêðåìåíòàëüíûõ áýêàïîâ â Percona XtraBackup è MySQL Enterprise Backup: I ñêîïèðîâàòü âñå ñòðàíèöû, äëÿ êîòîðûõ LSNîáíîâëåíèÿ LSNïîëíîãî áýêàïà Ïðîáëåìà äëÿ áîëüøèõ îáú¼ìîâ äàííûõ: I òðåáóåò ïîëíîãî ñêàíèðîâàíèÿ âñåõ äàííûõ
  25. 25. Èíêðåìåíòàëüíûå áýêàïû Percona Server: innodb_track_changed_pages=1 I ñîõðàíÿåò bitmap èçìåí¼ííûõ ñòðàíèö â îòäåëüíûõ ôàéëàõ Percona XtraBackup: I àâòîìàòè÷åñêè èñïîëüçóåò bitmap ôàéëû âìåñòî ïîëíîãî ñêàíèðîâàíèÿ
  26. 26. Èíêðåìåíòàëüíûå áýêàïû Percona Server: innodb_log_archive=1 I äóáëèðóåò REDO ëîã â îòäåëüíûõ ôàéëàõ Percona XtraBackup: I xtrabackup --prepare --log-arch-dir=/data/archived-logs/ ïðèêëàäûâàåò èçìåíåíèÿ áåç ïîëíîãî ñêàíèðîâàíèÿ
  27. 27. Ïðîâåðêà öåëîñòíîñòè Ïðîáëåìà: êàê ïðîâåðèòü, ÷òî ñ áýêàïîì ¾âñ¼ â ïîðÿäêå¿? I ïðîâåðêà êîíòðîëüíûõ ñóìì, ñïèñêà ôàéëîâ, è ò.ä. íå ãàðàíòèðóåò öåëîñòíîñòü I íå ñ ÷åì ñðàâíèâàòü:
  28. 28. Ïðîâåðêà öåëîñòíîñòè: ðåøåíèå Ïðåäëîæåíî ïîëüçîâàòåëåì â ñïèñêå ðàññûëêè: 1. Ñîçäàòü slave èç áýêàïà 1. pt-table-checksum äëÿ ïðîâåðêè öåëîñòíîñòè ðåïëèêàöèè 2. profit!
  29. 29. Îáëàêà Êòî ïîääåðæèâàåò cloud backups? I mysqldump/mydumper + ñòîðîííèå óòèëèòû (s3cmd) I MySQL Enterprise Backup ñ âåðñèè 3.10.2 I Percona XtraBackup 2.3 (ïîêà alpha)
  30. 30. Îáëàêà MySQL Enterprise Backup I ñåé÷àñ: Amazon S3 I â ïëàíàõ: ? Percona XtraBackup: I ñåé÷àñ: OpenStack I â ïëàíàõ: Amazon S3, Amazon Glacier
  31. 31. Îáëàêà: Percona XtraBackup Ñîçäàíèå áýêàïà: innobackupex --stream=xbstream /tmp | xbcloud put --storage=swift --swift-container=testbackup --swift-user=user --swift-password=password --swift-url=http://my.cloud.com/ --swift-key=testkey --parallel=10
  32. 32. Îáëàêà: Percona XtraBackup Âîññòàíîâëåíèå èç áýêàïà: xbcloud get --storage=swift --swift-container=testbackup --swift-user=user --swift-key=testkey --swift-url=http://my.cloud.com/ --parallel=10 | xbstream -x /data/backup innobackupex --apply-log /data/backup innobackupex --copy-back /data/backup
  33. 33. Âîïðîñû!

×