Tracking Page Changes for Your Database and Bitmap Backups

303 views
236 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
303
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Tracking Page Changes for Your Database and Bitmap Backups

  1. 1. Tracking Page Changesfor Your Database and BackupsLaurynasBiveinis
  2. 2. 2 Agenda • Incremental XtraBackup: performance • Incremental XtraBackup with bitmaps: performance • Server overhead • INFORMATION_SCHEMA.INNODB_CHANGED_PAGES • Implementation – Bitmap file format – New server thread
  3. 3. 3 Incremental XtraBackup: Performance • Does time to backup depend on the % of changed data? 100% 90% 80% 70% Backup Time 60% 50% 40% 30% 20% 10% 0% 0.10% 1.00% 10.00% 100.00% Delta Size
  4. 4. 4 Incremental XtraBackup: How Data Page Copying Works Can we avoid reading the old pages? Base Backup LSN = 1000 table.ibd LSN = 950 read LSN = 960 read LSN LSN = 1002 read > write LSN = 1003 read 1000 write LSN = 940 read ? Table.ibd.delta LSN = 1010 read write
  5. 5. 5 Incremental XtraBackup: Can We Avoid Reading the Old Pages? • http://bit.ly/FBIncBackup
  6. 6. 6 Incremental XtraBackup: Can We Avoid Reading the Old Pages? • How do we know which pages to read then? • Two ways to get the modification LSN of a page: – It is written on the page, - or - – We can figure it out from the redo log • The log is cyclical, we must, in the server, save the info before it is overwritten • --innodb-track-changed-pages=TRUE – Percona Server 5.1.67-14.4, 5.5.29-30.0, 5.6. – Percona XtraBackup 2.1 – zero configuration!
  7. 7. 7 Incremental XtraBackup with Changed Page Tracking … Changed pages between Percona LSNs 980 and 1020: Base Backup Server 1002, 1003, 1010 LSN = 1000 ... table.ibd LSN = 950 LSN = 960 LSN LSN = 1002 read > write LSN = 1003 read 1000 write LSN = 940 ? Table.ibd.delta LSN = 1010 read write
  8. 8. 8 Incremental XtraBackup with Changed Page Tracking: Performance 100% 90% 80% 70% Backup Time 60% Full Scan 50% Bitmap 40% 30% 20% 10% 0% 0.00% 0.01% 1.00% 100.00% Delta Size
  9. 9. 9 Percona Server with Changed Page Tracking: Server Overhead • Nothing is ever free! – But the price might be very well acceptable • Potential overhead #1: extra disk space requirements • Potential overhead #2: extra code running in the server
  10. 10. 10 Percona Server with Changed Page Tracking: Server Overhead Log and bitmap file size comparison 800 Log bytes / bitmap byte 700 600 500 400 300 200 100 0 1 2 3 4 5 6 7 8 Bitmap file #
  11. 11. 11 Percona Server with Changed Page Tracking: Server Overhead • Impact on TPS and response time: – Its a wash – With --innodb_flush_method=fdatasync
  12. 12. 12 Bitmap File Naming & Sizing • ib_modified_log_<seq>_<LSN>.xdb – <Seq>: 1, 2, 3, ... – <LSN>: the server LSN at the file create time • Rotated on – Server start – innodb_max_bitmap_file_size
  13. 13. 13 Bitmap File Management • PURGE CHANGED_PAGE_BITMAPS BEFORE <lsn> – ib_1_8192.xdb – ib_2_10000.xdb – ib_3_20000.xdb – Full backup taken, LSN = 22000 – PURGE C_P_B BEFORE 22000; – ib_4_30000.xdb – Incremental backup taken, LSN = 33000 – PURGE C_P_B BEFORE 33000;
  14. 14. 14 INFORMATION_SCHEMA.INNODB_CHANGED_PAGES • Percona Server can read the bitmaps too SHOW CREATE TABLE INFORMATION_SCHEMA.INNODB_CHANGED_PAGES; CREATE TABLE `INNODB_CHANGED_PAGES` ( `space_id` int(11) unsigned NOT NULL DEFAULT 0, `page_id` int(11) unsigned NOT NULL DEFAULT 0, `start_lsn` bigint(21) unsigned NOT NULL DEFAULT 0, `end_lsn` bigint(21) unsigned NOT NULL DEFAULT 0 ) • start_lsn and end_lsn are always at the checkpoint boundary • Does not show the exact LSN of a change • Does not show the number of changes for one page • Does show the number of flushes for a page over the workload
  15. 15. 15 INFORMATION_SCHEMA.INNODB_CHANGED_PAGES SELECT * FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES; space_id page_id start_lsn end_lsn 0 0 8204 38470 0 1 8204 38470 5 0 8204 38470 5 3 8204 38470 0 1 38471 50000 5 3 38471 50000 5 3 50001 60000 • Dont query like that in production! – It will read all the bitmaps you have. Gigabytes, terabytes, ... – Add WHERE start_lsn > X AND end_lsn < Y (index condition pushdown implemented for this case)
  16. 16. 16 INFORMATION_SCHEMA.INNODB_CHANGED_PAGES • Which tables are written to? SELECT DISTINCT space_id FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES WHERE ...; space_id 0 10 SELECT DISTINCT t1.space_id AS space_id, t2.schema AS db, t2.name AS tname FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES AS t1, INFORMATION_SCHEMA.INNODB_SYS_TABLES AS t2 WHERE t1.space_id = t2.space AND t1.start_lsn >... space_id db tname 0 SYS_FOREIGN 0 SYS_FOREIGN_COLS 10 test foo
  17. 17. 17 INFORMATION_SCHEMA.INNODB_CHANGED_PAGES • What are the hottest tables? SELECT space_id, COUNT(space_id) AS number_of_flushes FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES GROUP BY space_id ORDER BY number_of_flushes DESC; space_id number_of_flushes 0 65 10 5 11 4
  18. 18. 18 INFORMATION_SCHEMA.INNODB_CHANGED_PAGES • What are the hottest pages? SELECT space_id, page_id, COUNT(page_id) AS number_of_flushes FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES GROUP BY space_id, page_id HAVING number_of_flushes > 2 ORDER BY number_of_flushes DESC LIMIT 8; space_id page_id number_of_flushes 0 5 3 0 7 3 0 0 2 0 11 2 10 3 2 0 1 2 0 12 2 0 2 2
  19. 19. 19 INFORMATION_SCHEMA.INNODB_CHANGED_PAGES • For complex queries, copy data first CREATE TEMPORARY TABLE icp ( space_id INT(11) NOT NULL, page_id INT(11) NOT NULL, start_lsn BIGINT(21) NOT NULL, end_lsn BIGINT(21) NOT NULL, INDEX page_id(space_id, page_id), INDEX start_lsn(start_lsn), INDEX end_lsn(end_lsn)) ENGINE=InnoDB; INSERT INTO icp SELECT * FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES WHERE start_lsn > 8000;
  20. 20. 20 INFORMATION_SCHEMA.INNODB_CHANGED_PAGES • For complex queries, copy data first EXPLAIN SELECT DISTINCT space_id FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES; id select_type table type possible_keys key key_len ref rows Extra 1 SIMPLE INNODB_CHANGED_PAGES ALL NULL NULL NULL NULL NULL Using temporary EXPLAIN SELECT DISTINCT space_id FROM icp; id select_type table type possible_keys key key_len ref rows Extra 1 SIMPLE icp index NULL page_id 8 NULL 74 Using index
  21. 21. 21 Implementation: File Format A sequence of per-checkpoint varying number of data pages: Data for checkpoint at LSN 9000 LSN 10000 LSN 10500 For each checkpoint: 4KB space, start page space, start page space, start page Each page contains a bitmap for the next 32480 pages in space starting from start page
  22. 22. 22 Implementation: Server Side • A new XtraDB thread – 1. Wait for log checkpoint completed event – 2. Read the log up to the checkpoint, write the bitmap – 3. goto 1 • Little data sharing with the rest of XtraDB – log_sys->mutex for: ● setting and getting LSNs; ● calculating log read offset from LSN. • Little extra code for the query threads – Unread log overwrite check – Firing of the log checkpoint completed event –
  23. 23. 23 Implementation: Things We Had to Account For • Maximum checkpoint age violation – Destroys untracked log data – Make effort to avoid, but in the end we allow to overwrite it – Responding server > fast backups • Crash recovery – Re-read the log if available
  24. 24. 24 Conclusions • Percona Server together with Percona XtraBackup: • Enable faster incremental backups • Enable more frequent incremental backups • Does not hurt server operation, but have to manage the bitmaps now • New INFORMATION_SCHEMA table for gaining insight into data change patterns • Thank you! Questions?

×