Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
InnoDB CompressionGetting it ready for Facebook scaleNizam Ordulu nizam.ordulu@fb.comSoftware Engineer, database engineeri...
Why use compression
Why use compression▪   Save disk space.▪   Buy fewer servers.▪   Buy better disks (SSD) without too much    increase in co...
Database Size
IOPS
Sysbench Benchmarks
SysbenchDefault table schema for sysbenchCREATE TABLE `sbtest` (     `id` int(10) unsigned NOT NULL auto_increment,     `k...
In-memory benchmarkConfiguration▪   Buffer pool size =1G.▪   16 tables.▪   250K rows on each table.▪   Uncompressed db siz...
In-memory benchmarkLoad Time                                             Time(s)8070605040                                ...
In-memory benchmarkDatabase size after load                                           Size (M)12001000 800 600            ...
In-memory benchmarkTransactions per second for reads (oltp.lua, read-only)                            Transactions Per Sec...
In-memory benchmarkInserts per second (insert.lua)                                     Inserts Per Second 60000 50000 4000...
IO-bound benchmark for insertsInserts per second (insert.lua)                             Inserts Per Second60000500004000...
InnoDB Compression
InnoDB CompressionBasics▪   16K Pages are compressed to 1K, 2K, 4K, 8K blocks.▪   Block size is specified during table cre...
InnoDB CompressionExampleCREATE TABLE `sbtest1` ( `id` int(10) unsigned NOT NULL AUTO_INCREMENT, `k` int(10) unsigned NOT ...
InnoDB CompressionPage Modification Log (mlog)▪   InnoDB does not recompress a page on every update.▪   Updates are append...
InnoDB CompressionPage Modification Log Example
InnoDB CompressionPage Modification Log Example
InnoDB CompressionPage Modification Log Example
InnoDB CompressionPage Modification Log Example
InnoDB CompressionCompression failures are bad▪   Compression failures:    ▪   waste CPU cycles,    ▪   cause mutex conten...
InnoDB CompressionUnzip LRU▪   A compressed block is decompressed when it is read.▪   Compressed and uncompressed copy are...
InnoDB CompressionCompressed pages written to redo log▪   Compressed pages are written to redo log.▪   Reasons for doing t...
InnoDB CompressionOfficial advice on tuning compressionIf the number of “successful” compression operations(COMPRESS_OPS_O...
Facebook Improvements
Facebook ImprovementsFinding bugs and testing new features▪   Expanded mtr test suite with crash-recovery and stress tests...
Facebook ImprovementsTable level compression statistics▪   Added the following columns to table_statistics:     ▪   COMPRE...
Facebook ImprovementsRemoval of compressed pages from redo log▪   Removed compressed page images from redo log.▪   Introdu...
Facebook ImprovementsAdaptive padding▪   Put less data on each page to prevent compression failures.▪   pad = 16K – (maxim...
Facebook ImprovementsAdaptive padding
Facebook ImprovementsAdaptive padding
Facebook ImprovementsAdaptive padding▪   Algorithm to determine pad per table:    ▪   Increase the pad until the compressi...
Facebook ImprovementsAdaptive padding on insert benchmark                                                     Inserts Per ...
Facebook ImprovementsCompression ops in insert benchmark140000012000001000000 800000                                      ...
Facebook ImprovementsTime spent for compression ops in insert benchmark12001000 800                                       ...
Facebook ImprovementsOther improvements▪   Amount of empty allocated pages: 10-15% to 2-5%.▪   Cache memory allocations fo...
Facebook ImprovementsFuture work▪   Make page_zip_compress() more efficient.▪   Test larger page sizes:32K, 64K.▪   Prefix...
Questions
nizam.ordulu@fb.com
Upcoming SlideShare
Loading in …5
×

Getting innodb compression_ready_for_facebook_scale

1,901 views

Published on

Published in: Technology
  • Be the first to comment

Getting innodb compression_ready_for_facebook_scale

  1. 1. InnoDB CompressionGetting it ready for Facebook scaleNizam Ordulu nizam.ordulu@fb.comSoftware Engineer, database engineering @Facebook4/11/12
  2. 2. Why use compression
  3. 3. Why use compression▪ Save disk space.▪ Buy fewer servers.▪ Buy better disks (SSD) without too much increase in cost.▪ Reduce IOPS.
  4. 4. Database Size
  5. 5. IOPS
  6. 6. Sysbench Benchmarks
  7. 7. SysbenchDefault table schema for sysbenchCREATE TABLE `sbtest` ( `id` int(10) unsigned NOT NULL auto_increment, `k` int(10) unsigned NOT NULL default 0, `c` char(120) NOT NULL default , `pad` char(60) NOT NULL default , PRIMARY KEY (`id`), KEY `k` (`k`));
  8. 8. In-memory benchmarkConfiguration▪ Buffer pool size =1G.▪ 16 tables.▪ 250K rows on each table.▪ Uncompressed db size = 1.1G.▪ Compressed db size = 600M.▪ In-memory benchmark.▪ 16 threads.
  9. 9. In-memory benchmarkLoad Time Time(s)8070605040 Time(s)302010 0 mysql-uncompressed mysql-compressed fb-mysql-uncompressed fb-mysql-compressed
  10. 10. In-memory benchmarkDatabase size after load Size (M)12001000 800 600 Size (M) 400 200 0 mysql-uncompressed mysql-compressed fb-mysql-uncompressed fb-mysql-compressed
  11. 11. In-memory benchmarkTransactions per second for reads (oltp.lua, read-only) Transactions Per Second (Read-Only)80007000600050004000 TPS300020001000 0 mysql-uncompressed mysql-compressed fb-mysql-uncompressed fb-mysql-compressed
  12. 12. In-memory benchmarkInserts per second (insert.lua) Inserts Per Second 60000 50000 40000 30000 IPS 20000 10000 0 mysql-uncompressed mysql-compressed fb-mysql-uncompressed fb-mysql-compressed (4X)
  13. 13. IO-bound benchmark for insertsInserts per second (insert.lua) Inserts Per Second60000500004000030000 IPS2000010000 0 mysql-uncompressed fb-mysql-uncompressed
  14. 14. InnoDB Compression
  15. 15. InnoDB CompressionBasics▪ 16K Pages are compressed to 1K, 2K, 4K, 8K blocks.▪ Block size is specified during table creation.▪ 8K is safest if data is not too compressible.▪ blobs and varchars increase compressibility.▪ In-memory workloads may require larger buffer pool.
  16. 16. InnoDB CompressionExampleCREATE TABLE `sbtest1` ( `id` int(10) unsigned NOT NULL AUTO_INCREMENT, `k` int(10) unsigned NOT NULL DEFAULT 0, `c` char(120) NOT NULL DEFAULT ’, `pad` char(60) NOT NULL DEFAULT , PRIMARY KEY (`id`), KEY `k_1` (`k`)) ENGINE=InnoDB ROW_FORMAT=COMPRESSEDKEY_BLOCK_SIZE=8
  17. 17. InnoDB CompressionPage Modification Log (mlog)▪ InnoDB does not recompress a page on every update.▪ Updates are appended to the modification log.▪ mlog is located in the bottom of the compressed page.▪ When mlog is full, page is recompressed.
  18. 18. InnoDB CompressionPage Modification Log Example
  19. 19. InnoDB CompressionPage Modification Log Example
  20. 20. InnoDB CompressionPage Modification Log Example
  21. 21. InnoDB CompressionPage Modification Log Example
  22. 22. InnoDB CompressionCompression failures are bad▪ Compression failures: ▪ waste CPU cycles, ▪ cause mutex contention.
  23. 23. InnoDB CompressionUnzip LRU▪ A compressed block is decompressed when it is read.▪ Compressed and uncompressed copy are both in memory.▪ Any update on the page is applied to both of the copies.▪ When it is time to evict a page: ▪ Evict an uncompressed copy if the system is IO-bound. ▪ Evict a page from the normal LRU if the system is CPU- bound.
  24. 24. InnoDB CompressionCompressed pages written to redo log▪ Compressed pages are written to redo log.▪ Reasons for doing this: ▪ Reuse redo logs even if the zlib version changes. ▪ Prevent against indeterminism in compression.▪ Increase in redo log writes.▪ Increase in checkpoint frequency.
  25. 25. InnoDB CompressionOfficial advice on tuning compressionIf the number of “successful” compression operations(COMPRESS_OPS_OK) is a high percentage of the totalnumber of compression operations (COMPRESS_OPS), then thesystem is likely performing well. If the ratio is low, then InnoDB isreorganizing, recompressing, and splitting B-tree nodes moreoften than is desirable. In this case, avoid compressing sometables, or increase KEY_BLOCK_SIZE for some of thecompressed tables. You might turn off compression for tablesthat cause the number of “compression failures” in yourapplication to be more than 1% or 2% of the total. (Such a failureratio might be acceptable during a temporary operation such as adata load).
  26. 26. Facebook Improvements
  27. 27. Facebook ImprovementsFinding bugs and testing new features▪ Expanded mtr test suite with crash-recovery and stress tests.▪ Simulate compression failures.▪ Fixed the bugs revealed by the tests and production servers.
  28. 28. Facebook ImprovementsTable level compression statistics▪ Added the following columns to table_statistics: ▪ COMPRESS_OPS, ▪ COMPRESS_OPS_OK, ▪ COMPRESS_USECS, ▪ UNCOMPRESS_OPS, ▪ UNCOMPRESS_USECS.
  29. 29. Facebook ImprovementsRemoval of compressed pages from redo log▪ Removed compressed page images from redo log.▪ Introduced a new log record for compression.
  30. 30. Facebook ImprovementsAdaptive padding▪ Put less data on each page to prevent compression failures.▪ pad = 16K – (maximum data size allowed on the uncompressed copy)
  31. 31. Facebook ImprovementsAdaptive padding
  32. 32. Facebook ImprovementsAdaptive padding
  33. 33. Facebook ImprovementsAdaptive padding▪ Algorithm to determine pad per table: ▪ Increase the pad until the compression failure rate reaches the specified level. ▪ Decrease padding if the failure rate is too low.▪ Adapts to the compressibility of data over time.
  34. 34. Facebook ImprovementsAdaptive padding on insert benchmark Inserts Per Second▪ Padding value for sbtable is 2432. 35000▪ Compression failure rate: 30000 ▪ mysql: 41%. 25000 ▪ fb-mysql: 5%. 20000 15000 10000 5000 0 mysql-compressed fb-mysql-compressed
  35. 35. Facebook ImprovementsCompression ops in insert benchmark140000012000001000000 800000 compress_ops_ok 600000 compress_ops_fail 400000 200000 0 mysql-compressed fb-mysql-compressed
  36. 36. Facebook ImprovementsTime spent for compression ops in insert benchmark12001000 800 compress_time(s) 600 decompress_time(s) 400 200 0 mysql-compressed fb-mysql-compressed
  37. 37. Facebook ImprovementsOther improvements▪ Amount of empty allocated pages: 10-15% to 2-5%.▪ Cache memory allocations for: ▪ compression buffers, ▪ decompression buffers, ▪ buffer page descriptors.▪ Hardware accelerated checksum for compressed pages.▪ Remove adler32 calls from zlib functions.
  38. 38. Facebook ImprovementsFuture work▪ Make page_zip_compress() more efficient.▪ Test larger page sizes:32K, 64K.▪ Prefix compression.▪ Other compression algorithms: snappy, quicklz etc.▪ 3X compression in production.
  39. 39. Questions
  40. 40. nizam.ordulu@fb.com

×