Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

InnoDB Scalability improvements in MySQL 8.0

326 views

Published on

Slides of Mydbops Database Meetup 4 held at Zenefits - Bangalore. InnoDB Scalability improvements in MySQL 8.0 presented by Karthik.P.R ,CEO Mydbops.

Published in: Technology
  • Be the first to comment

InnoDB Scalability improvements in MySQL 8.0

  1. 1. InnoDB scalability improvements in MySQL 8.0 Mydbops Meetup - 4 Presented by Karthik P R www.mydbops.com info@mydbops.com 1
  2. 2. About Me 2 Karthik.P R, Founder/CEO Mydbops, Ex-Yahoo!, Working with MySQL ecosystem from 2010, Juggles between Business leadership and hands on technical expertise and a Creative Team Builder.
  3. 3. About Mydbops ● Founded in 2015, HQ in Bangalore India with 150+ customer base across the globe. ● Mydbops is on Database Consulting with core specialization on MySQL and MongoDB Administration and Support. ● We have expert team with 20+ certified DBA’s providing full time support and currently managing 300+ servers on premises and cloud. ● We help organisations to architect and scale systems in MySQL/MongoDB by implementing the advanced technologies in industry which are completely open source. ● We are a leading solution provider in the market for all sort of cloud based database deployments and management. ● An AWS partner for databases. 3
  4. 4. Mydbops at a GlanceHappy Customers 4
  5. 5. Agenda ● InnoDB Architecture ( Overview ) ● InnoDB Read Scalability ● InnoDB Write Scalability ● Lock Scheduling ● InnoDB Features in 8.0 5
  6. 6. Innodb Architecture 6
  7. 7. InnoDB Architecture ● Structures In Memory ○ Buffer Pool ○ Log Buffer ● Structures on disk ○ Tablespaces ○ Redo log File ○ Undo log file ○ Temporary Tablespace ( Global / Session ) 7
  8. 8. InnoDB Architecture 8 Buffer Pool Redo Buffer Tablespaces (System, Undo, Files Per table, Tmp tables ) Redo logs Sequential Writes Random R/W
  9. 9. InnoDB Architecture 9
  10. 10. InnoDB Architecture Important Configuration Variables ● Innodb_buffer_pool_size ● Innodb_buffer_pool_instances ● Innodb_log_buffer_size ● innodb_flush_method ● innodb_flush_log_at_trx_commit ● Innodb_log_file_size ● Innodb_max_undo_log_size 10
  11. 11. Innodb Read Scalability 11
  12. 12. InnoDB Read Scalability ● MySQL 5.7 ○ Full RO Transaction ○ Faster Reads than writes ● MySQL 8.0 ○ RW Scalability ○ Efficient on same HW ○ High Concurrency Work load 12
  13. 13. InnoDB Read Scalability Read Scalability ● Single mutex Contention ( fil_sys_t::mutex) Solution : ● Sharding the fil_sys_t::mutex ● 64 Shards ● individual shar for REDO and UNDO Spaces 13
  14. 14. InnoDB Read Scalability 14 Read only workload ( 4K pages, IO bound,Point Select )
  15. 15. Innodb Write Scalability 15
  16. 16. InnoDB Write Scalability Redo log (WAL) - 5.7 ● Major Component to ensure ACID ● Ensure Changes are durable ● Crash Recovery ● Slower at high concurrency ( Modern Hardware ) ● Needs a new design 16
  17. 17. InnoDB Write Scalability Redo log (WAL) OLD Design ( 5.7 ) 17
  18. 18. InnoDB Write Scalability Lock Less REDO log ( Dedicated Thread ) ● log_writer ● log_flusher ● log_write_notifier ● log_flush_notifier ● log_checkpointer ● log_closer 18
  19. 19. InnoDB Write Scalability Redo log (WAL) Improvements (8.0) 19
  20. 20. InnoDB Write Scalability More option to Fine tune (spin waits and CPU usage ) ● innodb-log-wait-for-flush-spin-hwm ● innodb-log-spin-cpu-abs-lwm ● innodb-log-spin-cpu-pct-hwm ● innodb-log-buffer-size ( Dynamic resize ) Improved performance with innodb-flush-log-at-trx-commit=1 20
  21. 21. InnoDB Write Scalability Many experimental config in MySQL code # Not for production ● INNODB_LOG_RECENT_WRITTEN_SIZE ● INNODB_LOG_EVENTS ● INNODB_LOG_RECENT_CLOSED_SIZE ● INNODB_LOG_WAIT_FOR_WRITE_SPIN_DELAY ● INNODB_LOG_WAIT_FOR_WRITE_TIMEOUT ● INNODB_LOG_WAIT_FOR_FLUSH_SPIN_DELAY ● INNODB_LOG_WAIT_FOR_FLUSH_TIMEOUT ● INNODB_LOG_WRITE_MAX_SIZE ● INNODB_LOG_WRITER_SPIN_DELAY ● INNODB_LOG_WRITER_TIMEOUT ● INNODB_LOG_FLUSHER_SPIN_DELAY ● INNODB_LOG_FLUSHER_TIMEOUT ● INNODB_LOG_WRITE_NOTIFIER_SPIN_DELAY ● INNODB_LOG_WRITE_NOTIFIER_TIMEOUT ● INNODB_LOG_FLUSH_NOTIFIER_SPIN_DELAY ● INNODB_LOG_FLUSH_NOTIFIER_TIMEOUT ● INNODB_LOG_CLOSER_SPIN_DELAY ● INNODB_LOG_CLOSER_TIMEOUT mysql/storage/innobase/handler/ha_innodb.cc 21
  22. 22. InnoDB Write Scalability 22 R/W Workload ( Updates )
  23. 23. Lock Scheduling 23
  24. 24. Lock Scheduling CATS ( Contention Aware Transaction Scheduling ) ● University of Michigan Contribution. ● Philosophy: Not all transaction are same. ● FIFO ( Default MySQL 5.7 ) ● Switches between CATS from FIFO ( MySQL 8.0 ) ● >= 32 Waiting threads ● Reduces lock sys wait mutex ● First database to adopt this Philosophy in Transaction 24
  25. 25. Lock Scheduling 25
  26. 26. InnoDB Features 26
  27. 27. InnoDB Features Data Dictionary ( SDI ) ● A major overhaul in Data Dictionary ● ibd2sbi tool Alter Algorithm=Instant ● Change Index ● Rename Table ● Modify column ● Add/Drop Virtual columns 27
  28. 28. InnoDB Features Alter table add column ● Tencent Games ● Instant add column (at end) Temporary Tablespace ( Session ) ● Pool of 10 temporary tablespace ( .ibt ) ● Each of 80K ( 5 pages ) in size ● 400K Space ID is reserved 28
  29. 29. InnoDB Features Undo logs ● SQL to control Undo logs ○ Create UNDO Tablespace … ○ Alter UNDO Tablespace …. ○ Drop UNDO Tablespace .. ● Default 2 files ( 4 is better ) InnoDB Dedicated Server ● Better defaults ● InnoDB Buffer Pool / InnoDB log File Size / InnoDB Flush method 29
  30. 30. InnoDB Features InnoDB Parallel Thread Count ● Path to parallel query in InnoDB ● Innodb_parallel_read_threads ● Make count() and Check tables faster ● Default 4 threads 30
  31. 31. Resources ● https://mysqlserverteam.com/mysql-8-0-new-lock-free-scalable-wal-design/ ● https://www.percona.com/live/19/speaker/sunny-bains ● https://mysqlserverteam.com/contention-aware-transaction-scheduling-arriving-in-innodb-to-boost- performance/ ● https://dev.mysql.com/worklog/task/?id=10793 ● http://dimitrik.free.fr/blog/posts/mysql-performance-80-and-sysbench-oltp_rw-updatenokey.html ● https://dev.mysql.com/doc/refman/8.0/en/innodb-architecture.html ● https://fossies.org/linux/mysql/storage/innobase/handler/ha_innodb.cc 31
  32. 32. Questions Reach me : prk@mydbops.com 32
  33. 33. Thank You 33

×