MySQL Monitoring Mechanisms: How the different pieces fit together Mark Leith (@MarkLeith)
Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information p...
Program Agenda <ul><li>Overview of Past MySQL Monitoring
The New MySQL Monitoring Vision
Performance Schema
New InnoDB Instrumentation
Optimizer Trace
Enhanced EXPLAIN
Other Goodies
Summary </li></ul>
A long time ago in a galaxy far, far away.... MySQL Episode IV.I  The SHOW statement continues to rule supreme...  USE THE...
Overview of Past MySQL Monitoring Prior to 5.5 instrumentation within MySQL was limited <ul><li>Various SHOW statements al...
INFORMATION_SCHEMA for (mostly) object metadata
Slow / General query logs for statement history
EXPLAIN / SHOW PROFILE for statement profiling </li></ul>
SHOW Statements <ul><li>Great for simple commands to get metadata, i.e: </li></ul><ul><ul><li>SHOW CREATE TABLE ...;
SHOW TABLES; </li></ul></ul><ul><li>Not so great for scripting or tooling, i.e: </li></ul><ul><ul><li>SHOW ENGINE INNODB S...
Can not filter, or re-order columns
Can not use dynamically (prepared statements etc.) </li></ul></ul>
INFORMATION_SCHEMA <ul><li>Added within 5.0+ </li></ul><ul><li>It's just SQL and (virtual) tables.. </li><ul><li>Can be us...
Order output independently (columns, rows, etc.) </li></ul><li>Plugin interface available </li></ul><ul><li>Optimization p...
Slow / General Query Logging <ul><li>Available in all versions of MySQL
Allow analyzing problems after the fact </li></ul><ul><li>Not nearly enough information within them for serious diagnosis ...
Overhead can be excessive </li><ul><li>General query log bottleneck at start of statements
CSV based log tables in 5.1+ are not ideal </li></ul></ul>
EXPLAIN / SHOW PROFILE <ul><li>Good high-level overviews of how a query runs
Allows some response time break downs  </li></ul><ul><li>Table based outputs belie true execution flows
Not enough statistics are reported </li><ul><li>EXPLAIN doesn't show all optimizer decisions
EXPLAIN doesn't support all statement types
SHOW PROFILE often isn't fine grained enough </li></ul></ul>
MySQL Episode V.V A NEW HOPE
5.5+ Instrumentation <ul><li>5.5 saw a huge investment in instrumentation
The  Performance Schema  framework was introduced </li><ul><li>Many “finer detail” instrumentation points added
Focused on IO and internal synchronization points  </li></ul><li>New INFORMATION_SCHEMA tables for InnoDB </li><ul><li>Cur...
Compression statistics </li></ul></ul>
5.6 And Beyond Accelerating investment in instrumentation and tooling <ul><li>Fill more gaps in Performance Schema statist...
Expose even more detailed InnoDB statistics
Take the guess work out of optimizer workings
Increase flexibility of instrumentation configuration
Provide more structured data for richer tooling </li></ul>
5.6 And Beyond – Instrumentation Vision <ul><li>SHOW - Object Metdata, limited extension otherwise
INFORMATION_SCHEMA – Standard Object Metadata
INFORMATION_SCHEMA – Plugin Specific Extensions
PERFORMANCE_SCHEMA – Consolidated Statistics </li></ul>
Performance Schema
Performance Schema in 5.5 <ul><li>Foundation for a powerful instrumentation framework
Similar in goals to the “ Oracle Wait Interface ”
Tracks  Events  by time/latency (to picosecond precision)
Entirely non-locking, and in fixed memory
Implemented as a real storage engine </li><ul><li>No unique optimization paths
Not persistent </li></ul></ul>
Performance Schema in 5.5 What is an “ Event ”? <ul><li>At it's most basic, a measurement of a unit of work </li><ul><li>I...
In some we record other things, like bytes requested </li></ul><li>Naming roughly follows “ Linnean Taxinomy ” </li><ul><l...
Performance Schema in 5.5 <ul><li>First instrumentation is more MySQL developer focused
Started at some of the smallest internal units of work  </li><ul><li>Mutexes -  wait/synch/mutex/
Read/Write locks -  wait/synch/rwlock/
Conditions -  wait/synch/cond/ </li></ul><li>File IO instrumented as well, across all file types </li></ul><ul><ul><li>wai...
Performance Schema in 5.5 THREAD_ID:  17 EVENT_ID:  57265474 EVENT_NAME:  wait/synch/mutex/innodb/kernel_mutex SOURCE:  lo...
Performance Schema in 5.5 SELECT event_name, ROUND(SUM(sum_timer_wait) / 1000000000000, 2) total_sec, ROUND(MIN(min_timer_...
Upcoming SlideShare
Loading in...5
×

MySQL Monitoring Mechanisms

1,420

Published on

The 5.5 and 5.6 releases of MySQL introduce several new mechanisms that provide improved monitoring and performance tuning functionality. Examples are performance schemas, InnoDB metrics tables, optimizer trace, and extended explain functionality. This session outlines the vision for monitoring-related functionality in MySQL and presents an overview of the new mechanisms. It shows how these are integrated with MySQL management tools. Furthermore, it discusses how these mechanisms can be utilized by application developers, DBAs, and production engineers for tracking down performance issues and monitoring production systems.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,420
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
39
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

MySQL Monitoring Mechanisms

  1. 2. MySQL Monitoring Mechanisms: How the different pieces fit together Mark Leith (@MarkLeith)
  2. 3. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
  3. 4. Program Agenda <ul><li>Overview of Past MySQL Monitoring
  4. 5. The New MySQL Monitoring Vision
  5. 6. Performance Schema
  6. 7. New InnoDB Instrumentation
  7. 8. Optimizer Trace
  8. 9. Enhanced EXPLAIN
  9. 10. Other Goodies
  10. 11. Summary </li></ul>
  11. 12. A long time ago in a galaxy far, far away.... MySQL Episode IV.I The SHOW statement continues to rule supreme... USE THE FORCE LUKE!!
  12. 13. Overview of Past MySQL Monitoring Prior to 5.5 instrumentation within MySQL was limited <ul><li>Various SHOW statements allowed some global visibility
  13. 14. INFORMATION_SCHEMA for (mostly) object metadata
  14. 15. Slow / General query logs for statement history
  15. 16. EXPLAIN / SHOW PROFILE for statement profiling </li></ul>
  16. 17. SHOW Statements <ul><li>Great for simple commands to get metadata, i.e: </li></ul><ul><ul><li>SHOW CREATE TABLE ...;
  17. 18. SHOW TABLES; </li></ul></ul><ul><li>Not so great for scripting or tooling, i.e: </li></ul><ul><ul><li>SHOW ENGINE INNODB STATUS parsing hurts
  18. 19. Can not filter, or re-order columns
  19. 20. Can not use dynamically (prepared statements etc.) </li></ul></ul>
  20. 21. INFORMATION_SCHEMA <ul><li>Added within 5.0+ </li></ul><ul><li>It's just SQL and (virtual) tables.. </li><ul><li>Can be used dynamically (prepared statements etc.)
  21. 22. Order output independently (columns, rows, etc.) </li></ul><li>Plugin interface available </li></ul><ul><li>Optimization path is unique, and disk accesses are high with some tables (I_S.TABLES etc.) </li></ul>
  22. 23. Slow / General Query Logging <ul><li>Available in all versions of MySQL
  23. 24. Allow analyzing problems after the fact </li></ul><ul><li>Not nearly enough information within them for serious diagnosis (even with patches)
  24. 25. Overhead can be excessive </li><ul><li>General query log bottleneck at start of statements
  25. 26. CSV based log tables in 5.1+ are not ideal </li></ul></ul>
  26. 27. EXPLAIN / SHOW PROFILE <ul><li>Good high-level overviews of how a query runs
  27. 28. Allows some response time break downs </li></ul><ul><li>Table based outputs belie true execution flows
  28. 29. Not enough statistics are reported </li><ul><li>EXPLAIN doesn't show all optimizer decisions
  29. 30. EXPLAIN doesn't support all statement types
  30. 31. SHOW PROFILE often isn't fine grained enough </li></ul></ul>
  31. 32. MySQL Episode V.V A NEW HOPE
  32. 33. 5.5+ Instrumentation <ul><li>5.5 saw a huge investment in instrumentation
  33. 34. The Performance Schema framework was introduced </li><ul><li>Many “finer detail” instrumentation points added
  34. 35. Focused on IO and internal synchronization points </li></ul><li>New INFORMATION_SCHEMA tables for InnoDB </li><ul><li>Current transaction and locking statistics
  35. 36. Compression statistics </li></ul></ul>
  36. 37. 5.6 And Beyond Accelerating investment in instrumentation and tooling <ul><li>Fill more gaps in Performance Schema statistics
  37. 38. Expose even more detailed InnoDB statistics
  38. 39. Take the guess work out of optimizer workings
  39. 40. Increase flexibility of instrumentation configuration
  40. 41. Provide more structured data for richer tooling </li></ul>
  41. 42. 5.6 And Beyond – Instrumentation Vision <ul><li>SHOW - Object Metdata, limited extension otherwise
  42. 43. INFORMATION_SCHEMA – Standard Object Metadata
  43. 44. INFORMATION_SCHEMA – Plugin Specific Extensions
  44. 45. PERFORMANCE_SCHEMA – Consolidated Statistics </li></ul>
  45. 46. Performance Schema
  46. 47. Performance Schema in 5.5 <ul><li>Foundation for a powerful instrumentation framework
  47. 48. Similar in goals to the “ Oracle Wait Interface ”
  48. 49. Tracks Events by time/latency (to picosecond precision)
  49. 50. Entirely non-locking, and in fixed memory
  50. 51. Implemented as a real storage engine </li><ul><li>No unique optimization paths
  51. 52. Not persistent </li></ul></ul>
  52. 53. Performance Schema in 5.5 What is an “ Event ”? <ul><li>At it's most basic, a measurement of a unit of work </li><ul><li>In all cases this includes time taken to complete
  53. 54. In some we record other things, like bytes requested </li></ul><li>Naming roughly follows “ Linnean Taxinomy ” </li><ul><li>/Class/Order/Family/Genus/Species </li></ul></ul>
  54. 55. Performance Schema in 5.5 <ul><li>First instrumentation is more MySQL developer focused
  55. 56. Started at some of the smallest internal units of work </li><ul><li>Mutexes - wait/synch/mutex/
  56. 57. Read/Write locks - wait/synch/rwlock/
  57. 58. Conditions - wait/synch/cond/ </li></ul><li>File IO instrumented as well, across all file types </li></ul><ul><ul><li>wait/io/file/ </li></ul></ul><ul><ul><li>Tracks both time waited and bytes per request </li></ul></ul>
  58. 59. Performance Schema in 5.5 THREAD_ID: 17 EVENT_ID: 57265474 EVENT_NAME: wait/synch/mutex/innodb/kernel_mutex SOURCE: lock0lock.c:5489 TIMER_START: 1716417836794180 TIMER_END: 1716417836828110 TIMER_WAIT: 33930 SPINS: NULL OBJECT_SCHEMA: NULL OBJECT_NAME: NULL OBJECT_TYPE: NULL OBJECT_INSTANCE_BEGIN: 4307564056 NESTING_EVENT_ID: NULL OPERATION: lock NUMBER_OF_BYTES: NULL FLAGS: 0 Mutex Event ~33 nanosecs To acquire lock
  59. 60. Performance Schema in 5.5 SELECT event_name, ROUND(SUM(sum_timer_wait) / 1000000000000, 2) total_sec, ROUND(MIN(min_timer_wait) / 1000000, 2) min_usec, ROUND((SUM(sum_timer_wait) / SUM(COUNT_STAR))/1000000, 2) avg_usec, ROUND(MAX(max_timer_wait) / 1000000, 2) max_usec FROM performance_schema.events_waits_summary_global_by_event_name WHERE sum_timer_wait > 0 GROUP BY event_name ORDER BY total_sec DESC limit 10; Top waits by latency
  60. 61. Performance Schema in 5.5 THREAD_ID: 6 EVENT_ID: 1036 EVENT_NAME: wait/io/file/innodb/innodb_data_file SOURCE: os0file.c:4891 TIMER_START: 338825682207740 TIMER_END: 338825703463000 TIMER_WAIT: 21255260 SPINS: NULL OBJECT_SCHEMA: NULL OBJECT_NAME: /Users/mark/sandboxes/msb_5_5_16/data/ibdata1 OBJECT_TYPE: FILE OBJECT_INSTANCE_BEGIN: 4683519488 NESTING_EVENT_ID: NULL OPERATION: write NUMBER_OF_BYTES: 32768 FLAGS: 0 File IO Event ~21 microsecs To write 32K On this file
  61. 62. Performance Schema in 5.5 SELECT substring_index(file_name, '/', -1) AS file, count_read, count_write, ROUND(sum_number_of_bytes_read / (1024*1024)) read_mb, ROUND(sum_number_of_bytes_write / (1024*1024)) write_mb, ROUND((sum_number_of_bytes_read + sum_number_of_bytes_write) / (1024*1024*1024)) total_gb FROM performance_schema.file_summary_by_instance ORDER BY total_gb DESC LIMIT 10; Top files by IO consumption
  62. 63. Performance Schema in 5.5 <ul><li>Proved the model, can handle the “flood”
  63. 64. Already aided finding bottlenecks
  64. 65. But.. </li><ul><li>Needed more DBA/Developer focused statistics
  65. 66. Needed more control on who/what is traced
  66. 67. Needed a little tuning for high load systems </li></ul></ul>
  67. 68. Performance Schema in 5.6 <ul><li>Adding DBA / Developer focused instrumentation
  68. 69. With the ability to drill down between each level
  69. 70. Statements – Latency and key statistics </li><ul><li>Stages – Statement execution states (i.e. “ Sending data ”) </li><ul><li>Network IO / Idle time - at the TCP / socket layers
  70. 71. Table IO - at the storage engine handler layer
  71. 72. Table Locks - at the storage engine handler layer
  72. 73. .. as well as file IO , mutexes , conditions , rwlocks </li></ul></ul></ul>
  73. 74. Performance Schema Statement Statistics
  74. 75. Performance Schema Stages The “stage” (state) Within statement execution When it started, ended, and How long it took The statement ID it was related to
  75. 76. Performance Schema in 5.6 – Event Nesting <ul><li>Events are related in a hierarchy
  76. 77. Currently uses the “Adjacency List” model
  77. 78. EVENT_ID (child) -> NESTING_EVENT_ID (parent)
  78. 79. Investigating the “Nested Set” model </li></ul>
  79. 80. Performance Schema in 5.6 – Event Nesting
  80. 81. Performance Schema in 5.6 – Graph Events http://www.markleith.co.uk/?p=471 The Statement The Stages The individual wait events by type
  81. 82. Performance Schema in 5.6 – Graph Events
  82. 83. Performance Schema in 5.6 – Summaries Summarize (GROUP BY) each class of event by user, host, account ( [email_address] ), thread, and globally New Object, Network IO Table IO and Table Lock Wait summaries
  83. 84. Performance Schema Table IO Summary The table and it's high level aggregated stats A breakdown By table IO type
  84. 85. “ Not everything that can be counted counts, and not everything that counts can be counted.” Albert Einstein Yet 60 nanoseconds of time could mean SO MUCH!
  85. 86. Performance Schema Configuration <ul><li>There is always some overhead with instrumentation
  86. 87. Whether, and how, we time events can be important
  87. 88. Set expectations up front on both: </li><ul><li>What you want to monitor
  88. 89. How selectively you want to monitor it </li></ul></ul>
  89. 90. Performance Schema Overhead http://marcalff.blogspot.com/2011/06/performance-schema-overhead-tuning.html June 2011 Lots of tuning done since this. New benchmarks should be out soon
  90. 91. Performance Schema Setup
  91. 92. Performance Schema – What's Missing? <ul><li>Storage engine level lock latency – i.e. InnoDB row locks
  92. 93. User lock latency – i.e. SELECT GET_LOCK();
  93. 94. A normalized statement level digest
  94. 95. Transactions level statistics (tied to global transaction ID)
  95. 96. Memory allocations
  96. 97. Distribution statistics (histograms)
  97. 98. Background thread “stages”, server cache statistics
  98. 99. ...and...and...and... - We're listening, tell us your needs </li></ul>
  99. 100. New InnoDB Instrumentation
  100. 101. New InnoDB Instrumentation <ul><li>MySQL 5.6: 11 New INFORMATION_SCHEMA tables </li><ul><li>7 Data Dictionary related
  101. 102. 3 Buffer Pool related
  102. 103. 1 General Statistics gold mine </li></ul></ul><ul><li>MySQL 5.5: 7 New INFORMATION_SCHEMA tables </li><ul><li>3 Transaction / Locking related
  103. 104. 4 Compression related </li></ul></ul>
  104. 105. InnoDB System Tables http://dev.mysql.com/doc/refman/5.6/en/innodb-sys-tablestats-table.html View InnoDB's internal Data Dictionary and table stats
  105. 106. INNODB_SYS_TABLES[TATS] SELECT innodb_sys_tables.name, CASE flag >> 5 WHEN 0 THEN 'Antelope' WHEN 1 THEN 'Barracuda' WHEN 2 THEN 'Cheetah' END AS file_format, @page_flags := (flag >> 1) & 15 AS page_flags, @page_size := IF(@page_flags = 0,16384, 512 << @page_flags) AS page_size, ROUND(( @page_size * clust_index_size ) /(1024*1024)) AS pk_mb, ROUND(( @page_size * other_index_size ) /(1024*1024)) AS secidx_mb FROM innodb_sys_tables JOIN innodb_sys_tablestats USING (table_id) ORDER BY pk_mb DESC;
  106. 107. INNODB_BUFFER_PAGE Tables
  107. 108. INNODB_BUFFER_PAGE - Summarized http://dev.mysql.com/doc/refman/5.6/en/innodb-buffer-page-table.html SELECT page_type, page_state, table_name, index_name, SUM(data_size) / (1024*1024) AS used_mb, SUM( IF(compressed_size = 0, 16384, compressed_size) ) / (1024*1024) AS total_mb FROM innodb_buffer_page GROUP BY page_type, page_state, table_name, index_name HAVING total_mb > 0 ORDER BY total_mb DESC;
  108. 109. INNODB_METRICS <ul><li>Consolidated InnoDB statistics
  109. 110. 199 metrics currently collected ( suggest just one more! )
  110. 111. More flexible statistics collection than SHOW STATUS </li><ul><li>Can enable/disable individually
  111. 112. Track when / how long each has been enabled
  112. 113. Provide extra statistics inline per metric
  113. 114. Provide extra metadata inline per metric </li></ul></ul>
  114. 115. INNODB_METRICS – Metric Details http://dev.mysql.com/doc/refman/5.6/en/innodb-metrics-table.html The variable Stats since start Stats since “reset” How long enabled Stat Metadata
  115. 116. INNODB_METRICS – Configuration Enable named metric Disable named metric Enable metric “module” Reset metric “module”
  116. 117. INNODB_METRICS – Looking at Counters http://dev.mysql.com/doc/refman/5.6/en/innodb-metrics-table.html SELECT concat(subsystem,'/', name) AS event, count, ROUND(count / time_elapsed) AS avg_per_second FROM information_schema.innodb_metrics WHERE status = 'enabled' AND type LIKE '%counter%' OR type = 'set_owner' ORDER BY avg_per_second DESC;
  117. 118. Optimizer Trace
  118. 119. Optimizer Trace <ul><li>EXPLAIN tells you how a query will execute
  119. 120. Optimizer Trace tells you why it will execute that way
  120. 121. New INFORMATION_SCHEMA.OPTIMIZER_TRACE
  121. 122. Contains trace of optimizer decisions, in JSON format
  122. 123. Enabled per thread, on demand
  123. 124. Targeted for advanced users, or in use with bug reporting </li></ul>
  124. 125. Optimizer Trace Enable Tracing Run Statement Select the trace
  125. 126. Optimizer Trace Expose query transformations
  126. 127. Optimizer Trace Considered plans With costs Estimations Of rows And costs Whether Indexes are Usable or not
  127. 128. Enhanced EXPLAIN
  128. 129. Enhanced EXPLAIN Plans <ul><li>In the past EXPLAIN was only supported with SELECT
  129. 130. To EXPLAIN an UPDATE - “Re-write equivalent SELECT” </li><ul><li>However optimization paths may be different </li></ul><li>Now you can EXPLAIN: </li><ul><li>UPDATE | DELETE | REPLACE | INSERT </li></ul></ul>
  130. 131. Enhanced EXPLAIN Plans <ul><li>“ Structured EXPLAIN” is in development
  131. 132. A properly structured EXPLAIN plan (again, JSON)
  132. 133. Sorry – no examples yet, it's very early in implementation </li></ul>
  133. 134. Other Goodies
  134. 135. Instance UUIDs / SHOW SLAVE STATUS <ul><li>Automatically generated unique instance identifiers
  135. 136. Persisted in $datadir/auto.cnf
  136. 137. Can be used for reliable replication topology discovery </li><ul><li>Hard in past given IP/Host matching (virtual IPs?) </li></ul></ul>More configuration and thread statistics
  137. 138. Q&A
  1. Gostou de algum slide específico?

    Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

×