More Related Content
Similar to Performance schema and_ps_helper
Similar to Performance schema and_ps_helper(20)
Performance schema and_ps_helper
- 2. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2
Insert Picture Here
Performance Schema and
ps_helper
Mark Leith
Senior Software Development Manager @ Oracle
@MarkLeith
Http://www.markleith.co.uk
- 3. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
Program Agenda
Performance Schema Overview
ps_helper Overview
ps_helper Examples
A peak at what is coming in 5.7...
- 4. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4
Performance Schema Overview
Who has used the most time / resources?
What was executed to take those resources?
Where was time spent in execution?
When were these things executed?
What is Performance Schema?
- 5. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5
Insert Picture Here
Monitoring Pre Performance Schema
Performance Schema Overview
- 6. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6
Insert Picture Here
Or maybe ...
Performance Schema Overview
CC image courtesy of Ghostboy on Flickr
- 7. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7
Insert Picture Here
Monitoring Post Performance Schema
Performance Schema Overview
CC image courtesy of jurvetson on Flickr
- 8. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8
Performance Schema Overview
Exposed within performance_schema schema
Tables use new PERFORMANCE_SCHEMA storage engine
Records various run time statistics via in-built instrumentation points
- 9. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9
Performance Schema Overview
Performance Schema tracks latency for various events
Latency is exposed to picosecond precision (a trillionth of a second)
In 5.5:
– File I/O, Mutexes, Read/Write Locks, Conditions
In 5.6:
– Network I/O, Table I/O, Table Locks, Stages, Statements, Idle
Also track other data as appropriate, like bytes, source position, etc.
Instrumentation Points
- 10. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10
Performance Schema Overview
Idle idle (5.6)
Table Locks wait/lock/table/% (5.6)
Network IO wait/io/socket/% (5.6)
Table IO wait/io/table/% (5.6)
File IO wait/io/file/% (5.5)
Mutexes wait/synch/mutex/% (5.5)
Read/Write Locks wait/synch/rwlock/% (5.5)
Conditions wait/synch/cond/% (5.5)
Wait Events
- 11. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11
Performance Schema Overview
Statements have two types of event
– SQL Statements statement/sql/%
– COM Commands statement/com/%
Stages are the thread states (like SHOW PROFILE, but available
across connections)
– One form so far stage/sql/%
– Apart from... stage/mysys/Waiting for table level lock
Statement and Stage Events (5.6+)
- 12. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12
Performance Schema Overview
mysql> select * from performance_schema.events_waits_history
limit 1G
*************************** 1. row ***************************
THREAD_ID: 3
EVENT_ID: 11
END_EVENT_ID: 11
EVENT_NAME: wait/io/file/innodb/innodb_log_file
SOURCE: os0file.cc:5542
TIMER_START: 2543370511043700
TIMER_END: 2543370535624100
TIMER_WAIT: 24580400
SPINS: NULL
...
A Raw Wait Event
- 13. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13
Performance Schema Overview
mysql> select * from performance_schema.events_waits_history
limit 1G
*************************** 1. row ***************************
...
OBJECT_SCHEMA: NULL
OBJECT_NAME: /Users/mark/sb/msb_5_7_2/data/ib_logfile0
INDEX_NAME: NULL
OBJECT_TYPE: FILE
OBJECT_INSTANCE_BEGIN: 4771328832
NESTING_EVENT_ID: NULL
NESTING_EVENT_TYPE: NULL
OPERATION: write
NUMBER_OF_BYTES: 512
FLAGS: NULL
A Raw Wait Event
- 14. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14
Performance Schema Overview
mysql> select * from
performance_schema.events_stages_history_long limit 1G
*************************** 1. row ***************************
THREAD_ID: 25
EVENT_ID: 5199518
END_EVENT_ID: 5199536
EVENT_NAME: stage/sql/System lock
SOURCE: lock.cc:304
TIMER_START: 280557566125986000
TIMER_END: 280557566190403000
TIMER_WAIT: 64417000
NESTING_EVENT_ID: 5199509
NESTING_EVENT_TYPE: STATEMENT
A Raw Stage Event
- 15. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15
Performance Schema Overview
mysql> select * from
performance_schema.events_statements_history_long limit 1G
*************************** 1. row ***************************
THREAD_ID: 24
EVENT_ID: 3923
END_EVENT_ID: 4044
EVENT_NAME: statement/sql/insert_select
SOURCE: mysqld.cc:931
TIMER_START: 251016737474892000
TIMER_END: 251016738730372000
TIMER_WAIT: 1255480000
LOCK_TIME: 573000000
...
A Raw Statement Event
- 16. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16
Performance Schema Overview
mysql> select * from
performance_schema.events_statements_history_long limit 1G
*************************** 1. row ***************************
...
SQL_TEXT: insert into t2 select * from t1
DIGEST: e6f8db8a3f557ffbb2bf6a7b237cd897
DIGEST_TEXT: INSERT INTO `t2` SELECT * FROM `t1`
CURRENT_SCHEMA: test
OBJECT_TYPE: NULL
OBJECT_SCHEMA: NULL
OBJECT_NAME: NULL
OBJECT_INSTANCE_BEGIN: NULL
...
A Raw Statement Event
- 17. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17
Performance Schema Overview
mysql> select * from
performance_schema.events_statements_history_long limit 1G
*************************** 1. row ***************************
...
MYSQL_ERRNO: 0
RETURNED_SQLSTATE: 00000
MESSAGE_TEXT: Records: 5 Duplicates: 0 Warnings: 0
ERRORS: 0
WARNINGS: 0
ROWS_AFFECTED: 5
ROWS_SENT: 0
ROWS_EXAMINED: 5
...
A Raw Statement Event
- 18. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18
Performance Schema Overview
mysql> select * from
performance_schema.events_statements_history_long limit 1G
*************************** 1. row ***************************
...
CREATED_TMP_DISK_TABLES: 0
CREATED_TMP_TABLES: 0
SELECT_FULL_JOIN: 0
SELECT_FULL_RANGE_JOIN: 0
SELECT_RANGE: 0
SELECT_RANGE_CHECK: 0
SELECT_SCAN: 1
...
A Raw Statement Event
- 19. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19
Performance Schema Overview
mysql> select * from
performance_schema.events_statements_history_long limit 1G
*************************** 1. row ***************************
...
SORT_MERGE_PASSES: 0
SORT_RANGE: 0
SORT_ROWS: 0
SORT_SCAN: 0
NO_INDEX_USED: 1
NO_GOOD_INDEX_USED: 0
NESTING_EVENT_ID: NULL
NESTING_EVENT_TYPE: NULL
A Raw Statement Event
Performance Schema Overview
- 20. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20
Performance Schema Overview
Event Nesting
Performance Schema Overview
Events are linked in a hierarchy
Use NESTING_EVENT_ID to EVENT_ID
I.e Statements → Stages → Waits
- 21. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21
Performance Schema Overview
- 22. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22
Program Agenda
ps_helper Overview
- 23. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23
ps_helper Overview
A collection of Views, Functions and Stored Procedures
Designed to make reading raw Performance Schema data easier
Implements many common DBA and Developer use cases
http://www.markleith.co.uk/ps_helper/
https://github.com/MarkLeith/dbahelper/
What is ps_helper?
- 24. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24
ps_helper Overview
$ git clone https://github.com/MarkLeith/dbahelper.git dbahelper
$ cd dbahelper
$ mysql -u user -p < ps_helper_<version>.sql
<version> can be 55, 56 or 57, for 5.5, 5.6 and 5.7 respectively.
Loading ps_helper
- 25. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.25
ps_helper Overview
Once loaded, creates a new ps_helper schema containing all objects
mysql> select object_type, count
-> from ps_helper.schema_object_overview
-> where db = 'ps_helper';
+-------------+-------+
| object_type | count |
+-------------+-------+
| FUNCTION | 8 |
| VIEW | 49 |
| PROCEDURE | 12 |
+-------------+-------+
- 26. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.26
ps_helper Overview
Functions
●
Make the raw data more readable to a human
●
format_time() / format_bytes()
●
Compress data for CLI output
●
format_statement() / format_path()
●
Extract some data to assist JOIN etc.
●
extract_[schema | table]_from_file_name()
- 27. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27
ps_helper Overview
Procedures – Convenience for P_S Config
●
currently_enabled() / currently_disabled()
●
enable_current_thread() / disable_current_thread()
●
enable_background_threads() / disable_background_threads()
●
save_current_config() / reload_saved_config()
●
reset_to_default() / truncate_all()
●
only_enable()
- 28. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28
ps_helper Overview
Procedures – Analysis of Instrumentation
●
dump_thread_stack()
●
analyze_statement_digest()
- 29. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.29
ps_helper Overview
Views
Summarize raw Performance Schema data for certain use cases
Expose both formatted and raw views where formatting is used
- 30. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.30
Program Agenda
ps_helper Examples
- 31. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31
ps_helper Examples
User Analysis
+--------------------------------+
| Tables_in_ps_helper |
+--------------------------------+
| user_summary |
| user_summary_by_stages |
| user_summary_by_statement_type |
+--------------------------------+
High level User Overview
What Statements User uses
Where User Waits Most
- 32. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.32
user_summary
Summary of user activity per user
mysql> select * from user_summaryG
*************************** 1. row ***************************
user: root
total_statements: 3072
total_latency: 00:04:44.92
avg_latency: 92.75 ms
current_connections: 1
total_connections: 10
unique_hosts: 1
- 33. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.33
user_summary_by_stages
Breakdown of the most expensive stages of execution per user
mysql> select * from user_summary_by_stages;
+------+--------------------------------+-------+-----------+-----------+
| user | event_name | count | wait_sum | wait_avg |
+------+--------------------------------+-------+-----------+-----------+
| root | stage/sql/creating table | 8 | 1.42 s | 178.10 ms |
| root | stage/sql/System lock | 28 | 246.30 ms | 8.80 ms |
| root | stage/sql/checking permissions | 59 | 205.34 ms | 3.48 ms |
| root | stage/sql/Opening tables | 40 | 28.55 ms | 713.79 us |
| root | stage/sql/query end | 45 | 18.51 ms | 411.42 us |
| root | stage/sql/updating | 4 | 16.94 ms | 4.23 ms |
| root | stage/sql/statistics | 18 | 11.96 ms | 664.43 us |
| root | stage/sql/init | 75 | 3.57 ms | 47.62 us |
| root | stage/sql/Sending data | 18 | 1.28 ms | 71.21 us |
- 34. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34
user_summary_by_statement_type
Breakdown of statement types and their stats per user
mysql> select * from user_summary_by_statement_type;
+------+-----------------------+-------+---------------+-------------+--------------+-----------+----------------------+---------------+------------+
| user | statement | count | total_latency | max_latency | lock_latency | rows_sent | rows_examined | rows_affected | full_scans |
+------+-----------------------+-------+---------------+-------------+--------------+-----------+----------------------+---------------+------------+
| root | create_view | 879 | 00:02:16.98 | 966.39 ms | 1.11 s | 0 | 0 | 0 | 0 |
| root | call_procedure | 5 | 00:02:06.75 | 00:01:05.80 | 00:02:06.13 | 0 | 11414288391619301677 | 0 | 2 |
| root | select | 114 | 6.15 s | 1.57 s | 1.00 s | 878 | 26038 | 0 | 65 |
| root | Field List | 181 | 2.90 s | 514.46 ms | 498.74 ms | 0 | 0 | 0 | 0 |
| root | drop_view | 858 | 2.05 s | 223.08 ms | 0 ps | 0 | 0 | 0 | 0 |
| root | show_tables | 33 | 1.80 s | 1.00 s | 814.10 ms | 795 | 795 | 0 | 33 |
| root | drop_function | 159 | 1.48 s | 966.11 ms | 1.38 s | 0 | 0 | 0 | 0 |
| root | create_table | 8 | 1.43 s | 520.83 ms | 0 ps | 0 | 0 | 0 | 0 |
| root | drop_db | 13 | 1.22 s | 249.29 ms | 1.15 s | 0 | 0 | 514 | 0 |
| root | show_fields | 14 | 913.79 ms | 424.08 ms | 514.58 ms | 192 | 192 | 0 | 14 |
| root | show_databases | 19 | 662.52 ms | 479.86 ms | 333.05 ms | 95 | 95 | 0 | 19 |
| root | create_procedure | 235 | 510.45 ms | 137.29 ms | 417.21 ms | 0 | 0 | 0 | 0 |
| root | set_option | 72 | 396.58 ms | 140.56 ms | 0 ps | 0 | 0 | 0 | 0 |
- 35. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35
ps_helper Examples
IO Analysis +------------------------------+
| Tables_in_ps_helper |
+------------------------------+
| io_by_thread_by_latency |
| io_global_by_file_by_bytes |
| io_global_by_file_by_latency |
| io_global_by_wait_by_bytes |
| io_global_by_wait_by_latency |
+------------------------------+
IO Breakdown by File
IO Breakdown by IO Event
IO Breakdown by Thread
- 36. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.36
io_by_thread_by_latency
Threads waiting the longest on file IO
mysql> select * from io_by_thread_by_latency limit 1G
*************************** 1. row ***************************
user: root@localhost
count_star: 8153
total_latency: 16.54 s
min_latency: 448.63 ns
avg_latency: 1.36 ms
max_latency: 279.73 ms
thread_id: 20
processlist_id: 1
- 37. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.37
io_global_by_file_by_latency
Files that have the highest latency
mysql> select * from io_global_by_file_by_latency limit 1G
*************************** 1. row ***************************
File: @@datadir/ps_helper/statement_analysis_raw.frm~
count_star: 30
total_latency: 513.96 ms
count_read: 0
read_latency: 0 ps
count_write: 5
write_latency: 132.13 us
count_misc: 25
misc_latency: 513.83 ms
- 38. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.38
io_global_by_file_by_bytes
Files doing the most IO by bytes
mysql> select * from io_global_by_file_by_bytes limit 1G
*************************** 1. row ***************************
file: @@datadir/ibtmp1
count_read: 0
total_read: 0 bytes
avg_read: 0 bytes
count_write: 48
total_written: 13.06 MiB
avg_write: 278.67 KiB
total: 13.06 MiB
write_pct: 100.00
- 39. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.39
io_global_by_wait_by_latency
File IO events with highest latency
mysql> select * from io_global_by_wait_by_latency limit 1G
*************************** 1. row ***************************
event_name: sql/file_parser
count_star: 2066
total_latency: 15.91 s
avg_latency: 7.70 ms
max_latency: 279.73 ms
read_latency: 0 ps
write_latency: 9.80 ms
misc_latency: 15.90 s
...
- 40. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.40
io_global_by_wait_by_latency cont..
mysql> select * from io_global_by_wait_by_latency limit 1G
*************************** 1. row ***************************
...
count_read: 0
total_read: 0 bytes
avg_read: 0 bytes
count_write: 281
total_written: 878.16 KiB
avg_written: 3.13 KiB
File IO events with highest latency
- 41. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.41
latest_file_io
mysql> select * from latest_file_io limit 10;
+-----------------------------------+------------------------------+-----------+-----------+-----------+
| thread | file | latency | operation | requested |
+-----------------------------------+------------------------------+-----------+-----------+-----------+
| service_manager@localhost:59760:3 | @@tmpdir/#sql11e48_20_3f.MYI | 22.33 us | write | 124 bytes |
| service_manager@localhost:59760:3 | @@tmpdir/#sql11e48_20_3f.MYI | 2.34 us | write | 2 bytes |
| service_manager@localhost:59760:3 | @@tmpdir/#sql11e48_20_3f.MYI | 113.38 us | close | NULL |
| service_manager@localhost:59760:3 | @@tmpdir/#sql11e48_20_3f.MYD | 39.82 us | close | NULL |
| service_manager@localhost:59760:3 | @@tmpdir/#sql11e48_20_3f.MYI | 48.53 us | delete | NULL |
| service_manager@localhost:59760:3 | @@tmpdir/#sql11e48_20_3f.MYD | 358.41 us | delete | NULL |
| srv_master_thread:16 | @@datadir/ib_logfile0 | 22.27 us | write | 512 bytes |
| srv_master_thread:16 | @@datadir/ib_logfile0 | 298.87 us | sync | NULL |
| service_manager@localhost:59760:3 | @@tmpdir/#sql11e48_20_41.MYI | 233.38 us | create | NULL |
| service_manager@localhost:59760:3 | @@tmpdir/#sql11e48_20_41.MYD | 119.15 us | create | NULL |
+-----------------------------------+------------------------------+-----------+-----------+-----------+
The latest File IO events in the history
- 42. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.42
Table Usage Stats
Index Usage Stats
Object Overviews
ps_helper Examples
Schema Analysis +-------------------------------------+
| Tables_in_ps_helper |
+-------------------------------------+
| schema_index_statistics |
| schema_object_overview |
| schema_table_statistics |
| schema_table_statistics_with_buffer |
| schema_tables_with_full_table_scans |
| schema_unused_indexes |
+-------------------------------------+
- 43. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.43
schema_object_overview
mysql> select * from schema_object_overview;
+--------------------+---------------+-------+
| db | object_type | count |
+--------------------+---------------+-------+
| common_schema | BASE TABLE | 18 |
| common_schema | FUNCTION | 70 |
| common_schema | INDEX (BTREE) | 49 |
| common_schema | PROCEDURE | 131 |
| common_schema | VIEW | 62 |
| information_schema | SYSTEM VIEW | 60 |
| mysql | BASE TABLE | 28 |
Overview of objects by type and count
- 44. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.44
schema_table_statistics_with_buffer
mysql> select * from schema_table_statistics_with_bufferG
*************************** 1. row ***************************
table_schema: mem__quan
table_name: example_statements
rows_fetched: 544
fetch_latency: 672.82 ms
rows_inserted: 10815
insert_latency: 00:07:12.10
rows_updated: 522
update_latency: 11.88 s
rows_deleted: 0
delete_latency: 0 ps
Table IO and File IO per table, along with InnoDB buffer use
- 45. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.45
schema_table_statistics_with_buffer cont...
mysql> select * from schema_table_statistics_with_bufferG
*************************** 1. row ***************************
...
io_read_requests: 36
io_read: 88.60 KiB
io_read_latency: 192.23 us
io_write_requests: 1947
io_write: 34.02 MiB
io_write_latency: 73.89 ms
io_misc_requests: 142
io_misc_latency: 1.48 s
...
Table IO and File IO per table, along with InnoDB buffer use
- 46. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.46
schema_table_statistics_with_buffer cont...
mysql> select * from schema_table_statistics_with_bufferG
*************************** 1. row ***************************
...
innodb_buffer_allocated: 4.93 MiB
innodb_buffer_data: 6.32 MiB
innodb_buffer_pages: 631
innodb_buffer_pages_hashed: 631
innodb_buffer_pages_old: 631
innodb_buffer_rows_cached: 10450
Table IO and File IO per table, along with InnoDB buffer use
- 47. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.47
schema_index_statistics
mysql> select * from schema_index_statistics limit 1G
*************************** 1. row ***************************
table_schema: mem__inventory
table_name: mysqlserver
index_name: PRIMARY
rows_selected: 2619
select_latency: 36.46 s
rows_inserted: 0
insert_latency: 0 ps
rows_updated: 494
update_latency: 2.35 s
rows_deleted: 0
delete_latency: 0 ps
Index IO by index
- 48. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.48
schema_unused_indexes
mysql> select * from schema_unused_indexes limit 20;
+-------------------+-----------------------------------+--------------------+
| object_schema | object_name | index_name |
+-------------------+-----------------------------------+--------------------+
| mem__config | user_form_defaults | FKC1AEF1F9E7EE2CFB |
| mem__enterprise | mos_service_requests | PRIMARY |
| mem__enterprise | whats_new_entries | entryId |
| mem__enterprise | whats_new_entries | PRIMARY |
| mem__events | actions | PRIMARY |
| mem__events | action_logs | policyId |
| mem__events | action_logs | ts |
+-------------------+-----------------------------------+--------------------+
Indexes that have had no IO
- 49. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.49
ps_helper Examples
Statement Analysis
+---------------------------------------------+
| Tables_in_ps_helper |
+---------------------------------------------+
| statement_analysis |
| statements_with_errors_or_warnings |
| statements_with_full_table_scans |
| statements_with_runtimes_in_95th_percentile |
| statements_with_sorting |
| statements_with_temp_tables |
+---------------------------------------------+
Normalized Statement
Overview
Statements by Usage
- 50. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.50
statement_analysis
mysql> select * from statement_analysis limit 1G
*************************** 1. row ***************************
query: SELECT * FROM ( SELECT `digest ... ` ,
`sum_no_index_used` AS ...
full_scan: *
exec_count: 99
err_count: 0
warn_count: 0
total_latency: 59.47 s
max_latency: 2.47 s
avg_latency: 600.74 ms
lock_latency: 69.14 ms
...
Statistics on Normalize Statements, sorted by highest latency
- 51. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.51
statement_analysis cont...
mysql> select * from statement_analysis limit 1G
*************************** 1. row ***************************
...
rows_sent: 36136
rows_sent_avg: 365
rows_scanned: 1520720
tmp_tables: 693
tmp_disk_tables: 297
rows_sorted: 38566
sort_merge_passes: 0
digest: 50761c6a1818824328745d8a136b9ed6
Statistics on Normalize Statements, sorted by highest latency
- 52. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.52
statement_with_full_table_scans
mysql> select * from statements_with_full_table_scans limit 1G
*************************** 1. row ***************************
query: SELECT COUNT ( * ) FROM `INFOR ... NE = ? AND
`SUPPORT` IN (...)
exec_count: 1206
no_index_used_count: 1206
no_good_index_used_count: 0
no_index_used_pct: 100
rows_sent: 1206
rows_examined: 10854
digest: 491ee7143ca1d98f36c24d7eb6d25272
Statements that scan tables, sorted by pct of scans and exec count
- 53. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.53
statement_with_temp_tables
mysql> select * from statements_with_temp_tables limit 1G
*************************** 1. row ***************************
query: SELECT * FROM `schema_object_o ... MA` ,
`information_schema` ...
exec_count: 1
memory_tmp_tables: 192
disk_tmp_tables: 33
avg_tmp_tables_per_query: 192
tmp_tables_to_disk_pct: 17
digest: c463377d1d2bce2b32cf0fe84d414669
Statements that use temporary tables, sorted by most disk tables
- 54. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.54
ps_helper Examples
Statement Analysis SPs
analyze_statement_digest()
dump_thread_stack()
Analyze Specific Statement
Analyze Specific Thread
- 55. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.55
analyze_statement_digest()
Analyzes live traffic looking for a certain statement
Captures statistics on each statement if finds matching the digest
Returns a report of the statistics
- A summary
- A break down for the longest running example
- An EXPLAIN (if the statement is not truncated)
Overview
- 56. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.56
analyze_statement_digest()
in_digest – The statement digest to analyze
in_runtime – How long to run analysis for
in_interval – How often to snapshot data
in_start_fresh – Whether to truncate all P_S data first
in_auto_enable – Whether to auto enabled required consumers
Parameters
- 57. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.57
analyze_statement_digest()
mysql> call analyze_statement_digest('6134e9d6f25eb8e6cddf11f6938f202a', 60, 0.1, true, true);
+--------------------+
| SUMMARY STATISTICS |
+--------------------+
| SUMMARY STATISTICS |
+--------------------+
1 row in set (59.93 sec)
+------------+-----------+-----------+-----------+---------------+------------+------------+
| executions | exec_time | lock_time | rows_sent | rows_examined | tmp_tables | full_scans |
+------------+-----------+-----------+-----------+---------------+------------+------------+
| 360 | 1.41 s | 138.39 ms | 720 | 0 | 0 | 0 |
+------------+-----------+-----------+-----------+---------------+------------+------------+
1 row in set (59.93 sec)
+--------------------------------+-------+-----------+
| event_name | count | latency |
+--------------------------------+-------+-----------+
| stage/sql/Sending data | 14 | 1.10 s |
| stage/sql/update | 198 | 667.34 ms |
| stage/sql/removing tmp table | 9 | 240.48 ms |
...
- 58. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.58
analyze_statement_digest()
+---------------------------+
| LONGEST RUNNING STATEMENT |
+---------------------------+
| LONGEST RUNNING STATEMENT |
+---------------------------+
1 row in set (59.95 sec)
+-----------+-----------+-----------+-----------+---------------+------------+-----------+
| thread_id | exec_time | lock_time | rows_sent | rows_examined | tmp_tables | full_scan |
+-----------+-----------+-----------+-----------+---------------+------------+-----------+
| 23277 | 77.64 ms | 97.00 us | 2 | 0 | 0 | 0 |
+-----------+-----------+-----------+-----------+---------------+------------+-----------+
1 row in set (59.95 sec)
+-------------------------------------------------------------------------------------------+
| sql_text |
+-------------------------------------------------------------------------------------------+
| /* mem dbpool.default */ insert into `mem__quan`.`normalized_statements_by_server_by_schema`
(firstSeen, lastSeen, normalized_statement_id, `schema`, server, id) values (1378983745000,
1378998600000, x'808ACEB88FC4B45BC47C4FE9D86C7F26', 'mem', '753c939e-1b99-11e3-b6d4-
bc761a1f4f2f', x'7CFE81F98B1E3FE0895D1AA2C39B326D') ON DUPLICATE KEY UPDATE ...
+-------------------------------------------------------------------------------------------+
- 59. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.59
analyze_statement_digest()
+--------------------------------+----------+
| event_name | latency |
+--------------------------------+----------+
| stage/sql/init | 10.41 ms |
| stage/sql/checking permissions | 2.35 us |
| stage/sql/Opening tables | 10.32 us |
| stage/sql/init | 13.53 us |
| stage/sql/System lock | 3.66 us |
| stage/sql/update | 61.74 ms |
| stage/sql/end | 1.11 us |
| stage/sql/query end | 1.31 us |
| stage/sql/closing tables | 5.44 ms |
| stage/sql/freeing items | 23.31 us |
+--------------------------------+----------+
10 rows in set (59.99 sec)
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| 1 | SIMPLE | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
- 60. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.60
dump_thread_stack()
Monitors a specific thread
Captures as much information about it's activity as is possible
Returns .dot formatted file, that can graph the event hierarchy
Overview
- 61. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.61
dump_thread_stack()
in_thread_id – The thread to analyze
in_outfile – The file to dump the data too (INTO OUTFILE)
in_max_runtime – How long to run analysis for
in_interval – How often to snapshot data
in_start_fresh – Whether to truncate all P_S data first
in_auto_setup – Whether to auto enabled required config
in_debug – Whether to also print debug info (source etc.)
Parameters
- 62. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.62
analyze_statement_digest()
mysql> call dump_thread_stack(27768, '/tmp/stack_27768.dot', 60, 0.1, true, true, true);
+------------------------------------------------+
| Info |
+------------------------------------------------+
| Data collection starting for THREAD_ID = 27768 |
+------------------------------------------------+
1 row in set (4.82 sec)
+---------------------------------------------+
| Info |
+---------------------------------------------+
| Stack trace written to /tmp/stack_27768.dot |
+---------------------------------------------+
1 row in set (60.90 sec)
+--------------------------------------------------------+
| Convert to PDF |
+--------------------------------------------------------+
| dot -Tpdf -o /tmp/stack_27768.pdf /tmp/stack_27768.dot |
+--------------------------------------------------------+
1 row in set (60.90 sec)
+--------------------------------------------------------+
| Convert to PNG |
+--------------------------------------------------------+
| dot -Tpng -o /tmp/stack_27768.png /tmp/stack_27768.dot |
+--------------------------------------------------------+
1 row in set (60.90 sec)+
- 63. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.63
analyze_statement_digest()
- 64. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.64
Program Agenda
A peak at what is coming in 5.7...
- 65. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.65
A peak at what is coming in 5.7...
THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL
PRODUCT DIRECTION. IT IS INTENDED FOR INFORMATION
PURPOSES ONLY, AND MAY NOT BE INCORPORATED INTO ANY
CONTRACT. IT IS NOT A COMMITMENT TO DELIVER ANY MATERIAL,
CODE, OR FUNCTIONALITY, AND SHOULD NOT BE RELIED UPON IN
MAKING PURCHASING DECISIONS. THE DEVELOPMENT, RELEASE,
AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED
FOR ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION
OF ORACLE.
- 66. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.66
New in 5.7.2!
§ Memory Instrumentation
§ Replication Configuration / Status (SHOW SLAVE STATUS
replacements)
§ Sub-Statement Instrumentation (Stored Programs and their
statements)
- 67. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.67
memory_global_by_current_allocated
mysql> select * from memory_global_by_current_allocated;
+----------------------------------------+---------------+---------------+-------------------+------------+-------------+----------------+
| event_name | current_count | current_alloc | current_avg_alloc | high_count | high_alloc | high_avg_alloc |
+----------------------------------------+---------------+---------------+-------------------+------------+-------------+----------------+
| memory/mysys/KEY_CACHE | 3 | 8.00 MiB | 2.67 MiB | 3 | 8.00 MiB | 2.67 MiB |
| memory/sql/Query_cache | 1 | 1008.00 KiB | 1008.00 KiB | 1 | 1008.00 KiB | 1008.00 KiB |
| memory/sql/TABLE_SHARE::mem_root | 505 | 739.88 KiB | 1.46 KiB | 505 | 739.88 KiB | 1.46 KiB |
| memory/sql/TABLE | 459 | 612.96 KiB | 1.33 KiB | 490 | 968.63 KiB | 1.98 KiB |
| memory/sql/sp_head::main_mem_root | 32 | 334.97 KiB | 10.47 KiB | 32 | 334.97 KiB | 10.47 KiB |
| memory/mysys/lf_dynarray | 124 | 305.16 KiB | 2.46 KiB | 124 | 305.16 KiB | 2.46 KiB |
| memory/mysys/array_buffer | 146 | 275.92 KiB | 1.89 KiB | 158 | 285.92 KiB | 1.81 KiB |
| memory/sql/Filesort_buffer::sort_keys | 1 | 255.89 KiB | 255.89 KiB | 1 | 255.89 KiB | 255.89 KiB |
| memory/myisam/MI_INFO | 9 | 69.98 KiB | 7.78 KiB | 12 | 73.24 KiB | 6.10 KiB |
| memory/mysys/IO_CACHE | 3 | 64.01 KiB | 21.34 KiB | 4 | 144.00 KiB | 36.00 KiB |
| memory/sql/errmsgs | 1 | 62.43 KiB | 62.43 KiB | 1 | 62.43 KiB | 62.43 KiB |
| memory/sql/thd::main_mem_root | 6 | 55.83 KiB | 9.30 KiB | 268 | 321.86 KiB | 1.20 KiB |
| memory/sql/plugin_mem_root | 8 | 43.70 KiB | 5.46 KiB | 8 | 43.70 KiB | 5.46 KiB |
| memory/mysys/lf_slist | 1249 | 39.03 KiB | 32 bytes | 1249 | 39.03 KiB | 32 bytes |
| memory/sql/tz_storage | 1 | 31.97 KiB | 31.97 KiB | 1 | 31.97 KiB | 31.97 KiB |
| memory/myisam/MYISAM_SHARE | 9 | 28.91 KiB | 3.21 KiB | 12 | 37.84 KiB | 3.15 KiB |
| memory/mysys/TREE | 3 | 23.91 KiB | 7.97 KiB | 3 | 23.91 KiB | 7.97 KiB |
Memory allocations globally by allocation type