MySQL Performance Schema in MySQL 8.0

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
MySQL Performance Schema
A great insight of MySQL server execution
Mayank Prasad
MySQL Developer
Oracle, MySQL
January 26, 2019

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
2

Program Agenda
Need and Design
Instruments and instrumentation
Statistics tables
Use cases
What’s new in MySQL 8.0 GA
1
2
3
4
5
3

Program Agenda
Need and Design
Statistics tables
Use cases
1
2
3
4
5
4

Why Performance Schema?
?
5
Session
stuck?
Hot table?
Code
contention?
Slow
application?
Too much
disk spin?
Low
throughput?
High traffic
on link?
End User
MySQL Developer
Application Developer DBA
Storage
Network
System

What it is?
“Performance Schema is a mechanism to give user
an insight of what is happening behind the scene
when MySQL server is running.”
• Introduced in MySQL 5.5
• New storage engine : Performance Schema *
• New Database : performance_schema
• Statistics stored in tables (hard coded DDLs).
• Non persistent data
• SQL user interface
6
* No application change needed to use Performance Schema.

MySQL 5.7 Performance Schema : Design
Block Diagram
MySQL
Server
Instrumentation points
(P_S hooks)
P_S Internal Buffers
P_S
Storage
Engine
Storage
Engine
Interface
Statistics
Report
Fetch
Data
SQL Query
Collect Data
Store
Data
P_S
Tables
7

Program Agenda
Need and Design
Statistics tables
Use cases
1
2
3
4
5
8

Instruments
9
• Name of monitored activity.
• Tree like structure. Separated by ‘/’.
• Left to right : More generic to more specific.
• 1200+ instruments in MySQL 8.0.
• Stored in performance_schema.setup_instruments table.
wait/io/file/myisam/log
statement/sql/select

Instruments contd…
Table setup_instruments
10
SETUP_INSTRUMENTS
NAME ENABLED TIMED
statement/sql/select YES YES
statement/sql/create_table YES NO
statement/com/Create DB NO NO
… …
stage/sql/closing tables NO NO
stage/sql/Opening tables NO NO
stage/sql/optimizing YES YES

Program Agenda
Need and Design
Statistics tables
Use cases
1
2
3
4
5
11

Statistics Tables in Performance Schema
12
SETUP
TABLES
Instruments
Actors
Objects
Consumers
*TIMERS
EVENTS
TABLES
Transactions
Statements
Stages
Waits
Idle
Error
REPLICATION
SUMMARY
VARIABLES
System
Session
Variable info
LOCK TABLES
Metadata
locks
Table Handles
DATA_LOCKS
SUMMARY
TABLES
Events
Memory
File I/O,
Table I/O,
Table locks
Socket
Connection
…
CONNECTION
Attribute
Type
INSTANCE
TABLES
Mutex
RW_locks
File
Sockets
Cond
MISC
By_global By_thread By_user/host By_account By_digest

Program Agenda
Need and Design
Statistics tables
Use cases
1
2
3
4
5
13

What does Performance Schema provide …
update performance_schema.setup_instruments set ENABLED='YES', TIMED='YES';
14
Connection 1 (Thread 24)
start transaction;
insert into test.t1 values('11');
commit;
start transaction;
commit;
start transaction;
commit;
select * from test.t1;
start transaction;
commit;
start transaction;
commit;
Two
insertsThree
inserts
Latest
Statement

What does Performance Schema provide … (cont.)
Statements Statistics
* Timer unit in table output is PICOSECOND.
EVENTS_STATEMENTS_CURRENT
THREAD_ID 24 25
EVENT_NAME
statement/sql
/select
statement/sql/
commit
TIMER_WAIT* 876585000 15998287000
SQL_TEXT
select * from
test.t1
commit
ROWS_SENT 3 0
NO_INDEX_USED 0 0
SELECT_SCAN 1 0
15
EVENTS_STATEMENTS_SUMMARY_BY_THREAD_BY_EVENT
_NAME
THREAD_ID 24 25
EVENT_NAME
statement/sql
/insert
statement/sql
/insert
COUNT_STAR 3 2
SUM_TIMER_WAIT* 35181659000 3477432000
SUM_ROWS_AFFECTED 3 2
SUM_SELECT_SCAN 0 0
SUM_NO_INDEX_USED 0 0

What does Performance Schema provide … (cont.)
Statements Statistics (cont.)
16
EVENTS_STATEMENTS_SUMMARY_GLOBAL_BY_EVENT_NAME
EVENT_NAME statement/sql/insert statement/sql/commit
COUNT_STAR 5 5
SUM_TIMER_WAIT 38659091000 65812216000
… …
SUM_ROWS_AFFECTED 5 0
… … …

Use case 1
Problem statement
17
• Multiple queries running for long on MySQL Server
• Few long running query (taking lots of time)
• No idea which one
• No idea why
• What to do ? …

Use case 1
Diagnosis
18
– THREAD_ID: 25
– EVENT_ID: 89
– EVENT_NAME: statement/sql/select
– SQL_TEXT : select bla bla bla…;
• Wait ! There’s more!
– SELECT_SCAN : 1
– NO_INDEX_USED: 1
• Aha !!
SELECT * FROM events_statements_history WHERE TIMER_WAIT > ‘X’;

Use case 2
Statements giving errors ( or warnings)
19
SELECT DIGEST_TEXT, SCHEMA_NAME, COUNT_STAR, SUM_ERRORS, SUM_WARNINGS
FROM performance_schema.events_statements_summary_by_digest
WHERE SUM_ERRORS > 0 ORDER BY SUM_ERRORS DESC limit 1G;
EVENTS_STATEMENTS_SUMMARY_BY_DIGEST
DIGEST_TEXT CREATE TEMPORARY TABLE IF NOT ... _logs` ( `id` INT8 NOT NULL )!
SCHEMA_NAME mem
COUNT_STAR 1725
SUM_ERRORS 1725
SUM_WARNINGS 0
FIRST_SEEN 2014-05-20 10:42:32
LAST_SEEN 2014-05-21 18:39:22

Use case 3
Problem Statement
20
• Multithreaded environment
• My session is stuck
• No idea why
• What to do ? …

Use case 3
• What T1 is waiting for
– Say T1 is waiting for mutex_A (column OBJECT_INSTANCE_BEGIN)
• Lets see who has taken this mutex_A
– Ok, so thread T2 is holding mutex_A (column LOCKED_BY_THREAD_ID)
• Find out what thread t2 is waiting for
• And so on…
SELECT * FROM mutex_instances WHERE OBJECT_INSTANCE_BEGIN = mutex_A;
SELECT * FROM events_waits_current WHERE THREAD_ID = T2;
Diagnosis
21
SELECT * FROM events_waits_current WHERE THREAD_ID = T1;

Event Hierarchy
Session
Transaction
*Statement
Stage
Wait
sync, lock, i/o
* Statements for non-transactions tables are not part of Transaction instrumentation.
22

Event Hierarchy
23
event_id
nesting_event_id
event_id
nesting_event_id
event_id
nesting_event_id
event_id
nesting_event_id
event_id
nesting_event_id
event_id
nesting_event_id
event_id
nesting_event_id
event_id
nesting_event_id
Transactions Statements Stages Waits

Event Hierarchy (contd..)
An example
Start transaction;
Insert into test.t1 values(‘3’);
Commit;
24

Event Hierarchy (contd..)
An example
Start transaction;
Insert into test.t1 values(‘3’);
Commit;
25

Program Agenda
Need and Design
Statistics tables
Use cases
1
2
3
4
5
26

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 27
What’s new in MySQL 5.7
2727
EnhancementsNew
Instruments
transactionsMemory
usage
Stored
programs
Prepared
statements
Metadata
locks
Connection
type InnoDB
Stages
User
variables
Replication
summary
tables
History per
session
Scalable
memory
allocation
Configurable
digest size
Reduced
memory
foot print
MySQL 5.7
Global/Session
variables/status …

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 28
What’s new in MySQL 8.0
2828
EnhancementsNew
Instruments
Error
Variable info
DATA Locks
DATA Locks
Wait
Histograms
Plugin
component
Table service
Statement
Histograms
Digest
Query
Sample
PFS Indexes
MySQL 8.0
…

Server Error Instrumentation
• New instrument : error
• Errors on running server are instrumented
• Handled and unhandled
• Aggregated across several dimensions (user, account, session etc.)
29

Server Error Instrumentation (contd)
Example
30
mysql> select * from no_table;
ERROR 1146 (42S02): Table 'test.no_table' doesn't exist
mysql> use no_database;
ERROR 1049 (42000): Unknown database 'no_database‘
Thrice
error
Once
error

Server Error Instrumentation (contd)
Error stats at runtime
EVENTS_ERRORS_SUMMARY_BY_THREAD_BY_ERROR
THREAD_ID 46 47 47
ERROR_NUMBER 1146 1146 1049
ERROR_NAME ER_NO_SUCH_TABLE ER_NO_SUCH_TABLE ER_BAD_DB_ERROR
SQL_STATE 42S02 42S02 42000
SUM_ERROR_RAISED 2 1 1
31
EVENTS_ERRORS_SUMMARY_GLOBAL_BY_ERROR
ERROR_NUMBER 1146 1049
ERROR_NAME ER_NO_SUCH_TABLE ER_BAD_DB_ERROR
SQL_STATE 42S02 42000
SUM_ERROR_RAISED 3 1

Variable info
• Gives information about the server variables.
• Source [COMMAND_LINE, COMPILED, EXPLICIT, PERSISTED etc.]
• Min/max value
• Who/When
32

Variable info (contd)
VARIABLES_INFO
VARIABLE_NAME max_connections
VARIABLE_SOURCE COMPILED
VARIABLE_PATH
MIN_VALUE 1
MAX_VALUE 100000
SET_TIME NULL
SET_USER NULL
SET_HOST NULL
33
VARIABLES_INFO
VARIABLE_NAME max_connections
VARIABLE_SOURCE PERSISTED
VARIABLE_PATH
/home/mayank/mysql-
bin/data/mysqld-auto.cnf
MIN_VALUE 1
MAX_VALUE 100000
SET_TIME 2019-01-25 16:56:22.381152
SET_USER root
SET_HOST localhost
SET PERSIST max_connections=47;
<restart server>
SELECT * FROM variables_info WHERE
VARIABLE_NAME='max_connections';
SELECT * FROM variables_info WHERE
VARIABLE_NAME='max_connections';

INDEXES on PFS tables
• Improve performance of running Performance Schema queries on MySQL server.
• Indexes are fixed. Can't be added/altered.
• Before, all rows fetched and then filtered.
• Now, rows are filtered at Performance Schema level.
34

INDEXES on PFS tables (contd)
Example
36
mysql> SELECT * FROM
performance_schema.variables_by_thread
WHERE thread_id = 49 AND
variable_name = 'time_zone';
+-----------+---------------+----------------+
| THREAD_ID | VARIABLE_NAME | VARIABLE_VALUE |
+-----------+---------------+----------------+
| 49 | time_zone | SYSTEM |
+-----------+---------------+----------------+
1 row in set (0.01 sec)
mysql> show session status
where variable_name like "%Handler_read%";
+-----------------------+-------+
| Variable_name | Value |
+-----------------------+-------+
| Handler_read_first | 0 |
| Handler_read_key | 1 |
| Handler_read_last | 0 |
| Handler_read_next | 0 |
| Handler_read_prev | 0 |
| Handler_read_rnd | 0 |
| Handler_read_rnd_next | 0 |
+-----------------------+-------+

DATA LOCKS
• Transaction A is locking row R,
• Transaction B is waiting on this very same row,
• B is effectively blocked by A.
• The instrumentation is to expose in performance_schema tables:
- which data is locked (R),
- who owns the lock (A),
- who waits for the data (B).
37

DATA LOCKS (contd)
Example
38
Session 1 (Thread Id 51)
mysql> create table test.t1 (c char(100));
Query OK, 0 rows affected (0.21 sec)
mysql> insert into t1 values ('blablabla');
Query OK, 1 row affected (0.10 sec)
mysql> SET autocommit=0;
mysql> select * from t1 for update;
+-----------+
| c |
+-----------+
| blablabla |
+-----------+

DATA LOCKS (contd)
39
Monitoring session
mysql> select ENGINE, ENGINE_TRANSACTION_ID, THREAD_ID, EVENT_ID,
OBJECT_SCHEMA, OBJECT_NAME, LOCK_TYPE, LOCK_MODE, LOCK_STATUS,
LOCK_DATA from performance_schema.data_locks;
+--------+-----------------------+-----------+----------+---------------+
| ENGINE | ENGINE_TRANSACTION_ID | THREAD_ID | EVENT_ID | OBJECT_SCHEMA |
+--------+-----------------------+-----------+----------+---------------+
| INNODB | 3610 | 51 | 23 | test |
| INNODB | 3610 | 51 | 23 | test |
| INNODB | 3610 | 51 | 23 | test |
+--------+-----------------------+-----------+----------+---------------+
-------------+-----------+-----------+-------------+------------------------+
OBJECT_NAME | LOCK_TYPE | LOCK_MODE | LOCK_STATUS | LOCK_DATA |
-------------+-----------+-----------+-------------+------------------------+
t1 | TABLE | IX | GRANTED | NULL |
t1 | RECORD | X | GRANTED | supremum pseudo-record |
t1 | RECORD | X | GRANTED | 0x000000000300 |
-----------+-----------+-------------+-------------+------------------------+

DATA LOCKS (contd)
Example
40
mysql> create table test.t1 (c char(100));
mysql> insert into t1 values ('blablabla');
Query OK, 1 row affected (0.10 sec)
mysql> SET autocommit=0;
+-----------+
| c |
+-----------+
| blablabla |
+-----------+
<It goes for wait>

DATA LOCKS (contd)
41
Monitoring session
mysql> select ENGINE, ENGINE_TRANSACTION_ID, THREAD_ID, EVENT_ID,
OBJECT_SCHEMA, OBJECT_NAME, LOCK_TYPE, LOCK_MODE, LOCK_STATUS,
LOCK_DATA from performance_schema.data_locks;
+--------+-----------------------+-----------+----------+---------------+
| ENGINE | ENGINE_TRANSACTION_ID | THREAD_ID | EVENT_ID | OBJECT_SCHEMA |
+--------+-----------------------+-----------+----------+---------------+
| INNODB | 3613 | 52 | 16 | test |
| INNODB | 3613 | 52 | 16 | test |
| INNODB | 3610 | 51 | 23 | test |
| INNODB | 3610 | 51 | 23 | test |
| INNODB | 3610 | 51 | 23 | test |
+--------+-----------------------+-----------+----------+---------------+
-------------+-----------+-----------+-------------+------------------------+
OBJECT_NAME | LOCK_TYPE | LOCK_MODE | LOCK_STATUS | LOCK_DATA |
-------------+-----------+-----------+-------------+------------------------+
t1 | RECORD | X | WAITING | 0x000000000300 |
t1 | RECORD | X | GRANTED | supremum pseudo-record |
t1 | RECORD | X | GRANTED | 0x000000000300 |
-----------+-----------+-------------+-------------+------------------------+

Statement Histogram
• Statement instrument gives
- Max/Min/Avg timings
• Categorize statements in different buckets based on time.
42

Statement Histogram (Contd)
43
EVENTS_STATEMENTS_HISTOGRAM_BY_DIGEST
SCHEMA_NAME test test test
DIGEST 50308…4b710 50308…4b710 2c88c…e0b4b
BUCKET_NUMBER 116 124 111
BUCKET_TIMER_LOW 1995262314 2884031503 1584893192
BUCKET_TIMER_HIGH 2089296130 3019951720 1659586907
BUCKET_QUANTILE 0.400000 1.000000 0.444444
40% of the queries of this digest are run
with latency lower then 2.08 ms.
100% of the queries of this digest are run
with latency lower then 3.01 ms.

Statement Histogram (Contd)
44
SCHEMA_NAME test
DIGEST 50308…4b710
COUNT_STAR 26
SUM_TIMER_WAIT 44055882000
MIN_TIMER_WAIT 1184073000
AVG_TIMER_WAIT 1694457000
MAX_TIMER_WAIT 2395033000
…
QUANTILE_95 2290867652
QUANTILE_99 2398832919
QUANTILE_999 2398832919
95% of the time query was executed less then 2.29 ms.
99% of the time query was executed less then 2.39 ms.
99.9% of the time query was executed less then 2.39 ms.

Plugin/Component TABLE service
• pfs_plugin_table : A new component service
• Two methods :
add_tables : plugin/component to add tables in performance schema
delete_tables : plugin/component to delete tables from performance schema
• An example plugin : pfs_example_plugin_employee implementation.
• An example component : pfs_example_component_population implementation.
• With above two examples, you can easily write your own tables to be exposed in PFS.
• Doxygen comment from code : Very detailed explanation with diagrams.
45

DIGEST QUERY SAMPLE
• A Sample query for every digest
• New variable : performance_schema_max_digest_sample_age
• Sample query gets updated
- After ‘performance_schema_max_digest_sample_age’ seconds.
- New query time exceeds max time till now.
46

DIGEST QUERY SAMPLE (contd)
Example
47
Client Session
mysql> select * from t1 where c1<3;
SCHEMA_NAME test
DIGEST 2c88c…e0b4b
DIGEST_TEXT SELECT * FROM `t1` WHERE `c1` < ?
QUERY_SAMPLE_TEXT select * from t1 where c1<3
QUERY_SAMPLE_SEEN 2019-01-25 14:58:52.792232
QUERY_SAMPLE_TIMER_WAIT 2231073000
Same
Digest

Instruments We have
• I/O operation
—file, table, network
• Locks
—mutex, r/w locks,
table locks, MDLs,
• Memory usage
• Connection type
• Errors
• DATA LOCKS
• Events
—transactions,
statements, stages,
waits, Idle
• Stored Programs
— Procedures,
Functions, Triggers,
Events
• User variables
• HISTOGRAMS
• Global/session variable
• Global/Session status
• Replication Summary Tables
• Individual and Historical stats
• Indexes on PFS tables
• Summarized stats by
- user
- host
- account
- thread
- global
48
100+ Tables and 1200+ Instruments

(earlier known as P_S Helper)
SYS Schema
Procedures Functions Views

Thanks you !
Questions ?

MySQL Performance Schema in MySQL 8.0

More Related Content

What's hot

Similar to MySQL Performance Schema in MySQL 8.0

Recently uploaded

MySQL Performance Schema in MySQL 8.0

Editor's Notes