The document discusses various tools from the Percona Toolkit that can be used for effective MySQL administration. It describes tools like pt-config-diff to find configuration differences, pt-query-digest to analyze MySQL queries from logs, pt-duplicate-key-checker to check for duplicate indexes, and pt-table-checksum to perform replication consistency checks. Installation instructions and usage examples are provided for some of the key tools.

1. Percona Toolkit for Effective
MySQL Administration
By
MyDBOPS Remote DBA
Team

2. ● MyDBOPS is on MySQL Consulting with core specialization on
MySQL Administration and Support.
● MyDBOPS is formed by a group of MySQL and open source
enthusiasts from TOP tech companies who have scaled TB’s of
data in MySQL.
● MyDBOPS was created with a motto of developing a Devops
model for MySQL administration. We help organisations to
scale in MySQL and implement the advance technologies in
MySQL.
About Us

3. Agenda
● Introduction
● Installation of Percona tool kit ( Linux )
● Tools
➢ pt-config-diff
➢ pt-query-digest
➢ pt-table-sync
➢ pt-summary
➢ pt-duplicate-key-checker
➢ pt-kill
➢ pt-mysql-summary
➢ pt-table-checksum
➢ pt-online-schema-change

4. INTRODUCTION
The Percona toolkit is a free, open source tool which used to monitor the
mysql server and make the complex MySQL operations easier than
normal way
Some of the operations :
● It verify master and slave data consistency.
● Find duplicate indexes.
● Summarize MySQL servers and linux server.
● Analyze queries from logs.
● It verify the config file differences.

5. Installing Percona Toolkit
Percona package repos for most of Linux Flavour.
$ sudo yum install percona-toolkit ( RPM Based server )
$ sudo dpkg install percona-toolkit ( Debian Based )
http://www.percona.com/software/repositories
Other download options available for RPM, DEB, tarball, or individual
tools.
http://www.percona.com/doc/percona-toolkit/ installation.html

6. Tools
pt-config-diff pt-query-digest pt-table-sync
pt-summary pt-duplicate-key-checker pt-kill
pt-mysql-summary pt-table-checksum pt-online-schema-change

7. pt-config-diff

8. Find The Differences

9. pt-config-diff
● It gives the differences of MySQL configuration files and server
variables. CONFIG can be a filename or a DSN. At least two
CONFIG sources must be given.
● Like standard Unix diff, there is no output if there are no differences.
Diff host1 config from SHOW VARIABLES against host2:
pt-config-diff “h=host1” “h=host2”
Diff config from [mysqld] section in my.cnf against host1 config:
pt-config-diff /etc/my.cnf h=host1
Diff the [mysqld] section of two option files:
pt-config-diff /etc/my-new.cnf /etc/my-old.cnf

10. Difference between two config files :
$ pt-config-diff /etc/my-old.cnf /etc/my-new.cnf
3 config differences
Variable /etc/my-old.cnf /etc/my-new.cnf
========================= =============== ===============
basedir /usr/local/mysql1 /usr/local/mysql2
datadir /var/lib/mysql1 /var/lib/mysql2
port 3306 3307
pt-config-diff

11. Difference between two hosts :
$ pt-config-diff “10.20.30.40,u=root,p=root” “10.20.30.41,u=root,p=root”
15 config differences
Variable Cent_Node1 khosla
========================= ========================= =========================
auto_increment_increment 3 1
back_log 750 150
binlog_cache_size 65536 32768
binlog_format ROW STATEMENT
character_sets_dir /usr/share/percona-xtr... /usr/share/mysql/chars...
datadir /data/mysql/ /var/lib/mysql/
event_scheduler OFF ON
expire_logs_days 14 0
explicit_defaults_for_... OFF ON
general_log_file /data/mysql/Cent_Node1... /var/lib/mysql/khosla.log
host_cache_size 778 628
hostname Cent_Node1 khosla
innodb_autoinc_lock_mode 2 1
innodb_buffer_pool_dum... ON OFF
innodb_buffer_pool_loa... ON OFF
pt-config-diff

12. pt-summary

13. pt-summary
● It summarizes the status and configuration of a server.
● It gives top running process and hardware informations.
● This tool works well on many types of Unix systems.
$ pt-summary
# Percona Toolkit System Summary Report ######################
Date | 2016-01-08 23:22:42 UTC (local TZ: IST +0530)
Hostname | lab1@mydbops.com
Uptime | 7:24, 4 users, load average: 0.48, 0.64, 0.58
System | Gigabyte Technology Co., Ltd.; To be filled by O.E.M.; vTo be filled by O.E.M.
(Desktop)
Service Tag | To be filled by O.E.M.
Platform | Linux
Release | Fedora release 21 (Twenty One)
Kernel | 4.1.13-100.fc21.x86_64
Architecture | CPU = 64-bit, OS = 64-bit
Threading | NPTL 2.20
Compiler | GNU CC version 4.9.2 20150212 (Red Hat 4.9.2-6).
SELinux | Enforcing
Virtualized | No virtualization detected

14. # Processor ##################################################
Processors | physical = 1, cores = 2, virtual = 2, hyperthreading = no
Speeds | 1x1695.468, 1x1698.632
Models | 2xIntel(R) Pentium(R) CPU G2030 @ 3.00GHz
Caches | 2x3072 KB
# Memory #####################################################
Total | 3.8G
Free | 1.5G
Used | physical = 1.7G, swap allocated = 3.9G, swap used = 2.2G, virtual = 3.9G
Buffers | 563.1M
Caches | 1.7G
Dirty | 76 kB
UsedRSS | 2.5G
Swappiness | 60
DirtyPolicy | 20, 10
DirtyStatus | 0, 0
pt-summary

15. Locator Size Speed Form Factor Type Type Detail
========= ======== ================= ============= ============= ===========
ChannelA-DIMM0 4096 MB 1333 MHz DIMM DDR3 Synchronous
ChannelB-DIMM0 {EMPTY} Unknown DIMM Unknown None
# Top Processes ##############################################
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2588 dhanase+ 20 0 1943588 111592 22592 S 20.0 2.8 8:06.26 gnome-shell
2925 dhanase+ 20 0 595064 20384 2332 S 6.7 0.5 10:11.08 skype
1 root 20 0 189044 4152 2148 S 0.0 0.1 0:02.40 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.02 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 0:00.20 ksoftirqd/0
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:+
7 root 20 0 0 0 0 R 0.0 0.0 0:07.49 rcu_sched
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh
9 root 20 0 0 0 0 S 0.0 0.0 0:06.04 rcuos/0
# Notable Processes ##########################################
PID OOM COMMAND
1443 -17 sshd
# The End ####################################################
pt-summary

16. pt-mysql-summary

17. pt-mysql-summary
● It summarizes the status and configuration of a MySQL database
server.
$ pt-mysql-summary
# Percona Toolkit MySQL Summary Report #######################
System time | 2016-01-08 23:22:20 UTC (local TZ: IST +0530)
# Instances ##################################################
Port Data Directory Nice OOM Socket
===== ========================== ==== === ======
3306 /var/lib/data 0 0 /tmp/mysql.sock
# MySQL Executable ###########################################
Path to executable | /usr/local/mysql/bin/mysqld
Has symbols | Yes
# Report On Port 3306 ########################################
User | root@lab1@mydbops.com
Time | 2016-01-09 04:52:19 (IST)
Hostname | lab1@mydbops.com
Version | 5.6.26 MySQL Community Server (GPL)
Built On | linux-glibc2.5 x86_64
Started | 2016-01-08 21:42 (up 0+07:09:24)
Databases | 4
Datadir | /var/lib/data/
Processes | 4 connected, 1 running
Replication | Is not a slave, has 0 slaves connected
Pidfile | /var/lib/data/localhost.pid (exists)

18. # Processlist ################################################
Command COUNT(*) Working SUM(Time) MAX(Time)
------------------------------ -------- ------- --------- ---------
Query 1 1 0 0
Sleep 3 0 60000 25000
User COUNT(*) Working SUM(Time) MAX(Time)
------------------------------ -------- ------- --------- ---------
asdfasfd 1 0 0 0
root 2 1 0 0
test 1 0 0 0
Host COUNT(*) Working SUM(Time) MAX(Time)
------------------------------ -------- ------- --------- ---------
localhost 4 1 0 0
db COUNT(*) Working SUM(Time) MAX(Time)
------------------------------ -------- ------- --------- ---------
NULL 4 1 0 0
State COUNT(*) Working SUM(Time) MAX(Time)
------------------------------ -------- ------- --------- ---------
3 0 0 0
init 1 1 0 0
pt-mysql-summary

19. # Configuration File #########################################
Config File | /etc/my.cnf
[mysqld]
basedir = /usr/local/mysql
datadir = /var/lib/data
port = 3306
server_id = 4
socket = /tmp/mysql.sock
innodb_flush_method = O_DIRECT
default-storage-engine = innodb
sql_mode = NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
# The End ####################################################
pt-mysql-summary

20. pt-query-digest

22. pt-query-digest
● It analyzes MySQL queries from slow, general, and binary log
files.
● It can also analyze queries from SHOW PROCESSLIST.
● By default, queries are grouped by fingerprint and reported in
descending order of query time (i.e. the slowest queries first).
● If no FILES are given, the tool reads STDIN.
● The optional DSN is used for certain options like --since and --
until.

23. $ pt-query-digest mysql-slow.log-20160108
# 490ms user time, 60ms system time, 26.68M rss, 215.63M vsz
# Current date: Sat Jan 2 10:27:51 2016
# Hostname: Cent_Node2
# Files: /data/log/mysql-slow.log-20160102
# Overall: 468 total, 15 unique, 0.01 QPS, 0.16x concurrency _____________
# Time range: 2016-01-01 10:00:07 to 22:55:06
# Attribute total min max avg 95% stddev median
# ============ ======= ======= ======= ======= ======= ======= =======
# Exec time 7460s 2s 369s 16s 144s 47s 3s
# Lock time 107ms 0 2ms 227us 568us 237us 185us
# Rows sent 4.14M 0 35.27k 9.07k 22.45k 11.74k 0.99
# Rows examine 1.11G 0 274.27M 2.43M 1.20M 17.07M 1.14M
# Rows affecte 7 0 1 0.01 0 0.12 0
# Bytes sent 751.05M 11 19.56M 1.60M 4.26M 2.29M 3.88k
# Query size 466.51k 6 9.31k 1020.74 1.78k 921.93 1.33k
pt-query-digest

24. # Profile
# Rank Query ID Response time Calls R/Call V/M Item
# ==== ================== =============== ===== ======== ===== ===========
# 1 0x813031B8BBC3B329 5189.9245 69.6% 154 33.7008 13... COMMIT
# 2 0x8618BCB1F7B137BF 709.1529 9.5% 3 236.3843 36.47 SELECT master.organization
master.organization_devices transaction_log.tx_master transaction_log.tx_result master.
geo_heirarchy_type master.geo_heirarchy
# 3 0xA0CA1BBEDCD1B80D 442.7764 5.9% 156 2.8383 0.16 SELECT master.
organization_attribute master.user_attribute master.organization_devices master.user_attribute
master.organization master.user master.mapping_organization_address master.address master.user
master.address master.mapping_organization_address
# MISC 0xMISC 298.8991 4.0% 16 18.6812 0.0 <9 ITEMS>
Explanations:
Column Meaning
============ ==========================================================
Rank The query's rank within the entire set of queries analyzed
Query ID The query's fingerprint
Response time The total response time, and percentage of overall total
Calls The number of times this query was executed
R/Call The mean response time per execution
V/M The Variance-to-mean ratio of response time
Item The distilled query
pt-query-digest

25. # Query 1: 0.00 QPS, 0.14x concurrency, ID 0x813031B8BBC3B329 at byte 722464
# Scores: V/M = 133.51
# Time range: 2016-07-01 11:14:29 to 21:33:15
# Attribute pct total min max avg 95% stddev median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count 32 154
# Exec time 69 5190s 2s 232s 34s 213s 67s 6s
# Lock time 0 0 0 0 0 0 0 0
# Rows sent 0 0 0 0 0 0 0 0
# Rows examine 0 0 0 0 0 0 0 0
# Rows affecte 0 0 0 0 0 0 0 0
# Bytes sent 0 1.65k 11 11 11 11 0 11
# Query size 0 924 6 6 6 6 0 6
# String:
# Databases transactio... (105/68%), collection... (10/6%)... 8 more
# Hosts 10.240.175.35 (137/88%), 10.240.213.146 (11/7%)... 1 more
# Last errno 0
# Users appuser (137/88%), np_user (11/7%)... 1 more
# Query_time distribution
# 1us
# 10us
# 1ms
# 10ms
# 1s ################################################################
# 10s+ #######################################
commitG
pt-query-digest

26. pt-query-digest
● One best tool a DBA should have.
● The queries can be visualized using Box anemometer.
● It over tcpdump too.

27. pt-duplicate-key-checker

28. pt-duplicate-key-checker
● It examines MySQL tables for duplicate or redundant indexes.
● Connection options are read from MySQL option files.
● It also gives the suggestions to drop those duplicate indexes.
● Dropping index can help you to save disk space.
● Optimizer can have better statistics to choose the right index if
duplicates were removed.

29. $ pt-duplicate-key-checker --database test
# ########################################################################
# test.t1
# ########################################################################
# name_id is a duplicate of name_idx
# Key definitions:
# KEY `name_id` (`Name`)
# KEY `name_idx` (`Name`),
# Column types:
# `name` varchar(20) default null
# To remove this duplicate index, execute:
ALTER TABLE `test`.`t1` DROP INDEX `name_id`;
# ########################################################################
# Summary of indexes
# ########################################################################
# Size Duplicate Indexes 69
# Total Duplicate Indexes 1
# Total Indexes 2
pt-duplicate-key-checker

30. pt-table-checksum

31. pt-table-checksum
● It performs an online replication consistency check by executing
checksum queries on the master and slave.
● The optional DSN specifies the master host.
Steps :
● First it checks for replication lag.
● Select datas in small chunks. ( mostly based on PK )
● Make checksum on chunk with replace into select.
● Displays the output.

32. $ pt-table-checksum u=pt_check,p=pt@check,P=3306 --set-vars innodb_lock_wait_timeout=200 --no-
check-binlog-format –databases=test --tables=t1
TS ERRORS DIFFS ROWS CHUNKS SKIPPED TIME TABLE
01-07T05:37:58 0 3 7365324 98 0 55.747 test.t1
pt-table-checksum

33. pt-table-sync

34. pt-table-sync
● Synchronizes the data between the master and slave servers.
● It is a very effective tool and it can be used along with pt-table-
checksum to make the data sync.
Steps :
● First it checks for replication lag and delay.
● Make the datas into small chunks.
● Then synchronizes the data.

35. $ pt-table-sync --sync-to-master u=pt_check,p=pt@check,P=3306,h=192.168.1.2 --databases=test
--tables=t1 --print > t1.sql
● The t1.sql contains the missing transactions from master.
● Import the sql file to slave. And then re-run the pt-table-
checksum.
$ pt-table-checksum u=pt_check,p=pt@check,P=3306 --set-vars innodb_lock_wait_timeout=200 --
no-check-binlog-format –databases=test --tables=t1
TS ERRORS DIFFS ROWS CHUNKS SKIPPED TIME TABLE
01-07T05:49:24 0 0 7365324 98 0 46.374 test.t1
● The data gets consistent.
pt-table-sync

36. pt-kill

37. pt-kill
● It kill MySQL queries that match certain criteria.
● You can specify by busy-time, match-command,interval etc.,
● It can be deamonized too.
For busy queries (Print):
$ pt-kill --busy-time 60 --print
For busy queries (Kill):
$ pt-kill --busy-time 60 --kill
For sleep process :
$ pt-kill --match-command Sleep --print

38. pt-online-schema-change

39. pt-online-schema-change
● It helps in modifying the table structure without much locking.
● Modifying,altering,adding a column and altering the table engine
can be easily done by this pt-osc.
● Support the table with foreign keys
● Support Galera Cluster too.
● 4X slower than normal alter.
● Data is copied to new table in small chunks.
Note : With latest MySQL 5.6 / 5.7 most of the DDL operation has become
online.

40. Process Flow in Online Schema Change

41. Altering Engine :
$ pt-online-schema-change --alter "ENGINE=INNODB" D=world,t=test --execute
Creating triggers...
Created triggers OK.
Copying approximately 3081121 rows...
Copying `world`.`test`: 0% 01:15:42 remain
Copying `world`.`test`: 1% 01:04:17 remain
Copying `world`.`test`: 2% 01:02:28 remain
Copying `world`.`test`: 3% 01:00:34 remain
....
...
Copying `world`.`test`: 97% 03:05 remain
Copying `world`.`test`: 97% 02:35 remain
Copying `world`.`test`: 99% 00:54 remain
Copying `world`.`test`: 99% 00:23 remain
2016-01-04T14:46:14 Copied rows OK.
2016-01-04T14:46:14 Analyzing new table...
2016-01-04T14:46:27 Swapping tables...
2016-01-04T14:46:27 Swapped original and new tables OK.
2016-01-04T14:46:27 Dropping old table...
2016-01-04T14:46:35 Dropped old table `world`.`_test_old` OK.
2016-01-04T14:46:36 Dropping triggers...
2016-01-04T14:46:36 Dropped triggers OK.
Successfully altered `world`.`test`.
pt-online-schema-change

42. How it works ?
● Create a new table with modified structure.
● Creates triggers for data changes. ( Insert/Update/delete )
● Copies data in small chunks.
● Swaps the table.
● Drop the old table and triggers.
pt-online-schema-change

43. More Tools
pt-archiver
pt-kill
pt-slave-find
pt-collect
pt-log-player
pt-slave-restart
pt-mext
pt-stalk
pt-deadlock-logger
pt-diskstats
pt-slave-delay
pt-visual-explain
pt-index-usage
Note:
Please read the manual and test it before using it on production.
pt-online-schema-change
pt-pmp
pt-fifo-split
pt-query-advisor
pt-tcp-model
pt-find
pt-trend
pt-fk-error-logger
pt-show-grants
pt-upgrade
pt-heartbeat
pt-sift
pt-variable-advisor

44. Thank You