Karwin Software Solutions LLC, profile picture
Uploaded byKarwin Software Solutions LLC
6,759 views

Survey of Percona Toolkit

The document summarizes the Percona Toolkit, which contains free and open source command-line tools for MySQL based on Percona's experience developing best practices. Some of the most popular tools are pt-summary, pt-mysql-summary, pt-stalk, pt-archiver, and pt-query-digest, which allow users to summarize MySQL servers, analyze queries from logs, and check for issues. The toolkit can be installed via package repositories or by downloading individual tools.

Embed presentation

Downloaded 177 times
1 / 105
2 / 105
3 / 105
4 / 105
5 / 105
6 / 105
7 / 105
8 / 105
9 / 105
10 / 105
11 / 105
12 / 105
13 / 105
14 / 105
15 / 105
16 / 105
17 / 105
18 / 105
19 / 105
20 / 105
21 / 105
22 / 105
23 / 105
24 / 105
25 / 105
26 / 105
27 / 105
28 / 105
29 / 105
30 / 105
31 / 105
32 / 105
33 / 105
34 / 105
35 / 105
36 / 105
37 / 105
38 / 105
39 / 105
40 / 105
41 / 105
42 / 105
43 / 105
44 / 105
45 / 105
46 / 105
47 / 105
48 / 105
49 / 105
50 / 105
51 / 105
52 / 105
53 / 105
54 / 105
55 / 105
56 / 105
57 / 105
58 / 105
59 / 105
60 / 105
61 / 105
62 / 105
63 / 105
64 / 105
65 / 105
66 / 105
67 / 105
68 / 105
69 / 105
70 / 105
71 / 105
72 / 105
73 / 105
74 / 105
75 / 105
76 / 105
77 / 105
78 / 105
79 / 105
80 / 105
81 / 105
82 / 105
83 / 105
84 / 105
85 / 105
86 / 105
87 / 105
88 / 105
89 / 105
90 / 105
91 / 105
92 / 105
93 / 105
94 / 105
95 / 105
96 / 105
97 / 105
98 / 105
99 / 105
100 / 105
101 / 105
102 / 105
103 / 105
104 / 105
105 / 105
Survey of Percona Toolkit: Command-line Tools for MySQL Bill Karwin, Percona Inc. Tuesday, December 4, 2012
Percona Toolkit • Free, open source tools based on Percona’s experience developing best practices for repetitive or complex MySQL operations. • Verify master and replica data consistency. • Efficiently archive rows. • Find duplicate indexes. • Summarize MySQL servers. • Analyze queries from logs and tcpdump. • Collect diagnostic information when problems occur. www.percona.com Tuesday, December 4, 2012
Installing www.percona.com Tuesday, December 4, 2012
Installing Percona Toolkit • Percona package repos for RHEL/CentOS or Debian/Ubuntu: $ sudo yum install percona-toolkit $ sudo dpkg install percona-toolkit • http://www.percona.com/software/repositories • Other download options available for RPM, DEB, tarball, or individual tools. • http://www.percona.com/doc/percona-toolkit/ installation.html www.percona.com Tuesday, December 4, 2012
Top Nine Popular Tools • pt-summary • pt-duplicate-key- checker • pt-mysql-summary • pt-table-checksum • pt-stalk • pt-table-sync • pt-archiver • pt-online-schema- • pt-query-digest change www.percona.com Tuesday, December 4, 2012
pt-summary http://www.percona.com/doc/percona-toolkit/pt-summary.html www.percona.com Tuesday, December 4, 2012
pt-summary • Summarize system information in a nice way. • Useful to verify operating system configuration, inspect many system attributes quickly. www.percona.com Tuesday, December 4, 2012
pt-summary $ pt-summary # Percona Toolkit System Summary Report ###################### Date | 2011-09-30 17:06:44 UTC (local TZ: PDT -0700) Hostname | huey.karwin.percona.com Uptime | 7:45, 1 user, load average: 0.04, 0.01, 0.00 System | innotek GmbH; VirtualBox; v1.2 () Service Tag | 0 Platform | Linux Release | CentOS release 5.6 (Final) Kernel | 2.6.18-238.19.1.el5 Architecture | CPU = 64-bit, OS = 64-bit Threading | NPTL 2.5 Compiler | GNU CC version 4.1.2 20080704 (Red Hat 4.1.2-51). SELinux | Enforcing Virtualized | VirtualBox # Processor ################################################## Processors | physical = 1, cores = 0, virtual = 1, hyperthreading = no Speeds | 1x2844.667 Models | 1xIntel(R) Core(TM) i7 CPU M 640 @ 2.80GHz Caches | 1x6144 KB www.percona.com Tuesday, December 4, 2012
pt-summary # Memory ##################################################### Total | 497.86M Free | 44.36M Used | physical = 453.50M, swap = 0.00k, virtual = 453.50M Buffers | 53.18M Caches | 260.91M Dirty | 48 kB UsedRSS | 113.4M Swappiness | vm.swappiness = 60 DirtyPolicy | vm.dirty_ratio = 40, vm.dirty_background_ratio = 10 DirtyStatus | vm.dirty_bytes = 0, vm.dirty_background_bytes = 0 www.percona.com Tuesday, December 4, 2012
pt-summary # Mounted Filesystems ######################################## Filesystem Size Used Type Opts Mountpoint /dev/mapper/VolGroup00-LogVol00 15G 14% ext3 rw / /dev/sda1 99M 21% ext3 rw /boot tmpfs 249M 0% tmpfs rw /dev/shm # Disk Schedulers And Queue Size ############################# hdc | [cfq] 128 sda | [cfq] 128 # Disk Partioning ############################################ Device Type Start End Size ============ ==== ========== ========== ================== /dev/sda Disk 17179869184 /dev/sda1 Part 1 13 98703360 /dev/sda2 Part 14 2088 17059230720 # Kernel Inode State ######################################### dentry-state | 35813 33772 45 0 0 0 file-nr | 510 0 49646 inode-nr | 29137 75 # LVM Volumes ################################################ LV VG Attr LSize Origin Snap% Move Log Copy% Convert LogVol00 VolGroup00 -wi-ao 14.88G LogVol01 VolGroup00 -wi-ao 1.00G www.percona.com Tuesday, December 4, 2012
pt-summary # Network Config ############################################# Controller | Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02) Controller | Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02) FIN Timeout | net.ipv4.tcp_fin_timeout = 60 Port Range | net.ipv4.ip_local_port_range = 32768 61000 # Interface Statistics ####################################### interface rx_bytes rx_packets rx_errors tx_bytes tx_packets tx_errors ========= ========= ========== ========== ========== ========== ========== lo 7000 60 0 7000 60 0 eth0 1250000 17500 0 3000000 15000 0 eth1 9000000 12500 0 600000 8000 0 # Network Connections ######################################## Connections from remote IP addresses 192.168.56.1 1 Connections to local IP addresses 192.168.56.111 1 Connections to top 10 local ports 22 1 States of connections ESTABLISHED 1 LISTEN 5 www.percona.com Tuesday, December 4, 2012
pt-summary # Top Processes ############################################## PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1 root 15 0 10372 688 572 S 0.0 0.1 0:00.44 init 2 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/0 3 root 34 19 0 0 0 S 0.0 0.0 0:00.20 ksoftirqd/0 4 root RT -5 0 0 0 S 0.0 0.0 0:00.03 watchdog/0 5 root 10 -5 0 0 0 S 0.0 0.0 0:07.58 events/0 6 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khelper 11 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kthread 15 root 10 -5 0 0 0 S 0.0 0.0 0:00.11 kblockd/0 16 root 20 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid # Simplified and fuzzy rounded vmstat (wait please) ########## procs ---swap-- -----io---- ---system---- --------cpu-------- r b si so bi bo ir cs us sy il wa st 2 0 0 0 10 8 1000 30 0 3 97 0 0 0 0 0 0 0 0 1000 30 0 2 98 0 0 0 0 0 0 0 0 1000 30 0 4 96 0 0 0 0 0 0 0 0 1000 30 0 2 98 0 0 0 0 0 0 0 0 1000 35 0 2 98 0 0 # The End #################################################### www.percona.com Tuesday, December 4, 2012
pt-mysql-summary http://www.percona.com/doc/percona-toolkit/pt-mysql-summary.html www.percona.com Tuesday, December 4, 2012
pt-mysql-summary • Summarize MySQL information in a nice way. • See current status at a glance. • Uses live information from running instance, because it could be different from /etc/my.cnf. • Organizes information in a consistent order, so you know where to find it. www.percona.com Tuesday, December 4, 2012
pt-mysql-summary $ pt-mysql-summary # Percona Toolkit MySQL Summary Report ####################### System time | 2011-09-30 17:57:07 UTC (local TZ: PDT -0700) # Instances ################################################## Port Data Directory Socket ===== ========================== ====== /var/lib/mysql # Report On Port 3306 ######################################## User | root@localhost Time | 2011-09-30 10:57:07 (PDT) Hostname | huey.karwin.percona.com Version | 5.1.58-community-log MySQL Built On | unknown-linux-gnu x86_64 Started | 2011-09-30 10:25 (up 0+00:31:18) Databases | 5 Datadir | /var/lib/mysql/ Processes | 1 connected, 1 running Replication | Is not a slave, has 0 slaves connected Pidfile | /var/lib/mysql/huey.karwin.percona.com.pid (exists) www.percona.com Tuesday, December 4, 2012
pt-mysql-summary # Processlist ################################################ Command COUNT(*) Working SUM(Time) MAX(Time) ------------------------------ -------- ------- --------- --------- Binlog Dump 4 4 1000000 350000 Query 1 1 0 0 Sleep 30 0 45 5 User COUNT(*) Working SUM(Time) MAX(Time) ------------------------------ -------- ------- --------- --------- appuser 29 0 0 0 repl 1 1 70000 70000 Host COUNT(*) Working SUM(Time) MAX(Time) ------------------------------ -------- ------- --------- --------- 192.168.56.127 29 11 0 0 192.168.56.128 1 1 70000 70000 db COUNT(*) Working SUM(Time) MAX(Time) ------------------------------ -------- ------- --------- --------- shopsite 29 0 0 0 NULL 1 1 100000 100000 www.percona.com Tuesday, December 4, 2012
pt-mysql-summary # Status Counters (Wait 10 Seconds) ########################## Variable Per day Per second 10 secs Bytes_received 150000000 1750 90 Bytes_sent 3000000 35 1500 . . . Handler_read_rnd_next 30000 30 Handler_write 2250000 25 30 . . . Queries 60000 2 Questions 20000 2 Select_scan 500 Sort_rows 175 Sort_scan 45 Table_locks_immediate 4000 Threads_created 450 Uptime 90000 1 1 www.percona.com Tuesday, December 4, 2012
pt-mysql-summary # Table cache ################################################ Size | 4 Usage | 100% # Key Percona Server features ################################ Table & Index Stats | Not Supported Multiple I/O Threads | Enabled Corruption Resilient | Not Supported Durable Replication | Not Supported Import InnoDB Tables | Not Supported Fast Server Restarts | Not Supported Enhanced Logging | Not Supported Replica Perf Logging | Not Supported Response Time Hist. | Not Supported Smooth Flushing | Not Supported HandlerSocket NoSQL | Not Supported Fast Maatkit Hashes | Unknown # Query cache ################################################ query_cache_type | ON Size | 0.0k Usage | 0% HitToInsertRatio | 0% www.percona.com Tuesday, December 4, 2012
pt-mysql-summary # Schema ##################################################### Would you like to mysqldump -d the schema and analyze it? y/n Y There are 5 databases. Would you like to dump all, or just one? Type the name of the database, or press Enter to dump all of them. sakila Database Tables Views SPs Trigs Funcs FKs Partn {chosen} 16 7 3 3 22 Database InnoDB MyISAM {chosen} 15 8 Database BTREE FULLTEXT {chosen} 63 1 st vt d t y d e s m c i b mi ai a e e e n e e h n l am rn t x a c u t d a t o le cy e t r i m i r b ls hi t m u it an i a m na rt m l i tm e n p t Database === === === === === === === === === === === === === === {chosen} 26 45 15 19 4 4 1 7 3 1 2 1 2 1 www.percona.com Tuesday, December 4, 2012
pt-mysql-summary # Noteworthy Technologies #################################### Full Text Indexing | Yes Geospatial Types | No Foreign Keys | Yes Partitioning | No SSL | No Explicit LOCK TABLES | No Delayed Insert | No XA Transactions | No NDB Cluster | No Prepared Statements | No www.percona.com Tuesday, December 4, 2012
pt-mysql-summary # InnoDB ##################################################### Version | 1.0.17 Buffer Pool Size | 16.0M Buffer Pool Fill | 45% Buffer Pool Dirty | 0% File Per Table | ON Page Size | 16k Log File Size | 2 * 5M = 10.0M Log Buffer Size | 8M Flush Method | O_DIRECT Flush Log At Commit | 2 XA Support | ON Checksums | ON Doublewrite | ON R/W I/O Threads | 4 4 I/O Capacity | 200 Thread Concurrency | 0 Concurrency Tickets | 500 Commit Concurrency | 0 Txn Isolation Level | REPEATABLE-READ Adaptive Flushing | ON Adaptive Checkpoint | 0 Checkpoint Age | 0k www.percona.com Tuesday, December 4, 2012
pt-mysql-summary InnoDB Queue | 0 queries inside InnoDB, 0 queries in queue Oldest Transaction | 0 Seconds History List Len | 6 Read Views | 1 Undo Log Entries | 0 transactions, 0 total undo, 0 max undo Pending I/O Reads | 0 buf pool reads, 0 normal AIO, 0 ibuf AIO, 0 preads Pending I/O Writes | 0 buf pool (0 LRU, 0 flush list, 0 page); 0 AIO, 0 sync, 0 log IO (0 log, 0 chkp); 0 pwrites Pending I/O Flushes | 0 buf pool, 0 log Transaction States | 1xnot started www.percona.com Tuesday, December 4, 2012
pt-mysql-summary # MyISAM ##################################################### Key Cache | 16.0k Pct Used | 20% Unflushed | 0% # Security ################################################### Users | 4 users, 0 anon, 0 w/o pw, 0 old pw Old Passwords | OFF # Binary Logging ############################################# Binlogs | 0 Zero-Sized | 0 Total Size | 0.0k binlog_format | STATEMENT expire_logs_days | 7 sync_binlog | 0 server_id | 0 binlog_do_db | binlog_ignore_db | www.percona.com Tuesday, December 4, 2012
pt-mysql-summary # Noteworthy Variables ####################################### Auto-Inc Incr/Offset | 1/1 default_storage_engine | 0 flush_time | 0 init_connect | 0 init_file | 0 sql_mode | 0 join_buffer_size | 128k sort_buffer_size | 64k read_buffer_size | 256k read_rnd_buffer_size | 256k bulk_insert_buffer | 0k max_heap_table_size | 16M tmp_table_size | 16M max_allowed_packet | 1M thread_stack | 256k log | OFF log_error | /var/lib/mysql/huey.karwin.percona.com.err log_warnings | 1 log_slow_queries | ON log_queries_not_using_indexes | OFF log_slave_updates | OFF www.percona.com Tuesday, December 4, 2012
pt-mysql-summary # Configuration File ######################################### Config File | Cannot autodetect, trying common locations Config File | /etc/my.cnf [client] port = 3306 [mysqld] skip-name-resolve skip-slave-start expire-logs-days = 7 slow-query-log = 1 long-query-time = 20000 default-storage-engine = innodb innodb_buffer_pool_size = 16M . . . # The End #################################################### www.percona.com Tuesday, December 4, 2012
pt-stalk http://www.percona.com/doc/percona-toolkit/pt-stalk.html www.percona.com Tuesday, December 4, 2012
pt-stalk • Wait for a problem to occur, then gather forensic data about MySQL and the system. • Good when you want to diagnose a problem, but you don’t know when it happens. www.percona.com Tuesday, December 4, 2012
What Data is Collected? 2012_11_30_17_49_13-df 2012_11_30_17_49_13-sysctl 2012_11_30_17_49_13-disk-space 2012_11_30_17_49_13-top 2012_11_30_17_49_13-diskstats 2012_11_30_17_49_13-trigger 2012_11_30_17_49_13-hostname 2012_11_30_17_49_13-variables 2012_11_30_17_49_13-innodbstatus1 2012_11_30_17_49_13-vmstat 2012_11_30_17_49_13-innodbstatus2 2012_11_30_17_49_13-vmstat-overall 2012_11_30_17_49_13-interrupts 2012_11_30_17_49_13-lsof 2012_11_30_17_49_13-meminfo 2012_11_30_17_49_13-mutex-status1 2012_11_30_17_49_13-mutex-status2 2012_11_30_17_49_13-mysqladmin 2012_11_30_17_49_13-netstat 2012_11_30_17_49_13-netstat_s 2012_11_30_17_49_13-opentables1 2012_11_30_17_49_13-opentables2 2012_11_30_17_49_13-output 2012_11_30_17_49_13-pmap 2012_11_30_17_49_13-processlist 2012_11_30_17_49_13-procstat 2012_11_30_17_49_13-procvmstat 2012_11_30_17_49_13-ps 2012_11_30_17_49_13-slabinfo www.percona.com Tuesday, December 4, 2012
Ad Hoc Usage $ pt-stalk --no-stalk 2012_11_30_17_49_13 Starting /usr/bin/pt-stalk --function=status -- variable=Threads_running --threshold=25 --match= --cycles=0 --interval=0 --iterations=1 --run-time=30 --sleep=0 --dest=/var/lib/pt-stalk --prefix= --notify-by-email= --log=/var/log/pt-stalk.log --pid=/var/run/pt- stalk.pid --plugin= 2012_11_30_17_49_13 Not stalking; collect triggered immediately 2012_11_30_17_49_13 Collect triggered 2012_11_30_17_49_13 Collector PID 2865 2012_11_30_17_49_13 Waiting up to 90 seconds for collectors to finish... 2012_11_30_17_50_43 Killing collector 2865 2012_11_30_17_50_43 Exiting because no more iterations 2012_11_30_17_50_43 /usr/bin/pt-stalk exit status 0 www.percona.com Tuesday, December 4, 2012
Background Usage • Run pt-stalk run as a daemon and let it wait. $ pt-stalk --daemonize ...event options... $ tail -f /var/log/pt-stalk.log 2012_11_30_20_37_30 Check results: Threads_running=1, matched=no, cycles_true=0 2012_11_30_20_37_31 Check results: Threads_running=1, matched=no, cycles_true=0 2012_11_30_20_37_32 Check results: Threads_running=1, matched=no, cycles_true=0 2012_11_30_20_37_33 Check results: Threads_running=1, matched=no, cycles_true=0 ... www.percona.com Tuesday, December 4, 2012
Function, Variable & Threshold • Function: the information source that pt-stalk polls for the trigger event. • status: SHOW GLOBAL STATUS • processlist: SHOW PROCESSLIST • filename: you can write a custom shell script www.percona.com Tuesday, December 4, 2012
Function, Variable & Threshold • Variable: what to watch in the information source. • status: watch the named status variable. • processlist: watch the named column. www.percona.com Tuesday, December 4, 2012
Function, Variable & Threshold • Threshold: if the watched value is greater than this threshold, a collect is triggered. • status: compare to the value of the given variable. • processlist: compare to the count of how many processes show the “match” value in the watched column. www.percona.com Tuesday, December 4, 2012
Example: Watch Status • Watch status and collect information when Threads_running is 25 or more. $ pt-stalk --function status ! --variable Threads_running ! --threshold 25 www.percona.com Tuesday, December 4, 2012
Example: Watch Processlist • Watch processlist and collect information when there are 10 or more processes with State=statistics $ pt-stalk --function processlist ! --variable State ! --match statistics ! --threshold 10 www.percona.com Tuesday, December 4, 2012
Example: Watch Custom Script • Watch processlist and collect information when there are 10 or more processes with State=statistics $ cat > purge_not_working.sh trg_plugin() { mysql $EXT_ARGV -E -e "SHOW ENGINE INNODB STATUS" | grep "^History list length" | awk '{print $4}' } $ pt-stalk --function purge_not_working.sh ! --threshold 200 www.percona.com Tuesday, December 4, 2012
pt-archiver http://www.percona.com/doc/percona-toolkit/pt-archiver.html www.percona.com Tuesday, December 4, 2012
pt-archiver • Archive rows from a MySQL table into another table or a file. • Works incrementally on chunks of rows. • Deletes data from source safely. www.percona.com Tuesday, December 4, 2012
Move Data • Copy data from one MySQL instance to another, then delete from the source: $ mysqldump -h huey -d imdb keyword | mysql -h dewey test $ pt-archiver --progress 10000 --source h=huey,D=imdb,t=keyword --dest h=dewey,D=test --where "1=1" --limit 1000 --commit-each TIME ELAPSED COUNT 2012-12-03T01:37:36 0 0 2012-12-03T01:37:43 6 10000 . . . www.percona.com Tuesday, December 4, 2012
Copy Data • Copy data from one MySQL instance to another, but do not delete data from the source: $ pt-archiver --progress 10000 --no-delete --source h=localhost,D=imdb,t=keyword --file '%t.csv' --where "1=1" --limit 1000 --commit-each TIME ELAPSED COUNT 2012-12-03T01:28:27 0 0 2012-12-03T01:28:28 0 10000 2012-12-03T01:28:28 0 20000 . . . www.percona.com Tuesday, December 4, 2012
Purge Data • Delete orphan rows (slowly): $ pt-archiver --progress 10000 --purge --source h=huey,D=imdb,t=person_info --where 'NOT EXISTS(SELECT * FROM name WHERE id=person_info.person_id)' www.percona.com Tuesday, December 4, 2012
Limitations • Destination table must exist. • Archiving related data across tables is awkward. • You can use WHERE with subqueries, but not JOIN. www.percona.com Tuesday, December 4, 2012
pt-query-digest http://www.percona.com/doc/percona-toolkit/pt-query-digest.html www.percona.com Tuesday, December 4, 2012
pt-query-digest • Analyze query execution logs and generate a query report, filter, replay, or transform queries. • If you learn only one tool in Percona Toolkit, make it this one! • Capture all traffic in the slow query log... temporarily. mysql> SET GLOBAL long_query_time=0; . . . wait for traffic . . . mysql> SET GLOBAL long_query_time=10; www.percona.com Tuesday, December 4, 2012
Report Output (1) $ pt-query-digest /var/lib/mysql/mysql-slow.log # 20.3s user time, 160ms system time, 29.92M rss, 2.34G vsz # Current date: Mon Aug 15 15:49:53 2011 # Hostname: huey.percona.com # Files: shopsite-slow.log # Overall: 88.68k total, 229 unique, 26.98 QPS, 245.51x concurrency ______ # Time range: 2011-08-15 16:00:43 to 16:55:30 # Attribute total min max avg 95% stddev median # ============ ======= ======= ======= ======= ======= ======= ======= # Exec time 806989s 2s 160s 9s 30s 11s 3s # Lock time 8s 21us 9ms 90us 159us 81us 76us # Rows sent 2.35M 0 368.61k 27.84 49.17 1.95k 0.99 # Rows examine 2.74G 0 737.23k 32.44k 101.89k 41.45k 11.91k # Query size 37.37M 42 16.77k 441.84 719.66 221.12 400.73 www.percona.com Tuesday, December 4, 2012
Report Output (2) # Profile # Rank Query ID Response time Calls R/Call Apdx V/M Item # ==== ================== ================= ===== ======== ==== ===== ==== # 1 0x2C28E6666E1DB80F 521215.0518 64.6% 19450 26.7977 0.01 2.34 SELECT campaign_user # 2 0xBAC856B3ED9D6303 145125.9331 18.0% 43282 3.3530 0.44 0.45 SELECT package_object # 3 0x39997372657D28E2 16694.7209 2.1% 1705 9.7916 0.06 9.10 SELECT plug_form # 4 0x3523ACB26E4C481A 14598.2371 1.8% 4740 3.0798 0.43 0.54 SELECT article_slideshow # 5 0xA69DF0D16A7026B2 12565.8977 1.6% 1316 9.5486 0.03 1.48 SELECT campaign_user # 6 0xB8356E351A6FFD21 12116.0409 1.5% 995 12.1769 0.02 1.58 SELECT category # 7 0x8F72E45EC91BC0F9 11491.9428 1.4% 3436 3.3446 0.39 0.66 SELECT package_page # 8 0x601559979824AADB 8302.8338 1.0% 2324 3.5726 0.41 0.44 SELECT template_item # 9 0xCEB19656E4165CFD 5189.4078 0.6% 678 7.6540 0.05 1.31 SELECT article # 10 0xCE5EE218C3751804 4890.2081 0.6% 1094 4.4700 0.27 1.18 SELECT article_resources www.percona.com Tuesday, December 4, 2012
Report Output (3) # Query 1: 6.77 QPS, 181.54x concurrency, ID 0x2C28E6666E1DB80F at byte 37195306 # This item is included in the report because it matches --limit. # Scores: Apdex = 0.01 [1.0], V/M = 2.34 # Query_time sparkline: | _^| # Time range: 2011-08-15 16:00:43 to 16:48:34 # Attribute pct total min max avg 95% stddev median # ============ === ======= ======= ======= ======= ======= ======= ======= # Count 21 19450 # Exec time 64 521215s 2s 55s 27s 40s 8s 26s # Lock time 29 2s 31us 9ms 120us 185us 112us 108us # Rows sent 0 18.99k 1 1 1 1 0 1 # Rows examine 70 1.94G 102.83k 105.55k 104.70k 101.89k 0 101.89k # Query size 19 7.22M 377 424 389.04 420.77 18.23 381.65 # String: # Databases shopsite # Users appuser # Query_time distribution # 1us # 10us # 100us # 1ms # 10ms # 100ms # 1s # # 10s+ ################################################################ www.percona.com Tuesday, December 4, 2012
Report Output (4) # Tables # SHOW TABLE STATUS FROM `shopsite` LIKE 'campaign_user'G # SHOW CREATE TABLE `shopsite`.`campaign_user`G # EXPLAIN /*!50100 PARTITIONS*/ SELECT COUNT(distinct email) AS count FROM campaign_user WHERE campaign_id = '1' AND misc_2 IS NULLG www.percona.com Tuesday, December 4, 2012
Query Review • Save each query type seen to a table. $ pt-query-digest --review h=dewey,D=percona,t=query_review /var/lib/mysql/mysql-slow.log • You can add notes to each query type. • When you analyze next week’s log, the report excludes previously reviewed queries. www.percona.com Tuesday, December 4, 2012
Other Options • Filtering queries. • Grouping queries. • Reading other sources of queries. • Including query EXPLAIN reports. • Recording query history for reviews, trending. • Community tools for browsing and visualizing query review & query history data: • https://github.com/kormoc/Query-Digest-UI • https://github.com/box/Anemometer www.percona.com Tuesday, December 4, 2012
Cautions • Processing large logs can be resource-intensive. • Copy logs to another server to avoid overloading your production site. • Reports contain real queries! • Can expose sensitive information. www.percona.com Tuesday, December 4, 2012
pt-duplicate-key-checker http://www.percona.com/doc/percona-toolkit/pt-duplicate-key-checker.html www.percona.com Tuesday, December 4, 2012
pt-duplicate-key-checker • Find duplicate indexes and foreign keys on MySQL tables. • MySQL permits you to create redundant keys. • Nearly every database has some. • Output is a series of ALTER TABLE statements ready to drop or reform duplicate indexes. www.percona.com Tuesday, December 4, 2012
pt-duplicate-key-checker $ pt-duplicate-key-checker # tezt.media_pictures ############################################################# # subject_node_id is a left-prefix of INDEX # Key definitions: # KEY `subject_node_id` (`subject_node_id`) # KEY `INDEX` USING BTREE (`subject_node_id`,`frame_id`,`file_id`,`source_id`), # Column types: #! `subject_node_id` int(11) unsigned default null #! `frame_id` smallint(6) unsigned not null #! `file_id` int(11) unsigned not null #! `source_id` int(11) not null # To remove this duplicate index, execute: ALTER TABLE `tezt`.`media_pictures` DROP INDEX `subject_node_id`; www.percona.com Tuesday, December 4, 2012
pt-duplicate-key-checker # Db.system_transaction ############################################################# # FOREIGN KEY A (`table_id`) REFERENCES `Db`.`table` (`id`) is a duplicate of FOREIGN KEY B (`table_id`) REFERENCES `Db`.`table` (`id`) # Key definitions: # CONSTRAINT `A` FOREIGN KEY (`table_id`) REFERENCES `table` (`id`) # CONSTRAINT `B` FOREIGN KEY (`table_id`) REFERENCES `table` (`id`) # Column types: #! `table_id` bigint(20) default null # To remove this duplicate foreign key, execute: ALTER TABLE `Db`.`system_transaction` DROP FOREIGN KEY `A`; # MySQL uses the A index for this foreign key constraint www.percona.com Tuesday, December 4, 2012
pt-table-checksum http://www.percona.com/doc/percona-toolkit/pt-table-checksum.html www.percona.com Tuesday, December 4, 2012
Data Drift • MySQL slaves may not be perfect replicas. • Non-deterministic statements. • Out-of-band changes directly on the slave. • Slave may lag and fail to keep up. • No built-in checking. • Are you using a slave for backups or reporting? www.percona.com Tuesday, December 4, 2012
pt-table-checksum • Perform an online replication consistency check, or checksum MySQL tables efficiently. • This is the solution to detect data drift. • Works by calculating checksums against “chunks” of rows. • The calculation propagates to slaves. www.percona.com Tuesday, December 4, 2012
Visualize This master slave replication www.percona.com Tuesday, December 4, 2012
Visualize This master slave replication www.percona.com Tuesday, December 4, 2012
Visualize This master slave replication www.percona.com Tuesday, December 4, 2012
Visualize This master slave replication www.percona.com Tuesday, December 4, 2012
Visualize This master slave replication www.percona.com Tuesday, December 4, 2012
Visualize This master slave replication www.percona.com Tuesday, December 4, 2012
Visualize This master slave replication www.percona.com Tuesday, December 4, 2012
Example $ pt-table-checksum TS ERRORS DIFFS ROWS CHUNKS SKIPPED TIME TABLE 12-01T11:00:13 0 0 633135 7 0 3.814 imdb.aka_name 12-01T11:00:15 0 0 290859 1 0 1.682 imdb.aka_title Checksumming imdb.cast_info: 24% 01:34 remain Checksumming imdb.cast_info: 48% 01:03 remain Checksumming imdb.cast_info: 75% 00:28 remain 12-01T11:02:13 0 0 22187768 163 0 118.059 imdb.cast_info 12-01T11:02:25 0 0 2406561 20 0 12.292 imdb.char_name 12-01T11:02:25 0 0 4 1 0 0.123 imdb.comp_cast_type 12-01T11:02:27 0 0 241457 1 0 1.291 imdb.company_name 12-01T11:02:27 0 0 4 1 0 0.033 imdb.company_type 12-01T11:02:27 0 0 97304 1 0 0.492 imdb.complete_cast 12-01T11:02:27 0 0 113 1 0 0.079 imdb.info_type 12-01T11:02:28 0 0 87520 1 0 0.367 imdb.keyword 12-01T11:02:28 0 0 7 1 0 0.027 imdb.kind_type 12-01T11:02:28 0 0 18 1 0 0.030 imdb.link_type 12-01T11:02:37 0 0 1965016 15 0 9.142 imdb.movie_companies Checksumming imdb.movie_info: 64% 00:16 remain 12-01T11:03:34 0 0 9748370 76 0 57.105 imdb.movie_info 12-01T11:03:38 0 0 934655 8 0 4.026 imdb.movie_info_idx 12-01T11:03:49 0 0 2776445 15 0 10.552 imdb.movie_keyword 12-01T11:03:52 0 0 922518 7 0 3.051 imdb.movie_link 12-01T11:04:07 0 0 2812743 25 0 15.817 imdb.name 12-01T11:04:29 0 0 2271731 22 0 21.495 imdb.person_info 12-01T11:04:29 0 0 12 1 0 0.015 imdb.role_type 12-01T11:04:39 0 0 1543719 17 0 10.189 imdb.title www.percona.com Tuesday, December 4, 2012
Let’s Break It • Delete 5% of data on the slave: mysql> DELETE FROM title WHERE RAND()*100 < 5; Query OK, 77712 rows affected (2.09 sec) www.percona.com Tuesday, December 4, 2012
Re-check $ pt-table-checksum --tables imdb.title TS ERRORS DIFFS ROWS CHUNKS SKIPPED TIME TABLE 12-03T05:04:26 0 14 1543719 16 0 10.512 imdb.title www.percona.com Tuesday, December 4, 2012
Check the Slave(s) mysql> SELECT db, tbl, SUM(this_cnt) AS total_rows, COUNT(*) AS chunks FROM percona.checksums WHERE ( master_cnt <> this_cnt OR master_crc <> this_crc OR ISNULL(master_crc) <> ISNULL(this_crc)) GROUP BY db, tbl; +------+-------+------------+--------+ | db | tbl | total_rows | chunks | +------+-------+------------+--------+ | imdb | title | 1466007 | 14 | +------+-------+------------+--------+ www.percona.com Tuesday, December 4, 2012
pt-table-sync http://www.percona.com/doc/percona-toolkit/pt-table-sync.html www.percona.com Tuesday, December 4, 2012
pt-table-sync • Synchronize MySQL table data efficiently. • This is the solution to correct data drift. www.percona.com Tuesday, December 4, 2012
Method 1: Sync Master to Slave(s) $ pt-table-sync --verbose --execute --replicate percona.checksums huey # Syncing via replication h=192.168.56.112 # DELETE REPLACE INSERT UPDATE ALGORITHM START END EXIT DATABASE.TABLE # 0 47 0 0 Chunk 05:05:46 05:05:47 2 imdb.title # 0 795 0 0 Chunk 05:05:47 05:05:49 2 imdb.title # 0 5070 0 0 Chunk 05:05:49 05:06:01 2 imdb.title # 0 6361 0 0 Chunk 05:06:01 05:06:16 2 imdb.title # 0 6867 0 0 Chunk 05:06:16 05:06:36 2 imdb.title # 0 7297 0 0 Chunk 05:06:36 05:06:55 2 imdb.title # 0 7504 0 0 Chunk 05:06:55 05:07:13 2 imdb.title # 0 7688 0 0 Chunk 05:07:13 05:07:34 2 imdb.title # 0 7346 0 0 Chunk 05:07:34 05:07:52 2 imdb.title # 0 7065 0 0 Chunk 05:07:52 05:08:10 2 imdb.title # 0 6937 0 0 Chunk 05:08:10 05:08:27 2 imdb.title # 0 6695 0 0 Chunk 05:08:27 05:08:43 2 imdb.title # 0 6765 0 0 Chunk 05:08:43 05:09:00 2 imdb.title # 0 1275 0 0 Chunk 05:09:00 05:09:04 2 imdb.title www.percona.com Tuesday, December 4, 2012
Method 2: Sync Slave to Master $ pt-table-sync --verbose --execute --sync-to-master h=dewey,D=imdb,t=title # Syncing D=imdb,P=5528,h=127.0.0.1,p=...,t=title,u=root # DELETE REPLACE INSERT UPDATE ALGORITHM START END EXIT DATABASE.TABLE # 0 23097 0 0 Chunk 16:07:21 16:08:21 2 imdb.title www.percona.com Tuesday, December 4, 2012
Method 3: Sync Two Hosts •pt-table-sync won’t let you clobber a slave by syncing it to some host other than its master. $ pt-table-sync --verbose --execute h=huey d=dewey --tables imdb.title Can't make changes on h=dewey because it's a slave. See the documentation section 'REPLICATION SAFETY' for solutions to this problem. at /usr/bin/pt- table-sync line 10642. www.percona.com Tuesday, December 4, 2012
Method 3: Sync Two Hosts •Now let’s try again, after a RESET SLAVE. $ pt-table-sync --verbose --execute h=huey h=dewey --tables imdb.title # Syncing h=dewey # DELETE REPLACE INSERT UPDATE ALGORITHM START END EXIT DATABASE.TABLE # 0 0 30867 0 Chunk 13:33:27 13:35:28 2 imdb.title www.percona.com Tuesday, December 4, 2012
pt-online-schema-change http://www.percona.com/doc/percona-toolkit/pt-online-schema-change.html www.percona.com Tuesday, December 4, 2012
pt-online-schema-change • Perform online, non-blocking ALTER TABLE. • Captures concurrent updates to a table while restructuring. • Some risks and caveats exist; please read the manual and test carefully. www.percona.com Tuesday, December 4, 2012
How MySQL Does ALTER TABLE • Lock the table. • Make a new, empty the table like the original. • Modify the columns of the new empty table. • Copy all rows of data from original to new table. • Swap the old and new tables. • Unlock the tables & drop the original. www.percona.com Tuesday, December 4, 2012
How pt-osc Does ALTER TABLE • Lock the table. • Make a new, empty the table like the original. • Modify the columns of the new empty table. • Copy all rows of data from original to new table. • Iterate over the table in chunks, in primary key order. • Use triggers to capture ongoing changes in the original, and apply them to the new table. • Swap the tables, then drop the original. • Unlock the tables. www.percona.com Tuesday, December 4, 2012
Visualize This cast_info www.percona.com Tuesday, December 4, 2012
Visualize This cast_info cast_info new www.percona.com Tuesday, December 4, 2012
Visualize This cast_info cast_info new www.percona.com Tuesday, December 4, 2012
Visualize This cast_info after cast_info trigger new www.percona.com Tuesday, December 4, 2012
Visualize This cast_info after cast_info trigger new www.percona.com Tuesday, December 4, 2012
Visualize This cast_info after cast_info trigger new www.percona.com Tuesday, December 4, 2012
Visualize This cast_info after cast_info trigger new www.percona.com Tuesday, December 4, 2012
Visualize This cast_info after cast_info trigger new www.percona.com Tuesday, December 4, 2012
Visualize This cast_info after cast_info trigger new www.percona.com Tuesday, December 4, 2012
Visualize This cast_info after cast_info trigger new www.percona.com Tuesday, December 4, 2012
Visualize This cast_info cast_info old www.percona.com Tuesday, December 4, 2012
Visualize This cast_info cast_info × old www.percona.com Tuesday, December 4, 2012
Example $ pt-online-schema-change --alter "ADD COLUMN Dummy INT NOT NULL" h=localhost,D=imdb,t=cast_info Exiting without altering `imdb`.`cast_info` because neither --dry-run nor -- execute was specified. Please read the tool's documentation carefully before using this tool. www.percona.com Tuesday, December 4, 2012
Example $ pt-online-schema-change --alter "ADD COLUMN Dummy INT NOT NULL" h=localhost,D=imdb,t=cast_info --execute Altering `imdb`.`cast_info`... Creating new table... Created new table imdb._cast_info_new OK. Altering new table... Altered `imdb`.`_cast_info_new` OK. Creating triggers... Created triggers OK. Copying approximately 22545051 rows... Copying `imdb`.`cast_info`: 10% 04:05 remain Copying `imdb`.`cast_info`: 19% 04:07 remain Copying `imdb`.`cast_info`: 28% 03:44 remain Copying `imdb`.`cast_info`: 37% 03:16 remain Copying `imdb`.`cast_info`: 47% 02:47 remain Copying `imdb`.`cast_info`: 56% 02:18 remain Copying `imdb`.`cast_info`: 64% 01:53 remain Copying `imdb`.`cast_info`: 73% 01:28 remain Copying `imdb`.`cast_info`: 82% 00:55 remain Copying `imdb`.`cast_info`: 91% 00:26 remain Copied rows OK. Swapping tables... Swapped original and new tables OK. Dropping old table... Dropped old table `imdb`.`_cast_info_old` OK. Dropping triggers... Dropped triggers OK. Successfully altered `imdb`.`cast_info`. www.percona.com Tuesday, December 4, 2012
Self-Adjusting • Copies rows in “chunks” which are sized dynamically by default. • The tool throttles itself back if load increases too much or if any replication slaves are lagging. • The tool tries to set its lock timeouts to let applications be more likely to succeed. www.percona.com Tuesday, December 4, 2012
Limitations • You can’t alter a table that already has triggers. • You can’t add a column that is NOT NULL without also declaring a DEFAULT value. • You can’t use replication filters. • If the table is referenced by any foreign keys, you must choose a method to resolve them. www.percona.com Tuesday, December 4, 2012
Why Shouldn’t I Use This? • Is your table small enough that ALTER is already quick enough? • Is your change already very quick, for example DROP KEY in InnoDB plugin? • Will pt-online-schema-change cause the change to take too long or increase the load too much? • Are you using MySQL 5.6, which supports online schema changes natively? www.percona.com Tuesday, December 4, 2012
Cautions • Execute pt-online-schema-change against a test instance first. • Back up your data before you execute a change like this (and verify the backup is viable). www.percona.com Tuesday, December 4, 2012
Top Nine Popular Tools • pt-summary • pt-duplicate-key- checker • pt-mysql-summary • pt-table-checksum • pt-stalk • pt-table-sync • pt-archiver • pt-online-schema- • pt-query-digest change www.percona.com Tuesday, December 4, 2012
Resources • Product site: http://www.percona.com/software/percona- toolkit • Percona support: http://www.percona.com/mysql-support • Community support: https://groups.google.com/forum/?fromgroups#! forum/percona-discussion • Bug tracker: https://bugs.launchpad.net/percona-toolkit www.percona.com Tuesday, December 4, 2012
Webinars • pt-stalk http://www.percona.com/webinars/2011-09-06- diagnosing-intermittent-mysql-problems • pt-table-checksum http://www.percona.com/webinars/2012-01-18- verifying-replication-integrity-with-percona-toolkit • pt-online-schema-change http://www.percona.com/webinars/2012-05-02- zero-downtime-schema-changes-in-mysql www.percona.com Tuesday, December 4, 2012
There Are Many More Tools pt-align pt-heartbeat pt-sift pt-config-diff pt-index-usage pt-slave-delay pt-deadlock-logger pt-ioprofile pt-slave-find pt-diskstats pt-kill pt-slave-restart pt-fifo-split pt-mext pt-table-usage pt-find pt-pmp pt-upgrade pt-fingerprint pt-query-advisor pt-variable-advisor pt-fk-error-logger pt-show-grants pt-visual-explain www.percona.com Tuesday, December 4, 2012
Percona Training for MySQL Senior Industry Experts In-Person and Online Classes Custom Onsite Training http://percona.com/training www.percona.com Tuesday, December 4, 2012
Visit Our Sponsors diamond platinum exhibitors and the rest! www.percona.com Tuesday, December 4, 2012
SQL Antipatterns http://www.pragprog.com/titles/bksqla/ www.percona.com Tuesday, December 4, 2012
License and Copyright Copyright 2012 Bill Karwin www.slideshare.net/billkarwin Released under a Creative Commons 3.0 License: http://creativecommons.org/licenses/by-nc-nd/3.0/ You are free to share - to copy, distribute and transmit this work, under the following conditions: Attribution. Noncommercial. No Derivative Works. You must attribute this work You may not use this work You may not alter, transform, to Bill Karwin. for commercial purposes. or build upon this work. www.percona.com Tuesday, December 4, 2012

More Related Content

PDF
How to Design Indexes, Really
byKarwin Software Solutions LLC
 
PDF
Deep dive into PostgreSQL statistics.
byAlexey Lesovsky
 
PDF
MySQL Index Cookbook
byMYXPLAIN
 
PDF
More mastering the art of indexing
byYoshinori Matsunobu
 
PDF
InnoDB Internal
bymysqlops
 
PPTX
Kafka replication apachecon_2013
byJun Rao
 
PDF
Understanding Presto - Presto meetup @ Tokyo #1
bySadayuki Furuhashi
 
PDF
MySQL: Indexing for Better Performance
byjkeriaki
 
How to Design Indexes, Really
byKarwin Software Solutions LLC
 
Deep dive into PostgreSQL statistics.
byAlexey Lesovsky
 
MySQL Index Cookbook
byMYXPLAIN
 
More mastering the art of indexing
byYoshinori Matsunobu
 
InnoDB Internal
bymysqlops
 
Kafka replication apachecon_2013
byJun Rao
 
Understanding Presto - Presto meetup @ Tokyo #1
bySadayuki Furuhashi
 
MySQL: Indexing for Better Performance
byjkeriaki
 

What's hot

PPT
Recovery of lost or corrupted inno db tables(mysql uc 2010)
byAleksandr Kuzminsky
 
PPTX
[135] 오픈소스 데이터베이스, 은행 서비스에 첫발을 내밀다.
byNAVER D2
 
PDF
MySQL Parallel Replication: inventory, use-case and limitations
byJean-François Gagné
 
PPTX
Oracle database performance tuning
byYogiji Creations
 
PPTX
Understanding isolation levels
byHieu Nguyen Trung
 
PPTX
MySQL Indexing - Best practices for MySQL 5.6
byMYXPLAIN
 
PPT
Oracle backup and recovery
byYogiji Creations
 
PDF
PostgreSQL Performance Tuning
byelliando dias
 
PPT
Ash masters : advanced ash analytics on Oracle
byKyle Hailey
 
PDF
MongoDB WiredTiger Internals
byNorberto Leite
 
PPTX
Oracle Database in-Memory Overivew
byMaria Colgan
 
PDF
Oracle 表格介紹
byChien Chung Shen
 
PDF
InnoDB Architecture and Performance Optimization, Peter Zaitsev
byFuenteovejuna
 
PPT
SQL Server Stored procedures
byRandy Riness @ South Puget Sound Community College
 
PPTX
Optimizing Apache Spark SQL Joins
byDatabricks
 
PDF
Tanel Poder - Performance stories from Exadata Migrations
byTanel Poder
 
PDF
MySQL 5.5 Guide to InnoDB Status
byKarwin Software Solutions LLC
 
PDF
Linux tuning to improve PostgreSQL performance
byPostgreSQL-Consulting
 
PDF
Oracle SGA 介紹
byChien Chung Shen
 
PDF
Spark Operator—Deploy, Manage and Monitor Spark clusters on Kubernetes
byDatabricks
 
Recovery of lost or corrupted inno db tables(mysql uc 2010)
byAleksandr Kuzminsky
 
[135] 오픈소스 데이터베이스, 은행 서비스에 첫발을 내밀다.
byNAVER D2
 
MySQL Parallel Replication: inventory, use-case and limitations
byJean-François Gagné
 
Oracle database performance tuning
byYogiji Creations
 
Understanding isolation levels
byHieu Nguyen Trung
 
MySQL Indexing - Best practices for MySQL 5.6
byMYXPLAIN
 
Oracle backup and recovery
byYogiji Creations
 
PostgreSQL Performance Tuning
byelliando dias
 
Ash masters : advanced ash analytics on Oracle
byKyle Hailey
 
MongoDB WiredTiger Internals
byNorberto Leite
 
Oracle Database in-Memory Overivew
byMaria Colgan
 
Oracle 表格介紹
byChien Chung Shen
 
InnoDB Architecture and Performance Optimization, Peter Zaitsev
byFuenteovejuna
 
SQL Server Stored procedures
byRandy Riness @ South Puget Sound Community College
 
Optimizing Apache Spark SQL Joins
byDatabricks
 
Tanel Poder - Performance stories from Exadata Migrations
byTanel Poder
 
MySQL 5.5 Guide to InnoDB Status
byKarwin Software Solutions LLC
 
Linux tuning to improve PostgreSQL performance
byPostgreSQL-Consulting
 
Oracle SGA 介紹
byChien Chung Shen
 
Spark Operator—Deploy, Manage and Monitor Spark clusters on Kubernetes
byDatabricks
 

Viewers also liked

PDF
Sql query patterns, optimized
byKarwin Software Solutions LLC
 
PDF
Percona toolkit
byKarwin Software Solutions LLC
 
PDF
InnoDB Locking Explained with Stick Figures
byKarwin Software Solutions LLC
 
PDF
Extensible Data Modeling
byKarwin Software Solutions LLC
 
PDF
SQL Outer Joins for Fun and Profit
byKarwin Software Solutions LLC
 
PDF
Mentor Your Indexes
byKarwin Software Solutions LLC
 
PDF
Schemadoc
byKarwin Software Solutions LLC
 
PDF
Sql Injection Myths and Fallacies
byKarwin Software Solutions LLC
 
PDF
Models for hierarchical data
byKarwin Software Solutions LLC
 
PDF
Requirements the Last Bottleneck
byKarwin Software Solutions LLC
 
PDF
Sql Antipatterns Strike Back
byKarwin Software Solutions LLC
 
PDF
Practical Object Oriented Models In Sql
byKarwin Software Solutions LLC
 
PDF
Full Text Search In PostgreSQL
byKarwin Software Solutions LLC
 
PDF
Hierarchical data models in Relational Databases
bynavicorevn
 
PDF
Trees and Hierarchies in SQL
byEduard Hildebrandt
 
KEY
Trees In The Database - Advanced data structures
byLorenzo Alberton
 
Sql query patterns, optimized
byKarwin Software Solutions LLC
 
Percona toolkit
byKarwin Software Solutions LLC
 
InnoDB Locking Explained with Stick Figures
byKarwin Software Solutions LLC
 
Extensible Data Modeling
byKarwin Software Solutions LLC
 
SQL Outer Joins for Fun and Profit
byKarwin Software Solutions LLC
 
Mentor Your Indexes
byKarwin Software Solutions LLC
 
Schemadoc
byKarwin Software Solutions LLC
 
Sql Injection Myths and Fallacies
byKarwin Software Solutions LLC
 
Models for hierarchical data
byKarwin Software Solutions LLC
 
Requirements the Last Bottleneck
byKarwin Software Solutions LLC
 
Sql Antipatterns Strike Back
byKarwin Software Solutions LLC
 
Practical Object Oriented Models In Sql
byKarwin Software Solutions LLC
 
Full Text Search In PostgreSQL
byKarwin Software Solutions LLC
 
Hierarchical data models in Relational Databases
bynavicorevn
 
Trees and Hierarchies in SQL
byEduard Hildebrandt
 
Trees In The Database - Advanced data structures
byLorenzo Alberton
 

Similar to Survey of Percona Toolkit

PDF
Loadays managing my sql with percona toolkit
byFrederic Descamps
 
PDF
Fosdem managing my sql with percona toolkit
byFrederic Descamps
 
PDF
unixtoolbox
bywensheng wei
 
ODP
Nagios Conference 2012 - Dan Wittenberg - Case Study: Scaling Nagios Core at ...
byNagios
 
PDF
KCC_Final.pdf
byOleg Sehelin
 
PDF
Jana treek 4
byJana Treek
 
PDF
Percona Toolkit for Effective MySQL Administration
byMydbops
 
PDF
2345014 unix-linux-bsd-cheat-sheets-i
byLogesh Kumar Anandhan
 
PPTX
Percona Live UK 2014 Part III
byAlkin Tezuysal
 
PPT
16aug06.ppt
byzagreb2
 
PPT
Servers and Processes: Behavior and Analysis
bydreamwidth
 
PDF
YOW2020 Linux Systems Performance
byBrendan Gregg
 
PDF
OSDC 2015: Georg Schönberger | Linux Performance Profiling and Monitoring
byNETWAYS
 
PDF
Linux Performance Profiling and Monitoring
byGeorg Schönberger
 
PDF
Unixtoolbox
byJake Shlayen
 
PDF
Unixtoolbox
byradikalzen
 
PDF
unixtoolbox.pdf
bysonipradeep
 
PDF
bysptlove
 
PDF
unixtoolbox.pdf
byqqlove2
 
PDF
byqqlove2
 
Loadays managing my sql with percona toolkit
byFrederic Descamps
 
Fosdem managing my sql with percona toolkit
byFrederic Descamps
 
unixtoolbox
bywensheng wei
 
Nagios Conference 2012 - Dan Wittenberg - Case Study: Scaling Nagios Core at ...
byNagios
 
KCC_Final.pdf
byOleg Sehelin
 
Jana treek 4
byJana Treek
 
Percona Toolkit for Effective MySQL Administration
byMydbops
 
2345014 unix-linux-bsd-cheat-sheets-i
byLogesh Kumar Anandhan
 
Percona Live UK 2014 Part III
byAlkin Tezuysal
 
16aug06.ppt
byzagreb2
 
Servers and Processes: Behavior and Analysis
bydreamwidth
 
YOW2020 Linux Systems Performance
byBrendan Gregg
 
OSDC 2015: Georg Schönberger | Linux Performance Profiling and Monitoring
byNETWAYS
 
Linux Performance Profiling and Monitoring
byGeorg Schönberger
 
Unixtoolbox
byJake Shlayen
 
Unixtoolbox
byradikalzen
 
unixtoolbox.pdf
bysonipradeep
 
bysptlove
 
unixtoolbox.pdf
byqqlove2
 
byqqlove2
 

Recently uploaded

PDF
Eredità digitale sugli smartphone: cosa resta di noi nei dispositivi mobili
bySpeck&Tech
 
PPTX
The Landscape of Computer Software Development Companies
byYash Singh
 
PPTX
Basics of Identity Access Management In mordern Infrastructure
byPrinceXavier18
 
PDF
Is It Possible to Have Wi-Fi Without an Internet Provider
bySidra Jefferi
 
PPTX
Chapter 3 Introduction to number system.pptx
byGetachewAbera9
 
PPTX
THIS IS CYBER SECURITY NOTES USED IN CLASS ON VARIOUS TOPICS USED IN CYBERSEC...
byMelissaGblinwon
 
PDF
Greetings All Students Update 3 by Mia Corp
by©LDMMIA, ©Reiki Yoga
 
PPTX
Unit-4-ARTIFICIAL NEURAL NETWORKS.pptx ANN ppt Artificial neural network
byssuserd4481a
 
PDF
Security Technologys: Access Control, Firewall, VPN
byShielaLasala
 
PDF
ElyriaSoftware — Powering the Future with Blockchain Innovation
byElyria Software
 
PDF
Decoding the DNA: The Digital Networks Act, the Open Internet, and IP interco...
byCSUC - Consorci de Serveis Universitaris de Catalunya
 
PDF
Vibe Coding vs. Spec-Driven Development [Free Meetup]
byHaim Michael
 
PDF
MuleSoft Vibes is an AI-powered assistant
byVikalp Bhalia
 
PDF
Knowing and Doing: Knowledge graphs, AI, and work
bymarainglezakis1
 
PPTX
Ethics in AI - Artificial Intelligence Fundamentals.pptx
byemenyiblessing
 
PDF
TrustArc Webinar - Looking Ahead: The 2026 Privacy Landscape
byTrustArc
 
PDF
The major tech developments for 2026 by Pluralsight, a research and training ...
byChris Skinner
 
PDF
Hybrid Cloud vs Multi-Cloud Strategy 2025
byTeleglobal International
 
PDF
API206-S: Transforming Supply Chains with Amazon Bedrock AgentCore - AWS re:I...
byChris Bingham
 
PDF
Safeguarding AI-Based Financial Infrastructure
byYasir Naveed Riaz
 
Eredità digitale sugli smartphone: cosa resta di noi nei dispositivi mobili
bySpeck&Tech
 
The Landscape of Computer Software Development Companies
byYash Singh
 
Basics of Identity Access Management In mordern Infrastructure
byPrinceXavier18
 
Is It Possible to Have Wi-Fi Without an Internet Provider
bySidra Jefferi
 
Chapter 3 Introduction to number system.pptx
byGetachewAbera9
 
THIS IS CYBER SECURITY NOTES USED IN CLASS ON VARIOUS TOPICS USED IN CYBERSEC...
byMelissaGblinwon
 
Greetings All Students Update 3 by Mia Corp
by©LDMMIA, ©Reiki Yoga
 
Unit-4-ARTIFICIAL NEURAL NETWORKS.pptx ANN ppt Artificial neural network
byssuserd4481a
 
Security Technologys: Access Control, Firewall, VPN
byShielaLasala
 
ElyriaSoftware — Powering the Future with Blockchain Innovation
byElyria Software
 
Decoding the DNA: The Digital Networks Act, the Open Internet, and IP interco...
byCSUC - Consorci de Serveis Universitaris de Catalunya
 
Vibe Coding vs. Spec-Driven Development [Free Meetup]
byHaim Michael
 
MuleSoft Vibes is an AI-powered assistant
byVikalp Bhalia
 
Knowing and Doing: Knowledge graphs, AI, and work
bymarainglezakis1
 
Ethics in AI - Artificial Intelligence Fundamentals.pptx
byemenyiblessing
 
TrustArc Webinar - Looking Ahead: The 2026 Privacy Landscape
byTrustArc
 
The major tech developments for 2026 by Pluralsight, a research and training ...
byChris Skinner
 
Hybrid Cloud vs Multi-Cloud Strategy 2025
byTeleglobal International
 
API206-S: Transforming Supply Chains with Amazon Bedrock AgentCore - AWS re:I...
byChris Bingham
 
Safeguarding AI-Based Financial Infrastructure
byYasir Naveed Riaz
 
In this document
Powered by AI

Overview of Percona Toolkit, free tools for MySQL operations like data consistency, archiving, and query analysis.

Instructions on installing Percona Toolkit on RHEL, CentOS, Debian, and Ubuntu using package managers or downloading directly.

List of the top nine popular tools in the Percona Toolkit, including pt-summary, pt-query-digest, and pt-archiver.

pt-summary tool provides system information with examples showing server attributes, uptime, memory, and processor details.

pt-mysql-summary tool summarizes MySQL server status, displaying configurations, process list, and other performance metrics.

pt-stalk collects forensic data upon problem occurrence, explaining its usage to trigger collection based on conditions.

pt-archiver archives rows safely from MySQL tables, detailing usage scenarios and limitations.

pt-query-digest analyzes query logs for performance insights, emphasizing its importance in identifying slow queries.

pt-duplicate-key-checker finds and provides solutions for duplicate indexes and foreign keys in MySQL tables.

pt-table-checksum performs consistency checks on MySQL replicas, explaining its role in maintaining database integrity.pt-table-sync synchronizes table data across MySQL databases, presenting multiple methods for syncing between servers.

pt-online-schema-change allows non-blocking table alterations, detailing processes and risks associated with its use.

A recap of additional tools in the Percona Toolkit, support resources, training, and acknowledgments.

Survey of Percona Toolkit

  • 1.
    Survey of PerconaToolkit: Command-line Tools for MySQL Bill Karwin, Percona Inc. Tuesday, December 4, 2012
  • 2.
    Percona Toolkit • Free, open source tools based on Percona’s experience developing best practices for repetitive or complex MySQL operations. • Verify master and replica data consistency. • Efficiently archive rows. • Find duplicate indexes. • Summarize MySQL servers. • Analyze queries from logs and tcpdump. • Collect diagnostic information when problems occur. www.percona.com Tuesday, December 4, 2012
  • 3.
    Installing www.percona.com Tuesday, December 4, 2012
  • 4.
    Installing Percona Toolkit • Percona package repos for RHEL/CentOS or Debian/Ubuntu: $ sudo yum install percona-toolkit $ sudo dpkg install percona-toolkit • http://www.percona.com/software/repositories • Other download options available for RPM, DEB, tarball, or individual tools. • http://www.percona.com/doc/percona-toolkit/ installation.html www.percona.com Tuesday, December 4, 2012
  • 5.
    Top Nine PopularTools • pt-summary • pt-duplicate-key- checker • pt-mysql-summary • pt-table-checksum • pt-stalk • pt-table-sync • pt-archiver • pt-online-schema- • pt-query-digest change www.percona.com Tuesday, December 4, 2012
  • 6.
    pt-summary http://www.percona.com/doc/percona-toolkit/pt-summary.html www.percona.com Tuesday, December 4, 2012
  • 7.
    pt-summary • Summarize system information in a nice way. • Useful to verify operating system configuration, inspect many system attributes quickly. www.percona.com Tuesday, December 4, 2012
  • 8.
    pt-summary $ pt-summary # Percona Toolkit System Summary Report ###################### Date | 2011-09-30 17:06:44 UTC (local TZ: PDT -0700) Hostname | huey.karwin.percona.com Uptime | 7:45, 1 user, load average: 0.04, 0.01, 0.00 System | innotek GmbH; VirtualBox; v1.2 () Service Tag | 0 Platform | Linux Release | CentOS release 5.6 (Final) Kernel | 2.6.18-238.19.1.el5 Architecture | CPU = 64-bit, OS = 64-bit Threading | NPTL 2.5 Compiler | GNU CC version 4.1.2 20080704 (Red Hat 4.1.2-51). SELinux | Enforcing Virtualized | VirtualBox # Processor ################################################## Processors | physical = 1, cores = 0, virtual = 1, hyperthreading = no Speeds | 1x2844.667 Models | 1xIntel(R) Core(TM) i7 CPU M 640 @ 2.80GHz Caches | 1x6144 KB www.percona.com Tuesday, December 4, 2012
  • 9.
    pt-summary # Memory ##################################################### Total | 497.86M Free | 44.36M Used | physical = 453.50M, swap = 0.00k, virtual = 453.50M Buffers | 53.18M Caches | 260.91M Dirty | 48 kB UsedRSS | 113.4M Swappiness | vm.swappiness = 60 DirtyPolicy | vm.dirty_ratio = 40, vm.dirty_background_ratio = 10 DirtyStatus | vm.dirty_bytes = 0, vm.dirty_background_bytes = 0 www.percona.com Tuesday, December 4, 2012
  • 10.
    pt-summary # Mounted Filesystems ######################################## Filesystem Size Used Type Opts Mountpoint /dev/mapper/VolGroup00-LogVol00 15G 14% ext3 rw / /dev/sda1 99M 21% ext3 rw /boot tmpfs 249M 0% tmpfs rw /dev/shm # Disk Schedulers And Queue Size ############################# hdc | [cfq] 128 sda | [cfq] 128 # Disk Partioning ############################################ Device Type Start End Size ============ ==== ========== ========== ================== /dev/sda Disk 17179869184 /dev/sda1 Part 1 13 98703360 /dev/sda2 Part 14 2088 17059230720 # Kernel Inode State ######################################### dentry-state | 35813 33772 45 0 0 0 file-nr | 510 0 49646 inode-nr | 29137 75 # LVM Volumes ################################################ LV VG Attr LSize Origin Snap% Move Log Copy% Convert LogVol00 VolGroup00 -wi-ao 14.88G LogVol01 VolGroup00 -wi-ao 1.00G www.percona.com Tuesday, December 4, 2012
  • 11.
    pt-summary # Network Config ############################################# Controller | Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02) Controller | Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02) FIN Timeout | net.ipv4.tcp_fin_timeout = 60 Port Range | net.ipv4.ip_local_port_range = 32768 61000 # Interface Statistics ####################################### interface rx_bytes rx_packets rx_errors tx_bytes tx_packets tx_errors ========= ========= ========== ========== ========== ========== ========== lo 7000 60 0 7000 60 0 eth0 1250000 17500 0 3000000 15000 0 eth1 9000000 12500 0 600000 8000 0 # Network Connections ######################################## Connections from remote IP addresses 192.168.56.1 1 Connections to local IP addresses 192.168.56.111 1 Connections to top 10 local ports 22 1 States of connections ESTABLISHED 1 LISTEN 5 www.percona.com Tuesday, December 4, 2012
  • 12.
    pt-summary # Top Processes ############################################## PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1 root 15 0 10372 688 572 S 0.0 0.1 0:00.44 init 2 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/0 3 root 34 19 0 0 0 S 0.0 0.0 0:00.20 ksoftirqd/0 4 root RT -5 0 0 0 S 0.0 0.0 0:00.03 watchdog/0 5 root 10 -5 0 0 0 S 0.0 0.0 0:07.58 events/0 6 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khelper 11 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kthread 15 root 10 -5 0 0 0 S 0.0 0.0 0:00.11 kblockd/0 16 root 20 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid # Simplified and fuzzy rounded vmstat (wait please) ########## procs ---swap-- -----io---- ---system---- --------cpu-------- r b si so bi bo ir cs us sy il wa st 2 0 0 0 10 8 1000 30 0 3 97 0 0 0 0 0 0 0 0 1000 30 0 2 98 0 0 0 0 0 0 0 0 1000 30 0 4 96 0 0 0 0 0 0 0 0 1000 30 0 2 98 0 0 0 0 0 0 0 0 1000 35 0 2 98 0 0 # The End #################################################### www.percona.com Tuesday, December 4, 2012
  • 13.
    pt-mysql-summary http://www.percona.com/doc/percona-toolkit/pt-mysql-summary.html www.percona.com Tuesday, December 4, 2012
  • 14.
    pt-mysql-summary • Summarize MySQL information in a nice way. • See current status at a glance. • Uses live information from running instance, because it could be different from /etc/my.cnf. • Organizes information in a consistent order, so you know where to find it. www.percona.com Tuesday, December 4, 2012
  • 15.
    pt-mysql-summary $ pt-mysql-summary # Percona Toolkit MySQL Summary Report ####################### System time | 2011-09-30 17:57:07 UTC (local TZ: PDT -0700) # Instances ################################################## Port Data Directory Socket ===== ========================== ====== /var/lib/mysql # Report On Port 3306 ######################################## User | root@localhost Time | 2011-09-30 10:57:07 (PDT) Hostname | huey.karwin.percona.com Version | 5.1.58-community-log MySQL Built On | unknown-linux-gnu x86_64 Started | 2011-09-30 10:25 (up 0+00:31:18) Databases | 5 Datadir | /var/lib/mysql/ Processes | 1 connected, 1 running Replication | Is not a slave, has 0 slaves connected Pidfile | /var/lib/mysql/huey.karwin.percona.com.pid (exists) www.percona.com Tuesday, December 4, 2012
  • 16.
    pt-mysql-summary # Processlist ################################################ Command COUNT(*) Working SUM(Time) MAX(Time) ------------------------------ -------- ------- --------- --------- Binlog Dump 4 4 1000000 350000 Query 1 1 0 0 Sleep 30 0 45 5 User COUNT(*) Working SUM(Time) MAX(Time) ------------------------------ -------- ------- --------- --------- appuser 29 0 0 0 repl 1 1 70000 70000 Host COUNT(*) Working SUM(Time) MAX(Time) ------------------------------ -------- ------- --------- --------- 192.168.56.127 29 11 0 0 192.168.56.128 1 1 70000 70000 db COUNT(*) Working SUM(Time) MAX(Time) ------------------------------ -------- ------- --------- --------- shopsite 29 0 0 0 NULL 1 1 100000 100000 www.percona.com Tuesday, December 4, 2012
  • 17.
    pt-mysql-summary # Status Counters (Wait 10 Seconds) ########################## Variable Per day Per second 10 secs Bytes_received 150000000 1750 90 Bytes_sent 3000000 35 1500 . . . Handler_read_rnd_next 30000 30 Handler_write 2250000 25 30 . . . Queries 60000 2 Questions 20000 2 Select_scan 500 Sort_rows 175 Sort_scan 45 Table_locks_immediate 4000 Threads_created 450 Uptime 90000 1 1 www.percona.com Tuesday, December 4, 2012
  • 18.
    pt-mysql-summary # Table cache ################################################ Size | 4 Usage | 100% # Key Percona Server features ################################ Table & Index Stats | Not Supported Multiple I/O Threads | Enabled Corruption Resilient | Not Supported Durable Replication | Not Supported Import InnoDB Tables | Not Supported Fast Server Restarts | Not Supported Enhanced Logging | Not Supported Replica Perf Logging | Not Supported Response Time Hist. | Not Supported Smooth Flushing | Not Supported HandlerSocket NoSQL | Not Supported Fast Maatkit Hashes | Unknown # Query cache ################################################ query_cache_type | ON Size | 0.0k Usage | 0% HitToInsertRatio | 0% www.percona.com Tuesday, December 4, 2012
  • 19.
    pt-mysql-summary # Schema ##################################################### Would you like to mysqldump -d the schema and analyze it? y/n Y There are 5 databases. Would you like to dump all, or just one? Type the name of the database, or press Enter to dump all of them. sakila Database Tables Views SPs Trigs Funcs FKs Partn {chosen} 16 7 3 3 22 Database InnoDB MyISAM {chosen} 15 8 Database BTREE FULLTEXT {chosen} 63 1 st vt d t y d e s m c i b mi ai a e e e n e e h n l am rn t x a c u t d a t o le cy e t r i m i r b ls hi t m u it an i a m na rt m l i tm e n p t Database === === === === === === === === === === === === === === {chosen} 26 45 15 19 4 4 1 7 3 1 2 1 2 1 www.percona.com Tuesday, December 4, 2012
  • 20.
    pt-mysql-summary # Noteworthy Technologies #################################### Full Text Indexing | Yes Geospatial Types | No Foreign Keys | Yes Partitioning | No SSL | No Explicit LOCK TABLES | No Delayed Insert | No XA Transactions | No NDB Cluster | No Prepared Statements | No www.percona.com Tuesday, December 4, 2012
  • 21.
    pt-mysql-summary # InnoDB ##################################################### Version | 1.0.17 Buffer Pool Size | 16.0M Buffer Pool Fill | 45% Buffer Pool Dirty | 0% File Per Table | ON Page Size | 16k Log File Size | 2 * 5M = 10.0M Log Buffer Size | 8M Flush Method | O_DIRECT Flush Log At Commit | 2 XA Support | ON Checksums | ON Doublewrite | ON R/W I/O Threads | 4 4 I/O Capacity | 200 Thread Concurrency | 0 Concurrency Tickets | 500 Commit Concurrency | 0 Txn Isolation Level | REPEATABLE-READ Adaptive Flushing | ON Adaptive Checkpoint | 0 Checkpoint Age | 0k www.percona.com Tuesday, December 4, 2012
  • 22.
    pt-mysql-summary InnoDB Queue | 0 queries inside InnoDB, 0 queries in queue Oldest Transaction | 0 Seconds History List Len | 6 Read Views | 1 Undo Log Entries | 0 transactions, 0 total undo, 0 max undo Pending I/O Reads | 0 buf pool reads, 0 normal AIO, 0 ibuf AIO, 0 preads Pending I/O Writes | 0 buf pool (0 LRU, 0 flush list, 0 page); 0 AIO, 0 sync, 0 log IO (0 log, 0 chkp); 0 pwrites Pending I/O Flushes | 0 buf pool, 0 log Transaction States | 1xnot started www.percona.com Tuesday, December 4, 2012
  • 23.
    pt-mysql-summary # MyISAM ##################################################### Key Cache | 16.0k Pct Used | 20% Unflushed | 0% # Security ################################################### Users | 4 users, 0 anon, 0 w/o pw, 0 old pw Old Passwords | OFF # Binary Logging ############################################# Binlogs | 0 Zero-Sized | 0 Total Size | 0.0k binlog_format | STATEMENT expire_logs_days | 7 sync_binlog | 0 server_id | 0 binlog_do_db | binlog_ignore_db | www.percona.com Tuesday, December 4, 2012
  • 24.
    pt-mysql-summary # Noteworthy Variables ####################################### Auto-Inc Incr/Offset | 1/1 default_storage_engine | 0 flush_time | 0 init_connect | 0 init_file | 0 sql_mode | 0 join_buffer_size | 128k sort_buffer_size | 64k read_buffer_size | 256k read_rnd_buffer_size | 256k bulk_insert_buffer | 0k max_heap_table_size | 16M tmp_table_size | 16M max_allowed_packet | 1M thread_stack | 256k log | OFF log_error | /var/lib/mysql/huey.karwin.percona.com.err log_warnings | 1 log_slow_queries | ON log_queries_not_using_indexes | OFF log_slave_updates | OFF www.percona.com Tuesday, December 4, 2012
  • 25.
    pt-mysql-summary # Configuration File ######################################### Config File | Cannot autodetect, trying common locations Config File | /etc/my.cnf [client] port = 3306 [mysqld] skip-name-resolve skip-slave-start expire-logs-days = 7 slow-query-log = 1 long-query-time = 20000 default-storage-engine = innodb innodb_buffer_pool_size = 16M . . . # The End #################################################### www.percona.com Tuesday, December 4, 2012
  • 26.
    pt-stalk http://www.percona.com/doc/percona-toolkit/pt-stalk.html www.percona.com Tuesday, December 4, 2012
  • 27.
    pt-stalk • Wait for a problem to occur, then gather forensic data about MySQL and the system. • Good when you want to diagnose a problem, but you don’t know when it happens. www.percona.com Tuesday, December 4, 2012
  • 28.
    What Data isCollected? 2012_11_30_17_49_13-df 2012_11_30_17_49_13-sysctl 2012_11_30_17_49_13-disk-space 2012_11_30_17_49_13-top 2012_11_30_17_49_13-diskstats 2012_11_30_17_49_13-trigger 2012_11_30_17_49_13-hostname 2012_11_30_17_49_13-variables 2012_11_30_17_49_13-innodbstatus1 2012_11_30_17_49_13-vmstat 2012_11_30_17_49_13-innodbstatus2 2012_11_30_17_49_13-vmstat-overall 2012_11_30_17_49_13-interrupts 2012_11_30_17_49_13-lsof 2012_11_30_17_49_13-meminfo 2012_11_30_17_49_13-mutex-status1 2012_11_30_17_49_13-mutex-status2 2012_11_30_17_49_13-mysqladmin 2012_11_30_17_49_13-netstat 2012_11_30_17_49_13-netstat_s 2012_11_30_17_49_13-opentables1 2012_11_30_17_49_13-opentables2 2012_11_30_17_49_13-output 2012_11_30_17_49_13-pmap 2012_11_30_17_49_13-processlist 2012_11_30_17_49_13-procstat 2012_11_30_17_49_13-procvmstat 2012_11_30_17_49_13-ps 2012_11_30_17_49_13-slabinfo www.percona.com Tuesday, December 4, 2012
  • 29.
    Ad Hoc Usage $ pt-stalk --no-stalk 2012_11_30_17_49_13 Starting /usr/bin/pt-stalk --function=status -- variable=Threads_running --threshold=25 --match= --cycles=0 --interval=0 --iterations=1 --run-time=30 --sleep=0 --dest=/var/lib/pt-stalk --prefix= --notify-by-email= --log=/var/log/pt-stalk.log --pid=/var/run/pt- stalk.pid --plugin= 2012_11_30_17_49_13 Not stalking; collect triggered immediately 2012_11_30_17_49_13 Collect triggered 2012_11_30_17_49_13 Collector PID 2865 2012_11_30_17_49_13 Waiting up to 90 seconds for collectors to finish... 2012_11_30_17_50_43 Killing collector 2865 2012_11_30_17_50_43 Exiting because no more iterations 2012_11_30_17_50_43 /usr/bin/pt-stalk exit status 0 www.percona.com Tuesday, December 4, 2012
  • 30.
    Background Usage • Run pt-stalk run as a daemon and let it wait. $ pt-stalk --daemonize ...event options... $ tail -f /var/log/pt-stalk.log 2012_11_30_20_37_30 Check results: Threads_running=1, matched=no, cycles_true=0 2012_11_30_20_37_31 Check results: Threads_running=1, matched=no, cycles_true=0 2012_11_30_20_37_32 Check results: Threads_running=1, matched=no, cycles_true=0 2012_11_30_20_37_33 Check results: Threads_running=1, matched=no, cycles_true=0 ... www.percona.com Tuesday, December 4, 2012
  • 31.
    Function, Variable &Threshold • Function: the information source that pt-stalk polls for the trigger event. • status: SHOW GLOBAL STATUS • processlist: SHOW PROCESSLIST • filename: you can write a custom shell script www.percona.com Tuesday, December 4, 2012
  • 32.
    Function, Variable &Threshold • Variable: what to watch in the information source. • status: watch the named status variable. • processlist: watch the named column. www.percona.com Tuesday, December 4, 2012
  • 33.
    Function, Variable &Threshold • Threshold: if the watched value is greater than this threshold, a collect is triggered. • status: compare to the value of the given variable. • processlist: compare to the count of how many processes show the “match” value in the watched column. www.percona.com Tuesday, December 4, 2012
  • 34.
    Example: Watch Status • Watch status and collect information when Threads_running is 25 or more. $ pt-stalk --function status ! --variable Threads_running ! --threshold 25 www.percona.com Tuesday, December 4, 2012
  • 35.
    Example: Watch Processlist • Watch processlist and collect information when there are 10 or more processes with State=statistics $ pt-stalk --function processlist ! --variable State ! --match statistics ! --threshold 10 www.percona.com Tuesday, December 4, 2012
  • 36.
    Example: Watch CustomScript • Watch processlist and collect information when there are 10 or more processes with State=statistics $ cat > purge_not_working.sh trg_plugin() { mysql $EXT_ARGV -E -e "SHOW ENGINE INNODB STATUS" | grep "^History list length" | awk '{print $4}' } $ pt-stalk --function purge_not_working.sh ! --threshold 200 www.percona.com Tuesday, December 4, 2012
  • 37.
    pt-archiver http://www.percona.com/doc/percona-toolkit/pt-archiver.html www.percona.com Tuesday, December 4, 2012
  • 38.
    pt-archiver • Archive rows from a MySQL table into another table or a file. • Works incrementally on chunks of rows. • Deletes data from source safely. www.percona.com Tuesday, December 4, 2012
  • 39.
    Move Data • Copy data from one MySQL instance to another, then delete from the source: $ mysqldump -h huey -d imdb keyword | mysql -h dewey test $ pt-archiver --progress 10000 --source h=huey,D=imdb,t=keyword --dest h=dewey,D=test --where "1=1" --limit 1000 --commit-each TIME ELAPSED COUNT 2012-12-03T01:37:36 0 0 2012-12-03T01:37:43 6 10000 . . . www.percona.com Tuesday, December 4, 2012
  • 40.
    Copy Data • Copy data from one MySQL instance to another, but do not delete data from the source: $ pt-archiver --progress 10000 --no-delete --source h=localhost,D=imdb,t=keyword --file '%t.csv' --where "1=1" --limit 1000 --commit-each TIME ELAPSED COUNT 2012-12-03T01:28:27 0 0 2012-12-03T01:28:28 0 10000 2012-12-03T01:28:28 0 20000 . . . www.percona.com Tuesday, December 4, 2012
  • 41.
    Purge Data • Delete orphan rows (slowly): $ pt-archiver --progress 10000 --purge --source h=huey,D=imdb,t=person_info --where 'NOT EXISTS(SELECT * FROM name WHERE id=person_info.person_id)' www.percona.com Tuesday, December 4, 2012
  • 42.
    Limitations • Destination table must exist. • Archiving related data across tables is awkward. • You can use WHERE with subqueries, but not JOIN. www.percona.com Tuesday, December 4, 2012
  • 43.
    pt-query-digest http://www.percona.com/doc/percona-toolkit/pt-query-digest.html www.percona.com Tuesday, December 4, 2012
  • 44.
    pt-query-digest • Analyze query execution logs and generate a query report, filter, replay, or transform queries. • If you learn only one tool in Percona Toolkit, make it this one! • Capture all traffic in the slow query log... temporarily. mysql> SET GLOBAL long_query_time=0; . . . wait for traffic . . . mysql> SET GLOBAL long_query_time=10; www.percona.com Tuesday, December 4, 2012
  • 45.
    Report Output (1) $ pt-query-digest /var/lib/mysql/mysql-slow.log # 20.3s user time, 160ms system time, 29.92M rss, 2.34G vsz # Current date: Mon Aug 15 15:49:53 2011 # Hostname: huey.percona.com # Files: shopsite-slow.log # Overall: 88.68k total, 229 unique, 26.98 QPS, 245.51x concurrency ______ # Time range: 2011-08-15 16:00:43 to 16:55:30 # Attribute total min max avg 95% stddev median # ============ ======= ======= ======= ======= ======= ======= ======= # Exec time 806989s 2s 160s 9s 30s 11s 3s # Lock time 8s 21us 9ms 90us 159us 81us 76us # Rows sent 2.35M 0 368.61k 27.84 49.17 1.95k 0.99 # Rows examine 2.74G 0 737.23k 32.44k 101.89k 41.45k 11.91k # Query size 37.37M 42 16.77k 441.84 719.66 221.12 400.73 www.percona.com Tuesday, December 4, 2012
  • 46.
    Report Output (2) #Profile # Rank Query ID Response time Calls R/Call Apdx V/M Item # ==== ================== ================= ===== ======== ==== ===== ==== # 1 0x2C28E6666E1DB80F 521215.0518 64.6% 19450 26.7977 0.01 2.34 SELECT campaign_user # 2 0xBAC856B3ED9D6303 145125.9331 18.0% 43282 3.3530 0.44 0.45 SELECT package_object # 3 0x39997372657D28E2 16694.7209 2.1% 1705 9.7916 0.06 9.10 SELECT plug_form # 4 0x3523ACB26E4C481A 14598.2371 1.8% 4740 3.0798 0.43 0.54 SELECT article_slideshow # 5 0xA69DF0D16A7026B2 12565.8977 1.6% 1316 9.5486 0.03 1.48 SELECT campaign_user # 6 0xB8356E351A6FFD21 12116.0409 1.5% 995 12.1769 0.02 1.58 SELECT category # 7 0x8F72E45EC91BC0F9 11491.9428 1.4% 3436 3.3446 0.39 0.66 SELECT package_page # 8 0x601559979824AADB 8302.8338 1.0% 2324 3.5726 0.41 0.44 SELECT template_item # 9 0xCEB19656E4165CFD 5189.4078 0.6% 678 7.6540 0.05 1.31 SELECT article # 10 0xCE5EE218C3751804 4890.2081 0.6% 1094 4.4700 0.27 1.18 SELECT article_resources www.percona.com Tuesday, December 4, 2012
  • 47.
    Report Output (3) # Query 1: 6.77 QPS, 181.54x concurrency, ID 0x2C28E6666E1DB80F at byte 37195306 # This item is included in the report because it matches --limit. # Scores: Apdex = 0.01 [1.0], V/M = 2.34 # Query_time sparkline: | _^| # Time range: 2011-08-15 16:00:43 to 16:48:34 # Attribute pct total min max avg 95% stddev median # ============ === ======= ======= ======= ======= ======= ======= ======= # Count 21 19450 # Exec time 64 521215s 2s 55s 27s 40s 8s 26s # Lock time 29 2s 31us 9ms 120us 185us 112us 108us # Rows sent 0 18.99k 1 1 1 1 0 1 # Rows examine 70 1.94G 102.83k 105.55k 104.70k 101.89k 0 101.89k # Query size 19 7.22M 377 424 389.04 420.77 18.23 381.65 # String: # Databases shopsite # Users appuser # Query_time distribution # 1us # 10us # 100us # 1ms # 10ms # 100ms # 1s # # 10s+ ################################################################ www.percona.com Tuesday, December 4, 2012
  • 48.
    Report Output (4) # Tables # SHOW TABLE STATUS FROM `shopsite` LIKE 'campaign_user'G # SHOW CREATE TABLE `shopsite`.`campaign_user`G # EXPLAIN /*!50100 PARTITIONS*/ SELECT COUNT(distinct email) AS count FROM campaign_user WHERE campaign_id = '1' AND misc_2 IS NULLG www.percona.com Tuesday, December 4, 2012
  • 49.
    Query Review • Save each query type seen to a table. $ pt-query-digest --review h=dewey,D=percona,t=query_review /var/lib/mysql/mysql-slow.log • You can add notes to each query type. • When you analyze next week’s log, the report excludes previously reviewed queries. www.percona.com Tuesday, December 4, 2012
  • 50.
    Other Options • Filtering queries. • Grouping queries. • Reading other sources of queries. • Including query EXPLAIN reports. • Recording query history for reviews, trending. • Community tools for browsing and visualizing query review & query history data: • https://github.com/kormoc/Query-Digest-UI • https://github.com/box/Anemometer www.percona.com Tuesday, December 4, 2012
  • 51.
    Cautions • Processing large logs can be resource-intensive. • Copy logs to another server to avoid overloading your production site. • Reports contain real queries! • Can expose sensitive information. www.percona.com Tuesday, December 4, 2012
  • 52.
    pt-duplicate-key-checker http://www.percona.com/doc/percona-toolkit/pt-duplicate-key-checker.html www.percona.com Tuesday, December 4, 2012
  • 53.
    pt-duplicate-key-checker • Find duplicate indexes and foreign keys on MySQL tables. • MySQL permits you to create redundant keys. • Nearly every database has some. • Output is a series of ALTER TABLE statements ready to drop or reform duplicate indexes. www.percona.com Tuesday, December 4, 2012
  • 54.
    pt-duplicate-key-checker $ pt-duplicate-key-checker # tezt.media_pictures ############################################################# # subject_node_id is a left-prefix of INDEX # Key definitions: # KEY `subject_node_id` (`subject_node_id`) # KEY `INDEX` USING BTREE (`subject_node_id`,`frame_id`,`file_id`,`source_id`), # Column types: #! `subject_node_id` int(11) unsigned default null #! `frame_id` smallint(6) unsigned not null #! `file_id` int(11) unsigned not null #! `source_id` int(11) not null # To remove this duplicate index, execute: ALTER TABLE `tezt`.`media_pictures` DROP INDEX `subject_node_id`; www.percona.com Tuesday, December 4, 2012
  • 55.
    pt-duplicate-key-checker # Db.system_transaction ############################################################# # FOREIGN KEY A (`table_id`) REFERENCES `Db`.`table` (`id`) is a duplicate of FOREIGN KEY B (`table_id`) REFERENCES `Db`.`table` (`id`) # Key definitions: # CONSTRAINT `A` FOREIGN KEY (`table_id`) REFERENCES `table` (`id`) # CONSTRAINT `B` FOREIGN KEY (`table_id`) REFERENCES `table` (`id`) # Column types: #! `table_id` bigint(20) default null # To remove this duplicate foreign key, execute: ALTER TABLE `Db`.`system_transaction` DROP FOREIGN KEY `A`; # MySQL uses the A index for this foreign key constraint www.percona.com Tuesday, December 4, 2012
  • 56.
    pt-table-checksum http://www.percona.com/doc/percona-toolkit/pt-table-checksum.html www.percona.com Tuesday, December 4, 2012
  • 57.
    Data Drift • MySQL slaves may not be perfect replicas. • Non-deterministic statements. • Out-of-band changes directly on the slave. • Slave may lag and fail to keep up. • No built-in checking. • Are you using a slave for backups or reporting? www.percona.com Tuesday, December 4, 2012
  • 58.
    pt-table-checksum • Perform an online replication consistency check, or checksum MySQL tables efficiently. • This is the solution to detect data drift. • Works by calculating checksums against “chunks” of rows. • The calculation propagates to slaves. www.percona.com Tuesday, December 4, 2012
  • 59.
    Visualize This master slave replication www.percona.com Tuesday, December 4, 2012
  • 60.
    Visualize This master slave replication www.percona.com Tuesday, December 4, 2012
  • 61.
    Visualize This master slave replication www.percona.com Tuesday, December 4, 2012
  • 62.
    Visualize This master slave replication www.percona.com Tuesday, December 4, 2012
  • 63.
    Visualize This master slave replication www.percona.com Tuesday, December 4, 2012
  • 64.
    Visualize This master slave replication www.percona.com Tuesday, December 4, 2012
  • 65.
    Visualize This master slave replication www.percona.com Tuesday, December 4, 2012
  • 66.
    Example $ pt-table-checksum TS ERRORS DIFFS ROWS CHUNKS SKIPPED TIME TABLE 12-01T11:00:13 0 0 633135 7 0 3.814 imdb.aka_name 12-01T11:00:15 0 0 290859 1 0 1.682 imdb.aka_title Checksumming imdb.cast_info: 24% 01:34 remain Checksumming imdb.cast_info: 48% 01:03 remain Checksumming imdb.cast_info: 75% 00:28 remain 12-01T11:02:13 0 0 22187768 163 0 118.059 imdb.cast_info 12-01T11:02:25 0 0 2406561 20 0 12.292 imdb.char_name 12-01T11:02:25 0 0 4 1 0 0.123 imdb.comp_cast_type 12-01T11:02:27 0 0 241457 1 0 1.291 imdb.company_name 12-01T11:02:27 0 0 4 1 0 0.033 imdb.company_type 12-01T11:02:27 0 0 97304 1 0 0.492 imdb.complete_cast 12-01T11:02:27 0 0 113 1 0 0.079 imdb.info_type 12-01T11:02:28 0 0 87520 1 0 0.367 imdb.keyword 12-01T11:02:28 0 0 7 1 0 0.027 imdb.kind_type 12-01T11:02:28 0 0 18 1 0 0.030 imdb.link_type 12-01T11:02:37 0 0 1965016 15 0 9.142 imdb.movie_companies Checksumming imdb.movie_info: 64% 00:16 remain 12-01T11:03:34 0 0 9748370 76 0 57.105 imdb.movie_info 12-01T11:03:38 0 0 934655 8 0 4.026 imdb.movie_info_idx 12-01T11:03:49 0 0 2776445 15 0 10.552 imdb.movie_keyword 12-01T11:03:52 0 0 922518 7 0 3.051 imdb.movie_link 12-01T11:04:07 0 0 2812743 25 0 15.817 imdb.name 12-01T11:04:29 0 0 2271731 22 0 21.495 imdb.person_info 12-01T11:04:29 0 0 12 1 0 0.015 imdb.role_type 12-01T11:04:39 0 0 1543719 17 0 10.189 imdb.title www.percona.com Tuesday, December 4, 2012
  • 67.
    Let’s Break It • Delete 5% of data on the slave: mysql> DELETE FROM title WHERE RAND()*100 < 5; Query OK, 77712 rows affected (2.09 sec) www.percona.com Tuesday, December 4, 2012
  • 68.
    Re-check $ pt-table-checksum --tables imdb.title TS ERRORS DIFFS ROWS CHUNKS SKIPPED TIME TABLE 12-03T05:04:26 0 14 1543719 16 0 10.512 imdb.title www.percona.com Tuesday, December 4, 2012
  • 69.
    Check the Slave(s) mysql> SELECT db, tbl, SUM(this_cnt) AS total_rows, COUNT(*) AS chunks FROM percona.checksums WHERE ( master_cnt <> this_cnt OR master_crc <> this_crc OR ISNULL(master_crc) <> ISNULL(this_crc)) GROUP BY db, tbl; +------+-------+------------+--------+ | db | tbl | total_rows | chunks | +------+-------+------------+--------+ | imdb | title | 1466007 | 14 | +------+-------+------------+--------+ www.percona.com Tuesday, December 4, 2012
  • 70.
    pt-table-sync http://www.percona.com/doc/percona-toolkit/pt-table-sync.html www.percona.com Tuesday, December 4, 2012
  • 71.
    pt-table-sync • Synchronize MySQL table data efficiently. • This is the solution to correct data drift. www.percona.com Tuesday, December 4, 2012
  • 72.
    Method 1: SyncMaster to Slave(s) $ pt-table-sync --verbose --execute --replicate percona.checksums huey # Syncing via replication h=192.168.56.112 # DELETE REPLACE INSERT UPDATE ALGORITHM START END EXIT DATABASE.TABLE # 0 47 0 0 Chunk 05:05:46 05:05:47 2 imdb.title # 0 795 0 0 Chunk 05:05:47 05:05:49 2 imdb.title # 0 5070 0 0 Chunk 05:05:49 05:06:01 2 imdb.title # 0 6361 0 0 Chunk 05:06:01 05:06:16 2 imdb.title # 0 6867 0 0 Chunk 05:06:16 05:06:36 2 imdb.title # 0 7297 0 0 Chunk 05:06:36 05:06:55 2 imdb.title # 0 7504 0 0 Chunk 05:06:55 05:07:13 2 imdb.title # 0 7688 0 0 Chunk 05:07:13 05:07:34 2 imdb.title # 0 7346 0 0 Chunk 05:07:34 05:07:52 2 imdb.title # 0 7065 0 0 Chunk 05:07:52 05:08:10 2 imdb.title # 0 6937 0 0 Chunk 05:08:10 05:08:27 2 imdb.title # 0 6695 0 0 Chunk 05:08:27 05:08:43 2 imdb.title # 0 6765 0 0 Chunk 05:08:43 05:09:00 2 imdb.title # 0 1275 0 0 Chunk 05:09:00 05:09:04 2 imdb.title www.percona.com Tuesday, December 4, 2012
  • 73.
    Method 2: SyncSlave to Master $ pt-table-sync --verbose --execute --sync-to-master h=dewey,D=imdb,t=title # Syncing D=imdb,P=5528,h=127.0.0.1,p=...,t=title,u=root # DELETE REPLACE INSERT UPDATE ALGORITHM START END EXIT DATABASE.TABLE # 0 23097 0 0 Chunk 16:07:21 16:08:21 2 imdb.title www.percona.com Tuesday, December 4, 2012
  • 74.
    Method 3: SyncTwo Hosts •pt-table-sync won’t let you clobber a slave by syncing it to some host other than its master. $ pt-table-sync --verbose --execute h=huey d=dewey --tables imdb.title Can't make changes on h=dewey because it's a slave. See the documentation section 'REPLICATION SAFETY' for solutions to this problem. at /usr/bin/pt- table-sync line 10642. www.percona.com Tuesday, December 4, 2012
  • 75.
    Method 3: SyncTwo Hosts •Now let’s try again, after a RESET SLAVE. $ pt-table-sync --verbose --execute h=huey h=dewey --tables imdb.title # Syncing h=dewey # DELETE REPLACE INSERT UPDATE ALGORITHM START END EXIT DATABASE.TABLE # 0 0 30867 0 Chunk 13:33:27 13:35:28 2 imdb.title www.percona.com Tuesday, December 4, 2012
  • 76.
    pt-online-schema-change http://www.percona.com/doc/percona-toolkit/pt-online-schema-change.html www.percona.com Tuesday, December 4, 2012
  • 77.
    pt-online-schema-change • Perform online, non-blocking ALTER TABLE. • Captures concurrent updates to a table while restructuring. • Some risks and caveats exist; please read the manual and test carefully. www.percona.com Tuesday, December 4, 2012
  • 78.
    How MySQL DoesALTER TABLE • Lock the table. • Make a new, empty the table like the original. • Modify the columns of the new empty table. • Copy all rows of data from original to new table. • Swap the old and new tables. • Unlock the tables & drop the original. www.percona.com Tuesday, December 4, 2012
  • 79.
    How pt-osc DoesALTER TABLE • Lock the table. • Make a new, empty the table like the original. • Modify the columns of the new empty table. • Copy all rows of data from original to new table. • Iterate over the table in chunks, in primary key order. • Use triggers to capture ongoing changes in the original, and apply them to the new table. • Swap the tables, then drop the original. • Unlock the tables. www.percona.com Tuesday, December 4, 2012
  • 80.
    Visualize This cast_info www.percona.com Tuesday, December 4, 2012
  • 81.
    Visualize This cast_info cast_info new www.percona.com Tuesday, December 4, 2012
  • 82.
    Visualize This cast_info cast_info new www.percona.com Tuesday, December 4, 2012
  • 83.
    Visualize This cast_info after cast_info trigger new www.percona.com Tuesday, December 4, 2012
  • 84.
    Visualize This cast_info after cast_info trigger new www.percona.com Tuesday, December 4, 2012
  • 85.
    Visualize This cast_info after cast_info trigger new www.percona.com Tuesday, December 4, 2012
  • 86.
    Visualize This cast_info after cast_info trigger new www.percona.com Tuesday, December 4, 2012
  • 87.
    Visualize This cast_info after cast_info trigger new www.percona.com Tuesday, December 4, 2012
  • 88.
    Visualize This cast_info after cast_info trigger new www.percona.com Tuesday, December 4, 2012
  • 89.
    Visualize This cast_info after cast_info trigger new www.percona.com Tuesday, December 4, 2012
  • 90.
    Visualize This cast_info cast_info old www.percona.com Tuesday, December 4, 2012
  • 91.
    Visualize This cast_info cast_info × old www.percona.com Tuesday, December 4, 2012
  • 92.
    Example $ pt-online-schema-change --alter "ADD COLUMN Dummy INT NOT NULL" h=localhost,D=imdb,t=cast_info Exiting without altering `imdb`.`cast_info` because neither --dry-run nor -- execute was specified. Please read the tool's documentation carefully before using this tool. www.percona.com Tuesday, December 4, 2012
  • 93.
    Example $ pt-online-schema-change --alter "ADD COLUMN Dummy INT NOT NULL" h=localhost,D=imdb,t=cast_info --execute Altering `imdb`.`cast_info`... Creating new table... Created new table imdb._cast_info_new OK. Altering new table... Altered `imdb`.`_cast_info_new` OK. Creating triggers... Created triggers OK. Copying approximately 22545051 rows... Copying `imdb`.`cast_info`: 10% 04:05 remain Copying `imdb`.`cast_info`: 19% 04:07 remain Copying `imdb`.`cast_info`: 28% 03:44 remain Copying `imdb`.`cast_info`: 37% 03:16 remain Copying `imdb`.`cast_info`: 47% 02:47 remain Copying `imdb`.`cast_info`: 56% 02:18 remain Copying `imdb`.`cast_info`: 64% 01:53 remain Copying `imdb`.`cast_info`: 73% 01:28 remain Copying `imdb`.`cast_info`: 82% 00:55 remain Copying `imdb`.`cast_info`: 91% 00:26 remain Copied rows OK. Swapping tables... Swapped original and new tables OK. Dropping old table... Dropped old table `imdb`.`_cast_info_old` OK. Dropping triggers... Dropped triggers OK. Successfully altered `imdb`.`cast_info`. www.percona.com Tuesday, December 4, 2012
  • 94.
    Self-Adjusting • Copies rows in “chunks” which are sized dynamically by default. • The tool throttles itself back if load increases too much or if any replication slaves are lagging. • The tool tries to set its lock timeouts to let applications be more likely to succeed. www.percona.com Tuesday, December 4, 2012
  • 95.
    Limitations • You can’t alter a table that already has triggers. • You can’t add a column that is NOT NULL without also declaring a DEFAULT value. • You can’t use replication filters. • If the table is referenced by any foreign keys, you must choose a method to resolve them. www.percona.com Tuesday, December 4, 2012
  • 96.
    Why Shouldn’t IUse This? • Is your table small enough that ALTER is already quick enough? • Is your change already very quick, for example DROP KEY in InnoDB plugin? • Will pt-online-schema-change cause the change to take too long or increase the load too much? • Are you using MySQL 5.6, which supports online schema changes natively? www.percona.com Tuesday, December 4, 2012
  • 97.
    Cautions • Execute pt-online-schema-change against a test instance first. • Back up your data before you execute a change like this (and verify the backup is viable). www.percona.com Tuesday, December 4, 2012
  • 98.
    Top Nine PopularTools • pt-summary • pt-duplicate-key- checker • pt-mysql-summary • pt-table-checksum • pt-stalk • pt-table-sync • pt-archiver • pt-online-schema- • pt-query-digest change www.percona.com Tuesday, December 4, 2012
  • 99.
    Resources • Product site: http://www.percona.com/software/percona- toolkit • Percona support: http://www.percona.com/mysql-support • Community support: https://groups.google.com/forum/?fromgroups#! forum/percona-discussion • Bug tracker: https://bugs.launchpad.net/percona-toolkit www.percona.com Tuesday, December 4, 2012
  • 100.
    Webinars • pt-stalk http://www.percona.com/webinars/2011-09-06- diagnosing-intermittent-mysql-problems • pt-table-checksum http://www.percona.com/webinars/2012-01-18- verifying-replication-integrity-with-percona-toolkit • pt-online-schema-change http://www.percona.com/webinars/2012-05-02- zero-downtime-schema-changes-in-mysql www.percona.com Tuesday, December 4, 2012
  • 101.
    There Are ManyMore Tools pt-align pt-heartbeat pt-sift pt-config-diff pt-index-usage pt-slave-delay pt-deadlock-logger pt-ioprofile pt-slave-find pt-diskstats pt-kill pt-slave-restart pt-fifo-split pt-mext pt-table-usage pt-find pt-pmp pt-upgrade pt-fingerprint pt-query-advisor pt-variable-advisor pt-fk-error-logger pt-show-grants pt-visual-explain www.percona.com Tuesday, December 4, 2012
  • 102.
    Percona Training forMySQL Senior Industry Experts In-Person and Online Classes Custom Onsite Training http://percona.com/training www.percona.com Tuesday, December 4, 2012
  • 103.
    Visit Our Sponsors diamond platinum exhibitors and the rest! www.percona.com Tuesday, December 4, 2012
  • 104.
    SQL Antipatterns http://www.pragprog.com/titles/bksqla/ www.percona.com Tuesday, December 4, 2012
  • 105.
    License and Copyright Copyright 2012 Bill Karwin www.slideshare.net/billkarwin Released under a Creative Commons 3.0 License: http://creativecommons.org/licenses/by-nc-nd/3.0/ You are free to share - to copy, distribute and transmit this work, under the following conditions: Attribution. Noncommercial. No Derivative Works. You must attribute this work You may not use this work You may not alter, transform, to Bill Karwin. for commercial purposes. or build upon this work. www.percona.com Tuesday, December 4, 2012