MySQLinsanity! This document provides an overview of Stanley Huang's MySQL performance tuning experience and expertise. It begins with introductions and background on Stanley Huang. It then discusses the typical phases of MySQL performance tuning projects, including SQL tuning and RDBMS tuning. Specific tips are provided around topics like slow query logging, index usage, partitioning, and server configuration. The document concludes with an invitation for questions.
Introduction into MySQL Query Tuning for Dev[Op]sSveta Smirnova
Percona Live Online 2021 talk: https://www.percona.com/resources/videos/introduction-mysql-query-tuning-for-devops
In this talk I will show how to get started with MySQL Query Tuning. I will make a short introduction into physical table structure and demonstrate how it may influence query execution time.
Then we will discuss basic query tuning instruments and techniques, mainly EXPLAIN command with its latest variations. You will learn how to understand its output and how to rewrite queries or change table structure to achieve better performance.
Introduction into MySQL Query Tuning for Dev[Op]sSveta Smirnova
Percona Live Online 2021 talk: https://www.percona.com/resources/videos/introduction-mysql-query-tuning-for-devops
In this talk I will show how to get started with MySQL Query Tuning. I will make a short introduction into physical table structure and demonstrate how it may influence query execution time.
Then we will discuss basic query tuning instruments and techniques, mainly EXPLAIN command with its latest variations. You will learn how to understand its output and how to rewrite queries or change table structure to achieve better performance.
MySQL exposes a collection of tunable parameters and indicators that is frankly intimidating. But a poorly tuned MySQL server is a bottleneck for your PHP application scalability. This session shows how to do InnoDB tuning and read the InnoDB status report in MySQL 5.5.
How to Avoid Pitfalls in Schema Upgrade with Percona XtraDB ClusterSveta Smirnova
Percona XtraDB Cluster (PXC) is a 100% synchronized cluster in regards to DML operations. It is ensured by the optimistic locking model and ability to rollback transaction which cannot be applied on all nodes. However, DDL operations are not transactional in MySQL. This adds complexity when you need to change the schema of the database.
Changes made by DDL may affect the results of the queries. Therefore all modifications must replicate on all nodes prior to the next data access. For operations that run momentarily, it can be easily achieved, but schema changes may take hours to apply. Therefore in addition to the safest synchronous blocking schema upgrade method: TOI, - PXC supports more relaxed, though not safe, method RSU.
RSU: Rolling Schema Upgrade is advertised to be non-blocking. But you still need to take care of updates, running while you are performing such an upgrade. Surprisingly, even updates on not related tables and schema can cause RSU operation to fail.
In this talk, I will uncover nuances of PXC schema upgrades and point to details you need to take special care about.
Further Information
Schema change is a frequent task, and many do not expect any surprises with it. However, the necessity to replay the changes to all synchronized nodes adds complexity. I made a webinar on a similar topic which was recorded and available for replay. Now I have found that I share a link to the webinar to my Support customers approximately once per week. Not having a good understanding of how schema change works in the cluster leads to lockups and operation failures. This talk will provide a checklist that will help to choose the best schema change method.
Presented at Percona Live Online: https://perconaliveonline2020.sched.com/event/ePm2/how-to-avoid-pitfalls-in-schema-upgrade-with-percona-xtradb-cluster
Introduction to MySQL Query Tuning for Dev[Op]sSveta Smirnova
To get data, we query the database. MySQL does its best to return requested bytes as fast as possible. However, it needs human help to identify what is important and should be accessed in the first place.
Queries, written smartly, can significantly outperform automatically generated ones. Indexes and Optimizer statistics, not limited to the Histograms only, help to increase the speed of the query a lot.
In this session, I will demonstrate by examples of how MySQL query performance can be improved. I will focus on techniques, accessible by Developers and DevOps rather on those which are usually used by Database Administrators. In the end, I will present troubleshooting tools which will help you to identify why your queries do not perform. Then you could use the knowledge from the beginning of the session to improve them.
How to Avoid Pitfalls in Schema Upgrade with GaleraSveta Smirnova
Galera Cluster for MySQL is a 100% synchronized cluster in regards to data modification operations (DML). It is ensured by the optimistic locking model and ability to rollback a transaction, which cannot be applied on all nodes. However, schema changes (DDL operations) are not transactional in MySQL, which adds complexity when you need to perform an upgrade or change schema of the database.
Changes made by DDL may affect results of the queries. Therefore all modifications must replicate on all nodes prior next data access. For operations which run momentarily it can be easily achieved, but schema changes may take hours to apply. Therefore in addition to safest synchronous blocking schema upgrade method TOI Galera also supports more relaxed, thought not safe, method RSU.
In her talk Sveta will describe which pitfalls you can hit while performing the change using one or another method, why and how to avoid them.
Presented at MariaDB Day Brussels 0202 2020: https://mariadb.org/mariadb-day-brussels-0202-2020-provisional-schedule/
How to migrate from MySQL to MariaDB without tearsSveta Smirnova
Presented at MariaDB Server Fest 2020: https://mariadb.org/fest2020/migrate-mysql/
MariaDB is a drop-in replacement for MySQL. Initial migration is simple: start MariaDB over the old MySQL datadir.
Later your application may notice that some features work differently than with MySQL. These are MariaDB improvements, so this is good and, likely the reason you migrated.
In this session, I will focus on the differences affecting application performance and behavior. In particular, features sharing the same name, but working differently.
Percona Live 2016 (https://www.percona.com/live/data-performance-conference-2016/sessions/why-use-explain-formatjson). Although EXPLAIN FORMAT=JSON was first presented a long time ago, there still aren't many resources that explain how and why to use it. The most advertised feature is visual EXPLAIN in MySQL Workbench, but this format can do more than create nice pictures. It prints additional information that can't be found in good old tabular EXPLAIN, and can help to solve many tricky performance issues. In this session, I will not only describe which additional information we can get with the new syntax, but also provide examples showing how to use it to diagnose production issues.
Talk at "Istanbul Tech Talks" in Istanbul, April, 17, 2018. http://www.istanbultechtalks.com/
In this talk I will show how to get started with MySQL Query Tuning. I will make short introduction into physical table structure and demonstrate how it may influence query execution time. Then we will discuss basic query tuning instruments and techniques, mainly EXPLAIN command with its latest variations. You will learn how to understand its output and how to rewrite query or change table structure to achieve better performance.
Facebook and Beyond - Lessons for Brand Engagement with Social Customers Lithium
The dividends for a well developed Facebook presence will ultimately depend on marketers investing in adopting sophisticated long-term strategies for customer engagement.
MySQL exposes a collection of tunable parameters and indicators that is frankly intimidating. But a poorly tuned MySQL server is a bottleneck for your PHP application scalability. This session shows how to do InnoDB tuning and read the InnoDB status report in MySQL 5.5.
How to Avoid Pitfalls in Schema Upgrade with Percona XtraDB ClusterSveta Smirnova
Percona XtraDB Cluster (PXC) is a 100% synchronized cluster in regards to DML operations. It is ensured by the optimistic locking model and ability to rollback transaction which cannot be applied on all nodes. However, DDL operations are not transactional in MySQL. This adds complexity when you need to change the schema of the database.
Changes made by DDL may affect the results of the queries. Therefore all modifications must replicate on all nodes prior to the next data access. For operations that run momentarily, it can be easily achieved, but schema changes may take hours to apply. Therefore in addition to the safest synchronous blocking schema upgrade method: TOI, - PXC supports more relaxed, though not safe, method RSU.
RSU: Rolling Schema Upgrade is advertised to be non-blocking. But you still need to take care of updates, running while you are performing such an upgrade. Surprisingly, even updates on not related tables and schema can cause RSU operation to fail.
In this talk, I will uncover nuances of PXC schema upgrades and point to details you need to take special care about.
Further Information
Schema change is a frequent task, and many do not expect any surprises with it. However, the necessity to replay the changes to all synchronized nodes adds complexity. I made a webinar on a similar topic which was recorded and available for replay. Now I have found that I share a link to the webinar to my Support customers approximately once per week. Not having a good understanding of how schema change works in the cluster leads to lockups and operation failures. This talk will provide a checklist that will help to choose the best schema change method.
Presented at Percona Live Online: https://perconaliveonline2020.sched.com/event/ePm2/how-to-avoid-pitfalls-in-schema-upgrade-with-percona-xtradb-cluster
Introduction to MySQL Query Tuning for Dev[Op]sSveta Smirnova
To get data, we query the database. MySQL does its best to return requested bytes as fast as possible. However, it needs human help to identify what is important and should be accessed in the first place.
Queries, written smartly, can significantly outperform automatically generated ones. Indexes and Optimizer statistics, not limited to the Histograms only, help to increase the speed of the query a lot.
In this session, I will demonstrate by examples of how MySQL query performance can be improved. I will focus on techniques, accessible by Developers and DevOps rather on those which are usually used by Database Administrators. In the end, I will present troubleshooting tools which will help you to identify why your queries do not perform. Then you could use the knowledge from the beginning of the session to improve them.
How to Avoid Pitfalls in Schema Upgrade with GaleraSveta Smirnova
Galera Cluster for MySQL is a 100% synchronized cluster in regards to data modification operations (DML). It is ensured by the optimistic locking model and ability to rollback a transaction, which cannot be applied on all nodes. However, schema changes (DDL operations) are not transactional in MySQL, which adds complexity when you need to perform an upgrade or change schema of the database.
Changes made by DDL may affect results of the queries. Therefore all modifications must replicate on all nodes prior next data access. For operations which run momentarily it can be easily achieved, but schema changes may take hours to apply. Therefore in addition to safest synchronous blocking schema upgrade method TOI Galera also supports more relaxed, thought not safe, method RSU.
In her talk Sveta will describe which pitfalls you can hit while performing the change using one or another method, why and how to avoid them.
Presented at MariaDB Day Brussels 0202 2020: https://mariadb.org/mariadb-day-brussels-0202-2020-provisional-schedule/
How to migrate from MySQL to MariaDB without tearsSveta Smirnova
Presented at MariaDB Server Fest 2020: https://mariadb.org/fest2020/migrate-mysql/
MariaDB is a drop-in replacement for MySQL. Initial migration is simple: start MariaDB over the old MySQL datadir.
Later your application may notice that some features work differently than with MySQL. These are MariaDB improvements, so this is good and, likely the reason you migrated.
In this session, I will focus on the differences affecting application performance and behavior. In particular, features sharing the same name, but working differently.
Percona Live 2016 (https://www.percona.com/live/data-performance-conference-2016/sessions/why-use-explain-formatjson). Although EXPLAIN FORMAT=JSON was first presented a long time ago, there still aren't many resources that explain how and why to use it. The most advertised feature is visual EXPLAIN in MySQL Workbench, but this format can do more than create nice pictures. It prints additional information that can't be found in good old tabular EXPLAIN, and can help to solve many tricky performance issues. In this session, I will not only describe which additional information we can get with the new syntax, but also provide examples showing how to use it to diagnose production issues.
Talk at "Istanbul Tech Talks" in Istanbul, April, 17, 2018. http://www.istanbultechtalks.com/
In this talk I will show how to get started with MySQL Query Tuning. I will make short introduction into physical table structure and demonstrate how it may influence query execution time. Then we will discuss basic query tuning instruments and techniques, mainly EXPLAIN command with its latest variations. You will learn how to understand its output and how to rewrite query or change table structure to achieve better performance.
Facebook and Beyond - Lessons for Brand Engagement with Social Customers Lithium
The dividends for a well developed Facebook presence will ultimately depend on marketers investing in adopting sophisticated long-term strategies for customer engagement.
Produced for academic purpose !
It can be very helpful to those whom trying to learn about the technology. This will defently give you an simple over view about SQlite
MySQL 5.7 innodb_enhance_partii_20160527Saewoong Lee
Release Date : 2016.05.27
Version : MySQL 5.7
Index :
- Part I : InnoDB Performance
- Part I : InnoDB Buffer Pool Flushing
- Part I : InnoDB internal Transaction General
- Part I : InnoDB Improved adaptive flushing
- Part II : InnoDB Online DDL
- Part II : Tablespace management
- Part II : InnoDB Bulk Load for Create Index
- Part II : InnoDB Temporary Tables
- Part II : InnoDB Full-Text CJK Support
- Part II : Support Syslog on Linux / Unix OS
- Part II : Performance_schema
- Part II : Useful tips
My talk for "MySQL, MariaDB and Friends" devroom at Fosdem on February 2, 2019
Born in 2010 in MySQL 5.5.3 as "a feature for monitoring server execution at a low level," grown in 5.6 times with performance fixes and DBA-faced features, in MySQL 5.7 Performance Schema is a mature tool, used by humans and more and more monitoring products. It becomes more popular over the years. In this talk I will give an overview of Performance Schema, focusing on its tuning, performance, and usability.
Performance Schema helps to troubleshoot query performance, complicated locking issues, memory leaks, resource usage, problematic behavior, caused by inappropriate settings and much more. It comes with hundreds of options which allow precisely tune what to instrument. More than 100 consumers store collected data.
Performance Schema is a potent tool. And very complicated at the same time. It does not affect performance in most cases and can slow down server dramatically if configured without care. It collects a lot of data, and sometimes this data is hard to read.
This talk will start from the introduction of how Performance Schema designed, and you will understand why it slowdowns server in some cases and does not affect your queries in others. Then we will discuss which information you can retrieve from Performance Schema and how to do it effectively.
I will cover its companion sys schema and graphical monitoring tools.
OSMC 2008 | Monitoring MySQL by Geert VanderkelenNETWAYS
Monitoring MySQL has a long history within Nagios. Several plugins are available already. In addition to that, there are probably lots of plugins that have been developed by the community. We take a look at some of these and discuss what kind of additional useful information could be pulled out of a MySQL Server for monitoring it even better. A simple example on how to write such plugins will be shown, also using NDB API for monitoring MySQL Cluster. Now that MySQL Enterprise Monitor (MEM) is available, we'll go through the possibilities for combining the two platforms. We will also discuss the NDOUtils for storing configuration and event data using MySQL.
This talk starts with a brief overview of MySQL itself: some history, where it's heading too, and why it is so successful.
MariaDB 10.5 new features for troubleshooting (mariadb server fest 2020)Valeriy Kravchuk
The recently released MariaDB 10.5 GA includes many new, useful features, but I’d like to concentrate on those helping DBAs and support engineers to find out what’s going on when a problem occurs.
Specifically I present and discuss the Performance Schema updates to match MySQL 5.7 instrumentation, new tables in the INFORMATION_SCHEMA to monitor the internals of a generic thread pool and improvements of ANALYZE for statements.
MySQL® 5.7 is a great release which has a lot to offer, especially in the development and replication areas. It provides a lot of new optimizer features for developers to take advantage of, a much more powerful GIS function and high performance JSON data type, allowing for a more powerful store for semi-structured data. It also features dramatically improved Performance Schema, Parallel and Multi-Source replication, allowing you to scale much further than ever before, just to give you a taste. In this webinar, we will provide an overview of the most important MySQL 5.7 features.
This webinar will be part of a 3-part series which will include MySQL 5.7 for Developers and MySQL 5.7 for DBAs.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™UiPathCommunity
In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni.
📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath:
Autopilot per Studio Web
Autopilot per Studio
Autopilot per Apps
Clipboard AI
GenAI applicata alla Document Understanding
👨🏫👨💻 Speakers:
Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant
Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath
Andrei Tasca, RPA Solutions Team Lead @NTT Data
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Enhancing Performance with Globus and the Science DMZGlobus
ESnet has led the way in helping national facilities—and many other institutions in the research community—configure Science DMZs and troubleshoot network issues to maximize data transfer performance. In this talk we will present a summary of approaches and tips for getting the most out of your network infrastructure using Globus Connect Server.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
2. Topic
▪ Who am I?
▪ MySQL performance tuning experience sharing.
▪ Q & A.
2
3. Who am I?
▪ Stanley Huang (黃偉倫)
▪ My email
▪ wenlien1001@gmail.com
▪ My Blog
▪ http://stanley-huang.blogspot.com
3
4. MySQL performance tuning experience
sharing
▪ What happened to the product?
▪ Too many data and have query performance issue
▪ Performance tuning phases
1. SQL tuning
▪ Over 90%, bad SQLs make great impact on
performance
▪ Over 90%, do not using the correct index(es) in bad
SQLs
2. RDBMS tuning
4
5. MySQL SQL tuning(1/2)
▪ Enable slow query log and find out the top 10 slow
queries
▪ Add two settings in my.cnf
▪ log-slow-queries ## default off
▪ long-query-time=5 ## default 10
▪ [data folder]/[host name]-slow.log
5
6. MySQL SQL tuning(2/2)
▪ Perform full-text search on InnoDB
(select … like ‘*pattern*’)
1. Change storage engine from InnoDB to MyISAM
2. Hybrid storage engine, create a new MyISAM table and move
data need to be full-text search from InnoDB to MyISAM
▪ Too many indexes
1. Create more indexes won’t enhance performance but impact on
writing.
2. (Suggestion) Don’t create more than 5 indexes on one table.
InnoDB in MySQL 5.6 has supported full-text search
6
7. Useful Tips For Networking System
1. IP Support
2. Mac Support
7
8. IP address support
Build In Function: inet_ntoa()/inet_aton()
mysql> CREATE TABLE testIP (
ip_int int unsigned DEFAULT NULL,
ip_char char(15) DEFAULT NULL,
index ip_int (ip_int),
index ip_char (ip_char)
) ENGINE=InnoDB;
mysql> insert into testIP valuse(inet_aton('216.18.50.126'),'216.18.50.126');
mysql> select inet_ntoa(ip_int),ip_char from testIP;
+-------------------+---------------+
| inet_ntoa(ip_int) | ip_char |
+-------------------+---------------+
| 216.18.50.126 | 216.18.50.126 |
+-------------------+---------------+
8
9. Execution Plain
mysql> explain select * from testIP where ip_char='216.18.50.126';
+----+-------------+--------+------+---------------+---------+---------+-------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------+------+---------------+---------+---------+-------+------+-------------+
| 1 | SIMPLE | testIP | ref | ip_char | ip_char | 16 | const | 1 | Using where |
+----+-------------+--------+------+---------------+---------+---------+-------+------+-------------+
mysql> explain select * from testIP where ip_int=inet_aton('216.18.50.126');
+----+-------------+--------+------+---------------+--------+---------+-------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------+------+---------------+--------+---------+-------+------+-------------+
| 1 | SIMPLE | testIP | ref | ip_int | ip_int | 5 | const | 1 | Using where |
+----+-------------+--------+------+---------------+--------+---------+-------+------+-------------+
9
10. Mac address support??
Build In Function: hex()
mysql> create table testMAC(mac_bit bit(48), mac_char char(17),
index(mac_bit), index(mac_char));
mysql> insert into testMAC values (x'00241DDC5548', '00:24:1D:DC:55:48');
mysql> select hex(mac_bit), mac_char from testMAC;
+--------------+-------------------+
| hex(mac_bit) | mac_char |
+--------------+-------------------+
| 241DDC5548 | 00:24:1D:DC:55:48 |
+--------------+-------------------+
Build In Function seems not enough~
1. create MySQL stored function
2. create UDF (User Define Function)
10
11. Execution Plain
mysql> explain select hex(mac_bit), mac_char from testMAC where
mac_char='00:24:1D:DC:55:48';
+----+-------------+----------+------+---------------+----------+---------+-------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------+------+---------------+----------+---------+-------+------+-------------+
| 1 | SIMPLE | testMAC2 | ref | mac_char | mac_char | 18 | const | 1 | Using where |
+----+-------------+----------+------+---------------+----------+---------+-------+------+-------------+
mysql> explain select hex(mac_bit), mac_char from testMAC where mac_bit=x'00241DDC5548';
+----+-------------+----------+------+---------------+---------+---------+-------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------+------+---------------+---------+---------+-------+------+-------------+
| 1 | SIMPLE | testMAC2 | ref | mac_bit | mac_bit | 7 | const | 1 | Using where |
+----+-------------+----------+------+---------------+---------+---------+-------+------+-------------+
11
12. Stored Function for MAC address
## create stored function ether_atob(), from ascii to bit
## must define "deterministic", or explain will not use index
delimiter //
drop function if exists ether_atob//
create function ether_atob(sAscii char(17))
returns bit(48)
deterministic
Begin
declare bReturn bit(48);
set bReturn=unhex(replace(sAscii,':',''));
return bReturn;
end//
delimiter ;
12
13. Stored Function for MAC address
(cont.)
## create stored function ether_btoa(), from bit to ascii
## must define "deterministic", or explain will not use index
delimiter //
drop function if exists ether_btoa//
create function ether_btoa(sBit bit(48))
returns char(17)
deterministic
begin
declare sReturn char(17);
set sReturn=lpad(hex(sBit),12,'0');
set sReturn=concat_ws(':', substr(sReturn,1,2), substr(sReturn,3,2),
substr(sReturn,5,2),
substr(sReturn,7,2), substr(sReturn,9,2), substr(sReturn,11,2));
return sReturn;
end//
delimiter ;
13
14. Stored Function for MAC address
(cont.)
mysql> create table ether_table (b bit(48), a char(17), index(b), index(a));
Query OK, 0 rows affected (0.67 sec)
mysql> insert into ether_table values
(ether_atob('00:CD:EF:00:CD:EF'),'00:CD:EF:00:CD:EF');
Query OK, 1 row affected (0.01 sec)
mysql> select ether_btoa(b), a from ether_table
where b=ether_atob('00:CD:EF:00:CD:EF');
+----------------+-------------------+
| ether_btoa(b) | a |
+----------------+-------------------+
| 00:CD:EF:00:CD:EF | 00:CD:EF:00:CD:EF |
+----------------+-------------------+
1 rows in set (0.01 sec )
14
15. Stored Function for MAC address
(cont.)
mysql> explain select ether_btoa(b), a from ether_table where
b=ether_atob('00:CD:EF:00:CD:EF');
+----+-------------+-------------+------+---------------+------+---------+-------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------+------+---------------+------+---------+-------+------+-------------+
| 1 | SIMPLE | ether_table | ref | b |b |7 | const | 1 | Using where |
+----+-------------+-------------+------+---------------+------+---------+-------+------+-------------+
1 row in set (0.00 sec)
15
17. Using UDF on MySQL Database
Make sure your MySQL supports UDF.
mysql> create function lib_mysqludf_log_info returns string soname
'lib_mysqludf_log.so';
ERROR 1126 (HY000): Can't open shared library 'lib_mysqludf_log.so' (errno:
0 feature disabled)
17
19. Characters of Stored Function, UDF
and Native Function
Method Speed Language Development Maintenance
Stored slow SQL ~minutes easy
Function (for small
functions)
UDF fast C ~hour hard
Native fast C Major pain hard
Function
19
20. Compare with UDF and Stored
Functions
Reference: Roland Bouman's blog
http://rpbouman.blogspot.com/2008/03/
udfs-at-mysql-users-conference.html
20
21. MySQL performance tuning
▪ Separate disk I/O
▪ Separate disk I/O of bin log files and data files to different disks
▪ MyISAM configuration
▪ key_buffer_size=128M # default is 8M
▪ bulk_insert_buffer_size=4194304 # default is 8M
▪ InnoDB configuration
▪ innodb_buffer_pool_size=32M # default
▪ 8Minnodb_log_buffer_size=8M # default 1M
▪ Prevent large single large file of InnoDB tablespace
▪ innodb_file_per_table
21
22. MySQL upgrade from 5.0 to 5.1
▪ In internal tests, MySQL 5.1 demonstrates, on average, a 15% gain
in total performance over MySQL 5.0. (There really is the free lunch)
22
24. What's wrong with “my” SQL
mysql> select count(1) from test_partition_wp where year(c3)=1995;
+----------+
| count(1) |
+----------+
| 47358 |
+----------+
1 row in set (0.58 sec)
mysql> select count(1) from test_partition_np where year(c3)=1995;
+----------+
| count(1) |
+----------+
| 47358 |
+----------+
1 row in set (0.53 sec) -- Faster then partition table, why??
24
25. Execution Plan
mysql> explain partitions select count(1) from test_partition_wp where
year(c3)=1995G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: test_partition_wp
partitions: p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 12006556
Extra: Using where
1 row in set (0.00 sec)
25
26. The correct SQL
mysql> select count(1) from test_partition_wp where c3 between '1995/01/01'
and '1995/12/31';
+----------+
| count(1) |
+----------+
| 47358 |
+----------+
1 row in set (0.04 sec) -- That's what I want!!
mysql> select count(1) from test_partition_np where c3 between '1995/01/01' and
'1995/12/31';
+----------+
| count(1) |
+----------+
| 47358 |
+----------+
1 row in set (0.62 sec)
26
27. Execution Plan
mysql> explain partitions select count(1) from test_partition_wp where c3
between '1995/01/01' and '1995/12/31'G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: test_partition_wp
partitions: p1
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 12006556
Extra: Using where
1 row in set (0.00 sec)
27
28. Performance Enhancement With Table
Partitioning
After we use partition on the large tables, the performance raise 11-45% up.
1. 5.1-partition 377.4146 tableA
5.1 687.4756 tableA
(687.4756-377.4146) / 687.4756 = 0.45
2. 5.1-partition 171.7333 tableB
5.1 193.9878 tableB
(193.9878-171.7333) / 193.9878 = 0.11
3. 5.1-partition 22.0741 tableC
5.1 34.4792 tableC
(34.4792-22.0741) / 34.4792 = 0.36
4. 5.1-partition 1.8519 tableD
5.1 3.3750 tableD
(3.3750-1.8519) / 3.3750 = 0.45
28
29. MySQL Performance Tips Summary
▪ Use EXPLAIN to profile the query.
▪ Always have slow query log.
▪ Avoid using wildcards at the start of LIKE queries.
▪ Isolate the workloads.
▪ Using data partitions..
▪ Don’t duplicate indexes.
▪ Use INET_ATON and INET_NTOA.
▪ Hire a MySQL Certified DBA.
http://forge.mysql.com/wiki/Top10SQLPerformanceTips
29