Adrian Hardy's slides from PHPNW08
Once you have your query returning the correct results, speed becomes an important factor. Speed can either be an issue from the outset, or can creep in as your dataset grows. Understanding the EXPLAIN command is essential to helping you solve and even anticipate slow queries.
Associated video: http://blip.tv/file/1791781
Advanced MySQL Query Tuning - talk at Percona Live and MySQL Meetup tour.
Tuning Queries and Schema/Indexes can significantly increase performance of your application and decrease response times.
This year I will cover new MySQL 5.6 and 5.7 algorithms that has been designed to improve query performance and simply tuning.
Topics:
1. Group by and order by optimizations
2. MySQL temporary tables and filesort
3. Using covered indexes to optimize your queries
4. Loose and tight index scan in MySQL
5. Using summary tables to optimize your reporting queries
6. New MySQL 5.6 and 5.7 Optimizer features and improvements
[오픈소스컨설팅]Day #1 MySQL 엔진소개, 튜닝, 백업 및 복구, 업그레이드방법Ji-Woong Choi
MySQL 소개
간략한 소개
version history
MySQL 사용처
제품 군 변화
시장 변화
MySQL 구성
MySQL 클라이언트 / 서버 개념
클라이언트 프로그램
MySQL 설치
MySQL 버전
MySQL 설치
MySQL 환경 설정
환경설정, 변수 설정
MySQL 스토리지 엔진 소개
MySQL tuning 소개 및 방법
데이터 백업/복구 방법
백업
복구
MySQL Upgrade
MySQL Performance Tuning: Top 10 Tips intended for PHP, Ruby and Java developers on performance tuning and optimization of MySQL. We will cover the deadly mistakes to be avoided. We will take real life examples of optimizing application many times. Here is the summary of what we intend to cover:
• Selection of Storage Engine
• Schema Optimization
• Server Tuning
• Hardware Selection and Tuning
• Effective uses of Index, when to use and when not to use.
• Partitions
• Speeding up using Stored Procedures
• Implementing prepared statements?
• Deadly Sins to be avoided
• Performance Tuning and Benchmarking Tools
MySQL users commonly ask: Here's my table, what indexes do I need? Why aren't my indexes helping me? Don't indexes cause overhead? This talk gives you some practical answers, with a step by step method for finding the queries you need to optimize, and choosing the best indexes for them.
Adrian Hardy's slides from PHPNW08
Once you have your query returning the correct results, speed becomes an important factor. Speed can either be an issue from the outset, or can creep in as your dataset grows. Understanding the EXPLAIN command is essential to helping you solve and even anticipate slow queries.
Associated video: http://blip.tv/file/1791781
Advanced MySQL Query Tuning - talk at Percona Live and MySQL Meetup tour.
Tuning Queries and Schema/Indexes can significantly increase performance of your application and decrease response times.
This year I will cover new MySQL 5.6 and 5.7 algorithms that has been designed to improve query performance and simply tuning.
Topics:
1. Group by and order by optimizations
2. MySQL temporary tables and filesort
3. Using covered indexes to optimize your queries
4. Loose and tight index scan in MySQL
5. Using summary tables to optimize your reporting queries
6. New MySQL 5.6 and 5.7 Optimizer features and improvements
[오픈소스컨설팅]Day #1 MySQL 엔진소개, 튜닝, 백업 및 복구, 업그레이드방법Ji-Woong Choi
MySQL 소개
간략한 소개
version history
MySQL 사용처
제품 군 변화
시장 변화
MySQL 구성
MySQL 클라이언트 / 서버 개념
클라이언트 프로그램
MySQL 설치
MySQL 버전
MySQL 설치
MySQL 환경 설정
환경설정, 변수 설정
MySQL 스토리지 엔진 소개
MySQL tuning 소개 및 방법
데이터 백업/복구 방법
백업
복구
MySQL Upgrade
MySQL Performance Tuning: Top 10 Tips intended for PHP, Ruby and Java developers on performance tuning and optimization of MySQL. We will cover the deadly mistakes to be avoided. We will take real life examples of optimizing application many times. Here is the summary of what we intend to cover:
• Selection of Storage Engine
• Schema Optimization
• Server Tuning
• Hardware Selection and Tuning
• Effective uses of Index, when to use and when not to use.
• Partitions
• Speeding up using Stored Procedures
• Implementing prepared statements?
• Deadly Sins to be avoided
• Performance Tuning and Benchmarking Tools
MySQL users commonly ask: Here's my table, what indexes do I need? Why aren't my indexes helping me? Don't indexes cause overhead? This talk gives you some practical answers, with a step by step method for finding the queries you need to optimize, and choosing the best indexes for them.
2021년 11월 18일(목)
- 14:00 ~ 15:00 MySQL Operator for Kubernetes
: Kubernetes 환경에서 MySQL에 대한 더 쉬운 운영
- 15:00 ~ 15:15 MySQL HA and Auto-Failover
: MySQL replication과 오픈소스 MHA를 통한 고가용성 확보
MySQL Administrator
Basic course
- MySQL 개요
- MySQL 설치 / 설정
- MySQL 아키텍처 - MySQL 스토리지 엔진
- MySQL 관리
- MySQL 백업 / 복구
- MySQL 모니터링
Advanced course
- MySQL Optimization
- MariaDB / Percona
- MySQL HA (High Availability)
- MySQL troubleshooting
네오클로바
http://neoclova.co.kr/
Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013Jaime Crespo
Tutorial delivered at Percona MySQL Conference Live London 2013.
It doesn't matter what new SSD technologies appear, or what are the latest breakthroughs in flushing algorithms: the number one cause for MySQL applications being slow is poor execution plan of SQL queries. While the latest GA version provided a huge amount of transparent optimizations -specially for JOINS and subqueries- it is still the developer's responsibility to take advantage of all new MySQL 5.6 features.
In this tutorial we will propose the attendants a sample PHP application with bad response time. Through practical examples, we will suggest step-by-step strategies to improve its performance, including:
* Checking MySQL & InnoDB configuration
* Internal (performance_schema) and external tools for profiling (pt-query-digest)
* New EXPLAIN tools
* Simple and multiple column indexing
* Covering index technique
* Index condition pushdown
* Batch key access
* Subquery optimization
Software Design Patterns in Laravel by Phill SparksPhill Sparks
Laravel makes use of quite a few well-established design patterns that promote reusable object-oriented code. Together, we will investigate the design patterns used in the core of Laravel 4 and discuss how they encourage reusable software.
24시간 365일 서비스를 위한 MySQL DB 이중화.
MySQL 이중화 방안들에 대해 알아보고 운영하면서 겪은 고민들을 이야기해 봅니다.
목차
1. DB 이중화 필요성
2. 이중화 방안
- HW 이중화
- MySQL Replication 이중화
3. 이중화 운영 장애
4. DNS와 VIP
5. MySQL 이중화 솔루션 비교
대상
- MySQL을 서비스하고 있는 인프라 담당자
- MySQL 이중화에 관심 있는 개발자
Learn how to load balance your applications following best practices with NGINX and NGINX Plus.
Join this webinar to learn:
- How to configure basic HTTP load balancing features
- The essential elements of load balancing: session persistence, health checks, and SSL termination
- How to load balance MySQL, DNS, and other common TCP/UDP applications
- How to have NGINX Plus automatically discover new service instances in an auto-scaling or microservices environment
How to Analyze and Tune MySQL Queries for Better Performanceoysteing
Tutorial at Oracle Open World 2015:
Performance of SQL queries plays a big role in application performance. If some queries execute slowly, these queries or the database schema may need tuning. This tutorial covers query processing, optimization methods, and how the MySQL optimizer chooses a specific plan to execute SQL. See demonstrations on how to use tools such as EXPLAIN (including the JSON-based variant), optimizer trace, and performance schema to analyze query plans. See how the Visual Explain functionality in MySQL Workbench helps you to visualize these plans. Based on the analysis, the tutorial covers how to take the next steps for performance tuning. It might mean forcing a particular index, changing the schema, or modifying configuration parameters.
The technology has almost written off MySQL as a database for new fancy NoSQL databases like MongoDB and Cassandra or even Hadoop for aggregation. But MySQL has a lot to offer in terms of 'ACID'ity, performance and simplicity. For many use-cases MySQL works well. In this week's ShareThis workshop we discuss different tips & techniques to improve performance and extend the lifetime of your MySQL deployment.
2021년 11월 18일(목)
- 14:00 ~ 15:00 MySQL Operator for Kubernetes
: Kubernetes 환경에서 MySQL에 대한 더 쉬운 운영
- 15:00 ~ 15:15 MySQL HA and Auto-Failover
: MySQL replication과 오픈소스 MHA를 통한 고가용성 확보
MySQL Administrator
Basic course
- MySQL 개요
- MySQL 설치 / 설정
- MySQL 아키텍처 - MySQL 스토리지 엔진
- MySQL 관리
- MySQL 백업 / 복구
- MySQL 모니터링
Advanced course
- MySQL Optimization
- MariaDB / Percona
- MySQL HA (High Availability)
- MySQL troubleshooting
네오클로바
http://neoclova.co.kr/
Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013Jaime Crespo
Tutorial delivered at Percona MySQL Conference Live London 2013.
It doesn't matter what new SSD technologies appear, or what are the latest breakthroughs in flushing algorithms: the number one cause for MySQL applications being slow is poor execution plan of SQL queries. While the latest GA version provided a huge amount of transparent optimizations -specially for JOINS and subqueries- it is still the developer's responsibility to take advantage of all new MySQL 5.6 features.
In this tutorial we will propose the attendants a sample PHP application with bad response time. Through practical examples, we will suggest step-by-step strategies to improve its performance, including:
* Checking MySQL & InnoDB configuration
* Internal (performance_schema) and external tools for profiling (pt-query-digest)
* New EXPLAIN tools
* Simple and multiple column indexing
* Covering index technique
* Index condition pushdown
* Batch key access
* Subquery optimization
Software Design Patterns in Laravel by Phill SparksPhill Sparks
Laravel makes use of quite a few well-established design patterns that promote reusable object-oriented code. Together, we will investigate the design patterns used in the core of Laravel 4 and discuss how they encourage reusable software.
24시간 365일 서비스를 위한 MySQL DB 이중화.
MySQL 이중화 방안들에 대해 알아보고 운영하면서 겪은 고민들을 이야기해 봅니다.
목차
1. DB 이중화 필요성
2. 이중화 방안
- HW 이중화
- MySQL Replication 이중화
3. 이중화 운영 장애
4. DNS와 VIP
5. MySQL 이중화 솔루션 비교
대상
- MySQL을 서비스하고 있는 인프라 담당자
- MySQL 이중화에 관심 있는 개발자
Learn how to load balance your applications following best practices with NGINX and NGINX Plus.
Join this webinar to learn:
- How to configure basic HTTP load balancing features
- The essential elements of load balancing: session persistence, health checks, and SSL termination
- How to load balance MySQL, DNS, and other common TCP/UDP applications
- How to have NGINX Plus automatically discover new service instances in an auto-scaling or microservices environment
How to Analyze and Tune MySQL Queries for Better Performanceoysteing
Tutorial at Oracle Open World 2015:
Performance of SQL queries plays a big role in application performance. If some queries execute slowly, these queries or the database schema may need tuning. This tutorial covers query processing, optimization methods, and how the MySQL optimizer chooses a specific plan to execute SQL. See demonstrations on how to use tools such as EXPLAIN (including the JSON-based variant), optimizer trace, and performance schema to analyze query plans. See how the Visual Explain functionality in MySQL Workbench helps you to visualize these plans. Based on the analysis, the tutorial covers how to take the next steps for performance tuning. It might mean forcing a particular index, changing the schema, or modifying configuration parameters.
The technology has almost written off MySQL as a database for new fancy NoSQL databases like MongoDB and Cassandra or even Hadoop for aggregation. But MySQL has a lot to offer in terms of 'ACID'ity, performance and simplicity. For many use-cases MySQL works well. In this week's ShareThis workshop we discuss different tips & techniques to improve performance and extend the lifetime of your MySQL deployment.
Polyglot Database - Linuxcon North America 2016Dave Stokes
Many Relation Databases are adding NoSQL features to their products. So what happens when you can get direct access to the data as a key/value pair, or you can store an entire document in a column of a relational table, and more
What Your Database Query is Really DoingDave Stokes
Do you ever wonder what your database servers is REALLY doing with that query you just wrote. This is a high level overview of the process of running a query
Accessibility: A Journey to Accessible Rich ComponentsAchievers Tech
Accessibility on the web can be a complex enigma to many web developers and designers. Achievers knows this first hand and in this talk Jason will cover how we approached making a large platform like Achievers accessible. He will also briefly cover some accessibility basics before digging into the journey of making our rich UI components accessible.
Jason Jang has been building websites since tables were all the rage. Along the way he's designed and developed both as a freelancer, and at various advertising agencies. His 10+ year journey landed him at Achievers where he's tackling the platform's accessibility by the horns.
Tips on how to prepare MySQL 5.7 GIS databases for the upgrade to MySQL 8.0 and the introduction of geography support.
Presentation given at the Pre-FOSDEM MySQL Day in Brussels, February 3, 2017.
MySQL/MariaDB replication is asynchronous. You can make replication faster by using better hardware (faster CPU, more RAM, or quicker disks), or you can use parallel replication to remove it single-threaded limitation; but lag can still happen. This talk is not about making replication faster, it is how to deal with its asynchronous nature, including the (in-)famous lag.
We will start by explaining the consequences of asynchronous replication and how/when lag can happen. Then, we will present the solution used at Booking.com to avoid both creating lag and minimize the consequence of stale reads on slaves (hint: this solution does not mean reading from the master because this does not scale).
Once all above is well understood, we will discuss how Booking.com’s solution can be improved: this solution was designed years ago and we would do this differently if starting from scratch today. Finally, I will present an innovative way to avoid lag: the no-slave-left-behind MariaDB patch.
Introduction to MySQL Query Tuning for Dev[Op]sSveta Smirnova
To get data, we query the database. MySQL does its best to return requested bytes as fast as possible. However, it needs human help to identify what is important and should be accessed in the first place.
Queries, written smartly, can significantly outperform automatically generated ones. Indexes and Optimizer statistics, not limited to the Histograms only, help to increase the speed of the query a lot.
In this session, I will demonstrate by examples of how MySQL query performance can be improved. I will focus on techniques, accessible by Developers and DevOps rather on those which are usually used by Database Administrators. In the end, I will present troubleshooting tools which will help you to identify why your queries do not perform. Then you could use the knowledge from the beginning of the session to improve them.
With the introduction of SQL Server 2012 data developers have new ways to interact with their databases. This session will review the powerful new analytic windows functions, new ways to generate numeric sequences and new ways to page the results of our queries. Other features that will be discussed are improvements in error handling and new parsing and concatenating features.
15 Ways to Kill Your Mysql Application Performanceguest9912e5
Jay is the North American Community Relations Manager at MySQL. Author of Pro MySQL, Jay has also written articles for Linux Magazine and regularly assists software developers in identifying how to make the most effective use of MySQL. He has given sessions on performance tuning at the MySQL Users Conference, RedHat Summit, NY PHP Conference, OSCON and Ohio LinuxFest, among others.In his abundant free time, when not being pestered by his two needy cats and two noisy dogs, he daydreams in PHP code and ponders the ramifications of __clone().
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
7. Column Index Type Definitions
• Column Index
– An index on a single column
• Compound Index
– An index on multiple columns
• Covering Index
– “Covers” all columns in a query
• Partial Index
– A subset of a column for the index
• E.g. Only the first 10 characters of a person’s name
8. Compound Index
• CREATE TABLE test (
id INT NOT NULL,
last_name CHAR(30) NOT NULL,
first_name CHAR(30) NOT NULL,
PRIMARY KEY (id),
INDEX name(last_name,first_name) );
• The name index is an index over the
last_name and first_name columns
9. What does that mean?
• Queries like:
– SELECT * FROM test WHERE first_name=‘Aris’ AND
last_name=‘Zakinthinos’;
– SELECT * FROM test WHERE last_name=‘York’;
– Will use the index
• But a query like:
– SELECT * FROM test WHERE first_name=‘Zak’;
– Will not
10. How Compound Indexes are Used
• If you have an index on (col1, col2,
col3)
• This Index will be used on queries for (col1),
(col1, col2) and (col1, col2, col3)
– Notice that the leftmost prefix must exist
• Only the col1 part of the index will be used for
queries for (col1, col3)
• This Index will not be used for queries for
(col2), (col3) and (col2, col3)
11. One Thing to Keep in Mind
• Remember that the index stores things in
sorted order so an n-field compound index is
the equivalent of sorting the data on n fields
• For example, for a 2 column index:
COL1 COL2
A 4
Z 3
A 5
Z 1
Index
(A,4)
(A,5)
(Z,1)
(Z,3)
12. Pro Tip
• MySQL always adds the primary key to the
end of your index
– You never have to add it to the end of your
compound key
13. Can you have too many indexes?
• YES!
• They take up space
– You want all your indexes to fit in memory
• They make inserts/deletes slower
– Remember that you need to insert/delete
into/from each index
16. What does that mean?
• How many tables are used
• How tables are joined
• How data is looked-up
• Possible and actual index use
• Length of index used
• Approximate number of rows examined
17. Why should I care?
• Ultimately, less server load leads to a better
user experience
– Could be the difference between usable and
bankrupt
• Impress your friends
• Get better jobs
18. Which queries should I examine?
• Every single one!
• If you have never used EXPLAIN:
– Start by looking at the items in the slow query log
– Or, execute SHOW FULL PROCESSLIST every
once in a while and grab a query that you see very
often
19. Pro Tip
• Using EXPLAIN during QA is better than in
production
• It avoids you having to say:
– “It’s not a problem—we call it the coffee break
feature.”
20. Can I run it on any query?
• Up to MySQL 5.6, it only worked on SELECT
queries
21. Anything else I should know?
• It doesn’t execute your query but MAY
execute portions of it. CAUTION!
– Nested subqueries are executed
23. Test Database
• We will be using MySQL’s sakalia test
database, available at:
http://dev.MySQL.com/doc/index-other.html
• It is a sample database for a DVD rental store
24. How do you use Explain?
• To get MySQL’s execution plan, simply put the
word ‘EXPLAIN’ in front of your select
statement:
EXPLAIN SELECT * FROM film f
JOIN film_actor fa ON fa.film_id = f.film_id
JOIN actor a ON a.actor_id = fa.actor_id;
28. Column Definitions
A sequential ID identifying the select in the query.
EXPLAIN SELECT * FROM film f
JOIN film_actor fa ON fa.film_id = f.film_id
JOIN actor a ON a.actor_id = fa.actor_id;
29. Column Definitions
The type of select you are doing. Common values:
• SIMPLE – Simple SELECT (not using UNION or subqueries)
• PRIMARY – Outermost SELECT
• UNION – Second or later SELECT in a UNION
• SUBQUERY – First SELECT in a subquery
• DEPENDENT SUBQUERY - First SELECT in subquery, dependent on outer query
• DERIVED - Derived table select (subquery in FROM clause)
30. Column Definitions
The Table or Alias this row refers to.
EXPLAIN SELECT * FROM film f
JOIN film_actor fa ON fa.film_id = f.film_id
JOIN actor a ON a.actor_id = fa.actor_id;
31. Column Definitions
The join type. Lots more to come on this.
EXPLAIN SELECT * FROM film f
JOIN film_actor fa ON fa.film_id = f.film_id
JOIN actor a ON a.actor_id = fa.actor_id;
32. Column Definitions
The possible indexes that could be used.
If NULL then no appropriate index was found.
EXPLAIN SELECT * FROM film f
JOIN film_actor fa ON fa.film_id = f.film_id
JOIN actor a ON a.actor_id = fa.actor_id;
33. Column Definitions
The Index that was used.
EXPLAIN SELECT * FROM film f
JOIN film_actor fa ON fa.film_id = f.film_id
JOIN actor a ON a.actor_id = fa.actor_id;
34. Column Definitions
The number of bytes MySQL uses from the index.
EXPLAIN SELECT * FROM film f
JOIN film_actor fa ON fa.film_id = f.film_id
JOIN actor a ON a.actor_id = fa.actor_id;
35. Column Definitions
The columns (or constants) form the index that are used.
EXPLAIN SELECT * FROM film f
JOIN film_actor fa ON fa.film_id = f.film_id
JOIN actor a ON a.actor_id = fa.actor_id;
36. Column Definitions
The approximate number of rows examined.
Note: this is just a guide.
EXPLAIN SELECT * FROM film f
JOIN film_actor fa ON fa.film_id = f.film_id
JOIN actor a ON a.actor_id = fa.actor_id;
37. Column Definitions
Information on how the tables are join.
Lots more to come on this!
EXPLAIN SELECT * FROM film f
JOIN film_actor fa ON fa.film_id = f.film_id
JOIN actor a ON a.actor_id = fa.actor_id;
39. Join Type – system/const
• At most one row returned
– E.g., … WHERE id=1
– Constant can be propagated through query
– Index must be either PRIMARY or UNIQUE
EXPLAIN SELECT * FROM rental WHERE rental_id = 10;
40. Join Type – eq_ref
• Index lookup returns exactly one row
– E.g., WHERE a.id=b.id
– Requires unique index on all parts of the key used
by the join
EXPLAIN SELECT * FROM customer c
JOIN address a ON c.address_id = a.address_id
41. Join Type – ref
• Index lookup that can return more than one
row
– Used when
• Either leftmost part of a unique key is used
• Or a non-unique or non-null key is used
EXPLAIN SELECT * FROM rental WHERE rental_id IN (10,11,12)
AND rental_date = '2006-02-01';
42. Join Type– ref_or_null
• Similar to ref but allows for null values or null
conditions
– Essentially an extra pass to look for nulls
EXPLAIN SELECT * FROM film WHERE
release_year = 2006 OR release_year IS NULL;
43. Join Type – index_merge
• Uses 2 separate indexes
– Extra field shows more info
– Can be one of:
• sort_union – OR condition on non-primary key fields
• union – OR condition using constants or ranges on primary
key fields
• intersection – AND condition with constants or range
conditions on primary key fields
EXPLAIN SELECT * FROM rental WHERE rental_id IN (10,11,12)
OR rental_date = '2006-02-01';
44. Join Type – range
• Access method for a range value in the where
clause (<, <=, >, >=, LIKE, IN or BETWEEN,
IN())
– Performs a partial index scan
– Lots of optimizations for this type of query
EXPLAIN SELECT * FROM rental WHERE rental_date
BETWEEN '2006-01-01' AND '2006-07-01';
45. Join Type – index
• Does an index scan
– You are doing a full scan of every record in the
index
– Better than ‘ALL’ but still requires a LOT of
resources
– Note: This is not the same as ‘USING INDEX’ in
Extras
EXPLAIN SELECT rental_date FROM rental;
46. Join Type – ALL
• Full table scan
– It will look at every record in the table
– Unless you want the whole table, it should be
avoided for all but the smallest of tables
EXPLAIN SELECT * FROM rental;
47. Join Type Summary
• From best to worst
– system/const
– eq_ref
– ref
– ref_or_null
– index_merge
– range
– index
– ALL
49. A More Complicated Query
• All identical id values are part of the same select
• This query has a bunch of UNIONs
• You can also see all of the join types used by this Query
51. Execution Order
• First thing to notice is that the order of execution is not the same as the query
• MySQL will rearrange your query to do what it thinks is optimal
• You will often disagree!
EXPLAIN SELECT * FROM film f
JOIN film_actor fa ON fa.film_id = f.film_id
JOIN actor a ON a.actor_id = fa.actor_id;
52. Query Processing
The above can be read as:
for (each row in table a [Actor] ) {
for (each row in fa [film_actor] matching a.actor_id) {
for the row in f [film] matching fa.film_id ) {
Add row to output
}
}
} EXPLAIN SELECT * FROM film f
JOIN film_actor fa ON fa.film_id = f.film_id
JOIN actor a ON a.actor_id = fa.actor_id;
53. Query Processing In General
for (each row in outer table matching where clause){
…
for (each row in inner table matching key value
and where clause) {
Add to output
}
}
54. rows is a critical performance indicator
• Total approximate rows is the PRODUCT of all
the rows with the same ID
• It is only an estimate
• This query estimated a total of 68,640 rows
• Actual rows examined 2,969,639
55. Remember: it is an estimate
Remember this query: Here is the rest:
This query executes in less than
200 ms because there are limit
clauses that only return a small
number of rows.
56. key_len
• This will tell you how much of the index it will use
– Really only useful if you have compound keys
• For a complete list of type to byte mapping, see:
– http://dev.mysql.com/doc/refman/5.6/en/storage-
requirements.html
• What’s missing from that page:
– VARCHAR(n)will always use a 2 byte length fields
– If a column can be NULL it takes one extra byte in the
index
• Warning! Multibyte characters make byte!=character
– UTF8 is 3 bytes
57. key_len example
EXPLAIN SELECT film_id, title FROM film WHERE description LIKE
'A Epic%' AND release_year=2006 AND language_id = 1;
CREATE TABLE `film` (
...
`description` text,
`release_year` year(4) DEFAULT NULL,
`language_id` tinyint(3) unsigned NOT NULL,
...
KEY `idx_compound_key` (`language_id`,`release_year`,`description`(20)),
) ENGINE=InnoDB AUTO_INCREMENT=1001 DEFAULT CHARSET=utf8;
tinyint – 1 byte year – 1 byte + 1 byte possible NULL
Description 20*3 bytes+2 byte
length+1 byte possible NULL
59. Pay Close Attention
• This column will give you a good sense of
what is going to happen during execution
60. Extra Column – The Bad
• Using temporary
– During execution, a temporary table was required
– Not horrible if the temporary table is small since it
will sit in memory
61. Extra Column – The Bad
• Using filesort
– Sorting was needed rather than using an index
– Not horrible if the result set is small
62. Extra Column – The Good
• Using index
– The query was satisfied using only an index
63. Extra Column – The Whatever
• Using where
– A where clause is used to restrict rows
– Unless you also see ‘Using Index,’ this means that
MySQL had to read the row from the database to
apply the where clause
65. Signs That Your Query Stinks
• No index is used – NULL in key column
• Large number of rows estimated
• Using temporary
• Using filesort
• Using derived tables – DERIVED in
select_type column
• Joining on derived tables
• Having dependent subqueries on a large result
set
66. Examples:
EXPLAIN SELECT * FROM film f WHERE release_year = 2006;
ALTER TABLE `sakila`.`film`
ADD INDEX `idx_release_year` (`release_year` ASC) ;
67. Examples:
EXPLAIN SELECT rating, COUNT(*) FROM film
WHERE rental_rate <1.00
GROUP BY rating;
• This is a terrible query.
• A full table scan that creates a temporary table and then sorts it.
• Why?
68. Grouping
• To execute this query, MySQL will build a
temporary table with all the rows that have
rental_rate <1.00
• It will then sort them by rating
• It will then count all of the items that have the
same rating
• To make this query fast you need all of the
ratings to be processed in order.
– That is, you have to avoid the build and sort step
69. Adding an Index
ALTER TABLE `sakila`.`film` ADD INDEX `idx_rating`
(`rating` ASC, `rental_rate` ASC) ;
EXPLAIN SELECT rating, COUNT(*) FROM film
WHERE rental_rate <1.00
GROUP BY rating;
• This works because MySQL can process all the rows with the same ‘rating’ sequentially.
• The second part of the index allows it to check the where clause from the index directly.
70. Optimizing GROUP BY
• All columns used in the GROUP BY must
come from the same index and the index must
store the keys in the order specified in the
GROUP BY
• If this isn’t true you will see a “Using
filesort” and/or “Using temporary”
71. Optimizing ORDER BY
• An index can be used with ORDER BY even
if the index doesn’t match the ORDER BY
columns exactly as long as the “missing” keys
are constants in the WHERE clause
–The order of the keys must match the order
of the ORDER BY clause
• If this isn’t true you will see a “Using
filesort” and/or “Using temporary”
72. Both GROUP BY and ORDER BY
EXPLAIN SELECT rating, COUNT(*) AS c FROM film
WHERE rental_rate <1.00
GROUP by rating
ORDER BY c;
• There is no way to avoid the ‘Using temporary’ and the ‘Using filesort’
• You are asking MySQL to first sort by rating to process the GROUP BY which it does using
an index.
• Then you are asking it to take that result and resort it from a computed field.
• An ORDER BY column list which is in a different order than the GROUP BY list will cause
problem.
73. What if I can’t make it better?
• Never give up!
– There is always a solution. You might not want to
do it, but there is always a solution.
• You might need to:
– Denormalize your data to allow you to construct a
better compound index
– Cache outside the database
– Pull some of the processing into your application
– Break up the query into smaller faster chunks
75. Test with Production Data
• Do not optimize with a subset of your data
• MySQL uses table statistics to determine its
execution plan
• If you optimize with test data your real world
performance might be completely different
76. PRACTICE!
• Like everything, you need to do it, to fully
understand it
• Don’t expect to be a MySQL ninja overnight
• It takes hard work but the benefits are worth
it
– 10 second searches optimized to 100 milliseconds