This document summarizes Peter Zaitsev's presentation on MySQL query optimization. It provides tips for optimizing queries such as avoiding unnecessary queries, caching results, simplifying queries, and optimizing queries by adding indexes and changing queries and schemas. The presentation also covers query execution plans, indexing techniques like covering indexes, and how to optimize queries using LIMIT, GROUP BY and other SQL clauses.
تأسست دار أوستن ماكولي بابليشرز في مدينة لندن عام 2006، وسرعان ما أصبحت إحدى دور النشر الواعدة في المملكة المتحدة، والتي عملت بجد لمنح المؤلفين أفضل فرصة للنجاح في السوق المزدحمة علي نحوٍ متزايد، وعزز هذا الفرصة لفتح فرع في مدينة نيويورك في الولايات المتحدة الأمريكية والشارقة في الإمارات العربية المتحدة. كما ارتبط اسمها بنشر الإبداع والمعرفة الحديثة الهادفة إلى مواكبة النهضة التي يشهدها العالم، ومن خلال سعيها الدائم إلى بناء سمعة مرموقة ووضع بصمة في صناعة النشر المحلية والعالمية.
https://austinmacauley.ae/ar/
Presented at the ICT and Language Education Conference, Kyoto, Japan. October 11th, 2015
http://kyotojalt.wix.com/ict-and-language-ed
Searching for a graded reader can be a challenging task for students in an extensive reading program. Student using the MoodleReader View Books Block will find the categories of genre, publisher, and level but cannot determine the number of copies, location, or availability of audio CDs. In contrast, the local library search engine may display the later, but then not have the original categories in the View Books Block. These discrepancies in the categories of the search methods are caused by a lack of metadata for each graded reader. This presentation will demonstrate a Moodle Database Activity as a robust search interface to augment the MoodleReader View Book Block or as a substitute for a local library search engine. The systematic collection of metadata for readers, organization in Google sheets, and importation to a Moodle Database Activity will be demonstrated. The addition of ebooks to the Database Activity will also be displayed with discussion of how to structure an extensive reading program that is both digital and print based. Finally, opportunities for collaboration will be discussed for amassing metadata for readers to share via Google sheets, exported database entries, or a Moodle Hub.
Advanced MySQL Query Tuning - talk at Percona Live and MySQL Meetup tour.
Tuning Queries and Schema/Indexes can significantly increase performance of your application and decrease response times.
This year I will cover new MySQL 5.6 and 5.7 algorithms that has been designed to improve query performance and simply tuning.
Topics:
1. Group by and order by optimizations
2. MySQL temporary tables and filesort
3. Using covered indexes to optimize your queries
4. Loose and tight index scan in MySQL
5. Using summary tables to optimize your reporting queries
6. New MySQL 5.6 and 5.7 Optimizer features and improvements
Building High Performance MySql Query Systems And Analytic Applicationsguest40cda0b
This presentation gives practical advice and tips on how to build high-performance read intensive databases, and discusses innovations such as column-oriented databases
تأسست دار أوستن ماكولي بابليشرز في مدينة لندن عام 2006، وسرعان ما أصبحت إحدى دور النشر الواعدة في المملكة المتحدة، والتي عملت بجد لمنح المؤلفين أفضل فرصة للنجاح في السوق المزدحمة علي نحوٍ متزايد، وعزز هذا الفرصة لفتح فرع في مدينة نيويورك في الولايات المتحدة الأمريكية والشارقة في الإمارات العربية المتحدة. كما ارتبط اسمها بنشر الإبداع والمعرفة الحديثة الهادفة إلى مواكبة النهضة التي يشهدها العالم، ومن خلال سعيها الدائم إلى بناء سمعة مرموقة ووضع بصمة في صناعة النشر المحلية والعالمية.
https://austinmacauley.ae/ar/
Presented at the ICT and Language Education Conference, Kyoto, Japan. October 11th, 2015
http://kyotojalt.wix.com/ict-and-language-ed
Searching for a graded reader can be a challenging task for students in an extensive reading program. Student using the MoodleReader View Books Block will find the categories of genre, publisher, and level but cannot determine the number of copies, location, or availability of audio CDs. In contrast, the local library search engine may display the later, but then not have the original categories in the View Books Block. These discrepancies in the categories of the search methods are caused by a lack of metadata for each graded reader. This presentation will demonstrate a Moodle Database Activity as a robust search interface to augment the MoodleReader View Book Block or as a substitute for a local library search engine. The systematic collection of metadata for readers, organization in Google sheets, and importation to a Moodle Database Activity will be demonstrated. The addition of ebooks to the Database Activity will also be displayed with discussion of how to structure an extensive reading program that is both digital and print based. Finally, opportunities for collaboration will be discussed for amassing metadata for readers to share via Google sheets, exported database entries, or a Moodle Hub.
Advanced MySQL Query Tuning - talk at Percona Live and MySQL Meetup tour.
Tuning Queries and Schema/Indexes can significantly increase performance of your application and decrease response times.
This year I will cover new MySQL 5.6 and 5.7 algorithms that has been designed to improve query performance and simply tuning.
Topics:
1. Group by and order by optimizations
2. MySQL temporary tables and filesort
3. Using covered indexes to optimize your queries
4. Loose and tight index scan in MySQL
5. Using summary tables to optimize your reporting queries
6. New MySQL 5.6 and 5.7 Optimizer features and improvements
Building High Performance MySql Query Systems And Analytic Applicationsguest40cda0b
This presentation gives practical advice and tips on how to build high-performance read intensive databases, and discusses innovations such as column-oriented databases
MYSQL Query Anti-Patterns That Can Be Moved to SphinxPythian
PalominoDB European Team lead, Vladimir Fedorkov will be discussing how to handle query bottlenecks that can result from increases in dataset and traffic
Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013Jaime Crespo
Tutorial delivered at Percona MySQL Conference Live London 2013.
It doesn't matter what new SSD technologies appear, or what are the latest breakthroughs in flushing algorithms: the number one cause for MySQL applications being slow is poor execution plan of SQL queries. While the latest GA version provided a huge amount of transparent optimizations -specially for JOINS and subqueries- it is still the developer's responsibility to take advantage of all new MySQL 5.6 features.
In this tutorial we will propose the attendants a sample PHP application with bad response time. Through practical examples, we will suggest step-by-step strategies to improve its performance, including:
* Checking MySQL & InnoDB configuration
* Internal (performance_schema) and external tools for profiling (pt-query-digest)
* New EXPLAIN tools
* Simple and multiple column indexing
* Covering index technique
* Index condition pushdown
* Batch key access
* Subquery optimization
Query Optimization with MySQL 5.7 and MariaDB 10: Even newer tricksJaime Crespo
Tutorial delivered at Percona Live London 2014, where we explore new features and techniques for faster queries with MySQL 5.6 and 5.7 and MariaDB 10, including the newest options in MySQL 5.7.5 and MariaDB 10.1.
Download here the virtual machine with the example database: http://dbahire.com/pluk14
Update: WordPress has a workaround for STRICT mode: https://core.trac.wordpress.org/ticket/26847
Let's get into several common types of queries that developers struggle with, showing SQL solutions, and then analyze them for optimal efficiency. I'll cover Exclusion Join, Random Selection, Greatest-Per-Group, Dynamic Pivot, and Relational Division.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Communications Mining Series - Zero to Hero - Session 1
Zurich2007 MySQL Query Optimization
1. MySQL Queries Optimization
Date and Location: Zurich Fun tour
November 21 2007
Presenting:
Peter Zaitsev,
Percona
2. -2-
Who do we have here ?
• MySQL Developers ?
• MySQL DBAs ?
• Managers ?
Название презентации или конференции (заполняется в колонтитулах)
3. -3-
Query Optimization Basics
• Avoid the query
• Cache the query
Simplify the query
•
• Optimize the query
Название презентации или конференции (заполняется в колонтитулах)
4. -4-
Avoid the Query
• Do you really need this query run ?
– Web applications quite frequently run queries and do not
use results
• Can you get the same results from others queries
result set ?
• Can you join several queries into one ?
Название презентации или конференции (заполняется в колонтитулах)
5. -5-
Cache the Query
• We assume you could not cache your page or page
block which is even better
• Objects are often better to cache than query results
– Contain results of multiple queries
– Ready to use; save on result processing overhead
• Memcache is the leader though many variants
• Summary/Cache tables is another form of caching
Название презентации или конференции (заполняется в колонтитулах)
6. -6-
Simplify the Query
• Simplify = make query to do less work for you
• Are you really using all rows query delivers ?
What is about columns ?
•
• Can you possibly get rid of some joins to tables you
do not use ?
Название презентации или конференции (заполняется в колонтитулах)
7. -7-
Optimize the Query
• Do it when you're sure query does just what it needs
to
• Adding Indexes
• Changing Query
• Using special Tricks
• There is a whole next section about it
Название презентации или конференции (заполняется в колонтитулах)
8. -8-
Do we really look at it this way?
• No we often have to take things from the bottom
• Look at MySQL Traffic (log and profile query load)
– We have great patch for MySQL 5.0 to log queries with
microsecond accuracy and a lot of other info
• Find queries which
– Are simply too long (60 sec interactive search query)
– Queries which cause most load on the server
• Fix them
– Change queries and schema, cache them, rework
application architecture if required
Название презентации или конференции (заполняется в колонтитулах)
9. -9-
Query Optimization
• Learn how MySQL Optimizer works
• Learn how MySQL Can execute queries
– Could be it simply can't do what you want it to do
• Especially frequent problem with Oracle converts
• Differ bad indexing from complex queries
• Have valid expectations
– Group by of 10.000.000 rows will not be instant
• Learn to read EXPLAIN
• And profile queries to see what they do
Название презентации или конференции (заполняется в колонтитулах)
10. -10-
Bad indexes or Bad Queries
• SELECT * FROM USER WHERE NAME =”Peter”
– If this query does full table scan you have bad indexing
• The query only needs rows with NAME=”Peter” to execute
– Add the index
• SELECT AVG(AGE) FROM USER GROUP BY CITY
– This is complex query which needs to traverse all rows
– You can make it to traverse shorter index records
• Add index on (CITY,AGE)
– But you can't limit it to just few records
• Will need to rethink schema and add cache/summary table
Название презентации или конференции (заполняется в колонтитулах)
11. -11-
How we build summary tables ?
• Use triggers in MySQL 5.0
– Easy to use (single place code change)
– Reliable as no updates can slip through
– Can get expensive for heavy updates
• Application live Updates
– More complex and tricky but allows optimization
• Ie keeping counts on app side and updating once
• Periodically Refresh
– Great if you can afford a bit stale data
– Simple and does not affect updates
– May be too slow on large data sizes
Название презентации или конференции (заполняется в колонтитулах)
12. -12-
Indexing Basics
• MySQL Can only use Prefixes of the index
• Index (A,B) can be used for
– A=5, A=5 and B=5, A=5 and B>6
• But Can't be used for
– B=6, B<2
• Only Equality/List allows second key part usage
– A=5 and B>6 - will use 2 keyparts
– A IN (1,2) and B=2 will use 2 key parts
– A>5 and B=2 will use 1 key part only
• B=2 will be checked while reading row/index only
Название презентации или конференции (заполняется в колонтитулах)
13. -13-
Using Indexes for Order By
• Even more restricted !
• Having same index (A,B)
A=5 ORDER BY B - Will use index
•
• A>5 ORDER BY B - Will not use index
• A IN (1,2) ORDER BY B – Does not help either
A>5 ORDER BY A - Will
•
ORDER BY A ASC B DESC – Does not
•
– You have to do sorting in the same order for it to use index
Название презентации или конференции (заполняется в колонтитулах)
14. -14-
Covering Indexes
• Very Powerful, often forgotten choice
• Allows to read data from index only, not touching
data file
• Helps because index is typically smaller and it is
sorted
– (appropriate data likely needs random access)
• SELECT NAME WHERE LOGIN=”Jack234”
– KEY(LOGIN,NAME) avoid to skip data read
• It is all or none in MySQL 5.0
– Either all columns should be checked from the index or
data will be read
Название презентации или конференции (заполняется в колонтитулах)
15. -15-
LIMIT
• Limit result set. Do not fetch 100 rows if you use 10
– Though “prefetching and caching” can be helpful ie for
search applications
• LIMIT helps the most when you have index used for
sorting
– “Using Temporary”, “Using Filesort” take out most of the
LIMIT benefit
• Beware of large Limit
– LIMIT 100000,10 will fetch and discard first 100000 rows
– Do not use LIMIT cycle in batch applications
Название презентации или конференции (заполняется в колонтитулах)
16. -16-
More about LIMIT
• Limit and Rankings
• If you can precompute positions do it
– WHERE POS BETWEEN 1001 and 1010 works much
better than LIMIT 1000,10
• Beware SQL_CALC_FOUND_ROWS
– Extra count(*) may be faster because it can often use
covering index
• Still very slow for large result sets though
• Protect your application from large limits
– People may not go to page 500 but search engine bots
well may do.
Название презентации или конференции (заполняется в колонтитулах)
17. -17-
GROUP BY
• Nasty one
– hides complexity as you get just couple of rows back
• Multiple ways to execute GROUP BY
– Index traversal (or skip-scan)
• If index fully matches GROUP BY Clause
– Using temporary table
• Good for small result set, hint SQL_SMALL_RESULT
– Using filesort
• Good for large result set, hint SQL_BIG_RESULT
• Can use ORDER BY NULL to avoid extra sort
– MySQL always sorts data for group by otherwise
Название презентации или конференции (заполняется в колонтитулах)
18. -18-
JOINs
• Joins are very expensive
– 10-20 for in memory accesses, 100-1000 for disk
• MySQL only has (Optimized) nested loops joins
• If you're to traverse through a lot of rows –
denormalize
– There are other benefits as more flexible indexing
strategies as well.
• “Delayed Join” - Join only to get full rows when you
need
Название презентации или конференции (заполняется в колонтитулах)
19. -19-
Delayed Join Example
• SELECT visitor_id, url FROM (SELECT id FROM log
WHERE ip=”123.45.67.89” ORDER BY ts DESC
LIMIT 50,10) l JOIN log ON (l.id=log.id) JOIN url on
(url.id=log.url_id) ORDER BY TS DESC;
• Looks silly but it works well
Fetch from log records using covering index on (IP,TS,ID)
–
– Perform LIMIT exclusion traversing index only
– When do self join to get other columns from log table
And more info, such as page url from the joined url page
–
Название презентации или конференции (заполняется в колонтитулах)
20. -20-
Sub Queries
• Can be poorly optimized in MySQL 5.0
– Though good work in progress for MySQL 6.0
• No In-Out transformation for subqueries
– SELECT * FROM A WHERE ID IN (SELECT ID FROM B)
• No “Caching” of non-scalar resultset even when
possible
• Subselects in FROM clause result in temporary
tables without any indexes
Название презентации или конференции (заполняется в колонтитулах)
21. -21-
Views and Unions
• VIEWS – Never help performance
– But can cause problems by hiding complexity
• Can be executed as MERGE or TEMORARY TABLE
• MySQL Can't push WHERE Clauses to temporary
table
– VIEW: SELECT country, COUNT(*) cnt FROM COUNTRY
GROUP BY country;
– SELECT * FROM v WHERE country=”USA”
• Will create temporary table populate and discard the rest.
– Same applies to ORDER BY/LIMIT w UNION
• UNION ALL also needs temporary table
Название презентации или конференции (заполняется в колонтитулах)
22. -22-
That is it !
• It was short
– But I hope you've learned something
– Contact me with questions
• pz@mysqlperformanceblog.com
– Our blog
• http://www.mysqlperformanceblog.com
– Commercial Services
• http://www.percona.com
Название презентации или конференции (заполняется в колонтитулах)