The Performance Schema in MySQL provides tables and instruments for troubleshooting issues like locks, I/O bottlenecks, slow queries, memory usage, and replication failures. It contains over 500 instruments in MySQL 5.6 and over 800 in 5.7. The tables provide visibility into the internal workings of MySQL to analyze and optimize performance.
Percona Live 2016 (https://www.percona.com/live/data-performance-conference-2016/sessions/why-use-explain-formatjson). Although EXPLAIN FORMAT=JSON was first presented a long time ago, there still aren't many resources that explain how and why to use it. The most advertised feature is visual EXPLAIN in MySQL Workbench, but this format can do more than create nice pictures. It prints additional information that can't be found in good old tabular EXPLAIN, and can help to solve many tricky performance issues. In this session, I will not only describe which additional information we can get with the new syntax, but also provide examples showing how to use it to diagnose production issues.
New features in Performance Schema 5.7 in actionSveta Smirnova
Tutorial for PerconaLive Amsterdam - September, 2015
Virtual machine can be downloaded from https://drive.google.com/file/d/0B67lYkmv0ZcsNE5wTFJUNC1sUlk/view?usp=sharing
Percona Live 2016 (https://www.percona.com/live/data-performance-conference-2016/sessions/why-use-explain-formatjson). Although EXPLAIN FORMAT=JSON was first presented a long time ago, there still aren't many resources that explain how and why to use it. The most advertised feature is visual EXPLAIN in MySQL Workbench, but this format can do more than create nice pictures. It prints additional information that can't be found in good old tabular EXPLAIN, and can help to solve many tricky performance issues. In this session, I will not only describe which additional information we can get with the new syntax, but also provide examples showing how to use it to diagnose production issues.
New features in Performance Schema 5.7 in actionSveta Smirnova
Tutorial for PerconaLive Amsterdam - September, 2015
Virtual machine can be downloaded from https://drive.google.com/file/d/0B67lYkmv0ZcsNE5wTFJUNC1sUlk/view?usp=sharing
Performance Schema for MySQL TroubleshootingSveta Smirnova
Percona Live (https://www.percona.com/live/data-performance-conference-2016/sessions/performance-schema-mysql-troubleshooting)
The performance schema in MySQL version 5.6, released in February, 2013, is a very powerful tool that can help DBAs discover why even the trickiest performance issues occur. Version 5.7 introduces even more instruments and tables. And while all these give you great power, you can get stuck choosing which instrument to use.
In this session, I will start with a description of a typical problem, then guide you how to use the performance schema to find out what causes the issue, the reason for unwanted behavior and how the received information can help you solve a particular problem.
Traditionally, performance schema sessions teach what is in contained in tables. I will, in contrast, start from a performance issue, then demonstrate which instruments and tables can help solve it. We will discuss how to setup the performance schema so that it has minimal impact on your server.
Character Encoding - MySQL DevRoom - FOSDEM 2015mushupl
Character encoding configuration in MySQL has always been a bit confusing. With too many options to set, unclear relationships between them, and the default settings that make MySQL incompatible with most languages, it is a headache to many users, many of whom end up with broken data. This lecture will provide an overview of the character set support in MySQL, guidelines on how to use it correctly, and will demonstrate several methods of detecting and repairing mangled data.
This slide deck describes the Flexviews materialized view toolkit for MySQL:
http://flexvie.ws
Learn how to use incrementally refreshable materialized views, and how they can improve your performance.
MySQL Performance - SydPHP October 2011Graham Weldon
A talk on optimisations around MySQL on the server side, and through the use of PHP extensions to reduce disk writes to provide for more IO access for MySQL. This was presented at SydPHP in October 2011
Performance Schema for MySQL TroubleshootingSveta Smirnova
Percona Live (https://www.percona.com/live/data-performance-conference-2016/sessions/performance-schema-mysql-troubleshooting)
The performance schema in MySQL version 5.6, released in February, 2013, is a very powerful tool that can help DBAs discover why even the trickiest performance issues occur. Version 5.7 introduces even more instruments and tables. And while all these give you great power, you can get stuck choosing which instrument to use.
In this session, I will start with a description of a typical problem, then guide you how to use the performance schema to find out what causes the issue, the reason for unwanted behavior and how the received information can help you solve a particular problem.
Traditionally, performance schema sessions teach what is in contained in tables. I will, in contrast, start from a performance issue, then demonstrate which instruments and tables can help solve it. We will discuss how to setup the performance schema so that it has minimal impact on your server.
Character Encoding - MySQL DevRoom - FOSDEM 2015mushupl
Character encoding configuration in MySQL has always been a bit confusing. With too many options to set, unclear relationships between them, and the default settings that make MySQL incompatible with most languages, it is a headache to many users, many of whom end up with broken data. This lecture will provide an overview of the character set support in MySQL, guidelines on how to use it correctly, and will demonstrate several methods of detecting and repairing mangled data.
This slide deck describes the Flexviews materialized view toolkit for MySQL:
http://flexvie.ws
Learn how to use incrementally refreshable materialized views, and how they can improve your performance.
MySQL Performance - SydPHP October 2011Graham Weldon
A talk on optimisations around MySQL on the server side, and through the use of PHP extensions to reduce disk writes to provide for more IO access for MySQL. This was presented at SydPHP in October 2011
Getting to Know MySQL Enterprise MonitorMark Leith
MySQL Enterprise Monitor is the monitoring and management solution for DBAs and developers delivered as part of MySQL Enterprise Edition. It provides background monitoring, alerting, trending, and analysis of the MySQL database and the statement traffic that is running within it.
View this session to learn how to install/configure, customize, and use MySQL Enterprise Monitor to suit your environment. Whether you use a single server or have hundreds of instances, MySQL Enterprise Monitor can provide great insights into how your environment is performing.
The 5.5 and 5.6 releases of MySQL introduce several new mechanisms that provide improved monitoring and performance tuning functionality. Examples are performance schemas, InnoDB metrics tables, optimizer trace, and extended explain functionality.
This session outlines the vision for monitoring-related functionality in MySQL and presents an overview of the new mechanisms. It shows how these are integrated with MySQL management tools. Furthermore, it discusses how these mechanisms can be utilized by application developers, DBAs, and production engineers for tracking down performance issues and monitoring production systems.
The slde contains an introduction to the global transaction identifiers(GTIDs) in MySQL Replication. The new protocol at re-connect, skipping transactions with GTIDS, replication filters, purging logs, backup/restore ets are covered here.
Priyanka, a MySQL cluster developer, presented MySQL cluster in the MySQL User camp. The slide deck contains an introduction to the cluster module- the architecture,
auto-sharding, failover etc in the cluster module.
MySQL 5.7 New Features for Developers session for DOAG (Oracle user group conference) in 2016. A similar version was also presented in Israel MySQL User Group on November 2016.
This presentation review new features in MySQL 5.7: Optimizer, InnoDB engine, JSON native data type, performance and sys schemas
Performance Schema is a powerful diagnostic instrument for:
- Query performance
- Complicated locking issues
- Memory leaks
- Resource usage
- Problematic behavior, caused by inappropriate settings
- More
It comes with hundreds of options which allow precisely tuning what to instrument. More than 100 consumers store collected data.
In this tutorial, we will try all the important instruments out. We will provide a test environment and a few typical problems which could be hardly solved without Performance Schema. You will not only learn how to collect and use this information but have experience with it.
Tutorial at Percona Live Austin 2019
My talk for "MySQL, MariaDB and Friends" devroom at Fosdem on February 2, 2019
Born in 2010 in MySQL 5.5.3 as "a feature for monitoring server execution at a low level," grown in 5.6 times with performance fixes and DBA-faced features, in MySQL 5.7 Performance Schema is a mature tool, used by humans and more and more monitoring products. It becomes more popular over the years. In this talk I will give an overview of Performance Schema, focusing on its tuning, performance, and usability.
Performance Schema helps to troubleshoot query performance, complicated locking issues, memory leaks, resource usage, problematic behavior, caused by inappropriate settings and much more. It comes with hundreds of options which allow precisely tune what to instrument. More than 100 consumers store collected data.
Performance Schema is a potent tool. And very complicated at the same time. It does not affect performance in most cases and can slow down server dramatically if configured without care. It collects a lot of data, and sometimes this data is hard to read.
This talk will start from the introduction of how Performance Schema designed, and you will understand why it slowdowns server in some cases and does not affect your queries in others. Then we will discuss which information you can retrieve from Performance Schema and how to do it effectively.
I will cover its companion sys schema and graphical monitoring tools.
MySQL Performance Schema in Action: the Complete TutorialSveta Smirnova
Performance Schema is powerful diagnostic instrument for:
- Query performance
- Complicated locking issues
- Memory leaks
- Resource usage
- Problematic behavior, caused by inappropriate settings
- More
It comes with hundreds of options which allow precisely tune what to instrument. More than 100 consumers store collected data.
In this tutorial we will try all important instruments out. We will provide test environment and few typical problems which could be hardly solved without Performance Schema. You will not only learn how to collect and use this information, but have experience with it.
Made it on PerconaLive Frankfurt, 2018: https://www.percona.com/live/e18/sessions/mysql-performance-schema-in-action-the-complete-tutorial
The MySQL sys schema was integrated fully into MySQL Server from version 5.7.7. Whether you are a DBA trying to determine where the resources are being used on your database instance and by whom, or a developer trying to figure out why your MySQL statements are running too slowly, the MySQL sys schema can help. Join this session to learn how to better use the MySQL sys schema to answer your day-to-day questions—from the original developer of the MySQL sys schema.
Demo on Performance Schema which I performed at DevOps Stage conference in Kiev on October 13, 2018. More at https://devopsstage.com/stranitsa-spikera/sveta-smirnova/
OSMC 2008 | Monitoring MySQL by Geert VanderkelenNETWAYS
Monitoring MySQL has a long history within Nagios. Several plugins are available already. In addition to that, there are probably lots of plugins that have been developed by the community. We take a look at some of these and discuss what kind of additional useful information could be pulled out of a MySQL Server for monitoring it even better. A simple example on how to write such plugins will be shown, also using NDB API for monitoring MySQL Cluster. Now that MySQL Enterprise Monitor (MEM) is available, we'll go through the possibilities for combining the two platforms. We will also discuss the NDOUtils for storing configuration and event data using MySQL.
This talk starts with a brief overview of MySQL itself: some history, where it's heading too, and why it is so successful.
MariaDB 10.5 new features for troubleshooting (mariadb server fest 2020)Valeriy Kravchuk
The recently released MariaDB 10.5 GA includes many new, useful features, but I’d like to concentrate on those helping DBAs and support engineers to find out what’s going on when a problem occurs.
Specifically I present and discuss the Performance Schema updates to match MySQL 5.7 instrumentation, new tables in the INFORMATION_SCHEMA to monitor the internals of a generic thread pool and improvements of ANALYZE for statements.
MySQL® 5.7 is a great release which has a lot to offer, especially in the development and replication areas. It provides a lot of new optimizer features for developers to take advantage of, a much more powerful GIS function and high performance JSON data type, allowing for a more powerful store for semi-structured data. It also features dramatically improved Performance Schema, Parallel and Multi-Source replication, allowing you to scale much further than ever before, just to give you a taste. In this webinar, we will provide an overview of the most important MySQL 5.7 features.
This webinar will be part of a 3-part series which will include MySQL 5.7 for Developers and MySQL 5.7 for DBAs.
MySQL 2024: Зачем переходить на MySQL 8, если в 5.х всё устраивает?Sveta Smirnova
25 октябрая 2023 года Oracle прекратила активную поддержку MySQL 5.7.
Это значит, что стоит присмотреться к улучшениям в версии 8:
- Новому системному словарю
- Современному SQL
- Поддержке JSON, NoSQL, MySQL Shell, и возможности работать с MySQL как с MongoDB
- Улучшениям в оптимизаторе запросов и диагностике
Мой доклад для разработчиков приложений под MySQL. Я не буду рассказывать как конфигурировать сервер и сфокусируюсь на его использовании.
Database in Kubernetes: Diagnostics and MonitoringSveta Smirnova
Kubernetes is the new cool in 2023. Many database installations are on Kubernetes now. And this creates challenges for Support engineers because traditional monitoring and diagnostic tools work differently on bare hardware and Kubernetes. In this session, I will focus on differences in methods we use to collect metrics, describe challenges that Percona Support hits when working with database installations on Kubernetes, and discuss how we resolve them. This talk will cover all database technologies we support: MySQL, MongoDB, and PostgreSQL.
Presented at Percona Live 2023
MySQL Database Monitoring: Must, Good and Nice to HaveSveta Smirnova
It is very easy to find if a database installation is having issues. You only need to enable Operating System monitoring. A disk, memory, or CPU usage change will alert you about the problems. But they would not show *why* the trouble happens. You need the help of database-specific monitoring tools.
As a Support Engineer, I am always very upset when handling complaints about the database behavior lacking specific database monitoring data because I cannot help!
There are two reasons database and system administrators do not enable necessary instrumentation. The first is a natural or expected performance impact. Second is the lack of knowledge on what needs to be on to resolve a particular issue.
In this talk, I will cover both concerns.
I will show which monitoring instruments will give information on what causes disk, memory, or CPU problems.
I will teach you how to use them.
I will uncover which performance impact these instruments have.
I will use both MySQL command-line client and open-source graphical instrument Percona Monitoring and Management (PMM) for the examples.
MySQL Cookbook 4th edition was released this summer. We are the book's authors and will show you how to "cook" MySQL. We will show you a few tasks with different priorities, such as JSON in MySQL for those who need flexibility, modern SQL for analytics, and Group Replication for high availability. We will also show how to write programs using JavaScript and Python languages, X DevAPI, and MySQL Shell. We will touch on some of the exciting features of MySQL Spatial Indexes and Geographical Data, Using a Full-Text Search, and more. We're hoping this talk will be interesting for both developers and administrators of MySQL.
MySQL performance can be improved by tuning queries, server options, and hardware. Traditionally it was an area of responsibility for three different roles: Development, DBA, and System Administrators. Now DevOps handle these all. But there is a gap. Knowledge gained by MySQL DBAs after years or focusing on a single product is hard to gain when you focus on more than one. This is why I am doing this session. I will show a minimal but most effective set of options to improve MySQL performance. For illustrations, I will use real user stories gained from my Support experience and Percona Kubernetes operators for PXC and MySQL.
MySQL Test Framework для поддержки клиентов и верификации баговSveta Smirnova
Talk for TestDriven Conf: https://tdconf.ru/2022/abstracts/8763
MySQL Test Framework (MTR) — это фреймворк для регрессионных тестов MySQL. Тесты для него пишут разработчики MySQL и запускаются во время подготовки к новым релизам.
MTR можно использовать и по-другому. Я его использую, чтобы тестировать проблемы, о которых сообщают клиенты, и подтверждать сообщения об ошибках (bug reports) одновременно на нескольких версиях MySQL.
При помощи MTR можно:
* программировать сложные развёртывания;
* тестировать проблему на нескольких версиях MySQL/Percona/MariaDB-серверов при помощи одной команды;
* тестировать несколько одновременных соединений;
* проверять ошибки и возвращаемые значения;
* работать с результатами запросов, хранимыми процедурами и внешними командами.
Тест может быть запущен на любой машине с MySQL-, Percona- или MariaDB-сервером.
Я покажу, как я работаю с MySQL Test Framework, и надеюсь, что вы тоже полюбите этот инструмент.
These slides are for my talk at Percona Live 2022: https://sched.co/10KEo
MySQL Cookbook 4th edition (https://www.target.com/p/mysql-cookbook-4th-edition-by-sveta-smirnova-alkin-tezuysal-paperback/-/A-85851771) is planned to be released this spring. I am one of the authors of the book and will show you how to "cook" MySQL. I will show you a few tasks with different priorities, such as JSON in MySQL for those who need flexibility; modern SQL for analytics, and Group Replication for high availability. I will also show how to write programs using JavaScript and Python languages, X DevAPI, and MySQL Shell. I expect this talk will be interesting for MySQL application developers.
Introduction into MySQL Query Tuning for Dev[Op]sSveta Smirnova
Percona Live Online 2021 talk: https://www.percona.com/resources/videos/introduction-mysql-query-tuning-for-devops
In this talk I will show how to get started with MySQL Query Tuning. I will make a short introduction into physical table structure and demonstrate how it may influence query execution time.
Then we will discuss basic query tuning instruments and techniques, mainly EXPLAIN command with its latest variations. You will learn how to understand its output and how to rewrite queries or change table structure to achieve better performance.
Talk for the DevOps Pro Moscow 2021: https://www.devopspro.ru/Sveta-Smirnova/
Производительность MySQL можно улучшить при помощи оптимизации запросов, настроек MySQL сервера и железа. Традиционно эти задачи распределялись между тремя ролями: Разработчик, Администратор баз данных и Системный Администратор. Теперь же все эти задачи решает DevOps, что непросто для одного человека. В этом докладе я расскажу об основных оптимизациях, которые решают большинство проблем производительности MySQL. Для иллюстраций я буду использовать реальные пользовательские истории и Percona Kubernetes Operator.
MySQL performance can be improved by tuning queries, server options, and hardware. Traditionally it was an area of responsibility of three different roles: Development, DBA and System Administrators. Now DevOps handle these all. But there is a gap. Knowledge, gained by MySQL DBAs after years or focus on the single product is hard to gain when you focus on more than one. This is why I am doing this session. I will show minimal, but the most effective, set of options which will improve MySQL performance. For illustrations, I will use real user stories, gained by my Support experience, and Kubernetes operators, now available from all main MySQL eco-system vendors: Oracle, MariaDB, and Percona.
Presented at Open Source Summit Europe 2020: https://sched.co/eCGf
How to Avoid Pitfalls in Schema Upgrade with Percona XtraDB ClusterSveta Smirnova
Percona XtraDB Cluster (PXC) is a 100% synchronized cluster in regards to DML operations. It is ensured by the optimistic locking model and ability to rollback transaction which cannot be applied on all nodes. However, DDL operations are not transactional in MySQL. This adds complexity when you need to change the schema of the database.
Changes made by DDL may affect the results of the queries. Therefore all modifications must replicate on all nodes prior to the next data access. For operations that run momentarily, it can be easily achieved, but schema changes may take hours to apply. Therefore in addition to the safest synchronous blocking schema upgrade method: TOI, - PXC supports more relaxed, though not safe, method RSU.
RSU: Rolling Schema Upgrade is advertised to be non-blocking. But you still need to take care of updates, running while you are performing such an upgrade. Surprisingly, even updates on not related tables and schema can cause RSU operation to fail.
In this talk, I will uncover nuances of PXC schema upgrades and point to details you need to take special care about.
Further Information
Schema change is a frequent task, and many do not expect any surprises with it. However, the necessity to replay the changes to all synchronized nodes adds complexity. I made a webinar on a similar topic which was recorded and available for replay. Now I have found that I share a link to the webinar to my Support customers approximately once per week. Not having a good understanding of how schema change works in the cluster leads to lockups and operation failures. This talk will provide a checklist that will help to choose the best schema change method.
Presented at Percona Live Online: https://perconaliveonline2020.sched.com/event/ePm2/how-to-avoid-pitfalls-in-schema-upgrade-with-percona-xtradb-cluster
How to migrate from MySQL to MariaDB without tearsSveta Smirnova
Presented at MariaDB Server Fest 2020: https://mariadb.org/fest2020/migrate-mysql/
MariaDB is a drop-in replacement for MySQL. Initial migration is simple: start MariaDB over the old MySQL datadir.
Later your application may notice that some features work differently than with MySQL. These are MariaDB improvements, so this is good and, likely the reason you migrated.
In this session, I will focus on the differences affecting application performance and behavior. In particular, features sharing the same name, but working differently.
Modern solutions for modern database load: improvements in the latest MariaDB...Sveta Smirnova
Presented at MariaDB Server Fest 2020: https://mariadb.org/fest2020/improvements/
MariaDB is famous for working well in high-performance environments. But our view of what to call high-performance changes over time. Every year we get faster data transfer speed; more devices connected to the Internet; more users and, as a result, more data.
Challenges, which developers have to solve, are getting harder. This session shows what engineers do to keep the product up to date, focusing on MariaDB improvements that make it different from its predecessor, MySQL.
How Safe is Asynchronous Master-Master Setup?Sveta Smirnova
Presented at Percona MySQL Tech Day on September 10, 2020: https://www.percona.com/tech-days#mysql
It is common knowledge that built-in asynchronous active-active replication is not safe. I remember times when the official MySQL User Reference Manual stated that such an installation is not recommended for production use. Some experts repeat this claim even now.
While this statement is generally true, I worked with thousands of shops that successfully avoided asynchronous replication limitations in active-active setups.
In this talk, I will show how they did it, demonstrate situations when asynchronous source-source replication is the best possible high availability option and beats such solutions as Galera or InnoDB Clusters. I will also cover common mistakes, leading to disasters.
Современному хайлоду - современные решения: MySQL 8.0 и улучшения PerconaSveta Smirnova
MySQL всегда использовали под высокой нагрузкой. Недаром эта база была и остаётся самым популярным бэкэндом для web. Однако наши представления о хайлоде с каждым годом расширяются. Большая скорость передачи данных -> больше устройств с подключением к интернет -> больше пользователей -> больше данных.
Задачи, стоящие перед разработчиками MySQL, с каждым годом усложняются.
В этом докладе я расскажу как менялись сценарии использования MySQL за [почти] 25 лет её истории и что делали инженеры, чтобы MySQL оставалась актуальной. Мы затронем такие темы, как работа с большим количеством активных соединений и высокими объёмами данных. Я покажу насколько современные версии лучше справляются с возросшими нагрузками.
Я надеюсь, что после моего доклада те слушатели, которые используют старые версии, захотят обновиться и те, кто уже обновились, узнают как использовать современный MySQL на полную мощность.
Прочитана на конференции OST 2020: https://ostconf.com/materials/2857#2857
How to Avoid Pitfalls in Schema Upgrade with GaleraSveta Smirnova
Galera Cluster for MySQL is a 100% synchronized cluster in regards to data modification operations (DML). It is ensured by the optimistic locking model and ability to rollback a transaction, which cannot be applied on all nodes. However, schema changes (DDL operations) are not transactional in MySQL, which adds complexity when you need to perform an upgrade or change schema of the database.
Changes made by DDL may affect results of the queries. Therefore all modifications must replicate on all nodes prior next data access. For operations which run momentarily it can be easily achieved, but schema changes may take hours to apply. Therefore in addition to safest synchronous blocking schema upgrade method TOI Galera also supports more relaxed, thought not safe, method RSU.
In her talk Sveta will describe which pitfalls you can hit while performing the change using one or another method, why and how to avoid them.
Presented at MariaDB Day Brussels 0202 2020: https://mariadb.org/mariadb-day-brussels-0202-2020-provisional-schedule/
How Safe is Asynchronous Master-Master Setup?Sveta Smirnova
It is common knowledge that built-in asynchronous master-master (active-active) replication is not safe. I remember times when the official MySQL User Reference Manual stated that such an installation is not recommended for production use. Some experts repeat this claim even now.
While this statement is generally true, I worked with thousands of shops that successfully avoided asynchronous replication limitations in active-active setups.
In this talk, I will show how they did it, demonstrate situations when asynchronous master-master replication is the best possible high availability option and beats such solutions as Galera or InnoDB Clusters. I will also cover common mistakes, leading to disasters.
Presented in "MySQL, MariaDB and Friends devroom" at Fosdem in 2020: https://fosdem.org/2020/schedule/event/mysql_master_master/
Introduction to MySQL Query Tuning for Dev[Op]sSveta Smirnova
To get data, we query the database. MySQL does its best to return requested bytes as fast as possible. However, it needs human help to identify what is important and should be accessed in the first place.
Queries, written smartly, can significantly outperform automatically generated ones. Indexes and Optimizer statistics, not limited to the Histograms only, help to increase the speed of the query a lot.
In this session, I will demonstrate by examples of how MySQL query performance can be improved. I will focus on techniques, accessible by Developers and DevOps rather on those which are usually used by Database Administrators. In the end, I will present troubleshooting tools which will help you to identify why your queries do not perform. Then you could use the knowledge from the beginning of the session to improve them.
Billion Goods in Few Categories: How Histograms Save a Life?Sveta Smirnova
We store data with an intention to use it: search, retrieve, group, sort... To do it effectively the MySQL Optimizer uses index statistics when compiles the query execution plan. This approach works excellently unless your data distribution is not even.
Last year I worked on several tickets where data follow the same pattern: millions of popular products fit into a couple of categories and rest used the rest. We had a hard time to find a solution for retrieving goods fast. We offered workarounds for version 5.7. However new MariaDB and MySQL 8.0 feature: histograms, - would work better, cleaner and faster. The idea of the talk was born.
Of course, histograms are not a panacea and do not help in all situations.
I will discuss:
how index statistics physically stored by the storage engine
which data exchanged with the Optimizer
why it is not enough to make correct index choice
when histograms can help and when they cannot
differences between MySQL and MariaDB histograms
A Billion Goods in a Few Categories: When Optimizer Histograms Help and When ...Sveta Smirnova
Last year this session’s speaker worked on several cases where data followed the same pattern: millions of popular products fit into a couple of categories, and the rest uses the rest. Her team had a hard time finding a solution for retrieving goods quickly. MySQL 8.0 has a feature that resolves such issues: optimizer histograms, storing statistics of an exact number of values in each data bucket. In real life, histograms don’t help with all queries accessing nonuniform data. How you write a statement, the number of rows in the table, data distribution: All of these may affect the use of histograms. This presentation shows examples demonstrating how the optimizer works in each case, describes how to create histograms, and covers differences between MySQL and Oracle implementations.
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Understanding Globus Data Transfers with NetSageGlobus
NetSage is an open privacy-aware network measurement, analysis, and visualization service designed to help end-users visualize and reason about large data transfers. NetSage traditionally has used a combination of passive measurements, including SNMP and flow data, as well as active measurements, mainly perfSONAR, to provide longitudinal network performance data visualization. It has been deployed by dozens of networks world wide, and is supported domestically by the Engagement and Performance Operations Center (EPOC), NSF #2328479. We have recently expanded the NetSage data sources to include logs for Globus data transfers, following the same privacy-preserving approach as for Flow data. Using the logs for the Texas Advanced Computing Center (TACC) as an example, this talk will walk through several different example use cases that NetSage can answer, including: Who is using Globus to share data with my institution, and what kind of performance are they able to achieve? How many transfers has Globus supported for us? Which sites are we sharing the most data with, and how is that changing over time? How is my site using Globus to move data internally, and what kind of performance do we see for those transfers? What percentage of data transfers at my institution used Globus, and how did the overall data transfer performance compare to the Globus users?
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Shahin Sheidaei
Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
A Comprehensive Look at Generative AI in Retail App Testing.pdfkalichargn70th171
Traditional software testing methods are being challenged in retail, where customer expectations and technological advancements continually shape the landscape. Enter generative AI—a transformative subset of artificial intelligence technologies poised to revolutionize software testing.
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar
The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month.
The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies.
However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News.
Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Experience our free, in-depth three-part Tendenci Platform Corporate Membership Management workshop series! In Session 1 on May 14th, 2024, we began with an Introduction and Setup, mastering the configuration of your Corporate Membership Module settings to establish membership types, applications, and more. Then, on May 16th, 2024, in Session 2, we focused on binding individual members to a Corporate Membership and Corporate Reps, teaching you how to add individual members and assign Corporate Representatives to manage dues, renewals, and associated members. Finally, on May 28th, 2024, in Session 3, we covered questions and concerns, addressing any queries or issues you may have.
For more Tendenci AMS events, check out www.tendenci.com/events
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...informapgpstrackings
Keep tabs on your field staff effortlessly with Informap Technology Centre LLC. Real-time tracking, task assignment, and smart features for efficient management. Request a live demo today!
For more details, visit us : https://informapuae.com/field-staff-tracking/
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
Globus Connect Server Deep Dive - GlobusWorld 2024
Performance Schema for MySQL Troubleshooting
1. Performance Schema for MySQL Troubleshooting
Sveta Smirnova
Senior Principal Technical Support Engineer
2. Overview
What is inside?
• MySQL 5.6
52 tables
554 instruments
31 variables
• MySQL 5.7
76 tables
893 instruments
40 variables
3. Overview
What can you troubleshoot?
• Internal server’s bottlenecks, caused by locks,
mutexes, IO
• Less optimal statements
• Most expensive operations
• Connection issues
• Memory usage
• Replication failures
• More
4. Basic setup
• Setup tables
UPDATE performance_schema.setup_instruments
SET enabled=’yes’, timed=’yes’
WHERE name LIKE ’stage/%’;
UPDATE performance_schema.setup_consumers
SET enabled=’yes’ WHERE name LIKE ...;
...
• Install sys schema
https://github.com/MarkLeith/mysql-sys
Install functions and run stored routine
$mysql < sys_57.sql
CALL sys.ps_setup_enable_consumers(’waits’);
CALL sys.ps_setup_enable_instrument(’’);
5. What happens inside MySQL server?
Available since version 5.5
• setup instruments.NAME
wait/io/file
• Operations with files
wait/io/socket
wait/io/table/sql/handler
wait/lock/table/sql/handler
wait/synch/cond
• InnoDB, MyISAM, sql
wait/synch/mutex
• sql, mysys, storage engines
wait/synch/rwlock/
• InnoDB, MyISAM, sql
6. What happens inside MySQL server?
Usage example
mysql> ALTER TABLE sbtest ADD KEY(c,pad);
...
• Run some load in parallel client
• Check Performance Schema using third connection
mysql> SELECT EVENT_NAME, COUNT_STAR,
-> AVG_TIMER_WAIT FROM performance_schema.
-> events_waits_summary_global_by_event_name
-> WHERE count_star>0
-> ORDER BY avg_timer_wait DESC, count_star DESCG
<See next slide>
8. What happens inside MySQL server?
Usage example, page 3
mysql> SELECT file, count_write cw,
-> total_written tw
-> FROM sys.io_global_by_file_by_bytes
-> WHERE count_write>0 ORDER BY count_write DESC;
+---------------------------------+-----+---------+
| file | cw | tw |
+---------------------------------+-----+---------+
| @@datadir/sbtest/sbtest.ibd |16830|479.53MiB|
| @@datadir/ibdata1 |4072 |372.44MiB|
| @@datadir/Innodb Merge Temp File| 1595| 1.56GiB |
...
9. Why your statements are slow?
Available since version 5.6
• events statements * tables
Statements
• statement/sql/delete
• statement/sql/select
• ...
Commands
• COM PING, COM QUIT, ...
• statement/com/Ping
• statement/com/Quit
Errors
• statement/sql/error
• statement/com/Error
• statement/sp/error
Stored Routines
• statement/sp
10. Statements tables: usage example
Which queries do not use indexes?
mysql> SELECT THREAD_ID AS TID, SUBSTR(SQL_TEXT, 1, 50)
-> AS SQL_TEXT, ROWS_SENT AS RS, ROWS_EXAMINED AS RE,
-> CREATED_TMP_TABLES, NO_INDEX_USED,
-> NO_GOOD_INDEX_USED FROM performance_schema.
-> events_statements_history WHERE NO_INDEX_USED=1
-> OR NO_GOOD_INDEX_USED=1G
********************** 1. row **********************
TID: 10124
SQL_TEXT: select emp_no, first_name,
last_name from employee
RS: 97750
RE: 397774
CREATED_TMP_TABLES: 0
NO_INDEX_USED: 1
NO_GOOD_INDEX_USED: 0
...
11. Statement tables: take it easy
Indes usage with sys schema
mysql> SELECT query, total_latency,
-> no_index_used_count, rows_sent,
-> rows_examined FROM
-> sys.statements_with_full_table_scans
-> WHERE db=’employees’ AND
-> query NOT LIKE ’%performance_schema%’G
********************** 1. row **********************
query: SELECT COUNT ( ‘emp_no‘ ) FROM ...
WHERE ‘title‘ = ?
total_latency: 805.37 ms
no_index_used_count: 1
rows_sent: 1
rows_examined: 397774
...
12. Statement tables
What else is worth attention?
• Field names
CREATED TMP DISK TABLES
CREATED TMP TABLES
SELECT FULL JOIN
SELECT RANGE CHECK
SELECT SCAN
SORT MERGE PASSES
SORT SCAN
• Views in sys schema
statement analysis
statements with runtimes in 95th percentile
statements with temp tables
statements with sorting
statements with full table scans
statements with errors or warnings
13. Which operations are most expensive?
Available since version 5.6
• events stages * tables
• Same information which you see in table
INFORMATION SCHEMA.PROCESSLIST or SHOW
PROCESSLIST output
init
executing
Opening tables
...
• Replacement for SHOW PROFILE
• Only server-level
• No information from storage engine in this table!
14. events stages * tables: usage example
Which states took critically long time?
mysql> SELECT eshl.event_name, sql_text,
-> eshl.timer_wait/1000000000000 wait_s
-> FROM performance_schema.events_stages_history_long
-> eshl JOIN performance_schema.
-> events_statements_history_long esthl ON
-> (eshl.nesting_event_id = esthl.event_id)
-> WHERE eshl.timer_wait > 1*10000000000G
********************** 1. row **********************
event_name: stage/sql/Sending data
sql_text: select count(emp_no) from employees
join salaries
using(emp_no) where hire_date = from_date
wait_s: 0.8170
1 row in set (0.00 sec)
15. events stages * tables
What else is worth attention?
• Everything, related to temporary tables
EVENT NAME LIKE ’stage/sql/%tmp%’
• Everything, related to locks
EVENT NAME LIKE ’stage/sql/%lock%’
• Everything in state ”Waiting for”
EVENT NAME LIKE ’stage/%/Waiting for%’
• Frequently met issues (based on MySQL Support
Team experience)
EVENT NAME=’stage/sql/end’
EVENT NAME=’stage/sql/freeing items’
EVENT NAME=’stage/sql/Sending data’
EVENT NAME=’stage/sql/cleaning up’
EVENT NAME=’stage/sql/closing tables’
16. Where is all your RAM?
Available since version 5.7
• Memory summary tables
• sys.memory * views
• Detailed information about memory usage of each
server’s internal
• Total amount of memory, allocated by the server
mysql> SELECT * FROM sys.memory_global_total;
+-----------------+
| total_allocated |
+-----------------+
| 458.44 MiB |
+-----------------+
17. Memory summary tables
Usage example
mysql> SELECT thread_id, user, current_avg_alloc, curre
-> FROM sys.memory_by_thread_by_current_bytes
-> WHERE thread_id IN 145, 146)G
********************** 1. row **********************
thread_id: 145
user: sql/slave_io
current_allocated: 1.04 GiB
current_avg_alloc: 2.64 KiB
********************** 2. row **********************
thread_id: 146
user: sql/slave_sql
current_allocated: 1.79 MiB
current_avg_alloc: 72 bytes
2 rows in set (0.11 sec)
18. Memory summary tables
What else is worth attention?
• Tables in Performance Schema
memory summary global by event name
memory summary by account by event name
memory summary by host by event name
memory summary by thread by event name
memory summary by user by event name
• Views in sys schema
memory global total
memory by thread by current bytes
memory by host by current bytes
memory by user by current bytes
memory global by current allocated
19. What did I miss?
You can do even more!
• MDL locks instrumentation
• host cache table and all information you need to
troubleshoot connection issues
• Replication tables: you don’t need to parse SHOW
TABLE STATUS output in your scripts anymore
• Hundreds of instruments
• How to tune Performance schema
23. The preceding is intended to outline our general product
direction. It is intended for information purposes only, and
may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality,
and should not be relied upon in making purchasing
decisions. The development, release, and timing of any
features or functionality described for Oracle’s products
remains at the sole discretion of Oracle.