This MariaDB workshop covers database concepts like tables, queries, indexes, and transactions. It discusses performance monitoring using the slow query log and tools like Percona Toolkit and Anemometer. Replication in MariaDB is explained, including traditional master-slave replication and Galera clusters. Hands-on exercises reinforce key concepts.
Introduction to MySQL Query Tuning for Dev[Op]sSveta Smirnova
To get data, we query the database. MySQL does its best to return requested bytes as fast as possible. However, it needs human help to identify what is important and should be accessed in the first place.
Queries, written smartly, can significantly outperform automatically generated ones. Indexes and Optimizer statistics, not limited to the Histograms only, help to increase the speed of the query a lot.
In this session, I will demonstrate by examples of how MySQL query performance can be improved. I will focus on techniques, accessible by Developers and DevOps rather on those which are usually used by Database Administrators. In the end, I will present troubleshooting tools which will help you to identify why your queries do not perform. Then you could use the knowledge from the beginning of the session to improve them.
Percona Live 2016 (https://www.percona.com/live/data-performance-conference-2016/sessions/why-use-explain-formatjson). Although EXPLAIN FORMAT=JSON was first presented a long time ago, there still aren't many resources that explain how and why to use it. The most advertised feature is visual EXPLAIN in MySQL Workbench, but this format can do more than create nice pictures. It prints additional information that can't be found in good old tabular EXPLAIN, and can help to solve many tricky performance issues. In this session, I will not only describe which additional information we can get with the new syntax, but also provide examples showing how to use it to diagnose production issues.
Introduction
Definitions
Why ansible
Understanding YAML
YAML Data presentation syntax
Basics getting started
Setting up ansible
Managing configuration and Inventory
Ad-hoc commands
Working with modules
Understanding playbooks
Variables, includes, imports and facts
Understanding Jinja2 templates
Working with roles
Understanding role structure
Creating roles
Deploying roles with ansible galaxy
Using the ansible galaxy CLI utility
Introduction to MySQL Query Tuning for Dev[Op]sSveta Smirnova
To get data, we query the database. MySQL does its best to return requested bytes as fast as possible. However, it needs human help to identify what is important and should be accessed in the first place.
Queries, written smartly, can significantly outperform automatically generated ones. Indexes and Optimizer statistics, not limited to the Histograms only, help to increase the speed of the query a lot.
In this session, I will demonstrate by examples of how MySQL query performance can be improved. I will focus on techniques, accessible by Developers and DevOps rather on those which are usually used by Database Administrators. In the end, I will present troubleshooting tools which will help you to identify why your queries do not perform. Then you could use the knowledge from the beginning of the session to improve them.
Percona Live 2016 (https://www.percona.com/live/data-performance-conference-2016/sessions/why-use-explain-formatjson). Although EXPLAIN FORMAT=JSON was first presented a long time ago, there still aren't many resources that explain how and why to use it. The most advertised feature is visual EXPLAIN in MySQL Workbench, but this format can do more than create nice pictures. It prints additional information that can't be found in good old tabular EXPLAIN, and can help to solve many tricky performance issues. In this session, I will not only describe which additional information we can get with the new syntax, but also provide examples showing how to use it to diagnose production issues.
Introduction
Definitions
Why ansible
Understanding YAML
YAML Data presentation syntax
Basics getting started
Setting up ansible
Managing configuration and Inventory
Ad-hoc commands
Working with modules
Understanding playbooks
Variables, includes, imports and facts
Understanding Jinja2 templates
Working with roles
Understanding role structure
Creating roles
Deploying roles with ansible galaxy
Using the ansible galaxy CLI utility
PHP classes in mumbai, Introduction to PHP/MYSQL..
best PHP/MYSQL classes in mumbai with job assistance.
our features are:
expert guidance by IT industry professionals
lowest fees of 5000
practical exposure to handle projects
well equiped lab
after course resume writing guidance
For more Visit: http://vibranttechnologies.co.in/php-classes-in-mumbai.html or http://phptraining.vibranttechnologies.co.in
Rich Model And Layered Architecture in SF2 ApplicationKirill Chebunin
Presentation for Symfony Camp UA 2012.
* What are Rich Model, Service Layer & Layered Architecture
* Layered architecture in Sf2 Application
* Integration with 3rd party bundles
Here are the slides that I gave for The Arizona Software Community meetup.
http://www.meetup.com/azsoftcom/events/222936544/
This was a gentle introduction to some of the features in EcmaScript 2015 and how and why you may use them.
From ReactPHP to Facebook Hack's Async implementation and many more, asynchronous programming has been a 'hot' topic lately. But how well does async programming support work in PHP and what can you actually use it for in your projects ? Let's look at some real-world use cases and how they leverage the power of async to do things you didn't know PHP could do.
MySQL® 5.7 is a great release which has a lot to offer, especially in the development and replication areas. It provides a lot of new optimizer features for developers to take advantage of, a much more powerful GIS function and high performance JSON data type, allowing for a more powerful store for semi-structured data. It also features dramatically improved Performance Schema, Parallel and Multi-Source replication, allowing you to scale much further than ever before, just to give you a taste. In this webinar, we will provide an overview of the most important MySQL 5.7 features.
This webinar will be part of a 3-part series which will include MySQL 5.7 for Developers and MySQL 5.7 for DBAs.
PHP classes in mumbai, Introduction to PHP/MYSQL..
best PHP/MYSQL classes in mumbai with job assistance.
our features are:
expert guidance by IT industry professionals
lowest fees of 5000
practical exposure to handle projects
well equiped lab
after course resume writing guidance
For more Visit: http://vibranttechnologies.co.in/php-classes-in-mumbai.html or http://phptraining.vibranttechnologies.co.in
Rich Model And Layered Architecture in SF2 ApplicationKirill Chebunin
Presentation for Symfony Camp UA 2012.
* What are Rich Model, Service Layer & Layered Architecture
* Layered architecture in Sf2 Application
* Integration with 3rd party bundles
Here are the slides that I gave for The Arizona Software Community meetup.
http://www.meetup.com/azsoftcom/events/222936544/
This was a gentle introduction to some of the features in EcmaScript 2015 and how and why you may use them.
From ReactPHP to Facebook Hack's Async implementation and many more, asynchronous programming has been a 'hot' topic lately. But how well does async programming support work in PHP and what can you actually use it for in your projects ? Let's look at some real-world use cases and how they leverage the power of async to do things you didn't know PHP could do.
MySQL® 5.7 is a great release which has a lot to offer, especially in the development and replication areas. It provides a lot of new optimizer features for developers to take advantage of, a much more powerful GIS function and high performance JSON data type, allowing for a more powerful store for semi-structured data. It also features dramatically improved Performance Schema, Parallel and Multi-Source replication, allowing you to scale much further than ever before, just to give you a taste. In this webinar, we will provide an overview of the most important MySQL 5.7 features.
This webinar will be part of a 3-part series which will include MySQL 5.7 for Developers and MySQL 5.7 for DBAs.
15 Ways to Kill Your Mysql Application Performanceguest9912e5
Jay is the North American Community Relations Manager at MySQL. Author of Pro MySQL, Jay has also written articles for Linux Magazine and regularly assists software developers in identifying how to make the most effective use of MySQL. He has given sessions on performance tuning at the MySQL Users Conference, RedHat Summit, NY PHP Conference, OSCON and Ohio LinuxFest, among others.In his abundant free time, when not being pestered by his two needy cats and two noisy dogs, he daydreams in PHP code and ponders the ramifications of __clone().
MySQL for business developer - Titouan BENOITTitouan BENOIT
MySQL introduction for business developer.
In this presentation you will learn to:
– Make basics queries to search data into MySQL databases
– Extract data from MySQL databases
– Use PHPMyAdmin
– Compute statistics on data (count, average, …)
Building an Analytic Extension to MySQL with ClickHouse and Open SourceAltinity Ltd
This is a joint webinar Percona - Altinity.
In this webinar we will discuss suggestions and tips on how to recognize when MySQL is overburdened with analytics and can benefit from ClickHouse’s unique capabilities.
We will then walk through important patterns for integrating MySQL and ClickHouse which will enable the building of powerful and cost-efficient applications that leverage the strengths of both databases.
Building an Analytic Extension to MySQL with ClickHouse and Open Source.pptxAltinity Ltd
Building an Analytic Extension to MySQL with ClickHouse and Open Source
In this webinar Percona and Altinity offer suggestions and tips on how to recognize when MySQL is overburdened with analytics and can benefit from ClickHouse’s unique capabilities.
Also, they will walk you through important patterns for integrating MySQL and ClickHouse which will enable the building of powerful and cost-efficient applications that leverage the strengths of both databases.
Learn how to avoid common mistakes, drill down to understand the true performance impact, and measure performance gains from corrections. Practical examples on configuration, schema, application code changes. Real-life case studies show disk space reduction, increases in throughput, and reduced query execution times.
EEvolution slides from EEUK2013 to use as a reference to our talk. Let us know if you need a hand with anything or further explanation... we know it was quite a heavy presentation.
Beyond php - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Beyond php - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Watch the full webinar at: http://embt.co/1pb4Zb4
This presentation is a must-see for anyone interested in Oracle 12! Dan is an Oracle ACE Director and has assembled this presentation with fresh and inside information from Oracle Corp and OOW13. Dan has pulled his top Oracle 12 features from the plethora of new features available and documented in his user group presentations "Oracle 12c New Features for Developers" and "Oracle 12c New Features for DBA's".
Top 10 features will include:
New SQL Syntax
New SQL and PL/SQL Limits
Pluggable Database
New Packages
Deprecated Features
New SQL Tuning Features
This presentation covers new SQL & PL/SQL syntax and options, the container DB of course, new SQL optimizer features, deprecated features, hints, and more. If you're supporting applications, then you won't want to miss this webinar!
Using MySQL without Maatkit is like taking a photo without removing the camera's lens cap. Professional MySQL experts use this toolkit to help keep complex MySQL installations running smoothly and efficiently. This session will show you practical ways to use Maatkit every day.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
2. Outline
- Tables and DDL
- Queries and DML
- Indexes and compound indexes
- Transactions and how they work, isolation levels
- Authorization and authentication, client protocol
3. Outline
- Basics of performance monitoring
- Notion of replication, types of replication
- Traditional replication in details
- Galera cluster and how it works
- MMM, PRM and query proxying
4. What’s in a box?
- Ubuntu 16.04.2
- Python 2.7.12
- MariaDB 10.0.29
- Sakila DB, Employees DB
- Percona Toolkit 2.2.16
- Anemometer
5. How to use Vagrant
- Create an empty folder
- Download https://goo.gl/ap6r6E there (rename it to
‘Vagrantfile’)
- Run ‘vagrant up’ in that folder
- Wait until a VM starts
- Run ‘vagrant ssh’ to get in
- My .mysql_history: https://goo.gl/AyrTW7
6. - Tables and DDL
- Queries and DML
- Indexes and compound indexes
- Transactions and how they work, isolation levels
- Authorization and authentication, client protocol
7. What is a table?
- A collection of related data
8. What is a table?
- A collection of related data
- Consists of columns and rows
12. How to create a table?
CREATE TABLE language (
language_id TINYINT UNSIGNED NOT NULL AUTO_INCREMENT,
name CHAR(20) NOT NULL,
last_update TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (language_id)
)ENGINE=InnoDB DEFAULT CHARSET=utf8;
13. Primary keys
- Identify a record uniquely
- So, adding two equal keys is not possible
- Can be natural like “passport number”
- Or surrogate
- Surrogate keys are auto-generated on the DB side
14. Natural PKs can be composite
CREATE TABLE film_actor (
actor_id SMALLINT UNSIGNED NOT NULL,
film_id SMALLINT UNSIGNED NOT NULL,
last_update TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (actor_id,film_id),
KEY idx_fk_film_id (`film_id`),
CONSTRAINT fk_film_actor_actor FOREIGN KEY (actor_id) REFERENCES actor (actor_id),
CONSTRAINT fk_film_actor_film FOREIGN KEY (film_id) REFERENCES film (film_id)
)ENGINE=InnoDB DEFAULT CHARSET=utf8;
15. Autoincrement primary keys
- Are surrogate
- Are 1,2,3,4 or 8 bytes long
- BTW, INT(10) is 4 bytes long
- Are incremented on every INSERT
- Should be always used
- BTW, InnoDB table is a clustered index* around its PK
- If no explicit PK exists 6-byte row ID will be used
17. - Tables and DDL
- Queries and DML
- Indexes and compound indexes
- Transactions and how they work, isolation levels
- Authorization and authentication, client protocol
18. A trivial SELECT query
- SELECT * FROM employees WHERE hire_date='1986-06-26'
- Please, never use “SELECT *”, always select certain
columns!
- A slightly better version:
- SELECT emp_no, first_name, last_name FROM employees
WHERE hire_date='1986-06-26';
19. Using a single table is impractical*
- Four types of JOINs:
- INNER JOIN
- LEFT OUTER JOIN
- RIGHT OUTER JOIN
- CROSS JOIN
- Left and right outer joins are equivalent
20. Left outer join example
- SELECT e.emp_no, first_name, last_name, salary FROM
employees e LEFT OUTER JOIN salaries s on e.emp_no =
s.emp_no WHERE hire_date='1986-06-26';
- This query selects an employee even if no payment
records exist in the salaries table
21. Aggregate queries and GROUP BY
- SELECT e.emp_no, first_name, last_name, SUM(salary)
FROM employees e LEFT OUTER JOIN salaries s on
e.emp_no = s.emp_no WHERE hire_date='1986-06-26'
GROUP BY e.emp_no;
22. How to get people w/no salary recs
- INSERT INTO employees(emp_no, first_name, last_name)
VALUES(600000, 'Alex', 'Chistyakov');
- Let’s count number of salary records using COUNT()
aggregate function
23. HAVING is like WHERE
- SELECT e.emp_no, first_name, last_name,
COUNT(salary) FROM employees e LEFT OUTER JOIN
salaries s on e.emp_no = s.emp_no GROUP BY e.emp_no
HAVING COUNT(salary) = 0;
24. Another way to do the same
- SELECT e.emp_no, first_name, last_name,
COUNT(salary) FROM employees e LEFT OUTER JOIN
salaries s on e.emp_no = s.emp_no WHERE s.emp_no IS
NULL;
- This query is more optimal*
25. Exercise #2
- Write a SELECT query which get all employees with total
sum of all salary records greater than 40000
26. - Tables and DDL
- Queries and DML
- Indexes and compound indexes
- Transactions and how they work, isolation levels
- Authorization and authentication, client protocol
27. Why indexes?
- Latency Numbers Every Programmer Should Know:
https://goo.gl/v4CEWU
- Indexes helps to avoid unnecessary disk operations
28. How indexes work?
- Index is a data structure optimized for search
- There are several types of indexes: hash indexes, B-tree
indexes
- Hash indexes allow to find exact rows
- B-tree indexes allow to find ranges
- InnoDB and Aria support B-tree indexes only
30. “SQL Tuning” by Dan Tow
- https://goo.gl/jRbD5H
- A must read for every DBA!
- Discusses how to build effective
indexes in great details
- Unfortunately does not cover
aggregate functions and sorting
31. Column cardinality
- Cardinality is a measure of data uniqueness
- Columns with more unique values have higher cardinality
- Columns with few unique values have lower cardinality
32. A composite index
- Covers two or more columns
- Allows to find rows by subsequently applying a filter
column-by-column
- Order of columns in a composite index matters!
33. Index selectivity
- An ability of a certain condition to filter
- Is expressed as a number of columns after filtering
divided by a total number of columns
- Lower values mean greater selectivity
- Some authors define selectivity as a total number of
columns divided by a resulting number of columns
34. Building a good composite index
- Columns with higher individual selectivity should go first
in a composite index
- Non-selective columns should be the latest
35. Functional indexes
- Original MySQL does not have functional indexes
- MariaDB adds support for virtual columns
- Functional indexes can be created over virtual columns
36. Virtual column example
- ALTER TABLE employees ADD lower_last_name
varchar(16) GENERATED ALWAYS AS (lower(last_name))
PERSISTENT;
- CREATE INDEX lower_last_name ON
employees(lower_last_name);
- SELECT e.emp_no, first_name, last_name FROM
employees e WHERE lower_last_name LIKE 'chistya%';
37. Let’s add %
- SELECT e.emp_no, first_name, last_name FROM
employees e WHERE lower_last_name LIKE '%chistya%';
- This will always lead to a full scan in current MariaDB
and MySQL implementations
- Full Text Search engine should be used instead
- I recommend Sphinx or Solr
38. Using ORDER BY
- In most real life cases can’t be covered by an index
- Dan Tow doesn’t consider these cases at all
- No good solution exists
39. Exercise #3
- Write a select which gets all salary records for the
employee w/emp_no = 10001 ordered by amount of the
salary record
- Create a covering index for this query
40. Things not to do in your life
- Please never ever do ORDER BY RAND()!
- How to do it properly: get a good random number on the
client side
- LIMIT 50 OFFSET 5000000 is the next thing not to do
- How to do it properly: “emp_no > $last_emp_no LIMIT
50”
41. - Tables and DDL
- Queries and DML
- Indexes and compound indexes
- Transactions and how they work, isolation levels
- Authorization and authentication, client protocol
42. A bit of history
- MySQL supported pluggable storage engines for years
- Two most notable were MyISAM and InnoDB
- MyISAM did not support transactions in any way
- InnoDB was transactional
43. MariaDB engines
- Many mysql.* tables are still in MyISAM format
- Aria storage engine emerged and is optionally
transactional in a crash-proof sense (does not support
explicit transactions though)
- InnoDB fully supports transactions
- I recommend to use InnoDB
44. A bit of InnoDB internals
- /var/lib/mysql/ib_logfile[01] are InnoDB redo logs
- The redo log works as a circular buffer
- It’s not practical to set the InnoDB log size
(innodb_log_file_size) to more than 128M
- This change requires restart
45. Generic recovery process
- Works the same way for any engine with WAL/redo
log/intent log/whatever
- The service starts after crash
- Log records are examined
- Finished transactions are applied to their final
destinations, unfinished ones are thrown out
- Aria performs these steps when in transaction mode too
46. COMMIT and auto-commit
- Every query starts and commits an implicit transaction
by default
- SET autocommit = 0; disables this
- START TRANSACTION or BEGIN should be used then to
start a transaction
- And COMMIT to finish it
- DDL statements perform COMMIT implicitly
47. ROLLBACK and savepoints
- ROLLBACK is used to abort a transaction
- Transactions can’t be nested but this behavior can be
emulated using savepoints
- SAVEPOINT label
- ROLLBACK TO label
- RELEASE SAVEPOINT label
48. A bit of InnoDB internals - MVCC
- MVCC stands for “Multiversion concurrency control”
- Records are declared dead but still occupy disk space
- InnoDB storage file never shrinks
- InnoDB uses a single file for everything by default and
this file can’t be compacted
49. It’s possible to overcome this
- innodb_file_per_table=1
- Every table will occupy a separate file (two separate
files in fact)
- Beware of Unix file descriptors limits!
- ulimit -n 65535 somewhere before starting mysqld_safe
50. Long transactions can be evil
- DDL statements require an exclusive lock on table
metadata
- An explicit transaction holds a read lock on every table it
uses
- If number of transactions per second is high enough the
DDL statement will wait forever
56. Exercise #4
- Open two different connections to the employees DB, set
autocommit to 0;
- Set isolation level to READ COMMITED in both windows,
select total number of employees whose names started
with Alex in the 1st session, delete the employee with ID
499559 in the 2nd session (don’t forget to COMMIT),
repeat the query in the 1st session
57. Exercise #4
- Set isolation level to REPEATABLE READ in both windows,
select total number of employees whose names started
with Alex in the 1st session, delete the employee with ID
499517 in the 2nd session (don’t forget to COMMIT),
repeat the query in the 1st session
58. Exercise #4
- Set isolation level to REPEATABLE READ in both windows,
select total number of employees whose names started
with Alex in the 1st session, insert an employee called
Alexis Doe in the 2nd session (don’t forget to COMMIT),
repeat the query in the 1st session
59. Exercise #4
- Set isolation level to SERIALIZABLE in both windows,
select total number of employees whose names started
with Alex in the 1st session, insert an employee called
Alex Didnotfail in the 2nd session (don’t forget to
COMMIT), repeat the query in the 1st session
60. - Tables and DDL
- Queries and DML
- Indexes and compound indexes
- Transactions and how they work, isolation levels
- Authorization and authentication, client protocol
61. mysql.user table
- Stores user privileges
- Can (but should not) be manipulated directly
- FLUSH PRIVILEGES rereads effective rights from it
- Uses MyISAM storage
62. GRANT statement
- Creates user accounts
- Grants privileges to them
- Is documented at https://goo.gl/zBHTd4
63. A superuser
- Has ALL PRIVILEGES ON *.*
- Has a number of SUPER privileges
64. A list of privileges
- Privileges can be global, database level, table level,
column level, function level and procedure level
- A list is available in GRANT command documentation
65. Default client credentials
- Can be set in ~/.my.cnf file like this:
[client]
user = root
password = Pheexaigee8a
66. Using views to limit rights
- Create a view using a privileged table columns
- Grant privileges to that view
67. Using stored procedures
- Create a stored procedure to perform AAA tasks
- Grant privileges to that stored procedure
68. MySQL wire protocol
- Is encrypted using a session key
- Can’t be easily proxied on L3 because of that
69. Exercise #5
- Grant all privileges on the employees.salaries table to a
user called “manager” with password da5ca9aeNgee%, a
user can connect from any host
- Create a view on a table employees consisting of emp_no
and the first and last names and grant a read privilege on
it to a user called “reader” with password eLegah0aez8a
70. - Basics of performance monitoring
- Notion of replication, types of replication
- Traditional replication in details
- Galera cluster and how it works
- MMM, PRM and query proxying
71. MySQL slow queries log
- The simplest way to do performance tuning
- Should be enabled in the MariaDB config file
- Slow queries will be written to a file for subsequent
analysis
72. Slow queries log config vars
- slow_query_log = on
- slow_query_log_file = /var/log/mysql/mariadb-slow.log
- long_query_time = 0.1
- log-queries-not-using-indexes
73. Analyzing the log w/Percona Tools
- pt-query-digest
- Documented at https://goo.gl/YCv1ya
- In the simplest case produces a textual report on most
time-consuming queries
74. Analyzing the log w/Anemometer
- Anemometer is a web-based slow query monitor created
at Box (https://github.com/box/Anemometer)
- Anemometer uses pt-query-digest to process the slow
query log internally
- Anemometer requires PHP, a webserver and a number of
other tools
- So, we use an Ansible role to simplify its deployment
75. Ansible role for Anemometer
- Ansible is a popular Configuration Management tool
- Ansible is written in Python and uses YAML as a
configuration description language
- A role for Anemometer is at https://goo.gl/us6V82
- This role works for Ubuntu 14.04 hosts and does not work
for 16.04 yet
- This is trivial to correct, expect a fix in a week
77. Partitioning and sharding
- Partitioning is a process of splitting a big table in smaller
subset on the same server
- Partitioning works well for time-series data
- Sharding is a process of splitting a big table in a number
of unrelated tables on different servers
- Sharding requires serious modifications of the app code
78. Partitioning in MariaDB
- MariaDB inherits MySQL support for partitioning
- Partitioning is documented at https://goo.gl/1CwIKX
- Certain limitations apply:
- Queries are not parallelized
- Partitioned table can’t contain or be referenced by
foreign keys
79. Partitioning in the real life
- Is tricky to set up properly
- Is often misused (I personally have never seen MySQL
partitioning set up properly)
- I strongly recommend not to use partitioning
80. Exercise #6
- Get familiar with the Anemometer tool
- Read and explain a query plan
81. - Basics of performance monitoring
- Notion of replication, types of replication
- Traditional replication in details
- Galera cluster and how it works
- MMM, PRM and query proxying
82. What is replication?
- Storing the same data on multiple MariaDB servers
- Establishing a master/slave relationship between the
original and the copies
- Distributing data modifications from a master node to
slave nodes
83. Master and slave nodes
- The master node gets data modification queries
(INSERTs, UPDATEs and DELETEs)
- The master node sends data changes to slaves
- Slave nodes are read-only and get updates from the
master
- Data modification on slave nodes is not prohibited in
MySQL/MariaDB world
84. Types of replication
- Replication can be synchronous or asynchronous
- Replication can also be master-slave or master-master
- All 4 options are possible: “synchronous master-slave”,
“asynchronous master-slave”, “synchronous
master-master” and “asynchronous master-master”
- Asynchronous master-slave is the default MariaDB setting
85. Master-slave and master-master
- There is only a single master in a MS replication topology
- There is more than one master in a MM setup
- A master should propagate data changes to all hosts in
the replication topology
- So, every master is also a slave in a MM setup
86. Sync or async
- Async: a transaction on a master is finished as soon as
it’s written to a transaction log on a master
- Semisync: a transaction on a master is finished only
after it’s written to a transaction log on one of slaves
- Sync: a transaction on a master is finished when it’s
acknowledged and committed on all slaves
87. Replication lag
- Replication lag is a delay between the same operations
on a master and on a slave
- Replication lag is meaningful for async replication only
- Replication lag should be minimized
88. Multi-master replication scalability
- Multi-master replication does not scale on writes!
- It’s a popular belief that it does (because there is more
than one master)
- But every master should perform exactly the same set of
write operations!
89. Multi-master tips and tricks
- Avoid writing to the same table on different masters!
- Split your schema to several non-related table sets
logically bound to different services if possible
- Work with these table sets on different masters
independently
90. - Basics of performance monitoring
- Notion of replication, types of replication
- Traditional replication in details
- Galera cluster and how it works
- MMM, PRM and query proxying
91. The binary log
- The binary log stores data modification events (both DDL
and DML changes)
- The binary log is storage neutral (works for Aria, InnoDB,
etc.)
- The binary log is not a transaction log
- The binary log can store events in 3 different formats
92. Binary log formats
- SBR (statement-based replication)
- RBR (row-based replication)
- Mixed (stores statements or rows when appropriate)
- Mixed seems to be the best of both worlds
- But it is not, in fact (avoid using it)
93. Statement-based replication
- Stores INSERT/UPDATE/DELETE and
CREATE/DROP/TRUNCATE statements as is
- Requires less space in the log
- Is not 100% accurate for all statements
94. SBR non-determinism
- INSERT INTO t1(c1, mtime) VALUES(1, NOW())
- NOW() can be different on master and slave
- INSERT INTO t2(c1, c2) VALUES(1, RAND())
- RAND() is definitely different on master and slave
- Fixes are trivial - master should send exact values
- DELETE FROM t1 LIMIT 10; - fix is not trivial
95. SBR is broken (mixed is broken too)
- Error 1062 (Duplicate entry NNN for key X)
- But why?..I just inserted a bunch of rows!
- This is a bug somehow related to range locking on a
primary key on slave side
- There is a lot of instructions on the Internet, something
like “set slave-skip-errors to 1062”
96. Never trust random Internet guys
- Don’t do “slave-skip-errors”
- To fix this bug properly…
- NEVER USE SBR OR MIXED LOG FORMATS, USE RBR!
- The only problem is that RBR is broken too
97. The binary log concept is broken
- Correctly implemented binary log stores physical changes
to the storage layer (WAL records)
- MySQL historically used pluggable storage layers, some of
them were non-transactional
- The binary log is on the wrong abstraction layer
- This can’t be easily fixed
98. RBR is broken (much less than SBR)
- DELETE FROM t1; generates a lot of rows to be written to
the binary log
- The slave can begin lagging
- A slave SQL thread uses indexes to apply row deltas
- Having a primary key is inevitable!
- It’s better to use surrogate keys
99. libslave
- A library to mimic a MySQL slave
- https://github.com/tarantool/libslave
- Can be embedded to an app, allows an app to connect to
the MySQL master and read the binlog
100. Cascading replication topologies
- Replication can (and should be)
cascaded (5 slaves on a single master
is a bad idea)
- A slave can be a master for a slave
- Config should be tweaked:
log-slave-updates=1
101. Replication rings
- If you absolutely need
master-master, you can have one
- Every master should have its own
key space
- auto_increment_offset=1
auto_increment_increment=10
102. Semisync replication
- Added since MariaDB 5.5, declared stable since 10.1.3
- Documented at https://goo.gl/wuiKfJ
- If a slave fails to acknowledge before a certain timeout,
a master switches to async automatically and switches
back when a slave catches up
103. Parallel replication
- Traditional MariaDB replication uses a single SQL thread
on the slave side
- Starting with 10.0.5 it’s possible to use several threads
- Documented at https://goo.gl/0p4SH9
104. Delayed replication
- Replication is not a backup!
- Delayed replication is (well, can be)
- Introduced in MariaDB 10.2.3
- Documented at https://goo.gl/BZguD9
- Replication delay can be achieved using pt-slave-delay
tool from Percona Toolkit
105. GTID
- Globally unique binlog events identification
- Introduced in 10.0.2
- Documented at https://goo.gl/xgJ27M
- Has a number of significant benefits: slave server can be
easily reconnected to another master, slave log position
is saved in a transactional way
106. - Basics of performance monitoring
- Notion of replication, types of replication
- Traditional replication in details
- Galera cluster and how it works
- MMM, PRM and query proxying
107. WSREP
- WSREP is a library for distributing working sets
- The Galera cluster is built around that library
108. The Galera cluster
- Is InnoDB-only
- Is semisync
- Does not use traditional replication at all
109. A common Galera cluster setup
- Two master nodes and one arbiter node
- The arbiter node does not store anything
110. - Basics of performance monitoring
- Notion of replication, types of replication
- Traditional replication in details
- Galera cluster and how it works
- MMM, PRM and query proxying