This document discusses MySQL indexes. It begins by describing the different storage engines in MySQL, including MyISAM and InnoDB. It then covers InnoDB storage architecture and how InnoDB interacts with the file system. The main types of indexes in MySQL are described as B-tree, hash, R-tree and full-text indexes. B-tree indexes are discussed in more detail, including how they support different query types and their limitations. Other topics covered include clustered indexes, useful index-related commands like EXPLAIN, and indexing strategies.
Have you ever wondered about the internal structure of indexes in postgres? What's the difference between a btree and a gin index? How are indexes searches etc. ? This talk is about this.
Advanced MySQL Query Tuning - talk at Percona Live and MySQL Meetup tour.
Tuning Queries and Schema/Indexes can significantly increase performance of your application and decrease response times.
This year I will cover new MySQL 5.6 and 5.7 algorithms that has been designed to improve query performance and simply tuning.
Topics:
1. Group by and order by optimizations
2. MySQL temporary tables and filesort
3. Using covered indexes to optimize your queries
4. Loose and tight index scan in MySQL
5. Using summary tables to optimize your reporting queries
6. New MySQL 5.6 and 5.7 Optimizer features and improvements
Have you ever wondered about the internal structure of indexes in postgres? What's the difference between a btree and a gin index? How are indexes searches etc. ? This talk is about this.
Advanced MySQL Query Tuning - talk at Percona Live and MySQL Meetup tour.
Tuning Queries and Schema/Indexes can significantly increase performance of your application and decrease response times.
This year I will cover new MySQL 5.6 and 5.7 algorithms that has been designed to improve query performance and simply tuning.
Topics:
1. Group by and order by optimizations
2. MySQL temporary tables and filesort
3. Using covered indexes to optimize your queries
4. Loose and tight index scan in MySQL
5. Using summary tables to optimize your reporting queries
6. New MySQL 5.6 and 5.7 Optimizer features and improvements
MySQL Database Monitoring: Must, Good and Nice to HaveSveta Smirnova
It is very easy to find if a database installation is having issues. You only need to enable Operating System monitoring. A disk, memory, or CPU usage change will alert you about the problems. But they would not show *why* the trouble happens. You need the help of database-specific monitoring tools.
As a Support Engineer, I am always very upset when handling complaints about the database behavior lacking specific database monitoring data because I cannot help!
There are two reasons database and system administrators do not enable necessary instrumentation. The first is a natural or expected performance impact. Second is the lack of knowledge on what needs to be on to resolve a particular issue.
In this talk, I will cover both concerns.
I will show which monitoring instruments will give information on what causes disk, memory, or CPU problems.
I will teach you how to use them.
I will uncover which performance impact these instruments have.
I will use both MySQL command-line client and open-source graphical instrument Percona Monitoring and Management (PMM) for the examples.
Presentation that I gave as a guest lecture for a summer intensive development course at nod coworking in Dallas, TX. The presentation targets beginning web developers with little, to no experience in databases, SQL, or PostgreSQL. I cover the creation of a database, creating records, reading/querying records, updating records, destroying records, joining tables, and a brief introduction to transactions.
This presentation focuses on optimization of queries in MySQL from developer’s perspective. Developers should care about the performance of the application, which includes optimizing SQL queries. It shows the execution plan in MySQL and explain its different formats - tabular, TREE and JSON/visual explain plans. Optimizer features like optimizer hints and histograms as well as newer features like HASH joins, TREE explain plan and EXPLAIN ANALYZE from latest releases are covered. Some real examples of slow queries are included and their optimization explained.
MySQL Indexing : Improving Query Performance Using Index (Covering Index)Hemant Kumar Singh
Query performance can be enhanced by a major factor if Database Indexed are used properly. The main aim of this slide was to explain the benefits of Covering Index, but ended up writing everything I knew.
Here is the summary of what I have covered in this slide:-
1. What affects Database performance
2. What is Database Index
3. Types Of Database Index
4. Column Index
5. Composite Index
6. Covering Index
7. Indexing Guidelines
It would be interesting to know these as well -
Best practices for Indexing in Database(RDBMS)
Best practices for Indexing in MySQL
Best practices for Indexing in PostgreSQL
Best practices for Database Modeling
Best practices for SQL Query Construction
Performance impact of Indexing on Query Performance
Performance impact of Indexing on INSERT Queries
Consolidate all these knowledge and you should be happy to see the overall performance gain in your SQL Query and hence overall application will run faster.
Performance Schema is a powerful diagnostic instrument for:
- Query performance
- Complicated locking issues
- Memory leaks
- Resource usage
- Problematic behavior, caused by inappropriate settings
- More
It comes with hundreds of options which allow precisely tuning what to instrument. More than 100 consumers store collected data.
In this tutorial, we will try all the important instruments out. We will provide a test environment and a few typical problems which could be hardly solved without Performance Schema. You will not only learn how to collect and use this information but have experience with it.
Tutorial at Percona Live Austin 2019
How to Analyze and Tune MySQL Queries for Better Performanceoysteing
Tutorial at Oracle Open World 2015:
Performance of SQL queries plays a big role in application performance. If some queries execute slowly, these queries or the database schema may need tuning. This tutorial covers query processing, optimization methods, and how the MySQL optimizer chooses a specific plan to execute SQL. See demonstrations on how to use tools such as EXPLAIN (including the JSON-based variant), optimizer trace, and performance schema to analyze query plans. See how the Visual Explain functionality in MySQL Workbench helps you to visualize these plans. Based on the analysis, the tutorial covers how to take the next steps for performance tuning. It might mean forcing a particular index, changing the schema, or modifying configuration parameters.
Using and Creating SQL Functions with Ammar Hassan Brohi.
String Functions
Numeric Functions
String / Number Conversion Functions
Group Functions
Date and Time Functions
Date Conversion Functions
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
MySQL Database Monitoring: Must, Good and Nice to HaveSveta Smirnova
It is very easy to find if a database installation is having issues. You only need to enable Operating System monitoring. A disk, memory, or CPU usage change will alert you about the problems. But they would not show *why* the trouble happens. You need the help of database-specific monitoring tools.
As a Support Engineer, I am always very upset when handling complaints about the database behavior lacking specific database monitoring data because I cannot help!
There are two reasons database and system administrators do not enable necessary instrumentation. The first is a natural or expected performance impact. Second is the lack of knowledge on what needs to be on to resolve a particular issue.
In this talk, I will cover both concerns.
I will show which monitoring instruments will give information on what causes disk, memory, or CPU problems.
I will teach you how to use them.
I will uncover which performance impact these instruments have.
I will use both MySQL command-line client and open-source graphical instrument Percona Monitoring and Management (PMM) for the examples.
Presentation that I gave as a guest lecture for a summer intensive development course at nod coworking in Dallas, TX. The presentation targets beginning web developers with little, to no experience in databases, SQL, or PostgreSQL. I cover the creation of a database, creating records, reading/querying records, updating records, destroying records, joining tables, and a brief introduction to transactions.
This presentation focuses on optimization of queries in MySQL from developer’s perspective. Developers should care about the performance of the application, which includes optimizing SQL queries. It shows the execution plan in MySQL and explain its different formats - tabular, TREE and JSON/visual explain plans. Optimizer features like optimizer hints and histograms as well as newer features like HASH joins, TREE explain plan and EXPLAIN ANALYZE from latest releases are covered. Some real examples of slow queries are included and their optimization explained.
MySQL Indexing : Improving Query Performance Using Index (Covering Index)Hemant Kumar Singh
Query performance can be enhanced by a major factor if Database Indexed are used properly. The main aim of this slide was to explain the benefits of Covering Index, but ended up writing everything I knew.
Here is the summary of what I have covered in this slide:-
1. What affects Database performance
2. What is Database Index
3. Types Of Database Index
4. Column Index
5. Composite Index
6. Covering Index
7. Indexing Guidelines
It would be interesting to know these as well -
Best practices for Indexing in Database(RDBMS)
Best practices for Indexing in MySQL
Best practices for Indexing in PostgreSQL
Best practices for Database Modeling
Best practices for SQL Query Construction
Performance impact of Indexing on Query Performance
Performance impact of Indexing on INSERT Queries
Consolidate all these knowledge and you should be happy to see the overall performance gain in your SQL Query and hence overall application will run faster.
Performance Schema is a powerful diagnostic instrument for:
- Query performance
- Complicated locking issues
- Memory leaks
- Resource usage
- Problematic behavior, caused by inappropriate settings
- More
It comes with hundreds of options which allow precisely tuning what to instrument. More than 100 consumers store collected data.
In this tutorial, we will try all the important instruments out. We will provide a test environment and a few typical problems which could be hardly solved without Performance Schema. You will not only learn how to collect and use this information but have experience with it.
Tutorial at Percona Live Austin 2019
How to Analyze and Tune MySQL Queries for Better Performanceoysteing
Tutorial at Oracle Open World 2015:
Performance of SQL queries plays a big role in application performance. If some queries execute slowly, these queries or the database schema may need tuning. This tutorial covers query processing, optimization methods, and how the MySQL optimizer chooses a specific plan to execute SQL. See demonstrations on how to use tools such as EXPLAIN (including the JSON-based variant), optimizer trace, and performance schema to analyze query plans. See how the Visual Explain functionality in MySQL Workbench helps you to visualize these plans. Based on the analysis, the tutorial covers how to take the next steps for performance tuning. It might mean forcing a particular index, changing the schema, or modifying configuration parameters.
Using and Creating SQL Functions with Ammar Hassan Brohi.
String Functions
Numeric Functions
String / Number Conversion Functions
Group Functions
Date and Time Functions
Date Conversion Functions
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
4. Storage Engine: MyISAM
Table lock
No automated data recovery
No transactions
Only indexes are cached in memory
Compact storage
Friday, December 30, 11
5. Storage Engine: InnoDB
Transactional
Foreign keys
Multiversioning
Clustered by primary key
No cached count(*)
Blocking AUTO_INCREMENT
Optimized cache
Friday, December 30, 11
6. InnoDB Storage Architecture
Tablespaces Segment
leaf node segment
extent extent
non-leaf node segment
extent extent
rollback node segment
...
Extent
Row
Page
trx id
row row
row id
row row
roll pointer
...
col 1 ... col n 64 pages
Friday, December 30, 11
7. InnoDB and File system
File
system
-‐>
InnoDB
-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐
disk
partition
-‐>
tablespace
file
-‐>
segment
inode
-‐>
fsp0fsp.c
'inode'
fs
space
allocation
unit
-‐>
extent
disk
block
-‐>
page
(16
kB)
Friday, December 30, 11
15. B+tree indexes: Demo
CREATE TABLE People (
last_name varchar(50) not null,
first_name varchar(50) not null,
dob date not null,
gender enum('m', 'f') not null,
key(last_name, first_name, dob)
);
Friday, December 30, 11
17. B+Tree: Types of queries
Match the full value
where last_name=? and first_name=? and bod=?
Match a leftmost prefix
where last_name=?
where last_name=?
Friday, December 30, 11
18. B+Tree: Types of queries
Match a column prefix
where last_name like ”Liang%”
Match a range of values
where last_name > ?
Friday, December 30, 11
19. B+Tree: Types of queries
Match one part exactly and match a range
on another
where last_name=? and first_name>?
Index only queries
select first_name where last_name=?
Friday, December 30, 11
20. B+Tree: Limitations
Doesn’t start from leftmost
where first_name=?
Skip columns in the index
where last_name=? and bod=?
Friday, December 30, 11
21. B+Tree: Limitations
More than one range conditions
where last_name>? and first_name>?
Can’t optimize to the right of the first range
conditon
where last_name>? and first_name=?
Friday, December 30, 11
26. Useful commands
show index from TABLE;
explain [extended] SELECT * FROM TABLE;
UPDATE, DELETE convert to SELECT
Friday, December 30, 11
27. Explain
select_tpe
simple, primary, subquery, derived, union
type
all < index < range < ref < eq_ref < const,
system < null
Friday, December 30, 11
28. Explain
Extra
using where
using index
using filesort
using temporary
Friday, December 30, 11