This slide deck describes the Flexviews materialized view toolkit for MySQL:
http://flexvie.ws
Learn how to use incrementally refreshable materialized views, and how they can improve your performance.
This slide deck describes the Flexviews materialized view toolkit for MySQL:
http://flexvie.ws
Learn how to use incrementally refreshable materialized views, and how they can improve your performance.
Silicon Valley Code Camp 2015 - Advanced MongoDB - The SequelDaniel Coupal
MongoDB presentation from Silicon Valley Code Camp 2015.
Walkthrough developing, deploying and operating a MongoDB application, avoiding the most common pitfalls.
This presentation was prepared for a Webcast where John Yerhot, Engine Yard US Support Lead, and Chris Kelly, Technical Evangelist at New Relic discussed how you can scale and improve the performance of your Ruby web apps. They shared detailed guidance on issues like:
Caching strategies
Slow database queries
Background processing
Profiling Ruby applications
Picking the right Ruby web server
Sharding data
Attendees will learn how to:
Gain visibility on site performance
Improve scalability and uptime
Find and fix key bottlenecks
See the on-demand replay:
http://pages.engineyard.com/6TipsforImprovingRubyApplicationPerformance.html
MySQL® 5.7 is a great release which has a lot to offer, especially in the development and replication areas. It provides a lot of new optimizer features for developers to take advantage of, a much more powerful GIS function and high performance JSON data type, allowing for a more powerful store for semi-structured data. It also features dramatically improved Performance Schema, Parallel and Multi-Source replication, allowing you to scale much further than ever before, just to give you a taste. In this webinar, we will provide an overview of the most important MySQL 5.7 features.
This webinar will be part of a 3-part series which will include MySQL 5.7 for Developers and MySQL 5.7 for DBAs.
Performance Schema is a powerful diagnostic instrument for:
- Query performance
- Complicated locking issues
- Memory leaks
- Resource usage
- Problematic behavior, caused by inappropriate settings
- More
It comes with hundreds of options which allow precisely tuning what to instrument. More than 100 consumers store collected data.
In this tutorial, we will try all the important instruments out. We will provide a test environment and a few typical problems which could be hardly solved without Performance Schema. You will not only learn how to collect and use this information but have experience with it.
Tutorial at Percona Live Austin 2019
My talk for "MySQL, MariaDB and Friends" devroom at Fosdem on February 2, 2019
Born in 2010 in MySQL 5.5.3 as "a feature for monitoring server execution at a low level," grown in 5.6 times with performance fixes and DBA-faced features, in MySQL 5.7 Performance Schema is a mature tool, used by humans and more and more monitoring products. It becomes more popular over the years. In this talk I will give an overview of Performance Schema, focusing on its tuning, performance, and usability.
Performance Schema helps to troubleshoot query performance, complicated locking issues, memory leaks, resource usage, problematic behavior, caused by inappropriate settings and much more. It comes with hundreds of options which allow precisely tune what to instrument. More than 100 consumers store collected data.
Performance Schema is a potent tool. And very complicated at the same time. It does not affect performance in most cases and can slow down server dramatically if configured without care. It collects a lot of data, and sometimes this data is hard to read.
This talk will start from the introduction of how Performance Schema designed, and you will understand why it slowdowns server in some cases and does not affect your queries in others. Then we will discuss which information you can retrieve from Performance Schema and how to do it effectively.
I will cover its companion sys schema and graphical monitoring tools.
Demo on Performance Schema which I performed at DevOps Stage conference in Kiev on October 13, 2018. More at https://devopsstage.com/stranitsa-spikera/sveta-smirnova/
MySQL Performance Schema in Action: the Complete TutorialSveta Smirnova
Performance Schema is powerful diagnostic instrument for:
- Query performance
- Complicated locking issues
- Memory leaks
- Resource usage
- Problematic behavior, caused by inappropriate settings
- More
It comes with hundreds of options which allow precisely tune what to instrument. More than 100 consumers store collected data.
In this tutorial we will try all important instruments out. We will provide test environment and few typical problems which could be hardly solved without Performance Schema. You will not only learn how to collect and use this information, but have experience with it.
Made it on PerconaLive Frankfurt, 2018: https://www.percona.com/live/e18/sessions/mysql-performance-schema-in-action-the-complete-tutorial
Every website wants to become successful. Few websites however undertake the basic and fundamental steps to build a rock solid foundation to ensure a scalable
"Disaster is inevitable" and "To move forward you must first backup" should be known to all software developers. This presentation will discuss all the options for your valuable data assets in MySQL, and highlight how to maintain site reliability of your data
The History and Future of the MySQL ecosystemRonald Bradford
The history and future of the MySQL Ecosystem. This talk sub-titled “Spaghetti and MySQLBalls (with a side of greens)” detailed the beginnings of MySQL, the MySQL acquisition history, described the state of current MySQL versions/variants/forks, storage engines, related vendors, NoSQL and much more.
A video of the presentations is available on YouTube at http://www.youtube.com/watch?v=9mKwkbaB5X8&feature=youtu.be
Lessons Learned Managing Large AWS EnvironmentsRonald Bradford
How to you optimize management of 500+ AWS servers? In this presentation I share my experiences using Amazon Web Servers covering techniques for webscale. Learn how to optimized your cost, handle security, automate and be prepared for handling failure.
Monitoring your technology stack with New RelicRonald Bradford
There is no excuse to not have monitoring of your LAMP stack, NoSQL database like MongoDB/Redis/Cassandra/Memcache, Cloud services and much more when you can use the popular New Relic tool for free. As the MySQL plugin author I can offer the following link will give you access to free monitoring http://j.mp/newrelic-mysql There can never be an excuse to not know how your application is performing, from 1 server to 100+ servers.
While MySQL is a popular and widely used RDBMS, some default features and settings are very foreign in comparison with other commercial RDBMS products. In this discussion, Ronald Bradford will discuss some of the MySQL defaults including a non-transactional state, silent data truncations, date management, and transaction isolation options. These are all critical for data integrity and consistency. He will cover in-depth topics including SQL_MODE that saves the day. He will also cover character sets and collations and the best practices to ensure your UTF8 is stored and retrieved correctly.
Only after a successful preparation covered in IGNITION can you be ready for the implementation and management of a MySQL ecosystem and a successful launch of your product.
We Discuss:
* Escape Options – Before and after backup and recovery situations
* Good to Go – Knowing and confirming your MySQL environment is ready
* Full Throttle – Understanding and Improving MySQL database performance
* A Green Dashboard – Monitoring for Success
* The Human Factor – Nobody is perfect, dealing with the design changes
* Propellant – The murky mess of MySQL versions, patches and variants
* Best Practices – Proven techniques for consistency, automation and reproducibility
Note: This is volume 2 of a two part series
IGNITION is the preparation necessary for a successful launch of a MySQL ecosystem for an Oracle DBA. This volume covers the preparation needed to be ready for ongoing production administration of MySQL.
We discuss:
* Translation – Understanding the MySQL terminology
* Installation – Knowing the options for MySQL distributions
* Protection Detail – Security of MySQL information
* The Dashboard – Understanding what to monitor in MySQL
* Mechanics – Understanding more of MySQL Internals including storage engines
* Redundancy – Maintaining multiple copies via MySQL replication
* Checklists – Double checking and cross referencing your ecosystem
NOTE: This is Volume 1 of a two part series
This presentation discusses the current state of the Drizzle database project including the principles behind this leading open source project, and the healthy and growing community. For more information visit http://drizzle.org
There has been significant movement in recent times towards less structured approaches of storing and retrieving data. No longer the realm of Relational Databases, there is a new crop of structured key/value pair stores and unstructured data offerings. This closing panel debate at Open SQL Camp 2009 discussed the SQL v NoSQL topic.
Example section on MySQL for the Oracle DBA 1 day bootcamp.
In object management we look at the key SQL objects including what differs with Oracle and what is Oracle specific functionality.
We also look at the MySQL data dictionary, the INFORMATION_SCHEMA
2007 MySQL Conference and Expo 90 minute presentation specifically targeting Oracle Developers and DBAs. Topics included.
*DBA Tips, Tricks, Gotcha's & Tools
* Key Differences for Developers
* Migrating from Oracle to MySQL
MySQL for Oracle Developers and the companion MySQL for Oracle DBA's were two presentations for the 2006 MySQL Conference and Expo. These were specifically designed for Oracle resources to understand the usage, syntax and differences between MySQL and Oracle.
1. 10x Performance Improvements
in 10 steps
A MySQL Case Study
Ronald Bradford
http://ronaldbradford.com
MySQL Users Conference - 2010.04
2. Application
Typical Web 2.0 social media site (Europe based)
• Users - Visitors, Free Members, Paying Members
• Friends
• User Content - Video, Pictures
• Forums, Chat, Email
3. Server Environment
• 1 Master Database Server (MySQL 5.0.x)
• 3 Slave Database Servers (MySQL 5.0.x)
• 5 Web Servers (Apache/PHP)
• 1 Static Content Server (Nginx)
• 1 Mail Server
14. 2. Identify Problem SQL
Action 1
• Install maatkit - http://www.maatkit.org
• Install OS tcpdump (if necessary)
• Get sudo access to tcpdump
http://ronaldbradford.com/blog/take-a-look-at-mk-query-digest-2009-10-08/
15. # Rank Query ID Response time Calls R/Call Item
# ==== ================== ================ ======= ========== ====
# 1 0xB8CE56EEC1A2FBA0 14.0830 26.8% 78 0.180552 SELECT c u
# 2 0x195A4D6CB65C4C53 6.7800 12.9% 257 0.026381 SELECT u
# 3 0xCD107808735A693C 3.7355 7.1% 8 0.466943 SELECT c u
# 4 0xED55DD72AB650884 3.6225 6.9% 77 0.047046 SELECT u
# 5 0xE817EFFFF5F6FFFD 3.3616 6.4% 147 0.022868 SELECT UNION c
# 6 0x15FD03E7DB5F1B75 2.8842 5.5% 2 1.442116 SELECT c u
# 7 0x83027CD415FADB8B 2.8676 5.5% 70 0.040965 SELECT c u
# 8 0x1577013C472FD0C6 1.8703 3.6% 61 0.030660 SELECT c
# 9 0xE565A2ED3959DF4E 1.3962 2.7% 5 0.279241 SELECT c t u
# 10 0xE15AE2542D98CE76 1.3638 2.6% 6 0.227306 SELECT c
# 11 0x8A94BB83CB730494 1.2523 2.4% 148 0.008461 SELECT hv u
# 12 0x959C3B3A967928A6 1.1663 2.2% 5 0.233261 SELECT c t u
# 13 0xBC6E3F701328E95E 1.1122 2.1% 4 0.278044 SELECT c t u
16. # Query 2: 4.94 QPS, 0.13x concurrency, ID 0x195A4D6CB65C4C53 at byte 4851683
# This item is included in the report because it matches --limit.
# pct total min max avg 95% stddev median
# Count 3 257
# Exec time 10 7s 35us 492ms 26ms 189ms 78ms 332us
# Time range 2009-10-16 11:48:55.896978 to 2009-10-16 11:49:47.760802
# bytes 2 10.75k 41 43 42.85 42.48 0.67 42.48
# Errors 1 none
# Rows affe 0 0 0 0 0 0 0 0
# Warning c 0 0 0 0 0 0 0 0
# Query_time distribution
# 1us
# 10us #
# 100us ################################################################
# 1ms ####
# 10ms ###
# 100ms ########
# 1s
# 10s+
# Tables
# SHOW TABLE STATUS LIKE 'u'G
# SHOW CREATE TABLE `u`G
# EXPLAIN
SELECT ... FROM u ...G
17. 2. Identify Problem SQL
Action 2
• Wrappers to capture SQL
• Re-run on single/multiple servers
• e.g. Different slave configurations
18. 2. Identify Problem SQL
Tip
• Enable General Query Log in Development/Testing
• Great for testing Batch Jobs
19. 2. Identify Problem SQL
Action 3
Application Logic
• Show total master/slave SQL statements executed
• Show all SQL with execution time (admin user only)
Tip
• Have abstracted class/method to execute ALL SQL
26. 4. The Art of Indexes
• Different Types
• Column
• Concatenated
• Covering
• Partial
http://ronaldbradford.com/blog/understanding-different-mysql-index-implementations-2009-07-22/
27. 4. The Art of Indexes
Action 1
• EXPLAIN Output
• Possible keys
• Key used
• Key length
• Using Index
28. 4. The Art of Indexes Tip
• Generally only 1 index used per table
• Make column NOT NULL when possible
• Statistics affect index choices
• Storage engines affect operations
29. Before (7.88 seconds) After (0.04 seconds)
*************************** 2. row ** *************************** 2. row ***
id: 2 id: 2
select_type: DEPENDENT SUBQUERY select_type: DEPENDENT SUBQUERY
table: h_p table: h_p
type: ALL type: index_subquery
possible_keys: NULL possible_keys: UId
key: NULL key: UId
key_len: NULL key_len: 4
ref: NULL ref: func
rows: 33789 rows: 2
Extra: Using where Extra: Using index
ALTER TABLE h_p ADD INDEX (UId);
30. mysql> explain SELECT UID, FUID, COUNT(*) AS Count FROM f
GROUP BY UID, FUID ORDER BY Count DESC LIMIT 2000G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: f
type: index
possible_keys: NULL
key: UID
key_len: 8
ref: NULL
rows: 2151326
Extra: Using index; Using temporary; Using filesort
ALTER TABLE f DROP INDEX UID,
ADD INDEX (UID,FUID)
31. 4. The Art of Indexes
Indexes can hurt performance
35. 6. Improving SQL
• Poor SQL Examples
• ORDER BY RAND()
• SELECT *
• Lookup joins
• ORDER BY
The database is best for storing
and retrieving data not logic
37. 7. Storage Engines
• MyISAM is default
• Table level locking
• Concurrent SELECT statements
• INSERT/UPDATE/DELETE blocked by long running SELECT
• All SELECT’s blocked by INSERT/UPDATE/DELETE
• Supports FULLTEXT
38. 7. Storage Engines
• InnoDB supports transactions
• Row level locking with MVCC
• Does not support FULLTEXT
• Different memory management
• Different system variables
39. 7. Storage Engines
• There are other storage engines
• Memory
• Archive
• Blackhole
• Third party
40. 7. Storage Engines
Using Multiple Engines
• Different memory management
• Different system variables
• Different monitoring
• Affects backup strategy
44. 8. Caching
Action 1
• Memcache is your friend http://memcached.org/
-
• Cache query results
• Cache lookup data (eliminate joins)
• Cache aggregated per user information
• Caching Page Content
• Top rated (e.g. for 5 minutes)
45. 8. Caching
Action 2
• MySQL has a Query Cache
• Determine the real benefit
• Turn on or off dynamically
• SET GLOBAL query_cache_size = 1024*1024*32;
46. 8. Caching Tip
The best performance
improvement for an SQL
statement is to eliminate it.
48. 9. Sharding
• Application level horizontal and vertical partitioning
• Vertical Partitioning
• Grouping like structures together (e.g. logging, forums)
• Horizontal Partitioning
• Affecting a smaller set of users (i.e. not 100%)
49. 9. Sharding
Action 1
• Separate Logging
• Reduced replication load on primary server
56. 11. Front End Improvements
• Know your total website load time http://getfirebug.com/
-
• How much time is actually database related?
• Reduce HTML page size - 15% improvement
• Remove full URL’s, inline css styles
• Reduce/combine css & js files
• Identify blocking elements (e.g. js)
57. 11. Front End Improvements
• Split static content to different ServerName
• Spread static content over multiple ServerNames (e.g. 3)
• Sprites - Combining lightweight images http://spriteme.org/
-
• Cookie-less domain name for static content
59. Before
• Users experienced slow or unreliable load times
• Management could observe, but no quantifiable details
• Concern over load for increased growth
• Release of some new features on hold
60. Now
• Users experienced consistent load times (~60ms)
• Quantifiable and visible real-time results
• Far greater load now supported (Clients + DB)
• Better testability and verification for scaling
• New features can be deployed
Server (Ping, Uptime)
Database Processlist (Total/Active/Idle/Locked)
Database Replication (Up,Lag)
Web Server (Page Load x 3, Page Size, http processes)
Traditional Existing Methods
InnoDb, the PK value is stored
Too many indexes
Disk volume
Slows DML
Index with unused columns
Text fields
network bandwidth
temporary tables
filesort