MySQL Scalability Mistakes - OTN

Common MySQL Scalability Mistakes OTN LAD Tour South America 2010.10 Ronald Bradford http://ronaldbradford.com Ronald Bradford www.RonaldBradford.com Common MySQL Scalability Mistakes - 2010.10

9 Common MySQL Scalability Mistakes - 2010.10

PROBLEM My website seems to freeze or responds randomly? Common MySQL Scalability Mistakes - 2010.10

CAUSE The default MyISAM storage engine uses exclusive table locks for DML. Common MySQL Scalability Mistakes - 2010.10

SOLUTION Optimize blocking query performance Use a transactional engine with MVCC and row based locking to address the LOCKING issue Common MySQL Scalability Mistakes - 2010.10

EXAMPLE End user report that selects all customer, order, order lines and order history data and performs poor joins. This takes shared read locks blocking future write locks then future reads. Common MySQL Scalability Mistakes - 2010.10

WHY MySQL is unique in that it offers different mechanisms for storing and retrieving data, each with strengths and weaknesses. The DEFAULT is not always the best. Common MySQL Scalability Mistakes - 2010.10

HOW Optimize Blocker Indexes Limit query Summary table Change storage engine Common MySQL Scalability Mistakes - 2010.10

PROBLEM Why is my database so large? Common MySQL Scalability Mistakes - 2010.10

SOLUTION Don’t store large static objects in the database Common MySQL Scalability Mistakes - 2010.10

EXAMPLE 80% of data is email content/ attachments 60% of data is PDF documents 30% of data is uncompressed large XML objects Common MySQL Scalability Mistakes - 2010.10

WHY Maximize memory usage for important data Reduce database recovery time Common MySQL Scalability Mistakes - 2010.10

HOW Compress large text data 90% saving on XML data Store static data in files Avoids DB handling overhead Common MySQL Scalability Mistakes - 2010.10

HOW Table Size per schema # Schema Table Usage SELECT table_schema,table_name,engine,row_format, table_rows, avg_row_length, (data_length+index_length)/1024/1024 as total_mb, (data_length)/1024/1024 as data_mb, (index_length)/1024/1024 as index_mb, CURDATE() AS today FROM information_schema.tables WHERE table_schema = DATABASE() ORDER BY 7 DESC; http://ronaldbradford.com/mysql-dba/ Common MySQL Scalability Mistakes - 2010.10

PROBLEM I can't access my website? Common MySQL Scalability Mistakes - 2010.10

TRUE STORY Common MySQL Scalability Mistakes - 2010.10

TRUE STORY Question: Common MySQL Scalability Mistakes - 2010.10

TRUE STORY Question: How do you know when your server is down or not accessible? Common MySQL Scalability Mistakes - 2010.10

TRUE STORY Question: How do you know when your server is down or not accessible? Answer: Common MySQL Scalability Mistakes - 2010.10

TRUE STORY Question: How do you know when your server is down or not accessible? Answer: The users will let us know. Common MySQL Scalability Mistakes - 2010.10

SOLUTION Integrated monitoring including graphical interface, real time analysis and notification Common MySQL Scalability Mistakes - 2010.10

HOW Monitoring/Alerting Graphical Historical Necessary Generally missing/incomplete Useless for real-time analysis Common MySQL Scalability Mistakes - 2010.10

HOW Dashboard The state of NOW Sampling at 1s/3s/5s e.g. 0.1% of throughput Common MySQL Scalability Mistakes - 2010.10

HOW Instrumentation Important to business viability e.g. orders per minute page load time Seamless implementation i.e. no code changes to view real-time extensible Common MySQL Scalability Mistakes - 2010.10

PROBLEM My replication slave can't keep up? Common MySQL Scalability Mistakes - 2010.10

SOLUTION Know the weakest link(s) of MySQL replication and don't exceed that, or cheat. Common MySQL Scalability Mistakes - 2010.10

WHY If replication can't catchup, slaves are useless. Backup & recovery may also suffer. Common MySQL Scalability Mistakes - 2010.10

HOW Master DML Statement Write Data/Redo Log Write Binary Log Return OK to client Common MySQL Scalability Mistakes - 2010.10

HOW Slave Detect master log change Retrieve binary log entry Write relay log (IO_THREAD) Read relay log Apply DML (SQL_THREAD) Write Data/redo log Common MySQL Scalability Mistakes - 2010.10

HOW Replication workarounds Restrict queries executed --ignore Different storage engines Different index structures Common MySQL Scalability Mistakes - 2010.10

EXPERT TIP Advanced workarounds RAID 0 (large number of slaves) Pre fetch thread Common MySQL Scalability Mistakes - 2010.10

PROBLEM My server has crashed with a hard drive failure Common MySQL Scalability Mistakes - 2010.10

TRUE STORY Question: Have you ever performed a database recovery? Answer: No, why? Common MySQL Scalability Mistakes - 2010.10

TRUE STORY Consultant: Do you know that your daily backups only recover the data up to that time, e.g. 1am. You know you have lost all your sales and changes since then. Customer: No, I didn’t know that. Common MySQL Scalability Mistakes - 2010.10

SOLUTION Have a DR Plan Documented Tested Timed Verified - End to End Common MySQL Scalability Mistakes - 2010.10

HOW Do you pass the MySQL backup/ recovery quiz? http://rb42.com/mysql-backup-quiz Common MySQL Scalability Mistakes - 2010.10

QUIZ 1. Do you have MySQL backups in place? 2. Do you backup ALL your MySQL data? 3. Do you have consistent MySQL backups? 4. Do you have backups that include both static snapshot and point in time transactions? 5. Do you review your backup logs EVERY SINGLE day or have tested backup monitoring in place? 6. Do you perform a test recovery of your static backup? 7. Do you perform a test recovery to point in time? 8. Do you time your backup and recovery process and review over time? 9. Do you have off-site copies of your backups? 10. Do you backup your primary binary logs? http://rb42.com/mysql-backup-quiz Common MySQL Scalability Mistakes - 2010.10

PROBLEM Why is my database executing 1,200 qps for 50 users? Common MySQL Scalability Mistakes - 2010.10

SOLUTION Determine what queries are running and why they are running? Common MySQL Scalability Mistakes - 2010.10

CAUSE Excessive SQL statements Duplicate Redundant Cachable Row at a time (RAT) Common MySQL Scalability Mistakes - 2010.10

WHY Reducing SQL load both improves performance now and provides greater capacity as you scale Common MySQL Scalability Mistakes - 2010.10

EXAMPLE RAT SELECT * FROM activities_theme WHERE theme_parent_id=0 SELECT * FROM activities_theme WHERE theme_parent_id=1 SELECT * FROM activities_theme WHERE theme_parent_id=2 SELECT * FROM activities_theme WHERE theme_parent_id=11 SELECT * FROM activities_theme WHERE theme_parent_id=16 SELECT * FROM activities_theme CAT WHERE theme_parent_id in (0,1,2,11,16) http://ronaldbradford.com/blog/optimizing-sql-performance-the-art-of-elimination-2010-07-08/ Common MySQL Scalability Mistakes - 2010.10

EXAMPLE Unnecessary 5 Query SELECT * FROM àrtist` 5 Query SELECT * FROM àrtist` 5 Query SELECT * FROM àrtist` Duplicate 5 Query SELECT * FROM àrtist` 5 Query SELECT * FROM àrtist` 5 Query SELECT * FROM àrtist` WHERE (ArtistID = 196 ) Only 2 queries 5 Query SELECT * FROM àrtist` WHERE (ArtistID = 2188 ) 5 Query SELECT * FROM àrtist` necessary or 1 CAT 5 Query SELECT * FROM àrtist` 5 Query SELECT * FROM àrtist` http://ronaldbradford.com/blog/optimizing-sql-performance-the-art-of-elimination-2010-07-08/ Common MySQL Scalability Mistakes - 2010.10

EXAMPLE SELECT pages_id, pages_livestats_code, pages_title, Unnecessary pages_parent, pages_exhibid, pages_theme, pages_accession_num FROM pages WHERE pages_id = 0 5 minutes 6000 executions 0 is out of bounds http://ronaldbradford.com/blog/optimizing-sql-performance-the-art-of-elimination-2010-07-08/ Common MySQL Scalability Mistakes - 2010.10

HOW Capture & Analyze DML is easy SELECT is harder http://www.slideshare.net/ronaldbradford/capturing-analyzing-and-optimizing-mysql Common MySQL Scalability Mistakes - 2010.10

HOW MySQL Binary Log (Archive Redo) mysqlbinlog mk-query-digest One Liner http://ronaldbradford.com/blog/mysql-dml-stats-per-table-2009-09-09/ Common MySQL Scalability Mistakes - 2010.10

HOW 55463 update sessions Top 2 queries = 57% 25574 insert into sessions 12820 update items 11636 insert into item_categories 7532 update users 5168 delete from item_categories 4076 update extended_item_infos 3701 insert into sphinx_new_items 3701 insert into mini_items 2190 update sweet_bars 1922 update chat_users 1662 update item_shipping_infos 1265 1260 update insert search_terms into images 81k of 141k 931 delete from item_shipping_infos 825 update booths 713 update booth_stats 574 update topics 540 update offers Common MySQL Scalability Mistakes - 2010.10

HOW Process List General Log tcpdump Application Common MySQL Scalability Mistakes - 2010.10

HOW Capturing, Analyzing and Optimizing your SQL http://www.slideshare.net/ronaldbradford/capturing-analyzing-and- optimizing-mysql Common MySQL Scalability Mistakes - 2010.10

EXPERT TIP /* Comment your queries */ The more products you have, the more developers you have, the more time you spend in code identification before you can even determine a resolution Common MySQL Scalability Mistakes - 2010.10

PROBLEM The database is slow. My webpage takes five seconds to load. Common MySQL Scalability Mistakes - 2010.10

SOLUTION Evaluate the time taken in the database and all stages in between Common MySQL Scalability Mistakes - 2010.10

EXAMPLE Client example showed a webpage taking 5 seconds to load. The html component was taking only 400 ms. Any MySQL performance improvement will only tune 8% of the total time. Common MySQL Scalability Mistakes - 2010.10

WHY Performance is important to end user Performance is perception Common MySQL Scalability Mistakes - 2010.10

HOW Firebug - http://getfirebug.com/ Httpwatch - http://httpwatch.com/ Page speed - http://code.google.com/speed/page-speed/ YSlow - http://developer.yahoo.com/yslow/ wget/curl Application code instrumentation Common MySQL Scalability Mistakes - 2010.10

EXPERT TIP http://www.stevesouders.com/ http://developer.yahoo.com/ performance/rules.html Common MySQL Scalability Mistakes - 2010.10

PROBLEM I want to add new H/W. How do I change my application to support this? Common MySQL Scalability Mistakes - 2010.10

SOLUTION Develop a seamless integration that requires no code changes, no downtime and very little additional physical resources. Common MySQL Scalability Mistakes - 2010.10

HOW Integrated monitoring and instrumentation Deployed from Day 1 Common MySQL Scalability Mistakes - 2010.10

HOW Seamless automated server deployment Version Control Build & Release Runtime config management Automated discovery Common MySQL Scalability Mistakes - 2010.10

HOW API One code path for business functionality Implied business documentation Enforced data exchange standard Testability Common MySQL Scalability Mistakes - 2010.10

HOW Different levels of data availability Read & Write Read Only No Access Cached Common MySQL Scalability Mistakes - 2010.10

HOW Different principles for scalability Read Scalability Write Scalability Caching Common MySQL Scalability Mistakes - 2010.10

REFERENCE Successful MySQL Scalability 1. Integrated monitoring & instrumentation 2. Seamless automated server deployment 3. Disaster is inevitable 4. Application Programming Interface 5. Support different levels of data availability 6. Support different scalability principles http://ronaldbradford.com/blog/successful-mysql-scalability-presentation-2010-09-17/ Common MySQL Scalability Mistakes - 2010.10

PROBLEM My website is slow? Common MySQL Scalability Mistakes - 2010.10

SOLUTION Seek professional advice. Hire for example Ronald Bradford. 20+ years of system architecture, database design and performance tuning. Employment as Consultant for Oracle Corporation (96-99) Employment as Senior Consultant for MySQL Inc (06-08) Common MySQL Scalability Mistakes - 2010.10

R Common MySQL Scalability Mistakes - 2010.10

RECAP Monitoring. Before, during and after NOW. You may not be able to predict the future but you can preempt the future. Choose the best product and features for you needs. The best SQL statement is the one you never have to execute. Common MySQL Scalability Mistakes - 2010.10

RECAP 3 levels of real time data access. Read/Write, Read and no access 3 aspects of scalability. Read, Write and Caching Operate below 90% capacity. That 10% is your insurance. Common MySQL Scalability Mistakes - 2010.10

CONTACT http://ronaldbradford.com me@ronaldbradford.com Common MySQL Scalability Mistakes - 2010.10

MySQL Scalability Mistakes - OTN

by Ronald Bradford

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 561

Statistics

MySQL Scalability Mistakes - OTN Presentation Transcript

http://ronaldbradford.com	558
http://translate.googleusercontent.com	3