Successful MySQL Scalability

6,423 views
6,325 views

Published on

Published in: Technology
0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,423
On SlideShare
0
From Embeds
0
Number of Embeds
1,607
Actions
Shares
0
Downloads
305
Comments
0
Likes
8
Embeds 0
No embeds

No notes for slide

Successful MySQL Scalability

  1. 1. Successful MySQL Scalability 2010.08 Ronald Bradford www.RonaldBradford.com @RonaldBradford #mysql
  2. 2. The one hour version
  3. 3. OUTLINE Necessary Principles System Architecture Data Availability Best Practices Being proactive Successful MySQL Scalability
  4. 4. 1 Successful MySQL Scalability
  5. 5. Action Integrated monitoring and instrumentation Successful MySQL Scalability
  6. 6. EXAMPLE Question: ✘ How do you know when your server is down or not accessible? Answer: The users will let us know. Successful MySQL Scalability
  7. 7. How Monitoring/Alerting Graphical Historical Necessary Generally missing/incomplete Useless for real-time analysis Successful MySQL Scalability
  8. 8. How Monitoring/Alerting TIV E C Graphical R E A Historical Necessary Generally missing/incomplete Useless for real-time analysis Successful MySQL Scalability
  9. 9. How Dashboard The state of NOW Sampling at 1s/3s/5s e.g. 0.1% of throughput http://rb42.com/monitoring-needs Successful MySQL Scalability
  10. 10. How Dashboard T I V E OAC The state of NOW P R Sampling at 1s/3s/5s e.g. 0.1% of throughput http://rb42.com/monitoring-needs Successful MySQL Scalability
  11. 11. How Instrumentation Important to business viability e.g. orders per minute (KPI) page load time Seamless implementation i.e. no code changes to view real-time extensible Successful MySQL Scalability
  12. 12. How Instrumentation T I V E OAC Important to business viability R e.g. orders per minute (KPI) P page load time Seamless implementation i.e. no code changes to view real-time extensible Successful MySQL Scalability
  13. 13. WHY Monitoring + Instrumentation = Preemptive Analysis Successful MySQL Scalability
  14. 14. How ✘ Successful MySQL Scalability
  15. 15. How ✔ < 1 second Successful MySQL Scalability
  16. 16. WHY Preemptive Analysis Fail early Faster user response Less physical resource impact Successful MySQL Scalability
  17. 17. TIP Have a status website allow for comments (e.g. blog) Have a public dashboard website Host them somewhere else! Successful MySQL Scalability
  18. 18. 2 Successful MySQL Scalability
  19. 19. Action Seamless automated server deployment Successful MySQL Scalability
  20. 20. Example Actual Client 100+ production database servers ✘ Full-time DBAs No automated software installation No single documented installation process No version control No auditing/verification Successful MySQL Scalability
  21. 21. How Basics Version Control Automated Build & Release Integrated Monitoring Successful MySQL Scalability
  22. 22. How Intermediate Health Checks Runtime config file management Benchmarking Successful MySQL Scalability
  23. 23. How Advanced Server request queue Automated discovery Proactive scaling (up & down) Successful MySQL Scalability
  24. 24. GOAL Consistent and timely provisioning Unattended installation with correct and reproducible ✔ configuration Automated discovery Successful MySQL Scalability
  25. 25. 3 Successful MySQL Scalability
  26. 26. Action Disaster is inevitable Successful MySQL Scalability
  27. 27. Example Question: ✘ Have you ever performed a database recovery? Answer: No, why? Successful MySQL Scalability
  28. 28. Example Consultant: ✘ Do you know that your daily backups only recover the data up to that time (e.g. 1 am). You know you have lost all your sales and data changes since then. Customer: No, I didn’t know that. Successful MySQL Scalability
  29. 29. HOW Have a DR plan Documented Tested Timed Verified Successful MySQL Scalability
  30. 30. HOW Test under production conditions System Load Database Size End to End Successful MySQL Scalability
  31. 31. GOAL Know and practice for disaster Provide confidence to management ✔ Be as confident about your recovery as your scalability Successful MySQL Scalability
  32. 32. SIDEBAR! What does your website look like when it’s down? Successful MySQL Scalability
  33. 33. SIDEBAR! ✘ Google Picasa Successful MySQL Scalability
  34. 34. SIDEBAR! ✘ Successful MySQL Scalability
  35. 35. SIDEBAR! ✘ Mozilla Successful MySQL Scalability
  36. 36. SIDEBAR! ✔ Successful MySQL Scalability
  37. 37. SIDEBAR! ✔ Successful MySQL Scalability
  38. 38. 4 Successful MySQL Scalability
  39. 39. Action Application Programming Interface Successful MySQL Scalability
  40. 40. Example Public API ✘ Application web site Batch processes 3 access paths to same data with different business rules Successful MySQL Scalability
  41. 41. Why One code path for business functionality Implied business documentation Enforced data exchange standard Testability Successful MySQL Scalability
  42. 42. GOAL Technology independence Business specification dependence (API) ✔ Stress testable Successful MySQL Scalability
  43. 43. SIDEBAR! Testing is not about what works, it’s about breaking your software Successful MySQL Scalability
  44. 44. 5 Successful MySQL Scalability
  45. 45. Action Support different levels of data availability Successful MySQL Scalability
  46. 46. HOW Data can be: Read & Write Read No Access Cached Successful MySQL Scalability
  47. 47. EXAMPLE Development Team: We need a maintenance window for software upgrades and new releases. Successful MySQL Scalability
  48. 48. EXAMPLE Management Team: No Downtime Successful MySQL Scalability
  49. 49. EXAMPLE Development Team: But we need this to fix problems and improve performance. Successful MySQL Scalability
  50. 50. EXAMPLE Management Team: No Downtime Successful MySQL Scalability
  51. 51. EXAMPLE What is your definition of no downtime? Successful MySQL Scalability
  52. 52. WHY Support downtime Software upgrades Support disasters Reduce high load Support partial functionality Successful MySQL Scalability
  53. 53. GOAL Meet the business needs of availability ✔ There are ways to give greater perception of availability Successful MySQL Scalability
  54. 54. 6 Successful MySQL Scalability
  55. 55. Action Support different scalability principles Successful MySQL Scalability
  56. 56. HOW Read Scalability Write Scalability Caching Successful MySQL Scalability
  57. 57. HOW Depends on your R/W ratio Depends on your business needs Depends on rate of data change (caching) Successful MySQL Scalability
  58. 58. ISSUES Consistency Data Interoperability Successful MySQL Scalability
  59. 59. GOAL ✔ Support large scale growth with appropriate software architecture Minimize human interaction Ongoing review and improvement Successful MySQL Scalability
  60. 60. SIDEBAR! Availability options & scalability principles = scalable datastore Successful MySQL Scalability
  61. 61. 7 Successful MySQL Scalability
  62. 62. What’s NEXT I only had one hour! Successful MySQL Scalability
  63. 63. What’s NEXT Reduce deployment time The right business metrics Team dynamics Managing risk Continual improvement Asynchronous Data interoperability Write once data ... Consistency Successful MySQL Scalability
  64. 64. R Successful MySQL Scalability
  65. 65. RECAP Monitoring. Before, during and after NOW. You may not be able to predict the future but you can preempt the future. Operate below 90%. That 10% is your insurance policy. Invest in insurance. Successful MySQL Scalability
  66. 66. RECAP Does your business exist without your data? How long can your site be unavailable before customers go elsewhere? Successful MySQL Scalability
  67. 67. RECAP Support 3 levels of real time data access Read/Write, Read and no access Support 3 different aspects of scalability Read, Write and Caching Successful MySQL Scalability
  68. 68. RECAP If you can’t drive without a mouse, find somebody that can. Successful MySQL Scalability
  69. 69. Reference The most common MySQL scalability mistakes, and how to avoid them. http://omniti.com/surge Successful MySQL Scalability
  70. 70. Reference ‣ 1. My website is slow? ‣ 2. I want to add new H/W. How do I change my application to support this? ‣ 3. The database is slow. My webpage takes five seconds to load. ‣ 4. Why is my database executing so many qps? Successful MySQL Scalability
  71. 71. Reference ‣ 5. My server has crashed with a hard drive failure ‣ 6. My replication slave can't keep up? ‣ 7. I can't access my website? ‣ 8. Why is my database so large? ‣ 9. My website seems to freeze or responds randomly? Successful MySQL Scalability
  72. 72. CONFERENCES MySQL Sunday O'Reilly MySQL Conference 2011 Colloborate 2011 Kaleidoscope (KScope) 2011 http://ronaldbradford.com/blog/2011-mysql-conferences-2010-09-07/ Successful MySQL Scalability
  73. 73. CONTACT http://ronaldbradford.com me@ronaldbradford.com Successful MySQL Scalability Successful MySQL Scalability

×