Successful Scalability Principles - Part 1

5,640 views

Published on

Every website wants to become successful. Few websites however undertake the basic and fundamental steps to build a rock solid foundation to ensure a scalable

Published in: Business, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,640
On SlideShare
0
From Embeds
0
Number of Embeds
3,239
Actions
Shares
0
Downloads
42
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Successful Scalability Principles - Part 1

  1. 1. Successful Scalability Principles - Part 1 Ronald Bradford http://ronaldbradford.com 2012.08 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  2. 2. OUTLINE Necessary Principles System Architecture Data Availability Best Practices Being proactive EffectiveMySQL.com - Performance, Scalability & Business Continuity
  3. 3. 1 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  4. 4. ACTION Integrated monitoring and instrumentation EffectiveMySQL.com - Performance, Scalability & Business Continuity
  5. 5. EXAMPLE Question: ✘ How do you know when your server is down or not accessible? Answer: The users will let us know. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  6. 6. HOW EffectiveMySQL.com - Performance, Scalability & Business Continuity
  7. 7. HOW Monitoring/Alerting Graphical Historical Necessary Generally missing/incomplete Useless for real-time analysis EffectiveMySQL.com - Performance, Scalability & Business Continuity
  8. 8. HOW E V I T C A E R Monitoring/Alerting Graphical Historical Necessary Generally missing/incomplete Useless for real-time analysis EffectiveMySQL.com - Performance, Scalability & Business Continuity
  9. 9. HOW http://rb42.com/monitoring-needs EffectiveMySQL.com - Performance, Scalability & Business Continuity
  10. 10. HOW Dashboard The state of NOW Near real-time statistics Sampling at 1s/3s/5s e.g. 0.1% of throughput http://rb42.com/monitoring-needs EffectiveMySQL.com - Performance, Scalability & Business Continuity
  11. 11. HOW Dashboard E V I T C A O R P The state of NOW Near real-time statistics Sampling at 1s/3s/5s e.g. 0.1% of throughput http://rb42.com/monitoring-needs EffectiveMySQL.com - Performance, Scalability & Business Continuity
  12. 12. HOW EffectiveMySQL.com - Performance, Scalability & Business Continuity
  13. 13. HOW Instrumentation Important to business viability e.g. orders per minute (KPI) page load time Seamless implementation i.e. no code changes to view real-time extensible EffectiveMySQL.com - Performance, Scalability & Business Continuity
  14. 14. HOW Instrumentation E V I T C A O R P Important to business viability e.g. orders per minute (KPI) page load time Seamless implementation i.e. no code changes to view real-time extensible EffectiveMySQL.com - Performance, Scalability & Business Continuity
  15. 15. Why Monitoring + Instrumentation = Preemptive Analysis EffectiveMySQL.com - Performance, Scalability & Business Continuity
  16. 16. How NOT TO ✘ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  17. 17. HOW TO ✔ < 1 second EffectiveMySQL.com - Performance, Scalability & Business Continuity
  18. 18. WHY Preemptive Analysis Fail early Faster user response Less physical resource impact EffectiveMySQL.com - Performance, Scalability & Business Continuity
  19. 19. TIP EffectiveMySQL.com - Performance, Scalability & Business Continuity
  20. 20. TIP Have a status website allow for comments (e.g. blog) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  21. 21. TIP Have a status website allow for comments (e.g. blog) Have a public dashboard website EffectiveMySQL.com - Performance, Scalability & Business Continuity
  22. 22. TIP Have a status website allow for comments (e.g. blog) Have a public dashboard website Host them somewhere else! EffectiveMySQL.com - Performance, Scalability & Business Continuity
  23. 23. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  24. 24. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  25. 25. 2 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  26. 26. ACTION Seamless automated server deployment EffectiveMySQL.com - Performance, Scalability & Business Continuity
  27. 27. EXAMPLE Actual Client 100+ production database servers ✘ Full-time DBAs No automated software installation No single documented installation process No version control No auditing/verification EffectiveMySQL.com - Performance, Scalability & Business Continuity
  28. 28. HOW Basics Version Control Automated Build & Release Integrated Monitoring EffectiveMySQL.com - Performance, Scalability & Business Continuity
  29. 29. HOW Intermediate Health Checks Runtime config file management Benchmarking EffectiveMySQL.com - Performance, Scalability & Business Continuity
  30. 30. HOW Advanced Server request queue Automated discovery Proactive scaling (up & down) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  31. 31. GOAL Consistent and timely provisioning Unattended installation with correct and reproducible configuration Automated discovery ✔ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  32. 32. 3 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  33. 33. ACTION Disaster is inevitable EffectiveMySQL.com - Performance, Scalability & Business Continuity
  34. 34. EXAMPLE Question: ✘ Have you ever performed a database recovery? Answer: No, why? EffectiveMySQL.com - Performance, Scalability & Business Continuity
  35. 35. EXAMPLE Consultant: ✘ Do you know that your daily backups only recover the data up to that time (e.g. 1 am).You know you have lost all your sales(aka $2 million sales today) and data changes since then. Customer: No, I didn’t know that. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  36. 36. HOW EffectiveMySQL.com - Performance, Scalability & Business Continuity
  37. 37. HOW Have a DR plan EffectiveMySQL.com - Performance, Scalability & Business Continuity
  38. 38. HOW Have a DR plan Documented EffectiveMySQL.com - Performance, Scalability & Business Continuity
  39. 39. HOW Have a DR plan Documented Tested EffectiveMySQL.com - Performance, Scalability & Business Continuity
  40. 40. HOW Have a DR plan Documented Tested Timed EffectiveMySQL.com - Performance, Scalability & Business Continuity
  41. 41. HOW Have a DR plan Documented Tested Timed Verified EffectiveMySQL.com - Performance, Scalability & Business Continuity
  42. 42. HOW EffectiveMySQL.com - Performance, Scalability & Business Continuity
  43. 43. HOW Test under production conditions EffectiveMySQL.com - Performance, Scalability & Business Continuity
  44. 44. HOW Test under production conditions System Load EffectiveMySQL.com - Performance, Scalability & Business Continuity
  45. 45. HOW Test under production conditions System Load Database Size EffectiveMySQL.com - Performance, Scalability & Business Continuity
  46. 46. HOW Test under production conditions System Load Database Size End to End EffectiveMySQL.com - Performance, Scalability & Business Continuity
  47. 47. GOAL Know and practice for disaster Provide confidence to management ✔ Be as confident about your recovery as your scalability EffectiveMySQL.com - Performance, Scalability & Business Continuity
  48. 48. SIDEBAR! What does your website look like when it’s down? EffectiveMySQL.com - Performance, Scalability & Business Continuity
  49. 49. SIDEBAR ✘ Google Picasa EffectiveMySQL.com - Performance, Scalability & Business Continuity
  50. 50. SIDEBAR ✘ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  51. 51. SIDEBAR ✘ Mozilla EffectiveMySQL.com - Performance, Scalability & Business Continuity
  52. 52. SIDEBAR ✔ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  53. 53. SIDEBAR ✔ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  54. 54. 4 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  55. 55. ACTION Application Programming Interface EffectiveMySQL.com - Performance, Scalability & Business Continuity
  56. 56. EXAMPLE ✘ 3 access paths to same data with different business rules Public API Application web site Batch processes EffectiveMySQL.com - Performance, Scalability & Business Continuity
  57. 57. WHY an API? One code path for business functionality Implied business documentation Enforced data exchange standard Testability EffectiveMySQL.com - Performance, Scalability & Business Continuity
  58. 58. GOAL Technology independence Business specification dependence (API) ✔ Stress testable EffectiveMySQL.com - Performance, Scalability & Business Continuity
  59. 59. REQUIREMENTS Input Specification Output Specification Standard for data communication Throughput needs Latency requirements EffectiveMySQL.com - Performance, Scalability & Business Continuity
  60. 60. SIDEBAR! Testing is not about what works, it’s about breaking your software EffectiveMySQL.com - Performance, Scalability & Business Continuity
  61. 61. 5 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  62. 62. ACTION Support different levels of data availability EffectiveMySQL.com - Performance, Scalability & Business Continuity
  63. 63. HOW EffectiveMySQL.com - Performance, Scalability & Business Continuity
  64. 64. HOW Data can be: EffectiveMySQL.com - Performance, Scalability & Business Continuity
  65. 65. HOW Data can be: Read & Write EffectiveMySQL.com - Performance, Scalability & Business Continuity
  66. 66. HOW Data can be: Read & Write Read EffectiveMySQL.com - Performance, Scalability & Business Continuity
  67. 67. HOW Data can be: Read & Write Read Cached EffectiveMySQL.com - Performance, Scalability & Business Continuity
  68. 68. HOW Data can be: Read & Write Read Cached No Access EffectiveMySQL.com - Performance, Scalability & Business Continuity
  69. 69. EXAMPLE Development Team: We need a maintenance window for software upgrades and new releases. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  70. 70. EXAMPLE Management Team: No Downtime EffectiveMySQL.com - Performance, Scalability & Business Continuity
  71. 71. EXAMPLE Development Team: But we need this to fix problems and improve performance. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  72. 72. EXAMPLE Management Team: No Downtime EffectiveMySQL.com - Performance, Scalability & Business Continuity
  73. 73. EXAMPLE What is your definition of no downtime? EffectiveMySQL.com - Performance, Scalability & Business Continuity
  74. 74. WHY Support downtime Software upgrades Support disasters Reduce high load Support partial functionality EffectiveMySQL.com - Performance, Scalability & Business Continuity
  75. 75. GOAL Meet the business needs of availability There are ways to give greater perception of availability ✔ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  76. 76. 6 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  77. 77. ACTION Support different scalability principles EffectiveMySQL.com - Performance, Scalability & Business Continuity
  78. 78. HOW Read Scalability Write Scalability Caching EffectiveMySQL.com - Performance, Scalability & Business Continuity
  79. 79. HOW Depends on your R/W ratio Depends on your business needs Depends on rate of data change (caching) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  80. 80. ISSUES Consistency Data Interoperability EffectiveMySQL.com - Performance, Scalability & Business Continuity
  81. 81. GOAL ✔ Support large scale growth with appropriate software architecture Minimize human interaction Ongoing review and improvement EffectiveMySQL.com - Performance, Scalability & Business Continuity
  82. 82. SIDEBAR! Availability options & scalability principles = scalable datastore EffectiveMySQL.com - Performance, Scalability & Business Continuity
  83. 83. + EffectiveMySQL.com - Performance, Scalability & Business Continuity
  84. 84. WHAT's NEXT I only had one hour! EffectiveMySQL.com - Performance, Scalability & Business Continuity
  85. 85. WHAT's NEXT Part 2 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  86. 86. WHAT's NEXT Part 2 Reduce deployment time The right business metrics Team dynamics Managing risk Continual improvement Data interoperability ... EffectiveMySQL.com - Performance, Scalability & Business Continuity
  87. 87. WHAT's NEXT Part 2 Reduce deployment time The right business metrics Team dynamics Managing risk Continual improvement Data interoperability ... Asynchronous Write once data Consistency EffectiveMySQL.com - Performance, Scalability & Business Continuity
  88. 88. C EffectiveMySQL.com - Performance, Scalability & Business Continuity
  89. 89. CONCLUsION Monitoring. Before, during and after NOW. You may not be able to predict the future but you can preempt the future. Operate below 90%. That 10% is your insurance policy. Invest in insurance. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  90. 90. CONCLUSION Does your business exist without your data? How long can your site be unavailable before customers go elsewhere? EffectiveMySQL.com - Performance, Scalability & Business Continuity
  91. 91. CONCLUSION Support 3 levels of real time data access Read/Write, Read and no access Support 3 different aspects of scalability Read, Write and Caching EffectiveMySQL.com - Performance, Scalability & Business Continuity
  92. 92. REFERENCE Presentation The most common MySQL scalability mistakes, and how to avoid them. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  93. 93. REFERENCE 1. My website is slow? 2. I want to add new H/W. How do I change my application to support this? 3. The database is slow. My webpage takes five seconds to load. 4. Why is my database executing so many qps? 5. My server has crashed with a hard drive failure 6. My replication slave can't keep up? 7. I can't access my website? 8. 8. Why is my database so large? 9. My website seems to freeze or responds randomly? EffectiveMySQL.com - Performance, Scalability & Business Continuity
  94. 94. Ronald Bradford http://effectiveMySQL.com EffectiveMySQL.com - Performance, Scalability & Business Continuity

×