Successful Scalability Principles - Part 1
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Successful Scalability Principles - Part 1

  • 1,382 views
Uploaded on

Every website wants to become successful. Few websites however undertake the basic and fundamental steps to build a rock solid foundation to ensure a scalable

Every website wants to become successful. Few websites however undertake the basic and fundamental steps to build a rock solid foundation to ensure a scalable

More in: Business , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,382
On Slideshare
801
From Embeds
581
Number of Embeds
3

Actions

Shares
Downloads
14
Comments
0
Likes
0

Embeds 581

http://ronaldbradford.com 561
http://next.ronaldbradford.com 19
https://www.linkedin.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Successful Scalability Principles - Part 1 Ronald Bradford http://ronaldbradford.com 2012.08 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 2. OUTLINE Necessary Principles System Architecture Data Availability Best Practices Being proactive EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 3. 1 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 4. ACTION Integrated monitoring and instrumentation EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 5. EXAMPLE Question: ✘ How do you know when your server is down or not accessible? Answer: The users will let us know. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 6. HOW EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 7. HOW Monitoring/Alerting Graphical Historical Necessary Generally missing/incomplete Useless for real-time analysis EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 8. HOW E V I T C A E R Monitoring/Alerting Graphical Historical Necessary Generally missing/incomplete Useless for real-time analysis EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 9. HOW http://rb42.com/monitoring-needs EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 10. HOW Dashboard The state of NOW Near real-time statistics Sampling at 1s/3s/5s e.g. 0.1% of throughput http://rb42.com/monitoring-needs EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 11. HOW Dashboard E V I T C A O R P The state of NOW Near real-time statistics Sampling at 1s/3s/5s e.g. 0.1% of throughput http://rb42.com/monitoring-needs EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 12. HOW EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 13. HOW Instrumentation Important to business viability e.g. orders per minute (KPI) page load time Seamless implementation i.e. no code changes to view real-time extensible EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 14. HOW Instrumentation E V I T C A O R P Important to business viability e.g. orders per minute (KPI) page load time Seamless implementation i.e. no code changes to view real-time extensible EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 15. Why Monitoring + Instrumentation = Preemptive Analysis EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 16. How NOT TO ✘ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 17. HOW TO ✔ < 1 second EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 18. WHY Preemptive Analysis Fail early Faster user response Less physical resource impact EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 19. TIP EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 20. TIP Have a status website allow for comments (e.g. blog) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 21. TIP Have a status website allow for comments (e.g. blog) Have a public dashboard website EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 22. TIP Have a status website allow for comments (e.g. blog) Have a public dashboard website Host them somewhere else! EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 23. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 24. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 25. 2 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 26. ACTION Seamless automated server deployment EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 27. EXAMPLE Actual Client 100+ production database servers ✘ Full-time DBAs No automated software installation No single documented installation process No version control No auditing/verification EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 28. HOW Basics Version Control Automated Build & Release Integrated Monitoring EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 29. HOW Intermediate Health Checks Runtime config file management Benchmarking EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 30. HOW Advanced Server request queue Automated discovery Proactive scaling (up & down) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 31. GOAL Consistent and timely provisioning Unattended installation with correct and reproducible configuration Automated discovery ✔ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 32. 3 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 33. ACTION Disaster is inevitable EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 34. EXAMPLE Question: ✘ Have you ever performed a database recovery? Answer: No, why? EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 35. EXAMPLE Consultant: ✘ Do you know that your daily backups only recover the data up to that time (e.g. 1 am).You know you have lost all your sales(aka $2 million sales today) and data changes since then. Customer: No, I didn’t know that. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 36. HOW EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 37. HOW Have a DR plan EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 38. HOW Have a DR plan Documented EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 39. HOW Have a DR plan Documented Tested EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 40. HOW Have a DR plan Documented Tested Timed EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 41. HOW Have a DR plan Documented Tested Timed Verified EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 42. HOW EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 43. HOW Test under production conditions EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 44. HOW Test under production conditions System Load EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 45. HOW Test under production conditions System Load Database Size EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 46. HOW Test under production conditions System Load Database Size End to End EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 47. GOAL Know and practice for disaster Provide confidence to management ✔ Be as confident about your recovery as your scalability EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 48. SIDEBAR! What does your website look like when it’s down? EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 49. SIDEBAR ✘ Google Picasa EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 50. SIDEBAR ✘ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 51. SIDEBAR ✘ Mozilla EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 52. SIDEBAR ✔ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 53. SIDEBAR ✔ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 54. 4 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 55. ACTION Application Programming Interface EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 56. EXAMPLE ✘ 3 access paths to same data with different business rules Public API Application web site Batch processes EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 57. WHY an API? One code path for business functionality Implied business documentation Enforced data exchange standard Testability EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 58. GOAL Technology independence Business specification dependence (API) ✔ Stress testable EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 59. REQUIREMENTS Input Specification Output Specification Standard for data communication Throughput needs Latency requirements EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 60. SIDEBAR! Testing is not about what works, it’s about breaking your software EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 61. 5 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 62. ACTION Support different levels of data availability EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 63. HOW EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 64. HOW Data can be: EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 65. HOW Data can be: Read & Write EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 66. HOW Data can be: Read & Write Read EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 67. HOW Data can be: Read & Write Read Cached EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 68. HOW Data can be: Read & Write Read Cached No Access EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 69. EXAMPLE Development Team: We need a maintenance window for software upgrades and new releases. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 70. EXAMPLE Management Team: No Downtime EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 71. EXAMPLE Development Team: But we need this to fix problems and improve performance. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 72. EXAMPLE Management Team: No Downtime EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 73. EXAMPLE What is your definition of no downtime? EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 74. WHY Support downtime Software upgrades Support disasters Reduce high load Support partial functionality EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 75. GOAL Meet the business needs of availability There are ways to give greater perception of availability ✔ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 76. 6 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 77. ACTION Support different scalability principles EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 78. HOW Read Scalability Write Scalability Caching EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 79. HOW Depends on your R/W ratio Depends on your business needs Depends on rate of data change (caching) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 80. ISSUES Consistency Data Interoperability EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 81. GOAL ✔ Support large scale growth with appropriate software architecture Minimize human interaction Ongoing review and improvement EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 82. SIDEBAR! Availability options & scalability principles = scalable datastore EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 83. + EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 84. WHAT's NEXT I only had one hour! EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 85. WHAT's NEXT Part 2 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 86. WHAT's NEXT Part 2 Reduce deployment time The right business metrics Team dynamics Managing risk Continual improvement Data interoperability ... EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 87. WHAT's NEXT Part 2 Reduce deployment time The right business metrics Team dynamics Managing risk Continual improvement Data interoperability ... Asynchronous Write once data Consistency EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 88. C EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 89. CONCLUsION Monitoring. Before, during and after NOW. You may not be able to predict the future but you can preempt the future. Operate below 90%. That 10% is your insurance policy. Invest in insurance. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 90. CONCLUSION Does your business exist without your data? How long can your site be unavailable before customers go elsewhere? EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 91. CONCLUSION Support 3 levels of real time data access Read/Write, Read and no access Support 3 different aspects of scalability Read, Write and Caching EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 92. REFERENCE Presentation The most common MySQL scalability mistakes, and how to avoid them. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 93. REFERENCE 1. My website is slow? 2. I want to add new H/W. How do I change my application to support this? 3. The database is slow. My webpage takes five seconds to load. 4. Why is my database executing so many qps? 5. My server has crashed with a hard drive failure 6. My replication slave can't keep up? 7. I can't access my website? 8. 8. Why is my database so large? 9. My website seems to freeze or responds randomly? EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 94. Ronald Bradford http://effectiveMySQL.com EffectiveMySQL.com - Performance, Scalability & Business Continuity