Download to read offline
Uploaded byTag1 Consulting, Inc.
Foreo Drives Millions of Dollars of Revenue During the Holiday Seasons with Tag1’s Technical Architecture and Leadership
The document discusses strategies for scaling applications and ensuring performance during high-traffic events, emphasizing a proactive approach to load testing and risk mitigation. It highlights the importance of maintaining performance targets, specifically for a Drupal 8 site during the 2019 holiday shopping season, while providing guidelines for effective test strategy, design, execution, and analysis. Key recommendations include using appropriate tools, incrementally scaling load tests, and focusing on critical paths to optimize performance and eliminate bottlenecks.
More Related Content
Similar to Foreo Drives Millions of Dollars of Revenue During the Holiday Seasons with Tag1’s Technical Architecture and Leadership
Foreo Drives Millions of Dollars of Revenue During the Holiday Seasons with Tag1’s Technical Architecture and Leadership
- 1. Scaling Applications and MissionCritical Events
- 2. TO FAIL WELL,FAIL FIRST BEFORE IT MATTERS, & FAIL AGAIN BEYOND EXPECTATION.
- 3. WHO ARE WE MichaelMeyers Managing Director Jeremy Andrews Founder & CEO ➔ CTO 1st venture backed Drupal startup ➔ CTO 1st top 100 Drupal site, #1 contributor to D7 ➔ VP @ Acquia - Dev Relations, Marketing, Products ➔ Advisor to Drupal Assoc. ➔ 15 Years with Drupal ➔ Performance, scalability & application security expert ➔ Linux firewalls and KernelTrap.org ➔ Orig. author Drupal installer, Throttle, pager logic, etc ➔ Maintain Memcache module ➔ Active Drupal contributor for 18 years
- 4. #2 All-time DrupalContributor 12% of Current Top 50 Contributors Architects of Fortune 500 sites Leaders from Drupal 4 to Drupal 8 Leaders in DevOps and Automated Testing Consulted for every major Drupal platform provider
- 5. Tag1 wrote thebook on performance and scalability Leaders from Drupal 4 to Drupal 8 Leaders in DevOps and Automated Testing Consulted for every major Drupal platform provider
- 6. IF TALK TIMEALLOWS Mention some other load testing clients w/ stats to establish & demonstrate world class expertise… ➔ ACLU 85x ➔ PAC12 ➔ WhiteHouse.gov ➔ Fortune Top 50 ➔ Performance book
- 7. AGENDA What you getin 30 minutes. ➔ Strategy, tools & technologies for load testing ➔ Best practices: an iterative approach to testing & tuning ➔ Load test and performance tuning tips and tricks ➔ How to plan for success & mitigate risk ➔ Q&A
- 9. Record Holiday Sales& Revenue in the US
- 10. 光棍节 Scaling Foreofor Global B2
- 11. Speed Matters It’s notjust about downtime ➔ Amazon ◆ A page load slow down of just 1s could cost $1.6 billion in sales each year ➔ Google ◆ A 400ms slowdown results in 440 million abandoned sessions per month - lots of lost ad revenue ➔ Walmart ◆ For every 100 ms of improvement, incremental revenue grew 1% Source: https://medium.com/@vikigreen/impact-of-slow-page-load-time-on-website-performance-40d5c9ce568a
- 12. Testing Life Cycle Iterate untilrisk is mitigated Implement Solutions Goals & Objectives Test Strategy Test DesignResults Analysis Test Execution
- 13. Goals & Objectives Test Strategy TestDesignResults Analysis Test Execution Goals & Objectives Business outcomes, performance targets Build a model for business $uccess Implement Solutions
- 14. FOREO: 2019 HolidayShopping Season Goals ➔ Business Goals ◆ Greatly exceed prior year’s traffic and sales metrics for maximum YoY growth ◆ Test and tune new Drupal 8 site ➔ Margin ◆ Support at least a 50% increase over projections ➔ Performance Objectives ◆ Maintain <2s page load times under peak loads
- 15. Goals & Objectives Test DesignResultsAnalysis Test Execution Test Strategy Test Strategy Scope Limitations Tooling Software & Env. Set Up Implement Solutions
- 16. II: Test StrategyStrategy:Scope Limitations ➔ Risk mitigation - you can’t do everything ➔ Identify and focus in on “critical paths” ➔ What aren’t you going to test? ➔ How are you going to test 3rd parties, APIs, etc. ➔ Destructive tests that create content & content removal
- 17. II: Test Strategy ●Risk Mitigation Planning: Balancing time, money, costs, etc. ○ How are you going to test APIs, 3rd party systems, etc? ● Picking the right tools ● Set up your Load Testing Environment ● Set up your Load Test Target ● Proactive, not Reactive: make testing a part of your dev process. Strategy: Picking the right tools - Locust ➔ Write tests & load plan in Python ➔ Leverage any Python library for integrations and testing ➔ Better than JMeter and other tools for creating plans and patterns ➔ Extensible flask-based UI ➔ Easy to distribute load generation
- 18. II: Test Strategy ●Risk Mitigation Planning: Balancing time, money, costs, etc. ○ How are you going to test APIs, 3rd party systems, etc? ● Picking the right tools ● Set up your Load Testing Environment ● Set up your Load Test Target ● Proactive, not Reactive: make testing a part of your dev process. Strategy: Set up Load Testing Environment ➔ Setup Locust ◆ How-to: https://tag1consulting.com/blog/distributed-load-testing-locust
- 19. II: Test Strategy ●Risk Mitigation Planning: Balancing time, money, costs, etc. ○ How are you going to test APIs, 3rd party systems, etc? ● Picking the right tools ● Set up your Load Testing Environment ● Set up your Load Test Target ● Proactive, not Reactive: make testing a part of your dev process. Strategy: Set up Load Test Target(s) ➔ Replica of Production is ideal, but not always an option ➔ Full stack & processes: CDN, reverse proxies, cron ➔ 3rd party APIs and systems ➔ Plan ahead, notify everyone (hosting platform, 3rd parties/partners, etc.) ➔ Monitor everything
- 20. FOREO: DevOps &Performance Testing ➔ Be proactive: really high traffic & really high stakes! ➔ Performance testing baked into to dev processes ➔ Integrated into CI/CD pipeline ➔ Auto accept/reject PRs based on performance
- 21. Test Design Goals & Objectives TestStrategy Test DesignResults Analysis Test Execution Test Design Load models & Create Tests Implement Solutions
- 22. II: Test Strategy ●Risk Mitigation Planning: Balancing time, money, costs, etc. ○ How are you going to test APIs, 3rd party systems, etc? ● Picking the right tools ● Set up your Load Testing Environment ● Set up your Load Test Target ● Proactive, not Reactive: make testing a part of your dev process. Test Design: Load Models ➔ Running tests independently ➔ Ratios for running tests concurrently ➔ Incremental ramp up of load
- 23. II: Test Strategy ●Risk Mitigation Planning: Balancing time, money, costs, etc. ○ How are you going to test APIs, 3rd party systems, etc? ● Picking the right tools ● Set up your Load Testing Environment ● Set up your Load Test Target ● Proactive, not Reactive: make testing a part of your dev process. Test Design: Creating Tests ➔ Check http headers, and page content, etc. ➔ Test static content as a % of total, roughly 10% after first pass ➔ Data parameterization and mapping ➔ Test your tests! ➔ Consistency is key - don’t make major changes when iterating
- 24. II: Test Strategy ●Risk Mitigation Planning: Balancing time, money, costs, etc. ○ How are you going to test APIs, 3rd party systems, etc? ● Picking the right tools ● Set up your Load Testing Environment ● Set up your Load Test Target ● Proactive, not Reactive: make testing a part of your dev process. Test Design: Creating Tests in Locust ➔ How-to: https://tag1consulting.com/blog/drupal-loadtest-locust ➔ Code your load model as a Python script ➔ Take advantage of Python libraries
- 25. FOREO: Creating Tests ➔Evaluate functional paths & analytics determine critical paths ◆ View category & product pages, add to cart, complete purchase ◆ Browse (non product) pages e.g., About, FAQ, etc. ◆ Auth: Support, Register products ➔ Handle localization for translated content ◆ Accommodate variances (e.g., submit button changes)
- 26. Test Execution Goals & Objectives TestStrategy Test DesignResults Analysis Test Execution Test Execution Establish Baselines Scale incrementally Monitor Implement Solutions
- 27. II: Test Strategy ●Risk Mitigation Planning: Balancing time, money, costs, etc. ○ How are you going to test APIs, 3rd party systems, etc? ● Picking the right tools ● Set up your Load Testing Environment ● Set up your Load Test Target ● Proactive, not Reactive: make testing a part of your dev process. Test Execution ➔ Scale up load incrementally - don’t go too fast, too soon ➔ Work through your load models ➔ Consider the length & timing of your test runs
- 28. II: Test Strategy ●Risk Mitigation Planning: Balancing time, money, costs, etc. ○ How are you going to test APIs, 3rd party systems, etc? ● Picking the right tools ● Set up your Load Testing Environment ● Set up your Load Test Target ● Proactive, not Reactive: make testing a part of your dev process. Test Execution: Monitoring in real-time ➔ Check everything from your infra to the edge ➔ Logging and monitoring ➔ Check test output: Locust logs, http headers and page content ➔ Confirm tests are hitting all aspects of your application
- 29. Results Analysis & Reporting Goals & Objectives TestStrategy Test DesignResults Analysis Test Execution Result Analysis Identify improvements Triage solutions Reporting Implement Solutions
- 30. II: Test Strategy ●Risk Mitigation Planning: Balancing time, money, costs, etc. ○ How are you going to test APIs, 3rd party systems, etc? ● Picking the right tools ● Set up your Load Testing Environment ● Set up your Load Test Target ● Proactive, not Reactive: make testing a part of your dev process. Results Analysis & Triage ➔ Focus on the major bottleneck - don’t over optimize ➔ Low hanging fruit
- 31. Reporting Leaders from Drupal 4to Drupal 8 Leaders in DevOps and Automated Testing Consulted for every major Drupal platform provider
- 32. Prioritizing Solutions Leaders from Drupal 4to Drupal 8 Leaders in DevOps and Automated Testing Consulted for every major Drupal platform provider Phase
- 33. Recommendations & Code Review Checklist Leaders fromDrupal 4 to Drupal 8 Leaders in DevOps and Automated Testing Consulted for every major Drupal platform provider Phase
- 34. Implement Solutions Triage & Patch Goals& Objectives Test Strategy Test DesignResults Analysis Test Execution Implement Solutions Eliminate Bottlenecks Low hanging fruit Incremental updates
- 35. II: Test Strategy ●Risk Mitigation Planning: Balancing time, money, costs, etc. ○ How are you going to test APIs, 3rd party systems, etc? ● Picking the right tools ● Set up your Load Testing Environment ● Set up your Load Test Target ● Proactive, not Reactive: make testing a part of your dev process. Implement Solutions ➔ Fix a problem, validate the solution. ➔ Do not make a lot of changes!
- 36. Examples Test Strategy TestDesignResults Analysis Test Execution Implement Solutions Goals & Objectives Implement Solutions Results Analysis Test Execution
- 37. EXAMPLE Foreo -Test Execution & Monitoring
- 38. EXAMPLE Foreo -Results Analysis: Opcode Cache
- 39. EXAMPLE Foreo -Implement Solution & Test
- 40. EXAMPLE Foreo -Results Analysis
- 41. EXAMPLE Foreo -Implement and Test
- 42. EXAMPLE Foreo -Analyze, Implement, Test...
- 43. PRO TIPS ✅ BeProactive, not Reactive ✅ Design & execute different load models ✅ Don’t make assumption - test entire stack ✅ Don’t replicate traffic 1:1 - focus on critical paths ✅ Be methodical ✅ Don’t scale load tests too fast, too soon ✅ Don’t make lots of changes. Make a change, test. ✅ Triage - focus on the biggest bang for your buck ✅ Iterate - bottlenecks block other bottlenecks.
- 44.
- 45. Subtitle Add speaker namehere Title slide Join us for contribution opportunities Friday, April 12, 2019 9:00-18:00 Room: 602 Mentored Contribution First Time Contributor Workshop General Contribution #DrupalContributions 9:00-12:00 Room: 606 9:00-18:00 Room: 6A
- 47. CONTACT US ATANY TIME Michael Meyers m@tag1consulting.com +1-917-402-8263 (toll free)