Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Taking the Guesswork Out of Performance Budgets

197 views

Published on

April 3, 2019 at the Perfmatters Conference in Redwood City, CA

Published in: Data & Analytics
  • Be the first to comment

Taking the Guesswork Out of Performance Budgets

  1. 1. Taking the Guesswork out of Performance Budgets Buddy Brewer — @bbrewer
  2. 2. Closed Won
  3. 3. Business
 Bookings, Revenue Sales Rep 
 Quota retirement, Cash in pocket
  4. 4. $124B
  5. 5. What’s the average number of calls my reps make each week? How many POCs started this month? What is the total bookings in outstanding proposals? Who sent the most emails this week? What percent of demos convert to POCs?
  6. 6. What’s the average number of third party calls in my pages? How big is the hero image on my home page? What’s the median time to first contentful paint on mobile? What are my top 10 slowest URLs? What percent of pages load in over 4 seconds?
  7. 7. It’s critical to know how a change in performance will change your business
  8. 8. Measure business and technical metrics together.
  9. 9. ✘ faster is better ✔ how much faster and how much better
  10. 10. 1. Identify a hero metric 2. Tie a performance budget to a business outcome 3. Focus on the slowest experiences Consider 3 Steps
  11. 11. Identify a hero metric 1
  12. 12. Domain Lookup, Connect Time, Connect End, SSL Handshake, Response Start, Response End, DOM Loading, DOM Complete, DOM Interactive, Start Render, First Contentful Paint, First Meaningful Paint, User Timing Metrics, DOM Content Loaded, Load Event, Time to Interactive
  13. 13. Tie technical and business metrics together at the visit level session id| … |load|fcp|fmp| … | session len | converted?
  14. 14. Identify a single desired business outcome that is captured by a metric
  15. 15. Look for technical metrics that significantly impact the business metric
  16. 16. Considerations for random forest classifiers Handles unbalanced data sets well Doesn’t require as much tuning as other methods Will tell you something about itself Watch out for metrics that aren’t collected broadly
  17. 17. Tie a Performance Budget to a Business Outcome2
  18. 18. Derive performance budget from incremental business improvement Suggestion: Start with 10% improvement on the business metric
  19. 19. Segment by key user populations Country Connection type Device type
  20. 20. Make a lot of small improvements
  21. 21. Focus on the slowest experiences 3
  22. 22. Fix the metric value and watch the percentile Fix the percentile and watch the metric value Optimize Monitor vs
  23. 23. Single Dimension Analysis
  24. 24. Ranking by multiple factors Dim: entry_page_type Val: item pages Dim: entry_url Val: https://example.com/page1234 Dim: entry_page_type Val: store locator pages Dim: country Val: US Dim: device_type Val: iPhone
  25. 25. Re-baseline performance budget at least quarterly
  26. 26. It’s critical to know how a change in performance will change your business
  27. 27. Start when you get back to work. Ask your business partners what metrics they use to define business success. Include these in your RUM data.
  28. 28. Resources Project Jupyter: https://jupyter.org/ Sklearn’s Random Forest Classfier: https://scikit- learn.org/stable/modules/generated/ sklearn.ensemble.RandomForestClassifier.html Example code: https://gist.github.com/bbrewer/ 17b29c0bbfb77dc76f5bbaa54c2c7ea4
  29. 29. Don't participate, lead.

×