1. Taking the Guesswork out of
Performance Budgets
Buddy Brewer — @bbrewer

3. Closed Won

4. Business
Bookings, Revenue
Sales Rep
Quota retirement, Cash in pocket

5. $124B

6. What’s the average number of calls my reps make each week?
How many POCs started this month?
What is the total bookings in outstanding proposals?
Who sent the most emails this week?
What percent of demos convert to POCs?

7. What’s the average number of third party calls in my pages?
How big is the hero image on my home page?
What’s the median time to first contentful paint on mobile?
What are my top 10 slowest URLs?
What percent of pages load in over 4 seconds?

8. It’s critical to know how a
change in performance
will change your business

9. Measure business
and technical
metrics together.

10. ✘ faster is better
✔ how much faster and
how much better

11. 1. Identify a hero metric
2. Tie a performance budget to a business outcome
3. Focus on the slowest experiences
Consider 3 Steps

12. Identify a hero metric
1

13. Domain Lookup, Connect Time, Connect End, SSL Handshake, Response Start,
Response End, DOM Loading, DOM Complete, DOM Interactive, Start Render, First
Contentful Paint, First Meaningful Paint, User Timing Metrics, DOM Content Loaded,
Load Event, Time to Interactive

15. Tie technical and business metrics
together at the visit level
session id| … |load|fcp|fmp| … | session len | converted?

16. Identify a single desired business outcome
that is captured by a metric

17. Look for technical metrics that
significantly impact the business metric

20. Considerations for random forest classifiers
Handles unbalanced data sets well
Doesn’t require as much tuning as other methods
Will tell you something about itself
Watch out for metrics that aren’t collected broadly

21. Tie a Performance Budget
to a Business Outcome2

23. Derive performance budget from
incremental business improvement
Suggestion: Start with 10% improvement on the
business metric

25. Segment by key user populations
Country
Connection type
Device type

26. Make a lot of small improvements

27. Focus on the slowest experiences
3

28. Fix the metric value and watch the percentile
Fix the percentile and watch the metric value
Optimize
Monitor
vs

30. Single Dimension
Analysis

31. Ranking by multiple factors
Dim: entry_page_type
Val: item pages
Dim: entry_url
Val: https://example.com/page1234
Dim: entry_page_type
Val: store locator pages
Dim: country
Val: US
Dim: device_type
Val: iPhone

32. Re-baseline performance
budget at least quarterly

33. It’s critical to know how a
change in performance
will change your business

34. Start when you get back to work.
Ask your business partners what
metrics they use to define business
success. Include these in your RUM
data.

35. Resources
Project Jupyter: https://jupyter.org/
Sklearn’s Random Forest Classfier: https://scikit-
learn.org/stable/modules/generated/
sklearn.ensemble.RandomForestClassifier.html
Example code: https://gist.github.com/bbrewer/
17b29c0bbfb77dc76f5bbaa54c2c7ea4

36. Don't participate, lead.