You want a single, unicorn metric that magically sums up the user experience, business value, and numbers that DevOps cares about, but so far, you're just not getting it. So where do you start? In this talk at the 2015 Velocity conference in Santa Clara, Cliff Crocker and I walked through various metrics that answer performance questions from multiple perspectives -- from designer and DevOps to CRO and CEO.

1. Metrics, metrics everywhere
(but where the heck do you start?)

2. @tameverts @cliffcrocker
#velocityconf

4. Who cares about performance today?
How do I measure performance?
How fast am I?
How fast should I be?
How do I get there?

6. The myth of a single metric

8. Start render DNS TCP TTFB
DOM loading DOM ready Page load Fully loaded
User timing Resource timing Requests Bytes in
Speed Index Pagespeed score 1s = $$ DOM elements
DOM size Visually complete Redirect SSL negotiation

9. Who cares about performance?

10. “47% of consumers expect a web
page to load in 2 seconds or less.”
Akamai, 2009

12. 1s = $27M DNS
144ms
Start render
1.59s
Hero image render
2.01s

13. How do I measure performance?

17. Androiddevicefragmentation
OpenSignal,August2014

18. RUM versus plus synthetic

19. RUM 101

20. Technology for collecting performance metrics
directly from the end user’s browser
Involves instrumenting your site via JavaScript
Measurements are fired across the network to a
collection point through a small request object
(beacon)

21. What makes RUM great
 Always on
 Every user, every browser, everywhere
 Able to capture human behavior/events
 Only getting better

22. Questions your RUM data can answer

23. What are
my users’
environments?

24. How do visitors move
through my site?

25. How are my
third-party scripts
performing in
real time?

26. What impact does
performance have
on my business?

27. Synthetic Monitoring 101

28. Uses automated agents (bots) to measure your
website from different physical locations
A set “path” or URL is defined
Tests are run either ad hoc or scheduled,
and data is collected

29. What makes synthetic monitoring great
 Rich data collected (waterfall, video,
filmstrip, HTTP headers)
 Consistent “clean room” baseline
 Nothing to install
 Doesn’t require users / ability to
measure pre-production and
competition

30. Questions your synthetic data can answer

31. How do I compare to the competition?

32. How does the
design of my
pages affect
performance?

33. How does
the newest
version
of my site
compare
to previous
versions?

34. How well am I sticking to my performance budget?

35. What if my third parties fail?
Original: 3.5s
SPOF: 22.7s

37. 36© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Why are these numbers so different?
I don’t trust your data. Your numbers are wrong.
How are you calculating page load time?
I can’t share two sets of numbers with the business?

38. “But it loads so much faster for me!?!”
2015 Macbook Pro
Warm browser cache
FIOS
X86 – Windows 7 VM
Completely cold cache/DNS
Throttled bandwidth

39. boomerang.js
Episodes

40. W3C Performance
Working Group

43. How fast am I?

44. Navigation Timing API

45. Browser support for Navigation Timing

46. 45© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Network operations
Front-end developer
Marketing and site operations
Designer
C-level

47. Use case: Measure
network performance

48. I need visibility into…
 issues with authoritative DNS servers
 impact of denial of service attacks
on end users
 efficiency of connection re-use
 tier 1 connectivity issues (load balancer,
web server)

49. Start render DNS TCP TTFB
DOM loading DOM ready Page load Fully loaded
User timing Resource timing Requests Bytes in
Speed Index Pagespeed score 1s = $$ DOM elements
DOM size Visually complete Redirect SSL negotiation

50. Measuring DNS and TCP
function getPerf() {
var timing = window.performance.timing;
return {
dns: timing.domainLookupEnd -
timing.domainLookupStart};
connect: timing.connectEnd - timing.connectStart};
}

51. What’s with all those zeros!
 Connection reuse
 DNS caching
 Prefetching

52. Focus on higher percentiles
85th percentile
Median (50th)

53. Use case: Measure
front-end browser events

54. How do I…
 understand the impact of back-end
versus front-end on page speed?
 investigate how DOM complexity affects
performance?
 measure a “fully loaded” page?

55. Start render DNS TCP TTFB
DOM load event DOM ready Page load Fully loaded
User timing Resource timing Requests Bytes in
Speed Index Pagespeed score 1s = $$ DOM elements
DOM size Visually complete Redirect SSL negotiation

56. Isolate front-end vs. back-end

57. Isolate front-end vs. back-end
function getPerf() {
var timing = window.performance.timing;
return {
ttfb: timing.responseStart - timing.connectEnd};
basePage: timing.responseEnd - timing.responseStart};
frontEnd: timing.loadEventStart -
timing.responseEnd};
}

59. Front-end
Back-end

60. Investigate DOM events
function getPerf() {
var timing = window.performance.timing;
return {
DLoading: timing.domLoading –
timing.navigationStart};
DInt: timing.domInteractive –
timing.navigationStart};
DContLoaded: timing.domContentLoadedEventEnd –
timing.navigationStart;
DContLoadTime: timing.domContentLoadedEventEnd –
timing.domContentLoadedEventStart};
DComplete: timing.domComplete -
timing.navigationStart};
PLoad: timing.loadEventStart -

61. 2618 DOM nodes
86 DOM nodes
Visualizing DOM complexity

62. Use case: Measure
object-level performance

63. I need to understand…
 how third-party content affects my
performance
 how long a group of assets takes to
download
 how assets served by a CDN are
performing

64. Start render DNS TCP TTFB
DOM loading DOM ready Page load Fully loaded
User timing Resource timing Requests Bytes in
Speed Index Pagespeed score 1s = $$ DOM elements
DOM size Visually complete Redirect SSL negotiation

65. Resource Timing interface

66. Browser support for Resource Timing

67. Cross-Origin Resource Sharing (CORS)
Start/End time only unless Timing-Allow-Origin
HTTP Response Header defined
Timing-Allow-Origin = "Timing-Allow-Origin" ":" origin-list-
or-null | "*"

68. Resource Timing
var rUrl = ‘http://www.akamai.com/images/img/cliff-crocker-
dualtone-150x150.png’;
var me = performance.getEntriesByName(rUrl)[0];
var timings = {
loadTime: me.duration,
dns: me.domainLookupEnd - me.domainLookupStart,
tcp: me.connectEnd - me.connectStart,
waiting: me.responseStart - me.requestStart,
fetch: me.responseEnd - me.responseStart
}
Measuring a single resource

69. Other uses for Resource Timing
 Slowest resources
 Time to first image (download)
 Response time by domain
 Time a group of assets
 Response time by initiator type (element type)
 Cache-hit ratio for resources
For examples see: https://github.com/lognormal/beyond-page-metrics

70. Using Resource Groups
PLT impact not due
resource groups
PLT impact
correlates with
improvement
from image domains

71. Use case: Measure
the user experience

72. I just need to understand…
 when users perceive the page to
be ready
 how long until my page begins
to render
 when content above the fold is visible

73. Start render DNS TCP TTFB
DOM loading DOM ready Page load Fully loaded
User timing Resource timing Requests Bytes in
Speed Index Pagespeed score 1s = $$ DOM elements
DOM size Visually complete Redirect SSL negotiation

74. The fallacy of “First Paint” in the wild
 Support for First Paint is not standardized between browsers
 Metric can be misleading (i.e. painting a white screen)

75. First Paint is not equal to Start Render!
Chrome – “First Paint” True Start Render

76. Start Render and filmstrips

77. User Timing Interface
 Allows developers to measure performance of
their applications through high-precision
timestamps
 Consists of “marks” and “measures”
 PerformanceMark: Timestamp
 PerformanceMeasure: Duration between
two given marks

78. Measure duration between two marks
performance.mark(“startTask”);
//Some stuff you want to measure happens here
performance.mark(“stopTask”);
//Measure the duration between the two marks
performance.measure(“taskDuration”,“startTask”,“stopTask”);

79. How long does it
take to display
the main product
image on my site?

80. Record when an image loads
<img src=“image1.gif” onload=“performance.mark(‘image1’)”>
For more interesting examples, see:
Measure hero image delay
http://www.stevesouders.com/blog/2015/05/12/hero-image-custom-metrics/
Measure aggregate times to get an “above fold time”
http://blog.patrickmeenan.com/2013/07/measuring-performance-of-user-
experience.html

81. How do I measure performance
when the onload event no longer
matters?
Use case: Measure
performance of SPAs

82. onload event
visible resources

83. Measuring SPAs
• Accept the fact that onload no longer matters
• Tie into routing events of SPA framework
• Measure downloads during soft refreshes
• (support in boomerang.js for Angular and other
SPA frameworks)
See: http://www.soasta.com/blog/angularjs-real-user-
monitoring-single-page-applications/

84. How fast should I be?

85. Use case: Measure
business impact

86. I need to understand…
 how performance affects business KPIs
 how our site compares to our
competitors

87. Start render DNS TCP TTFB
DOM loading DOM ready Page load Fully loaded
User timing Resource timing Requests Bytes in
Speed Index Pagespeed score 1s = $$ DOM elements
DOM size Visually complete Redirect SSL negotiation

90. 2% increase in conversions
for every 1 second of improvement

91. Cut load times in half
Increased sales by 13%

93. So what?
You must look at your own data.

96. Not all pages are created equal
For a typical
ecommerce site,
conversion rate
drops by up to 50%
when “browse”
pages increase from
1 to 6 seconds

97. Not all pages are created equal
However, there is
much less impact
to conversion
when “checkout”
pages degrade

98. Conversion Impact Score

99. How do I get there?

101. Create a performance budget
See:
Setting a Performance Budget
http://timkadlec.com/2013/01/setting-a-performance-budget/
Performance Budget Metrics
http://timkadlec.com/2014/11/performance-budget-metrics/

102. Set meaningful, page-specific SLAs

103. “Response time measured using resource timing from Chrome
browsers in the United States should not exceed a median
(50th percentile) of 100ms or a 95th percentile of 500ms for
a population of more than 500 users in a 24-hour period.”

104. “Vendor will make an effort to ensure average response
times for content is within reasonable limits.”

106. Chapter 8:
Changing Culture
at Your Organization

107. performancebacon.com
performancebeacon.com

108. Thanks!

109. Meet us at booth #801