Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
A Modern Approach to Performance Monitoring
Cliff Crocker 
@cliffcrocker 
SOASTA 
VP Product, mPulse
1. How fast am I? 
1. How fast should I be? 
2. How do I get there?
Synthetic vs. RUM
Synthetic 101 
Synthetic monitoring (for purposes of this discussion) refers to 
the use of automated agents (bots) to mea...
RUM 101 
Real User Measurement (RUM) is a technology for collecting 
performance metrics directly from the browser of an e...
RUM 
Cast a wide net 
• Identify key areas of concern 
• Understand real user impact 
• Map performance to human behavior ...
The Early Days of RUM 
• Round-trip time 
• Start/stop timers via JavaScript 
• Early contributors: 
• Steve Souders/Episo...
Navigation Timing
Browser Support for Navigation Timing 
!
© 2014 SOASTA CONFIDENTIAL - All rights reserved. 10 
Navigation Timing 
DNS: Domain Lookup Time 
function getPerfStats() ...
© 2014 SOASTA CONFIDENTIAL - All rights reserved. 11 
Navigation Timing 
TCP: Connection Time to Server 
function getPerfS...
© 2014 SOASTA CONFIDENTIAL - All rights reserved. 12 
Navigation Timing 
TTFB: Time to First Byte 
function getPerfStats()...
© 2014 SOASTA CONFIDENTIAL - All rights reserved. 13 
Navigation Timing 
Base Page 
function getPerfStats() { 
var timing ...
© 2014 SOASTA CONFIDENTIAL - All rights reserved. 14 
Navigation Timing 
Front-end Time 
function getPerfStats() { 
var ti...
© 2014 SOASTA CONFIDENTIAL - All rights reserved. 15 
Navigation Timing 
Page Load Time 
function getPerfStats() { 
var ti...
Measuring Assets 
• Strength of synthetic 
• Full visibility into asset performance 
• Images 
• JavaScript 
• CSS 
• HTML...
Object Level RUM
Browser Support for Resource Timing
CORS: Cross-Origin Resource Sharing 
Timing-Allow-Origin = "Timing-Allow-Origin" ":" origin-list-or- 
null | "*" 
Start/En...
Resource Timing 
var rUrl = ‘http://www.akamai.com/images/img/cliff-crocker-dualtone- 
150x150.png’; 
var me = performance...
Resource Timing 
• Slowest resources 
• Time to first image 
• Response time by domain 
• Time a group of assets 
• Respon...
Resource Timing 
• PerfMap - https://github.com/zeman/perfmap 
• Mark Zeman 
• Waterfall.js - https://github.com/andydavie...
1. How fast am I? 
1. How fast should I be? 
2. How do I get there?
Picking a Number 
• Industry benchmarks? 
• Apdex? 
• Analyst report? 
• Case studies?
“Synthetic monitoring shows you 
how you relate to your competitors, 
RUM shows you how you relate to 
your customers.” 
–...
Benchmarking
Benchmarking 
• http://soasta.com/mpulseUX
Benchmarking 
• Page construction 
• Requests 
• Images 
• Size 
• Other important metrics 
• Speedindex 
• Start Render 
...
Performance is a Business 
Problem
Yahoo! - 2008 
Increase of 400ms 
causes 5-9% increase in 
user abandonment 
http://www.slideshare.net/stubbornella/design...
Shopzilla - 2009 
A reduction in Page Load 
time of 5s increased site 
conversion 7-12%! 
http://assets.en.oreilly.com/1/e...
Walmart - 2012 
http://minus.com/msM8y8nyh#1e 
SF WebPerf – 
2012 
Up to 2% 
conversion drop for 
every additional 
second
So What?
SIMULATION
How Fast Should You Be? 
• Use synthetic measurement for benchmarking your 
competitors 
• Understand how fast your site n...
1. How fast am I? 
1. How fast should I be? 
2. How do I get there?
Real users are not normal
Page Load Times 
2.76s – Median 
17.26s – p98 
10.45s – p95
Page Load Times 
Platform Median 95th 
Percentile 
98th 
Percentile 
Mobile 3.62s 12.53s 20.04s 
Desktop 2.44s 9.31s 15.86...
Page Load Times 
OS Median 95th 
Percentile 
98th 
Percentile 
Windows 7 2.41s 9.29s 15.89s 
Mac OS X/10 2.30s 8.11s 13.45...
Other Factors 
• Geography 
• User Agent 
• Connection Type 
• Carrier/ISP 
• Device Type
Not All Pages are Created Equal 
For a typical 
eCommerce site, 
conversion rate 
drops by up to 50% 
when “browse” 
pages...
Not All Pages are Created Equal 
However, there is 
much less impact 
to conversion 
when “checkout” 
pages degrade
How Do I Get There? 
• Focus on the highest value opportunities/demographics 
• Identify key pages that have the most impa...
Thank You!
Images 
1. Modern (Title Slide): https://www.flickr.com/photos/looking4poetry/3477854720/in/photolist-6ijV3q-9t3sjb-cj8VWE...
A Modern Approach to Performance Monitoring
A Modern Approach to Performance Monitoring
A Modern Approach to Performance Monitoring
Upcoming SlideShare
Loading in …5
×

A Modern Approach to Performance Monitoring

3,336 views

Published on

Akamai Edge 14 - Discussion on RUM, Synthetic and setting realistic and meaningful performance goals.

Published in: Internet
  • How To Make Every Day Your Perfect Day By Raising Your Vibration ◆◆◆ http://scamcb.com/manifmagic/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

A Modern Approach to Performance Monitoring

  1. 1. A Modern Approach to Performance Monitoring
  2. 2. Cliff Crocker @cliffcrocker SOASTA VP Product, mPulse
  3. 3. 1. How fast am I? 1. How fast should I be? 2. How do I get there?
  4. 4. Synthetic vs. RUM
  5. 5. Synthetic 101 Synthetic monitoring (for purposes of this discussion) refers to the use of automated agents (bots) to measure your website from different physical locations. • A set ‘path’ or URL is defined • Tests are run either adhoc or scheduled and data is collected
  6. 6. RUM 101 Real User Measurement (RUM) is a technology for collecting performance metrics directly from the browser of an end user. • Involves instrumentation of your website via JavaScript • Measurements are fired across the network to a collection point through a small request object (beacon) <JS> <beacon>
  7. 7. RUM Cast a wide net • Identify key areas of concern • Understand real user impact • Map performance to human behavior & $$ Synthetic Diagnostic tool • Identify issues in a ‘lab’/remove variables • Reproduce a problem found with RUM http://www.flickr.com/photos/84338444@N00/with/3780079044/ http://www.flickr.com/photos/ezioman/
  8. 8. The Early Days of RUM • Round-trip time • Start/stop timers via JavaScript • Early contributors: • Steve Souders/Episodes • Philip Tellis/Boomerang.js • Both widely in use today
  9. 9. Navigation Timing
  10. 10. Browser Support for Navigation Timing !
  11. 11. © 2014 SOASTA CONFIDENTIAL - All rights reserved. 10 Navigation Timing DNS: Domain Lookup Time function getPerfStats() { var timing = window.performance.timing; return { dns: timing.domainLookupEnd - timing.domainLookupStart}; }
  12. 12. © 2014 SOASTA CONFIDENTIAL - All rights reserved. 11 Navigation Timing TCP: Connection Time to Server function getPerfStats() { var timing = window.performance.timing; return { connect: timing.connectEnd - timing.connectStart}; }
  13. 13. © 2014 SOASTA CONFIDENTIAL - All rights reserved. 12 Navigation Timing TTFB: Time to First Byte function getPerfStats() { var timing = window.performance.timing; return { ttfb: timing.responseStart - timing.connectEnd}; }
  14. 14. © 2014 SOASTA CONFIDENTIAL - All rights reserved. 13 Navigation Timing Base Page function getPerfStats() { var timing = window.performance.timing; return { basePage: timing.responseEnd - timing.responseStart}; }
  15. 15. © 2014 SOASTA CONFIDENTIAL - All rights reserved. 14 Navigation Timing Front-end Time function getPerfStats() { var timing = window.performance.timing; return { frontEnd: timing.loadEventStart - timing.responseEnd}; }
  16. 16. © 2014 SOASTA CONFIDENTIAL - All rights reserved. 15 Navigation Timing Page Load Time function getPerfStats() { var timing = window.performance.timing; return { load: timing.loadEventStart - timing.navigationStart}; }
  17. 17. Measuring Assets • Strength of synthetic • Full visibility into asset performance • Images • JavaScript • CSS • HTML • A lot of which is served by third-parties • CDN!
  18. 18. Object Level RUM
  19. 19. Browser Support for Resource Timing
  20. 20. CORS: Cross-Origin Resource Sharing Timing-Allow-Origin = "Timing-Allow-Origin" ":" origin-list-or- null | "*" Start/End time only unless Timing-Allow-Origin HTTP Response Header defined
  21. 21. Resource Timing var rUrl = ‘http://www.akamai.com/images/img/cliff-crocker-dualtone- 150x150.png’; var me = performance.getEntriesByName(url)[0]; var timings = { loadTime: me.duration, dns: me.domainLookupEnd - me.domainLookupStart, tcp: me.connectEnd - me.connectStart, waiting: me.responseStart - me.requestStart, fetch: me.responseEnd - me.responseStart }
  22. 22. Resource Timing • Slowest resources • Time to first image • Response time by domain • Time a group of assets • Response time by initiator type (element type) • Cache-hit ratio for resources For examples see: http://www.slideshare.net/bluesmoon/beyond-page-level-metrics
  23. 23. Resource Timing • PerfMap - https://github.com/zeman/perfmap • Mark Zeman • Waterfall.js - https://github.com/andydavies/waterfall • Andy Davies
  24. 24. 1. How fast am I? 1. How fast should I be? 2. How do I get there?
  25. 25. Picking a Number • Industry benchmarks? • Apdex? • Analyst report? • Case studies?
  26. 26. “Synthetic monitoring shows you how you relate to your competitors, RUM shows you how you relate to your customers.” – Buddy Brewer
  27. 27. Benchmarking
  28. 28. Benchmarking • http://soasta.com/mpulseUX
  29. 29. Benchmarking • Page construction • Requests • Images • Size • Other important metrics • Speedindex • Start Render • PageSpeed/Yslow scoring
  30. 30. Performance is a Business Problem
  31. 31. Yahoo! - 2008 Increase of 400ms causes 5-9% increase in user abandonment http://www.slideshare.net/stubbornella/designing-fast-websites-presentation
  32. 32. Shopzilla - 2009 A reduction in Page Load time of 5s increased site conversion 7-12%! http://assets.en.oreilly.com/1/event/29/Shopzilla%27s%20Site%20Redo%20- %20You%20Get%20What%20You%20Measure%20Presentation.ppt
  33. 33. Walmart - 2012 http://minus.com/msM8y8nyh#1e SF WebPerf – 2012 Up to 2% conversion drop for every additional second
  34. 34. So What?
  35. 35. SIMULATION
  36. 36. How Fast Should You Be? • Use synthetic measurement for benchmarking your competitors • Understand how fast your site needs to be to reach business goals/objectives with RUM • You must look at your own data
  37. 37. 1. How fast am I? 1. How fast should I be? 2. How do I get there?
  38. 38. Real users are not normal
  39. 39. Page Load Times 2.76s – Median 17.26s – p98 10.45s – p95
  40. 40. Page Load Times Platform Median 95th Percentile 98th Percentile Mobile 3.62s 12.53s 20.04s Desktop 2.44s 9.31s 15.86s
  41. 41. Page Load Times OS Median 95th Percentile 98th Percentile Windows 7 2.41s 9.29s 15.89s Mac OS X/10 2.30s 8.11s 13.45s iOS7 3.27s 10.64s 15.79s Android 4 4.06s 14.30s 27.93s iOS8 3.53s 11.54s 19.72s Windows 8 2.67s 10.75s 18,74s
  42. 42. Other Factors • Geography • User Agent • Connection Type • Carrier/ISP • Device Type
  43. 43. Not All Pages are Created Equal For a typical eCommerce site, conversion rate drops by up to 50% when “browse” pages increase from 1 to 6 seconds
  44. 44. Not All Pages are Created Equal However, there is much less impact to conversion when “checkout” pages degrade
  45. 45. How Do I Get There? • Focus on the highest value opportunities/demographics • Identify key pages that have the most impact on your KPIs • Prioritize based on reducing friction within the funnel or critical path
  46. 46. Thank You!
  47. 47. Images 1. Modern (Title Slide): https://www.flickr.com/photos/looking4poetry/3477854720/in/photolist-6ijV3q-9t3sjb-cj8VWE-MNjPA- 4yBqug-2rk5he-4nZVzJ-4yE45J-9t3Fvj-2rptr5-69ymdr-6XVTH-6QZ4dg-9t3HN5-8geUkB-6oEXs5-eH9zS-H2XTt-cKdJvb- 7GNKWx-cKdKZd-5Rw3qt-jfT5Dx-Jfiuy-bTop6R-2m8kAB-Jfiuu-7E8eMf-9jqD6-9hYAvd-Jfius-gsqr7U-7KiNAZ-8kzG9V-euXdb- 44qJNN-47jBuY-nsxZrZ-7cGjQL-4cfHKq-cGzbSC-aQtWPR-8y3sR3-6okjAW-5A7BSC-6aRsvh-eSMHkX-kxvdyK-9t3udu-c8EkYu 2. Man vs. Machine: https://www.flickr.com/photos/eogez/3289851965/in/photolist-61HmnB-9nkQu3-dfinF1-4GjK24-4GjL6Z- 4GoV6C-dibVCU-4GoTXo-4GjF3k-4GjH7Z-4GoR6h-ek6eT9-ek6eUU-4fADUx-4fEDV9-65JA6W-65Jsxu-65JDnj-4fAE12- 65JDTh-65JBZu-65Ek9M-65JtDu-65JuiN-65JzFS-65EfiH-65Jv63-65JxX7-65Enyc-65Eh4c-65Jxjm-65JwmU-65EhUP-65JCML- 65EjxP-65JBQf-65Ee4Z-65JD4b-65JBF1-65Ec1r-65EkPB-65Jy4o-65Eg96-65JAjb-65Emfe-65Efoz-65JBam-65Eniv-65JCrU- 65Eed2 3. https://www.flickr.com/photos/bradybd/2818154005/ 4. https://www.flickr.com/photos/perspective/149321089/ 5. Dartboard: http://en.wikipedia.org/wiki/Darts#mediaviewer/File:Darts_in_a_dartboard.jpg References: 1. http://calendar.perfplanet.com/2011/a-practical-guide-to-the-navigation-timing-api/ 2. http://www.slideshare.net/bluesmoon/beyond-page-level-metrics

×