Your SlideShare is downloading. ×
0
• Philip Tellis• @bluesmoon• philip@bluesmoon.info• SOASTA• boomerang         LSPE Meetup / 2013-01-17   Measuring the Per...
Measuring the Performance Experienced by               Real Users     Philip Tellis / philip@bluesmoon.info               ...
Real Users             LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Real Users   3
Real Users     • Get delighted     • Get frustrated     • Have credit cards     • ... that they are attached to     • Form...
Real Users     • Get delighted     • Get frustrated     • Have credit cards     • ... that they are attached to     • Form...
Real Users     • Get delighted     • Get frustrated     • Have credit cards     • ... that they are attached to     • Form...
Real Users     • Get delighted     • Get frustrated     • Have credit cards     • ... that they are attached to     • Form...
Real Users     • Get delighted     • Get frustrated     • Have credit cards     • ... that they are attached to     • Form...
What can we measure?LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Real Users   4
Anything that the user does!LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Real Users   5
Anything that the user does!           (in the browser)LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by...
But we really care about performance here. . .     LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Rea...
So let’s talk about HowLSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Real Users   7
Modern Browsers – no problem           LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Real Users   8
Modern Browsers – no problem           LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Real Users   9
Real Users don’t always use the latest browsers.      LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by ...
What about older browsers?    1   Get a timestamp before they leave a page    2   Save it in a cookie    3   Compare with ...
What about older browsers?    1   Get a timestamp before they leave a page    2   Save it in a cookie    3   Compare with ...
The magic. . . ...               LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Real Users   12
But it doesn’t need to be. . .        http://lognormal.github.com/boomerang/doc/              LSPE Meetup / 2013-01-17   M...
boomerang    • BSD licensed    • Measure load time, bandwidth, latency    • DNS lookup time, NavTiming, Connection Type   ...
And it beacons all of this back to a predefined URL        LSPE Meetup / 2013-01-17   Measuring the Performance Experienced...
So what then?LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Real Users   16
Sparse Distribution            LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Real Users   17
Log-normal distribution            LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Real Users   18
Bimodal distribution            LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Real Users   19
Distributions     • Sparse distribution suggests that you don’t have enough       data points     • Log-normal distributio...
Distributions are interesting, but how does it affect                   engagement?        LSPE Meetup / 2013-01-17   Meas...
So we compared load time to bounce rate   LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Real Users  ...
And we borrowed a biological term called LD50      LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Rea...
LD50       LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Real Users   24
This helps you fix only the stuff that’s hurting you.        LSPE Meetup / 2013-01-17   Measuring the Performance Experienc...
Did you know that IE7 seems to be faster than IE8?        LSPE Meetup / 2013-01-17   Measuring the Performance Experienced...
But that’s mostly IE9 users browsing in IE7           compatibility mode.    LSPE Meetup / 2013-01-17   Measuring the Perf...
Did you know that users bounce if a page is too                   fast?      LSPE Meetup / 2013-01-17   Measuring the Perf...
But that’s mostly because error pages are the fastest.         LSPE Meetup / 2013-01-17   Measuring the Performance Experi...
Did you know that user patience wrt performance is               a cultural thing?        LSPE Meetup / 2013-01-17   Measu...
It’s tied both to where a user is from, and also the                  type of service.       LSPE Meetup / 2013-01-17   Me...
Demo?LSPE Meetup / 2013-01-17    Measuring the Performance Experienced by Real Users   32
Summary   1   Measure the load time experienced by real users   2   Correlate it with other things that user did   3   Pro...
Summary   1   Measure the load time experienced by real users   2   Correlate it with other things that user did   3   Pro...
Summary   1   Measure the load time experienced by real users   2   Correlate it with other things that user did   3   Pro...
Thank You!                     Questions?LSPE Meetup / 2013-01-17   Measuring the Performance Experienced by Real Users   34
• Philip Tellis• @bluesmoon• philip@bluesmoon.info• www.SOASTA.com• boomerang• LogNormal Blog         LSPE Meetup / 2013-0...
Upcoming SlideShare
Loading in...5
×

Measuring the Performance Experienced by Real Users

1,477

Published on

My talk at the LSPE meetup in Jan 2013.

http://www.meetup.com/SF-Bay-Area-Large-Scale-Production-Engineering/events/92123162/

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,477
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
21
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "Measuring the Performance Experienced by Real Users"

  1. 1. • Philip Tellis• @bluesmoon• philip@bluesmoon.info• SOASTA• boomerang LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 1
  2. 2. Measuring the Performance Experienced by Real Users Philip Tellis / philip@bluesmoon.info LSPE Meetup / 2013-01-17 LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 2
  3. 3. Real Users LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3
  4. 4. Real Users • Get delighted • Get frustrated • Have credit cards • ... that they are attached to • Form an impression from experience LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3
  5. 5. Real Users • Get delighted • Get frustrated • Have credit cards • ... that they are attached to • Form an impression from experience LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3
  6. 6. Real Users • Get delighted • Get frustrated • Have credit cards • ... that they are attached to • Form an impression from experience LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3
  7. 7. Real Users • Get delighted • Get frustrated • Have credit cards • ... that they are attached to • Form an impression from experience LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3
  8. 8. Real Users • Get delighted • Get frustrated • Have credit cards • ... that they are attached to • Form an impression from experience LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3
  9. 9. What can we measure?LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 4
  10. 10. Anything that the user does!LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 5
  11. 11. Anything that the user does! (in the browser)LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 5
  12. 12. But we really care about performance here. . . LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 6
  13. 13. So let’s talk about HowLSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 7
  14. 14. Modern Browsers – no problem LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 8
  15. 15. Modern Browsers – no problem LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 9
  16. 16. Real Users don’t always use the latest browsers. LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 10
  17. 17. What about older browsers? 1 Get a timestamp before they leave a page 2 Save it in a cookie 3 Compare with timestamp when the page loads LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 11
  18. 18. What about older browsers? 1 Get a timestamp before they leave a page 2 Save it in a cookie 3 Compare with timestamp when the page loads Then there’s some magic to handle edge-cases LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 11
  19. 19. The magic. . . ... LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 12
  20. 20. But it doesn’t need to be. . . http://lognormal.github.com/boomerang/doc/ LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 13
  21. 21. boomerang • BSD licensed • Measure load time, bandwidth, latency • DNS lookup time, NavTiming, Connection Type • Page Parameters • And anything else you can write a plugin for LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 14
  22. 22. And it beacons all of this back to a predefined URL LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 15
  23. 23. So what then?LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 16
  24. 24. Sparse Distribution LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 17
  25. 25. Log-normal distribution LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 18
  26. 26. Bimodal distribution LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 19
  27. 27. Distributions • Sparse distribution suggests that you don’t have enough data points • Log-normal distribution is typical • Bi-modal distribution suggests two (or more) distributions combined LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 20
  28. 28. Distributions are interesting, but how does it affect engagement? LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 21
  29. 29. So we compared load time to bounce rate LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 22
  30. 30. And we borrowed a biological term called LD50 LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 23
  31. 31. LD50 LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 24
  32. 32. This helps you fix only the stuff that’s hurting you. LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 25
  33. 33. Did you know that IE7 seems to be faster than IE8? LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 26
  34. 34. But that’s mostly IE9 users browsing in IE7 compatibility mode. LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 27
  35. 35. Did you know that users bounce if a page is too fast? LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 28
  36. 36. But that’s mostly because error pages are the fastest. LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 29
  37. 37. Did you know that user patience wrt performance is a cultural thing? LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 30
  38. 38. It’s tied both to where a user is from, and also the type of service. LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 31
  39. 39. Demo?LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 32
  40. 40. Summary 1 Measure the load time experienced by real users 2 Correlate it with other things that user did 3 Profit LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 33
  41. 41. Summary 1 Measure the load time experienced by real users 2 Correlate it with other things that user did 3 Profit LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 33
  42. 42. Summary 1 Measure the load time experienced by real users 2 Correlate it with other things that user did 3 Profit LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 33
  43. 43. Thank You! Questions?LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 34
  44. 44. • Philip Tellis• @bluesmoon• philip@bluesmoon.info• www.SOASTA.com• boomerang• LogNormal Blog LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 35
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×