Your SlideShare is downloading. ×
0
BEYOND AVERAGES
Dan Kuebrich / @dankosaur
Beyond Averages
• Abstraction: summary statistics for performance data	

• Why performance data is hard	

• Visualizing da...
A few of my favorite abstractions
A few of my favorite abstractions
• Abstraction lets us trade information for
actionability

A few of my favorite abstractions
• Abstraction lets us trade information for
actionability


• Min, max, average (“mean”)...
A few of my favorite abstractions
• Abstraction lets us trade information for
actionability


• Min, max, average (“mean”)...
Averages: average at best
Averages: average at best
Averages: average at best
Averages: average at best
Averages vs Percentiles

[1, 16, 17, 19, 13, 5, 20, 3, 10, 14, 8]
Averages vs Percentiles

[1, 3, 5, 8, 10, 13, 14, 16, 17, 19, 20]
Averages vs Percentiles
Average (Mean): 11.54

[1, 3, 5, 8, 10, 13, 14, 16, 17, 19, 20]
Median (50th Percentile): 13

90th...
Percentiles: 1 of 100 slices
X

95%
Percentiles: 2 of 100 slices
X

95%

10%
Y
Percentiles: 2 of 100 slices
95%

10%
Percentiles: 2 of 100 slices
95%

10%
Percentiles: 2 of 100 slices
95%

10%
Computers are hard
• Rarely do we have a single distribution underlying the
data


• Different users, different requests, ...
Percentiles vs Distributions

http://en.wikipedia.org/wiki/Percentile
Percentiles vs Distributions

[13, 13, 13, 13, 13, 13, 19, 19, 19, 19, 19]
Median (50th Percentile): 13

90th Percentile: ...
Rarely do we have a single normal
distribution underlying the data
Median	

Mean	

90th
Median	

Mean	

90th
The Log-Normal Distribution

(source: http://www.geo.mtu.edu/volcanoes/vc_web/background/S_chem.html)
Log-Normal Distribution

(source: http://en.wikipedia.org/wiki/File:Comparison_mean_median_mode.svg)
Log-Normal Distribution

(source: http://en.wikipedia.org/wiki/File:Comparison_mean_median_mode.svg)
Log-Normal Distribution
Is there a place between Averageland
and “A Beautiful Mind”?

http://now-here-this.timeout.com/2012/10/07/crazy-walls-of-c...
(eg. # of calls)

Frequency	


Histograms

Value	

(eg. latency)
Populations revisited
95%

10%
(eg. # of calls)

Frequency	


Histograms

Value	

(eg. latency)
Populations re-revisited
95%

?

10%
(eg. # of calls)

Frequency	


3d Histograms?

Value	

(eg. latency)
(eg. # of calls)

Frequency	


3d Histograms?

Time

Value	

(eg. latency)
(eg. # of calls)

Frequency	


Heatmaps

Value	

(eg. latency)
(eg. # of calls)

Frequency	


Heatmaps

Value	

(eg. latency)
(eg. # of calls)

Frequency	


Heatmaps

Value	

(eg. latency)
(eg. latency)

Value	


Heatmaps

Time
(eg. latency)

Value	


Heatmaps

Time
Latency in the wild…

http://sciencefiction.com/2013/10/24/throwback-thursday-jurassic-park/
Average, or Absolute?
Average, or Absolute?
Multi-modal Data
Multi-modal Data
Multi-modal Drill-down
Multi-modal Drill-down
Multi-modal Drill-down
Long Tails and Outliers
Long Tails and Outliers
bottom 98%
Long Tails and Outliers
bottom 98%
Long Tails and Outliers
all of it
Added Population
Added Population
Added Population
Thanks!
Sign up for a free TraceView account at
appneta.com/products/traceview-free-account
Beyond Averages - Web Performance Meetup
Beyond Averages - Web Performance Meetup
Beyond Averages - Web Performance Meetup
Beyond Averages - Web Performance Meetup
Beyond Averages - Web Performance Meetup
Beyond Averages - Web Performance Meetup
Upcoming SlideShare
Loading in...5
×

Beyond Averages - Web Performance Meetup

229

Published on

When raw data becomes overwhelming, we turn to abstraction to understand our world. In examining the performance of our systems, the data is always overwhelming. Solutions like summary statistics have come to our rescue, and they are good—up to a point. In order to truly understand our systems, we need to know when and how to sidestep those abstractions,to get deep, detailed performance insight. At this meetup, I’ll explore techniques for visualizing the underlying structure of performance data and how this empowers drilling down to populations and individual samples in the data set.

Published in: Technology, Art & Photos
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
229
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Beyond Averages - Web Performance Meetup"

  1. 1. BEYOND AVERAGES Dan Kuebrich / @dankosaur
  2. 2. Beyond Averages • Abstraction: summary statistics for performance data • Why performance data is hard • Visualizing data distributions
  3. 3. A few of my favorite abstractions
  4. 4. A few of my favorite abstractions • Abstraction lets us trade information for actionability

  5. 5. A few of my favorite abstractions • Abstraction lets us trade information for actionability
 • Min, max, average (“mean”), quantiles, stdev

  6. 6. A few of my favorite abstractions • Abstraction lets us trade information for actionability
 • Min, max, average (“mean”), quantiles, stdev
 • That’s a great trade! • ... right?
  7. 7. Averages: average at best
  8. 8. Averages: average at best
  9. 9. Averages: average at best
  10. 10. Averages: average at best
  11. 11. Averages vs Percentiles [1, 16, 17, 19, 13, 5, 20, 3, 10, 14, 8]
  12. 12. Averages vs Percentiles [1, 3, 5, 8, 10, 13, 14, 16, 17, 19, 20]
  13. 13. Averages vs Percentiles Average (Mean): 11.54 [1, 3, 5, 8, 10, 13, 14, 16, 17, 19, 20] Median (50th Percentile): 13 90th Percentile: 19
  14. 14. Percentiles: 1 of 100 slices X 95%
  15. 15. Percentiles: 2 of 100 slices X 95% 10% Y
  16. 16. Percentiles: 2 of 100 slices 95% 10%
  17. 17. Percentiles: 2 of 100 slices 95% 10%
  18. 18. Percentiles: 2 of 100 slices 95% 10%
  19. 19. Computers are hard • Rarely do we have a single distribution underlying the data
 • Different users, different requests, different resources, different instances, different times
  20. 20. Percentiles vs Distributions http://en.wikipedia.org/wiki/Percentile
  21. 21. Percentiles vs Distributions [13, 13, 13, 13, 13, 13, 19, 19, 19, 19, 19] Median (50th Percentile): 13 90th Percentile: 19
  22. 22. Rarely do we have a single normal distribution underlying the data
  23. 23. Median Mean 90th
  24. 24. Median Mean 90th
  25. 25. The Log-Normal Distribution (source: http://www.geo.mtu.edu/volcanoes/vc_web/background/S_chem.html)
  26. 26. Log-Normal Distribution (source: http://en.wikipedia.org/wiki/File:Comparison_mean_median_mode.svg)
  27. 27. Log-Normal Distribution (source: http://en.wikipedia.org/wiki/File:Comparison_mean_median_mode.svg)
  28. 28. Log-Normal Distribution
  29. 29. Is there a place between Averageland and “A Beautiful Mind”? http://now-here-this.timeout.com/2012/10/07/crazy-walls-of-clues-from-tv-film-reviewed-by-carrie-from-homeland/
  30. 30. (eg. # of calls) Frequency Histograms Value (eg. latency)
  31. 31. Populations revisited 95% 10%
  32. 32. (eg. # of calls) Frequency Histograms Value (eg. latency)
  33. 33. Populations re-revisited 95% ? 10%
  34. 34. (eg. # of calls) Frequency 3d Histograms? Value (eg. latency)
  35. 35. (eg. # of calls) Frequency 3d Histograms? Time Value (eg. latency)
  36. 36. (eg. # of calls) Frequency Heatmaps Value (eg. latency)
  37. 37. (eg. # of calls) Frequency Heatmaps Value (eg. latency)
  38. 38. (eg. # of calls) Frequency Heatmaps Value (eg. latency)
  39. 39. (eg. latency) Value Heatmaps Time
  40. 40. (eg. latency) Value Heatmaps Time
  41. 41. Latency in the wild… http://sciencefiction.com/2013/10/24/throwback-thursday-jurassic-park/
  42. 42. Average, or Absolute?
  43. 43. Average, or Absolute?
  44. 44. Multi-modal Data
  45. 45. Multi-modal Data
  46. 46. Multi-modal Drill-down
  47. 47. Multi-modal Drill-down
  48. 48. Multi-modal Drill-down
  49. 49. Long Tails and Outliers
  50. 50. Long Tails and Outliers bottom 98%
  51. 51. Long Tails and Outliers bottom 98%
  52. 52. Long Tails and Outliers all of it
  53. 53. Added Population
  54. 54. Added Population
  55. 55. Added Population
  56. 56. Thanks! Sign up for a free TraceView account at appneta.com/products/traceview-free-account
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×