SlideShare a Scribd company logo
Introduction
               Statistics - I
               Statistics - II




The Statistics of Web Performance

 Philip Tellis / philip@bluesmoon.info


           ConFoo / 2010-03-12




       ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                              Statistics - I
                              Statistics - II


$ finger philip




      Philip Tellis
      philip@bluesmoon.info
      @bluesmoon
      yahoo
      geek




                      ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                          The goal
        Statistics - I
                          Performance Measurement
        Statistics - II




      Introduction




ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                        The goal
                      Statistics - I
                                        Performance Measurement
                      Statistics - II




Accurately measure page performance
    At least, as accurately as possible




              ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                        The goal
                      Statistics - I
                                        Performance Measurement
                      Statistics - II




Accurately measure page performance
    At least, as accurately as possible




              ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                              The goal
                            Statistics - I
                                              Performance Measurement
                            Statistics - II


Be unintrusive




  If you try to measure something accurately, you will change
  something related
                                                               – Heisenberg’s uncertainty principle




                    ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                            The goal
          Statistics - I
                            Performance Measurement
          Statistics - II




And one number to rule them all




  ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                             The goal
                           Statistics - I
                                             Performance Measurement
                           Statistics - II


Bandwidth




     Real bandwidth v/s advertised bandwidth
     Bandwidth to your server, not to the ISP
     Bandwidth during normal internet usage
         If the user’s always watching movies, you’re not winning




                   ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                             The goal
                           Statistics - I
                                             Performance Measurement
                           Statistics - II


Bandwidth




     Real bandwidth v/s advertised bandwidth
     Bandwidth to your server, not to the ISP
     Bandwidth during normal internet usage
         If the user’s always watching movies, you’re not winning




                   ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                                    The goal
                                  Statistics - I
                                                    Performance Measurement
                                  Statistics - II


Latency



        How long does it take a byte to get to the user?
              Wired, wireless, mobile, satellite?
              How many hops in between?
              Speed of light is constant
        This is not a battle we will soon win.
              When was the last time you heard latency mentioned in a
              TV ad?
  http://www.stuartcheshire.org/rants/Latency.html




                          ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                                    The goal
                                  Statistics - I
                                                    Performance Measurement
                                  Statistics - II


Latency



        How long does it take a byte to get to the user?
              Wired, wireless, mobile, satellite?
              How many hops in between?
              Speed of light is constant
        This is not a battle we will soon win.
              When was the last time you heard latency mentioned in a
              TV ad?
  http://www.stuartcheshire.org/rants/Latency.html




                          ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                                    The goal
                                  Statistics - I
                                                    Performance Measurement
                                  Statistics - II


Latency



        How long does it take a byte to get to the user?
              Wired, wireless, mobile, satellite?
              How many hops in between?
              Speed of light is constant
        This is not a battle we will soon win.
              When was the last time you heard latency mentioned in a
              TV ad?
  http://www.stuartcheshire.org/rants/Latency.html




                          ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                             The goal
                           Statistics - I
                                             Performance Measurement
                           Statistics - II


User perceived page load time



     Time from “click on a link” to “spinner stops spinning”
     This is what users notice
         Depends on how long your page takes to build
         Depends on what’s in your page
         Depends on how long components take to load
         Depends on how long the browser takes to execute and
         render




                   ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                             The goal
           Statistics - I
                             Performance Measurement
           Statistics - II




We need to measure real user data




   ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                          The goal
                        Statistics - I
                                          Performance Measurement
                        Statistics - II




The statistics apply to any kind of performance data though




                ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
        Statistics - I    Margin of Error
        Statistics - II   Central Tendency




      Statistics - I




ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                         Statistics - I    Margin of Error
                         Statistics - II   Central Tendency


Disclaimer




  I am not a statistician




                 ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                       Statistics - I    Margin of Error
                       Statistics - II   Central Tendency


Population



             All possible users of your system




               ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                       Statistics - I    Margin of Error
                       Statistics - II   Central Tendency


Sample



         Representative subset of the population




               ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                     Statistics - I    Margin of Error
                     Statistics - II   Central Tendency


Bad sample



                   Sometimes it’s not




             ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                                  Statistics - I    Margin of Error
                                  Statistics - II   Central Tendency


How to randomize?




        Pick 10% of users at random and always test them

                                                OR

        For each user, decide at random if they should be tested
  http://tech.bluesmoon.info/2010/01/statistics-of-performance-measurement.html




                          ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                            Statistics - I    Margin of Error
                            Statistics - II   Central Tendency


Select 10% of users - I



     if($sessionid % 10 === 0) {
        // instrument code for measurement
     }

      Once a user enters the measurement bucket, they stay
      there until they log out
      Fixed set of users, so tests may be more consistent
      Error in the sample results in positive feedback




                    ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                           Statistics - I    Margin of Error
                           Statistics - II   Central Tendency


Select 10% of users - II



     if(rand() < 0.1 * getrandmax()) {
        // instrument code for measurement
     }

      For every request, a user has a 10% chance of being
      tested
      Gets rid of positive feedback errors, but sample size !=
      10% of population




                   ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                       Statistics - I    Margin of Error
                       Statistics - II   Central Tendency


How big a sample is representative?




                      Select n such that
                          σ
                     1.96 √n ≤ 5%µ




               ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                          Statistics - I    Margin of Error
                          Statistics - II   Central Tendency


Standard Deviation

     Standard deviation tells you the spread of the curve
     The narrower the curve, the more confident you can be




                  ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                     Statistics - I    Margin of Error
                     Statistics - II   Central Tendency


MoE at 95% confidence




                                 σ
                           ±1.96 √n




             ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                          Statistics - I    Margin of Error
                          Statistics - II   Central Tendency


MoE & Sample size




   There is an inverse square root correlation between sample
                     size and margin of error




                  ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                      Statistics - I    Margin of Error
                      Statistics - II   Central Tendency




But wait... it’s not complicated enough.
    We have different types of margins of error
    ...more about that later




              ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                      Statistics - I    Margin of Error
                      Statistics - II   Central Tendency




But wait... it’s not complicated enough.
    We have different types of margins of error
    ...more about that later




              ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                      Statistics - I    Margin of Error
                      Statistics - II   Central Tendency




But wait... it’s not complicated enough.
    We have different types of margins of error
    ...more about that later




              ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                    Statistics - I    Margin of Error
                    Statistics - II   Central Tendency


Ding dong




            ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                         Statistics - I    Margin of Error
                         Statistics - II   Central Tendency


One number




    Mean (Arithmetic)
        Good for symmetric curves
        Affected by outliers


          Mean(10, 11, 12, 11, 109) = 30




                 ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                        Statistics - I    Margin of Error
                        Statistics - II   Central Tendency


One number




    Median
       Middle value measures central tendency well
       Not trivial to pull out of a DB


        Median(10, 11, 12, 11, 109) = 11




                ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                        Statistics - I    Margin of Error
                        Statistics - II   Central Tendency


One number




    Mode
       Not often used
       Multi-modal distributions suggest problems


           Mode(10, 11, 12, 11, 109) = 11




                ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                            Statistics - I    Margin of Error
                            Statistics - II   Central Tendency


Other numbers




     A percentile point in the distribution: 95th , 98.5th or 99th
          Used to find out the worst user experience
          Makes more sense if you filter data first


            P95th (10, 11, 12, 11, 109) = 12




                    ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                         Statistics - I    Margin of Error
                         Statistics - II   Central Tendency


Other means




    Geometric mean
       Good if your data is exponential in nature
       (with the tail on the right)


       GMean(10, 11, 12, 11, 109) = 16.68




                 ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                               Statistics - I    Margin of Error
                               Statistics - II   Central Tendency


Wait... how did I get that?




            N
                ΠN xi — could lead to overflow
                 i=1

            ΣN loge (xi )
             i=1
                 N
        e                   — computationally simpler




                       ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                               Statistics - I    Margin of Error
                               Statistics - II   Central Tendency


Wait... how did I get that?




            N
                ΠN xi — could lead to overflow
                 i=1

            ΣN loge (xi )
             i=1
                 N
        e                   — computationally simpler




                       ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                               Statistics - I    Margin of Error
                               Statistics - II   Central Tendency


Wait... how did I get that?




            N
                ΠN xi — could lead to overflow
                 i=1

            ΣN loge (xi )
             i=1
                 N
        e                   — computationally simpler




                       ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                               Statistics - I    Margin of Error
                               Statistics - II   Central Tendency


Wait... how did I get that?




            N
                ΠN xi — could lead to overflow
                 i=1

            ΣN loge (xi )
             i=1
                 N
        e                   — computationally simpler




                       ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                          Statistics - I    Margin of Error
                          Statistics - II   Central Tendency


Other means




   And there is also the Harmonic mean, but forget about that




                  ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                            Statistics - I    Margin of Error
                            Statistics - II   Central Tendency


...though consequently



  We have other margins of error
     Geometric margin of error
          Uses geometric standard deviation
      Median margin of error
          Uses ranges of actual values from data set
      Stick to the arithmetic MoE
      – simpler to calculate, simpler to read and not incorrect




                    ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction      Random Sampling
                            Statistics - I    Margin of Error
                            Statistics - II   Central Tendency


...though consequently



  We have other margins of error
     Geometric margin of error
          Uses geometric standard deviation
      Median margin of error
          Uses ranges of actual values from data set
      Stick to the arithmetic MoE
      – simpler to calculate, simpler to read and not incorrect




                    ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                          Filtering
        Statistics - I
                          The Log-Normal distribution
        Statistics - II




      Statistics - II




ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                     Filtering
                   Statistics - I
                                     The Log-Normal distribution
                   Statistics - II


Outliers




                                             Out of range data points
                                             Nothing you can fix here
                                             There’s even a book about
                                             them




           ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                     Filtering
                   Statistics - I
                                     The Log-Normal distribution
                   Statistics - II


Outliers




                                             Out of range data points
                                             Nothing you can fix here
                                             There’s even a book about
                                             them




           ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                     Filtering
                   Statistics - I
                                     The Log-Normal distribution
                   Statistics - II


Outliers




                                             Out of range data points
                                             Nothing you can fix here
                                             There’s even a book about
                                             them




           ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                     Filtering
                   Statistics - I
                                     The Log-Normal distribution
                   Statistics - II


Outliers




                                             Out of range data points
                                             Nothing you can fix here
                                             There’s even a book about
                                             them




           ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                             Filtering
                           Statistics - I
                                             The Log-Normal distribution
                           Statistics - II


DNS problems can cause outliers




     2 or 3 DNS servers for an ISP
     30 second timeout if first fails
     ... 30 second increase in page load time
     Maybe measure both and fix what you can
     http://nms.lcs.mit.edu/papers/dns-ton2002.pdf




                   ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                         Filtering
                       Statistics - I
                                         The Log-Normal distribution
                       Statistics - II


Band-pass filtering




               ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                            Filtering
                          Statistics - I
                                            The Log-Normal distribution
                          Statistics - II


Band-pass filtering




     Strip everything outside a reasonable range
         Bandwidth range: 4kbps - 4Gbps
         Page load time: 50ms - 120s
     You may need to relook at the ranges all the time




                  ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                         Filtering
                       Statistics - I
                                         The Log-Normal distribution
                       Statistics - II


IQR filtering




               ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                           Filtering
                         Statistics - I
                                           The Log-Normal distribution
                         Statistics - II


IQR filtering




           Here, we derive the range from the data




                 ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                           Filtering
         Statistics - I
                           The Log-Normal distribution
         Statistics - II




Let’s look at some real charts




 ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                          Filtering
                        Statistics - I
                                          The Log-Normal distribution
                        Statistics - II


Bandwidth distribution for web devs




                          x-axis is linear


                ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                          Filtering
                        Statistics - I
                                          The Log-Normal distribution
                        Statistics - II


Now let’s use log(kbps) instead of kbps




                     x-axis is exponential


                ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                            Filtering
                          Statistics - I
                                            The Log-Normal distribution
                          Statistics - II


Exponential == Geometric




     Categories/Buckets grow exponentially
     Data is related geometrically
     Use the geometric mean and geometric margin of error
                           gmean
         Error _range =              /gmoe , gmean ∗ gmoe
     Non-linear ranges are hard for humans to grok




                  ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                            Filtering
                          Statistics - I
                                            The Log-Normal distribution
                          Statistics - II


Exponential == Geometric




     Categories/Buckets grow exponentially
     Data is related geometrically
     Use the geometric mean and geometric margin of error
                           gmean
         Error _range =              /gmoe , gmean ∗ gmoe
     Non-linear ranges are hard for humans to grok




                  ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                            Filtering
                          Statistics - I
                                            The Log-Normal distribution
                          Statistics - II


Exponential == Geometric




     Categories/Buckets grow exponentially
     Data is related geometrically
     Use the geometric mean and geometric margin of error
                           gmean
         Error _range =              /gmoe , gmean ∗ gmoe
     Non-linear ranges are hard for humans to grok




                  ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
        Statistics - I
        Statistics - II




                  So...




ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                                       Statistics - I
                                       Statistics - II


Further reading

     Web Performance - Not a Simple Number
     http://www.netforecast.com/Articles/BCR+C25+Web+Performance+-+Not+A+Simple+Number.pdf

     Revisiting statistics for web performance (introduction to
     Log-Normal)
     http://home.pacbell.net/ciemo/statistics/WhatDoYouMean.pdf

     Random Sampling
     http://tech.bluesmoon.info/2010/01/statistics-of-performance-measurement.html

     Khan Academy’s tutorials on statistics
     http://khanacademy.com/

     Learning about Statistical Learning
     http://measuringmeasures.blogspot.com/2010/01/learning-about-statistical-learning.html

     Wikipedia articles on Random Sampling, Central Tendency,
     Standard Error, Confounding, Means and IQR

                               ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                         Statistics - I
                         Statistics - II


Summary



    Choose a reasonable sample size and sampling factor
    Tune sample size for minimal margin of error
    Decide based on your data whether to use mode, median
    or one of the means
    Figure out whether your data is Normal, Log-Normal or
    something else
    Filter out anomalous outliers




                 ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                             Statistics - I
                             Statistics - II


contact me




     Philip Tellis
     philip@bluesmoon.info
     bluesmoon.info
     @bluesmoon




                     ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                            Statistics - I
                            Statistics - II


Photo credits




     http://www.flickr.com/photos/leoffreitas/332360959/ by leoffreitas
     http://www.flickr.com/photos/cobalt/56500295/ by cobalt123
     http://www.flickr.com/photos/sophistechate/4264466015/ by Lisa
     Brewster
     http://www.flickr.com/photos/nchoz/243216008/ by nchoz




                    ConFoo / 2010-03-12       The Statistics of Web Performance
Introduction
                             Statistics - I
                             Statistics - II


List of figures




      http://en.wikipedia.org/wiki/File:Standard_deviation_diagram.svg
      http://en.wikipedia.org/wiki/File:Normal_Distribution_PDF.svg
      http://en.wikipedia.org/wiki/File:KilroySchematic.svg
      http://en.wikipedia.org/wiki/File:Boxplot_vs_PDF.png




                     ConFoo / 2010-03-12       The Statistics of Web Performance

More Related Content

Similar to The Statistics of Web Performance

Sunz2012 2011-2012 bi trends
Sunz2012   2011-2012 bi trendsSunz2012   2011-2012 bi trends
Sunz2012 2011-2012 bi trends
OptimalBI Limited
 
Meaure Marketing Online - IABC Ottawa
Meaure Marketing Online - IABC OttawaMeaure Marketing Online - IABC Ottawa
Meaure Marketing Online - IABC Ottawa
PublicInsite
 
Mobile Analytics
Mobile AnalyticsMobile Analytics
Mobile Analytics
tchenard
 
C 04-internet of things - horizon watch trend report (client version) 28jan2015
C 04-internet of things - horizon watch trend report (client version) 28jan2015C 04-internet of things - horizon watch trend report (client version) 28jan2015
C 04-internet of things - horizon watch trend report (client version) 28jan2015
Diane Shimmon
 
Attobahn_Reduced_Angel_070915
Attobahn_Reduced_Angel_070915Attobahn_Reduced_Angel_070915
Attobahn_Reduced_Angel_070915
Darryl Gray
 
Innovate 2013 session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3Innovate 2013   session 1243 mobile testing.v3
Innovate 2013 session 1243 mobile testing.v3
Leigh Williamson
 
Query at Speed of Thought
Query at Speed of ThoughtQuery at Speed of Thought
Query at Speed of Thought
MISNet - Integeo SE Asia
 
MeasureWorks eFinancials - Best practices for a successfull mobile experienc...
MeasureWorks eFinancials  - Best practices for a successfull mobile experienc...MeasureWorks eFinancials  - Best practices for a successfull mobile experienc...
MeasureWorks eFinancials - Best practices for a successfull mobile experienc...
MeasureWorks
 
IOT - The 3rd Internet Tsunami is Here
IOT - The 3rd Internet Tsunami is HereIOT - The 3rd Internet Tsunami is Here
IOT - The 3rd Internet Tsunami is Here
Dr. Mazlan Abbas
 
Cisco Video Data Explosion
Cisco Video Data ExplosionCisco Video Data Explosion
Cisco Video Data Explosion
menkento
 
Nosql Now 2015
Nosql Now 2015Nosql Now 2015
Nosql Now 2015
kaiyzen
 
Wireless Network Optimization (2010)
Wireless Network Optimization (2010)Wireless Network Optimization (2010)
Wireless Network Optimization (2010)
Marc Jadoul
 
Forrester wave business intelligence platforms
Forrester wave business intelligence platformsForrester wave business intelligence platforms
Forrester wave business intelligence platforms
divjeev
 
Forrester - Business Intelligence
Forrester - Business IntelligenceForrester - Business Intelligence
Forrester - Business Intelligence
None
 
The Forrester Wave: Enterprise Business Intelligence Platforms, Q4 2010
The Forrester Wave: Enterprise Business Intelligence Platforms, Q4 2010The Forrester Wave: Enterprise Business Intelligence Platforms, Q4 2010
The Forrester Wave: Enterprise Business Intelligence Platforms, Q4 2010
telcobiaas
 
G 2 Industry Analysis
G 2 Industry AnalysisG 2 Industry Analysis
G 2 Industry Analysis
RAJEEV RANJAN
 
Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...
Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...
Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...
Computaris
 
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
Bitbar
 
Data Pioneers - Roland Haeve (Atos Nederland) - Big data in organisaties
Data Pioneers - Roland Haeve (Atos Nederland) - Big data in organisatiesData Pioneers - Roland Haeve (Atos Nederland) - Big data in organisaties
Data Pioneers - Roland Haeve (Atos Nederland) - Big data in organisaties
Multiscope
 
The IoT Food Chain – Picking the Right Dining Partner is Important with Dean ...
The IoT Food Chain – Picking the Right Dining Partner is Important with Dean ...The IoT Food Chain – Picking the Right Dining Partner is Important with Dean ...
The IoT Food Chain – Picking the Right Dining Partner is Important with Dean ...
gogo6
 

Similar to The Statistics of Web Performance (20)

Sunz2012 2011-2012 bi trends
Sunz2012   2011-2012 bi trendsSunz2012   2011-2012 bi trends
Sunz2012 2011-2012 bi trends
 
Meaure Marketing Online - IABC Ottawa
Meaure Marketing Online - IABC OttawaMeaure Marketing Online - IABC Ottawa
Meaure Marketing Online - IABC Ottawa
 
Mobile Analytics
Mobile AnalyticsMobile Analytics
Mobile Analytics
 
C 04-internet of things - horizon watch trend report (client version) 28jan2015
C 04-internet of things - horizon watch trend report (client version) 28jan2015C 04-internet of things - horizon watch trend report (client version) 28jan2015
C 04-internet of things - horizon watch trend report (client version) 28jan2015
 
Attobahn_Reduced_Angel_070915
Attobahn_Reduced_Angel_070915Attobahn_Reduced_Angel_070915
Attobahn_Reduced_Angel_070915
 
Innovate 2013 session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3Innovate 2013   session 1243 mobile testing.v3
Innovate 2013 session 1243 mobile testing.v3
 
Query at Speed of Thought
Query at Speed of ThoughtQuery at Speed of Thought
Query at Speed of Thought
 
MeasureWorks eFinancials - Best practices for a successfull mobile experienc...
MeasureWorks eFinancials  - Best practices for a successfull mobile experienc...MeasureWorks eFinancials  - Best practices for a successfull mobile experienc...
MeasureWorks eFinancials - Best practices for a successfull mobile experienc...
 
IOT - The 3rd Internet Tsunami is Here
IOT - The 3rd Internet Tsunami is HereIOT - The 3rd Internet Tsunami is Here
IOT - The 3rd Internet Tsunami is Here
 
Cisco Video Data Explosion
Cisco Video Data ExplosionCisco Video Data Explosion
Cisco Video Data Explosion
 
Nosql Now 2015
Nosql Now 2015Nosql Now 2015
Nosql Now 2015
 
Wireless Network Optimization (2010)
Wireless Network Optimization (2010)Wireless Network Optimization (2010)
Wireless Network Optimization (2010)
 
Forrester wave business intelligence platforms
Forrester wave business intelligence platformsForrester wave business intelligence platforms
Forrester wave business intelligence platforms
 
Forrester - Business Intelligence
Forrester - Business IntelligenceForrester - Business Intelligence
Forrester - Business Intelligence
 
The Forrester Wave: Enterprise Business Intelligence Platforms, Q4 2010
The Forrester Wave: Enterprise Business Intelligence Platforms, Q4 2010The Forrester Wave: Enterprise Business Intelligence Platforms, Q4 2010
The Forrester Wave: Enterprise Business Intelligence Platforms, Q4 2010
 
G 2 Industry Analysis
G 2 Industry AnalysisG 2 Industry Analysis
G 2 Industry Analysis
 
Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...
Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...
Sandvine Webinar – Making Cents of Internet Phenomena Through Network Busines...
 
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
 
Data Pioneers - Roland Haeve (Atos Nederland) - Big data in organisaties
Data Pioneers - Roland Haeve (Atos Nederland) - Big data in organisatiesData Pioneers - Roland Haeve (Atos Nederland) - Big data in organisaties
Data Pioneers - Roland Haeve (Atos Nederland) - Big data in organisaties
 
The IoT Food Chain – Picking the Right Dining Partner is Important with Dean ...
The IoT Food Chain – Picking the Right Dining Partner is Important with Dean ...The IoT Food Chain – Picking the Right Dining Partner is Important with Dean ...
The IoT Food Chain – Picking the Right Dining Partner is Important with Dean ...
 

More from Philip Tellis

Improving D3 Performance with CANVAS and other Hacks
Improving D3 Performance with CANVAS and other HacksImproving D3 Performance with CANVAS and other Hacks
Improving D3 Performance with CANVAS and other Hacks
Philip Tellis
 
Frontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy PersonFrontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy Person
Philip Tellis
 
Frontend Performance: De débutant à Expert à Fou Furieux
Frontend Performance: De débutant à Expert à Fou FurieuxFrontend Performance: De débutant à Expert à Fou Furieux
Frontend Performance: De débutant à Expert à Fou Furieux
Philip Tellis
 
Frontend Performance: Expert to Crazy Person
Frontend Performance: Expert to Crazy PersonFrontend Performance: Expert to Crazy Person
Frontend Performance: Expert to Crazy Person
Philip Tellis
 
Beyond Page Level Metrics
Beyond Page Level MetricsBeyond Page Level Metrics
Beyond Page Level Metrics
Philip Tellis
 
Frontend Performance: Beginner to Expert to Crazy Person (San Diego Web Perf ...
Frontend Performance: Beginner to Expert to Crazy Person (San Diego Web Perf ...Frontend Performance: Beginner to Expert to Crazy Person (San Diego Web Perf ...
Frontend Performance: Beginner to Expert to Crazy Person (San Diego Web Perf ...
Philip Tellis
 
Frontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy PersonFrontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy Person
Philip Tellis
 
Frontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy PersonFrontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy Person
Philip Tellis
 
Frontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy PersonFrontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy Person
Philip Tellis
 
mmm... beacons
mmm... beaconsmmm... beacons
mmm... beacons
Philip Tellis
 
RUM Distillation 101 -- Part I
RUM Distillation 101 -- Part IRUM Distillation 101 -- Part I
RUM Distillation 101 -- Part I
Philip Tellis
 
Improving 3rd Party Script Performance With IFrames
Improving 3rd Party Script Performance With IFramesImproving 3rd Party Script Performance With IFrames
Improving 3rd Party Script Performance With IFrames
Philip Tellis
 
Extending Boomerang
Extending BoomerangExtending Boomerang
Extending Boomerang
Philip Tellis
 
Abusing JavaScript to measure Web Performance, or, "how does boomerang work?"
Abusing JavaScript to measure Web Performance, or, "how does boomerang work?"Abusing JavaScript to measure Web Performance, or, "how does boomerang work?"
Abusing JavaScript to measure Web Performance, or, "how does boomerang work?"
Philip Tellis
 
Abusing JavaScript to Measure Web Performance
Abusing JavaScript to Measure Web PerformanceAbusing JavaScript to Measure Web Performance
Abusing JavaScript to Measure Web Performance
Philip Tellis
 
Rum for Breakfast
Rum for BreakfastRum for Breakfast
Rum for Breakfast
Philip Tellis
 
Analysing network characteristics with JavaScript
Analysing network characteristics with JavaScriptAnalysing network characteristics with JavaScript
Analysing network characteristics with JavaScript
Philip Tellis
 
A Node.JS bag of goodies for analyzing Web Traffic
A Node.JS bag of goodies for analyzing Web TrafficA Node.JS bag of goodies for analyzing Web Traffic
A Node.JS bag of goodies for analyzing Web Traffic
Philip Tellis
 
Input sanitization
Input sanitizationInput sanitization
Input sanitization
Philip Tellis
 
Messing with JavaScript and the DOM to measure network characteristics
Messing with JavaScript and the DOM to measure network characteristicsMessing with JavaScript and the DOM to measure network characteristics
Messing with JavaScript and the DOM to measure network characteristics
Philip Tellis
 

More from Philip Tellis (20)

Improving D3 Performance with CANVAS and other Hacks
Improving D3 Performance with CANVAS and other HacksImproving D3 Performance with CANVAS and other Hacks
Improving D3 Performance with CANVAS and other Hacks
 
Frontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy PersonFrontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy Person
 
Frontend Performance: De débutant à Expert à Fou Furieux
Frontend Performance: De débutant à Expert à Fou FurieuxFrontend Performance: De débutant à Expert à Fou Furieux
Frontend Performance: De débutant à Expert à Fou Furieux
 
Frontend Performance: Expert to Crazy Person
Frontend Performance: Expert to Crazy PersonFrontend Performance: Expert to Crazy Person
Frontend Performance: Expert to Crazy Person
 
Beyond Page Level Metrics
Beyond Page Level MetricsBeyond Page Level Metrics
Beyond Page Level Metrics
 
Frontend Performance: Beginner to Expert to Crazy Person (San Diego Web Perf ...
Frontend Performance: Beginner to Expert to Crazy Person (San Diego Web Perf ...Frontend Performance: Beginner to Expert to Crazy Person (San Diego Web Perf ...
Frontend Performance: Beginner to Expert to Crazy Person (San Diego Web Perf ...
 
Frontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy PersonFrontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy Person
 
Frontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy PersonFrontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy Person
 
Frontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy PersonFrontend Performance: Beginner to Expert to Crazy Person
Frontend Performance: Beginner to Expert to Crazy Person
 
mmm... beacons
mmm... beaconsmmm... beacons
mmm... beacons
 
RUM Distillation 101 -- Part I
RUM Distillation 101 -- Part IRUM Distillation 101 -- Part I
RUM Distillation 101 -- Part I
 
Improving 3rd Party Script Performance With IFrames
Improving 3rd Party Script Performance With IFramesImproving 3rd Party Script Performance With IFrames
Improving 3rd Party Script Performance With IFrames
 
Extending Boomerang
Extending BoomerangExtending Boomerang
Extending Boomerang
 
Abusing JavaScript to measure Web Performance, or, "how does boomerang work?"
Abusing JavaScript to measure Web Performance, or, "how does boomerang work?"Abusing JavaScript to measure Web Performance, or, "how does boomerang work?"
Abusing JavaScript to measure Web Performance, or, "how does boomerang work?"
 
Abusing JavaScript to Measure Web Performance
Abusing JavaScript to Measure Web PerformanceAbusing JavaScript to Measure Web Performance
Abusing JavaScript to Measure Web Performance
 
Rum for Breakfast
Rum for BreakfastRum for Breakfast
Rum for Breakfast
 
Analysing network characteristics with JavaScript
Analysing network characteristics with JavaScriptAnalysing network characteristics with JavaScript
Analysing network characteristics with JavaScript
 
A Node.JS bag of goodies for analyzing Web Traffic
A Node.JS bag of goodies for analyzing Web TrafficA Node.JS bag of goodies for analyzing Web Traffic
A Node.JS bag of goodies for analyzing Web Traffic
 
Input sanitization
Input sanitizationInput sanitization
Input sanitization
 
Messing with JavaScript and the DOM to measure network characteristics
Messing with JavaScript and the DOM to measure network characteristicsMessing with JavaScript and the DOM to measure network characteristics
Messing with JavaScript and the DOM to measure network characteristics
 

Recently uploaded

Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
BibashShahi
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Precisely
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Alpen-Adria-Universität
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
Neo4j
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
saastr
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Pitangent Analytics & Technology Solutions Pvt. Ltd
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 

Recently uploaded (20)

Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 

The Statistics of Web Performance

  • 1. Introduction Statistics - I Statistics - II The Statistics of Web Performance Philip Tellis / philip@bluesmoon.info ConFoo / 2010-03-12 ConFoo / 2010-03-12 The Statistics of Web Performance
  • 2. Introduction Statistics - I Statistics - II $ finger philip Philip Tellis philip@bluesmoon.info @bluesmoon yahoo geek ConFoo / 2010-03-12 The Statistics of Web Performance
  • 3. Introduction The goal Statistics - I Performance Measurement Statistics - II Introduction ConFoo / 2010-03-12 The Statistics of Web Performance
  • 4. Introduction The goal Statistics - I Performance Measurement Statistics - II Accurately measure page performance At least, as accurately as possible ConFoo / 2010-03-12 The Statistics of Web Performance
  • 5. Introduction The goal Statistics - I Performance Measurement Statistics - II Accurately measure page performance At least, as accurately as possible ConFoo / 2010-03-12 The Statistics of Web Performance
  • 6. Introduction The goal Statistics - I Performance Measurement Statistics - II Be unintrusive If you try to measure something accurately, you will change something related – Heisenberg’s uncertainty principle ConFoo / 2010-03-12 The Statistics of Web Performance
  • 7. Introduction The goal Statistics - I Performance Measurement Statistics - II And one number to rule them all ConFoo / 2010-03-12 The Statistics of Web Performance
  • 8. Introduction The goal Statistics - I Performance Measurement Statistics - II Bandwidth Real bandwidth v/s advertised bandwidth Bandwidth to your server, not to the ISP Bandwidth during normal internet usage If the user’s always watching movies, you’re not winning ConFoo / 2010-03-12 The Statistics of Web Performance
  • 9. Introduction The goal Statistics - I Performance Measurement Statistics - II Bandwidth Real bandwidth v/s advertised bandwidth Bandwidth to your server, not to the ISP Bandwidth during normal internet usage If the user’s always watching movies, you’re not winning ConFoo / 2010-03-12 The Statistics of Web Performance
  • 10. Introduction The goal Statistics - I Performance Measurement Statistics - II Latency How long does it take a byte to get to the user? Wired, wireless, mobile, satellite? How many hops in between? Speed of light is constant This is not a battle we will soon win. When was the last time you heard latency mentioned in a TV ad? http://www.stuartcheshire.org/rants/Latency.html ConFoo / 2010-03-12 The Statistics of Web Performance
  • 11. Introduction The goal Statistics - I Performance Measurement Statistics - II Latency How long does it take a byte to get to the user? Wired, wireless, mobile, satellite? How many hops in between? Speed of light is constant This is not a battle we will soon win. When was the last time you heard latency mentioned in a TV ad? http://www.stuartcheshire.org/rants/Latency.html ConFoo / 2010-03-12 The Statistics of Web Performance
  • 12. Introduction The goal Statistics - I Performance Measurement Statistics - II Latency How long does it take a byte to get to the user? Wired, wireless, mobile, satellite? How many hops in between? Speed of light is constant This is not a battle we will soon win. When was the last time you heard latency mentioned in a TV ad? http://www.stuartcheshire.org/rants/Latency.html ConFoo / 2010-03-12 The Statistics of Web Performance
  • 13. Introduction The goal Statistics - I Performance Measurement Statistics - II User perceived page load time Time from “click on a link” to “spinner stops spinning” This is what users notice Depends on how long your page takes to build Depends on what’s in your page Depends on how long components take to load Depends on how long the browser takes to execute and render ConFoo / 2010-03-12 The Statistics of Web Performance
  • 14. Introduction The goal Statistics - I Performance Measurement Statistics - II We need to measure real user data ConFoo / 2010-03-12 The Statistics of Web Performance
  • 15. Introduction The goal Statistics - I Performance Measurement Statistics - II The statistics apply to any kind of performance data though ConFoo / 2010-03-12 The Statistics of Web Performance
  • 16. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Statistics - I ConFoo / 2010-03-12 The Statistics of Web Performance
  • 17. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Disclaimer I am not a statistician ConFoo / 2010-03-12 The Statistics of Web Performance
  • 18. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Population All possible users of your system ConFoo / 2010-03-12 The Statistics of Web Performance
  • 19. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Sample Representative subset of the population ConFoo / 2010-03-12 The Statistics of Web Performance
  • 20. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Bad sample Sometimes it’s not ConFoo / 2010-03-12 The Statistics of Web Performance
  • 21. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency How to randomize? Pick 10% of users at random and always test them OR For each user, decide at random if they should be tested http://tech.bluesmoon.info/2010/01/statistics-of-performance-measurement.html ConFoo / 2010-03-12 The Statistics of Web Performance
  • 22. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Select 10% of users - I if($sessionid % 10 === 0) { // instrument code for measurement } Once a user enters the measurement bucket, they stay there until they log out Fixed set of users, so tests may be more consistent Error in the sample results in positive feedback ConFoo / 2010-03-12 The Statistics of Web Performance
  • 23. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Select 10% of users - II if(rand() < 0.1 * getrandmax()) { // instrument code for measurement } For every request, a user has a 10% chance of being tested Gets rid of positive feedback errors, but sample size != 10% of population ConFoo / 2010-03-12 The Statistics of Web Performance
  • 24. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency How big a sample is representative? Select n such that σ 1.96 √n ≤ 5%µ ConFoo / 2010-03-12 The Statistics of Web Performance
  • 25. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Standard Deviation Standard deviation tells you the spread of the curve The narrower the curve, the more confident you can be ConFoo / 2010-03-12 The Statistics of Web Performance
  • 26. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency MoE at 95% confidence σ ±1.96 √n ConFoo / 2010-03-12 The Statistics of Web Performance
  • 27. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency MoE & Sample size There is an inverse square root correlation between sample size and margin of error ConFoo / 2010-03-12 The Statistics of Web Performance
  • 28. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency But wait... it’s not complicated enough. We have different types of margins of error ...more about that later ConFoo / 2010-03-12 The Statistics of Web Performance
  • 29. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency But wait... it’s not complicated enough. We have different types of margins of error ...more about that later ConFoo / 2010-03-12 The Statistics of Web Performance
  • 30. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency But wait... it’s not complicated enough. We have different types of margins of error ...more about that later ConFoo / 2010-03-12 The Statistics of Web Performance
  • 31. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Ding dong ConFoo / 2010-03-12 The Statistics of Web Performance
  • 32. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency One number Mean (Arithmetic) Good for symmetric curves Affected by outliers Mean(10, 11, 12, 11, 109) = 30 ConFoo / 2010-03-12 The Statistics of Web Performance
  • 33. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency One number Median Middle value measures central tendency well Not trivial to pull out of a DB Median(10, 11, 12, 11, 109) = 11 ConFoo / 2010-03-12 The Statistics of Web Performance
  • 34. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency One number Mode Not often used Multi-modal distributions suggest problems Mode(10, 11, 12, 11, 109) = 11 ConFoo / 2010-03-12 The Statistics of Web Performance
  • 35. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Other numbers A percentile point in the distribution: 95th , 98.5th or 99th Used to find out the worst user experience Makes more sense if you filter data first P95th (10, 11, 12, 11, 109) = 12 ConFoo / 2010-03-12 The Statistics of Web Performance
  • 36. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Other means Geometric mean Good if your data is exponential in nature (with the tail on the right) GMean(10, 11, 12, 11, 109) = 16.68 ConFoo / 2010-03-12 The Statistics of Web Performance
  • 37. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Wait... how did I get that? N ΠN xi — could lead to overflow i=1 ΣN loge (xi ) i=1 N e — computationally simpler ConFoo / 2010-03-12 The Statistics of Web Performance
  • 38. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Wait... how did I get that? N ΠN xi — could lead to overflow i=1 ΣN loge (xi ) i=1 N e — computationally simpler ConFoo / 2010-03-12 The Statistics of Web Performance
  • 39. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Wait... how did I get that? N ΠN xi — could lead to overflow i=1 ΣN loge (xi ) i=1 N e — computationally simpler ConFoo / 2010-03-12 The Statistics of Web Performance
  • 40. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Wait... how did I get that? N ΠN xi — could lead to overflow i=1 ΣN loge (xi ) i=1 N e — computationally simpler ConFoo / 2010-03-12 The Statistics of Web Performance
  • 41. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency Other means And there is also the Harmonic mean, but forget about that ConFoo / 2010-03-12 The Statistics of Web Performance
  • 42. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency ...though consequently We have other margins of error Geometric margin of error Uses geometric standard deviation Median margin of error Uses ranges of actual values from data set Stick to the arithmetic MoE – simpler to calculate, simpler to read and not incorrect ConFoo / 2010-03-12 The Statistics of Web Performance
  • 43. Introduction Random Sampling Statistics - I Margin of Error Statistics - II Central Tendency ...though consequently We have other margins of error Geometric margin of error Uses geometric standard deviation Median margin of error Uses ranges of actual values from data set Stick to the arithmetic MoE – simpler to calculate, simpler to read and not incorrect ConFoo / 2010-03-12 The Statistics of Web Performance
  • 44. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II Statistics - II ConFoo / 2010-03-12 The Statistics of Web Performance
  • 45. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II Outliers Out of range data points Nothing you can fix here There’s even a book about them ConFoo / 2010-03-12 The Statistics of Web Performance
  • 46. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II Outliers Out of range data points Nothing you can fix here There’s even a book about them ConFoo / 2010-03-12 The Statistics of Web Performance
  • 47. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II Outliers Out of range data points Nothing you can fix here There’s even a book about them ConFoo / 2010-03-12 The Statistics of Web Performance
  • 48. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II Outliers Out of range data points Nothing you can fix here There’s even a book about them ConFoo / 2010-03-12 The Statistics of Web Performance
  • 49. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II DNS problems can cause outliers 2 or 3 DNS servers for an ISP 30 second timeout if first fails ... 30 second increase in page load time Maybe measure both and fix what you can http://nms.lcs.mit.edu/papers/dns-ton2002.pdf ConFoo / 2010-03-12 The Statistics of Web Performance
  • 50. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II Band-pass filtering ConFoo / 2010-03-12 The Statistics of Web Performance
  • 51. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II Band-pass filtering Strip everything outside a reasonable range Bandwidth range: 4kbps - 4Gbps Page load time: 50ms - 120s You may need to relook at the ranges all the time ConFoo / 2010-03-12 The Statistics of Web Performance
  • 52. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II IQR filtering ConFoo / 2010-03-12 The Statistics of Web Performance
  • 53. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II IQR filtering Here, we derive the range from the data ConFoo / 2010-03-12 The Statistics of Web Performance
  • 54. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II Let’s look at some real charts ConFoo / 2010-03-12 The Statistics of Web Performance
  • 55. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II Bandwidth distribution for web devs x-axis is linear ConFoo / 2010-03-12 The Statistics of Web Performance
  • 56. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II Now let’s use log(kbps) instead of kbps x-axis is exponential ConFoo / 2010-03-12 The Statistics of Web Performance
  • 57. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II Exponential == Geometric Categories/Buckets grow exponentially Data is related geometrically Use the geometric mean and geometric margin of error gmean Error _range = /gmoe , gmean ∗ gmoe Non-linear ranges are hard for humans to grok ConFoo / 2010-03-12 The Statistics of Web Performance
  • 58. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II Exponential == Geometric Categories/Buckets grow exponentially Data is related geometrically Use the geometric mean and geometric margin of error gmean Error _range = /gmoe , gmean ∗ gmoe Non-linear ranges are hard for humans to grok ConFoo / 2010-03-12 The Statistics of Web Performance
  • 59. Introduction Filtering Statistics - I The Log-Normal distribution Statistics - II Exponential == Geometric Categories/Buckets grow exponentially Data is related geometrically Use the geometric mean and geometric margin of error gmean Error _range = /gmoe , gmean ∗ gmoe Non-linear ranges are hard for humans to grok ConFoo / 2010-03-12 The Statistics of Web Performance
  • 60. Introduction Statistics - I Statistics - II So... ConFoo / 2010-03-12 The Statistics of Web Performance
  • 61. Introduction Statistics - I Statistics - II Further reading Web Performance - Not a Simple Number http://www.netforecast.com/Articles/BCR+C25+Web+Performance+-+Not+A+Simple+Number.pdf Revisiting statistics for web performance (introduction to Log-Normal) http://home.pacbell.net/ciemo/statistics/WhatDoYouMean.pdf Random Sampling http://tech.bluesmoon.info/2010/01/statistics-of-performance-measurement.html Khan Academy’s tutorials on statistics http://khanacademy.com/ Learning about Statistical Learning http://measuringmeasures.blogspot.com/2010/01/learning-about-statistical-learning.html Wikipedia articles on Random Sampling, Central Tendency, Standard Error, Confounding, Means and IQR ConFoo / 2010-03-12 The Statistics of Web Performance
  • 62. Introduction Statistics - I Statistics - II Summary Choose a reasonable sample size and sampling factor Tune sample size for minimal margin of error Decide based on your data whether to use mode, median or one of the means Figure out whether your data is Normal, Log-Normal or something else Filter out anomalous outliers ConFoo / 2010-03-12 The Statistics of Web Performance
  • 63. Introduction Statistics - I Statistics - II contact me Philip Tellis philip@bluesmoon.info bluesmoon.info @bluesmoon ConFoo / 2010-03-12 The Statistics of Web Performance
  • 64. Introduction Statistics - I Statistics - II Photo credits http://www.flickr.com/photos/leoffreitas/332360959/ by leoffreitas http://www.flickr.com/photos/cobalt/56500295/ by cobalt123 http://www.flickr.com/photos/sophistechate/4264466015/ by Lisa Brewster http://www.flickr.com/photos/nchoz/243216008/ by nchoz ConFoo / 2010-03-12 The Statistics of Web Performance
  • 65. Introduction Statistics - I Statistics - II List of figures http://en.wikipedia.org/wiki/File:Standard_deviation_diagram.svg http://en.wikipedia.org/wiki/File:Normal_Distribution_PDF.svg http://en.wikipedia.org/wiki/File:KilroySchematic.svg http://en.wikipedia.org/wiki/File:Boxplot_vs_PDF.png ConFoo / 2010-03-12 The Statistics of Web Performance