The document discusses heavy-tailed distributions and their prevalence in computer networking. It begins with definitions of key concepts like outliers, heavy-tailed distributions, and how these distributions violate assumptions of traditional statistical analysis. Examples are given of heavy-tailedness in areas like web objects, video systems, and peer-to-peer networks. Specific distributions like Pareto and Weibull are mentioned as fitting networking metrics well. The document emphasizes that extreme observations are common in networks and should not be discarded without careful analysis.

1. The Tale of Heavy Tails in Computer Networking
Stenio Fernandes
CIn/UFPE, Recife, Brazil
Carleton University - ARS Lab – May 2016

2. Outline
 Essential Concepts and Terminology
| The heavy-tail phenomenon
| Outliers detection
| Heavy-tailed distributions and its variations (subclasses)
 Evidences of Heavy-Tailedness in Computer Networks
| Examples
2

3. Essential Concepts and Terminology
3

4. The heavy-tail phenomenon
4
• Heavy-tailedness in computer networking is like Ninjas,
they’re everywhere Internet meme
• Extreme observations must be taken carefully and very
seriously
• Dataset that exhibits very large observation values
makes descriptive and inferential statistical analysis
much more difficult
• It might not make sense to use traditional statistical
techniques and tools in these cases
• Some important initial questions:
• Are we confident to discard single, scattered, or
burst of observations that presents extreme values
due to uncontrolled factors?
• Are the extreme values come from valid
measurements?

5. Statistical black sheep
 This is what I call Statistical Black Sheep
| the ones that causes shame or embarrassment because of deviation from the accepted
standards of his or her group (Black Sheep definition on M-W)
 You can either keep or discard such measurement values
based on subjective analysis
| It is out of scope of your interest
• Ex.: mean value of cat videos length on YouTube
 Take-home lesson:
| do not disgrace the black sheep without proper reasons
 Decision can also be made based on rigorous statistical
analysis
| a quantitative analysis
| Recall that an outlier might be influential on regression modeling (more on that later) 5

6. It starts with outliers
Here is an Outlier
6
 Here is another one!

7. Outliers
 An observation can be considered as outlier if it falls below or
above certain limits
| detection is only an indication that you might want to think carefully about them
 There are a number of formal tests and rules of thumbs to detect
outliers in an observation variable
1. Grubbs’
2. Tietjen-Moore’s
3. Mahalanobis distance
4. Extreme Value Theory (EVT)
5. Generalized Extreme Studentized Deviate (ESD)
 Try to not be so picky when choosing the method
| simply because outlier detection and handling is an art
| a subjective approach plays an important role to accommodate outliers in your analysis 7

8. Outliers in Regression Models
8

9. Outliers
 Kurtosis: concrete idea
about the expected
number of outliers
| High (strong skewness) or low
(weak skewness)
 A general approach for
outlier detection
| identify values apart from the
central values( in terms of 𝜎)
| A common and simple approach
• define the fences as 𝜇 ± 3 × 𝜎
• 𝜇 is the sample mean
• 𝜎 is the sample standard deviation
(more conservative: use 4 𝜎) 9

10. Outliers and Heavy-Tails
 Verify if there are lots of observations outside the fences
| This might be indicating that the underlying phenomenon generates heavy-tailed data
• Your black sheep metamorphoses into a black swan
 If extreme values come from distributions with heavy tails
• Weibull, Gamma, Pareto
| Such events are not so rare
| They are likely to be part of the underlying phenomenon
 If you decided to keep the outliers
| You recognized them as part of the underlying data generation process
| You need to address them properly
 Why do we need to use other statistical measures when dealing
with heavy-tailed distributions?
10

11. Moments from Heavy-Tails Distributions
11

12. More on Heavy-Tails
 Classification
| Light or Thin tail
| Fat or Heavy tail
| Long tail
 Light and thin tailed distributions are always used as references
| Normal and Exponential distributions
| Definition: A probability distribution that has an exponentially decaying complementary CDF
 Heavy-tailed distributions are the general ones
| most formal analysis of heavy tailed distributions indeed deals with right heavy tailed
distributions with [0, ∞] support
| the term fat tail is not well accepted by the traditional (and more formal) communities of
statisticians and mathematicians, although is widely used in the finance one
12

13. Some Formal Stuff
 Some intuition behind the concepts
| Power-Law is a relation between two variables in a 𝑝 𝑥 ∝ 𝑐𝑓(𝑥) form, where 𝑓 𝑥 takes a
general form of 𝑥−𝛼
| There are dozens of power-law distributions
• Zipf and Pareto are the most well-known ones in the computer networking field
| They have interesting mathematical properties, such as the tails fall asymptotically according
to the power parameter
 The Pareto distribution became famous due to its capability to
fit in and model well real-world related problems
| The Pareto rule (or principle), aka the 80/20 rule, has been used to exemplify clearly that
phenomena of all sorts are running far from the Normal distribution.
• It is clear that the normal is not being Normal!
13

14. Some Formalities
 A non-negative random variable X, either continuous or discrete,
can be considered a Power-Law distribution if it follows
| 𝑃[𝑋 ≥ 𝑥] ∼ 𝑐𝑥−𝛼
| where c and 𝛼 are the constant parameters that characterize the distribution.
• 𝛼 is known as the scale parameter. Both constants are positive.
 Heavy-tailedness
| The tail of a function 𝐹(𝑥, ∞) is denoted by 𝐹 = 𝑃(𝑋 > 𝑥), where F is the distribution function
of a random variable X.
• F is (right) heavy-tailed if 𝐸 𝑒 𝜆𝑋
= ∞, for all λ > 0.
• The distribution is light-tailed when 𝐸 𝑒 𝜆𝑋
< ∞.
14

15. Some Formalities
 Long-tailedness
 𝐹 (the survival function) is long-tailed when
| lim
𝑥→∞
𝐹(𝑥+𝜆)
𝐹(𝑥)
= 1, 𝑓𝑜𝑟 𝑎𝑙𝑙 𝜆 > 0
• 𝐹 is a non-increasing function, so it converges to 1.
| Considering that the tail of 𝐹 has a polynomial decay rate −𝛼 (i.e., the tail index), its 𝑘 𝑡ℎ
moments are infinite for all 𝑘 > 𝛼.
 The Pareto case
| One interesting property of a Power-Law distribution is that if you take the logarithmic scale
plot (i.e., log-log) of the CCDF – in a rank plot - it should present a straight line
| Its density function is given by:
• 𝑝 𝑥 = 𝛼𝑘 𝛼 𝑥−𝛼−1
15

16. Some Formalities
 Pareto shows interesting features
| If 𝛼 ≤ 1, there is no first moment, i.e., its mean is infinite.
| In the case of if 0 < 𝛼 ≤ 2, its variance is also infinite (heavy-tail).
| A Pareto PDF is scale free
• In computer networking problems, it can capture self-similar behavior (aka fractal) in several layers
of the protocol stack
 A log-log view of the Pareto PDF reveals, as expected, a straight
line, as follows:
| ln 𝑝(𝑥) = −𝛼 − 1 ln 𝑥 + 𝛼 ln 𝑘 + ln 𝛼
 The second and third terms of the equation are constants.
| The relation between ln 𝑝(𝑥) and ln 𝑥 is linear, where −𝛼 − 1 is its slope.
• A simple approach for identifying the scale parameter is by means of linear regression.
16

17. 17

18. Take-home lesson
 The fact is that some universal statistical practices and theories
do not hold if the data follows a heavy-tailed distribution
| The Law of Large Numbers (LLN) and the Central Limit Theorem (CLT) do not hold when
dealing with heavy tailed distributions.
• This is due to the fact that their first or second moments are not finite, which is the fundamental
assumption that supports both LLN and CLT.
18

19. Evidences of Heavy-Tailedness in Computer Networks
19

20. Evidences of Heavy-tailedness
 Extreme events in nature occurs in both micro and macro scales
 a number of case studies and evidences of the occurrence of
extreme events
| nature (e.g., earthquakes, landslides, floods, droughts, storms)
| human-induced catastrophes (spills, nuclear accidents, dam ruptures, power outages)
| financial (e.g., wealth distribution. When the 0.1% richer has 50% of the world’s wealth)
| geo- and socio-political area (e.g., human fatalities in wars)
| online social network phenomenon (e.g., tweets like “the naked celebrity pics leak cracks
down the Internet”), which is known to causes spikes in traffic from time to time
 Extreme events in computer networking have been studied (by
measurements, modelling, and analyses) for decades
| Unfortunately, a number of network engineers and researchers still do not take such
phenomena carefully 20

21. Some Examples
 Power-Law distributions in Internet measurements
| web objects have a tight relation with long tails
• Images, Texts, Video, Embedded code
| modelling issues and implications for network planning and design (e.g., web caching architectures)
• Question like “What is the average size of web objects in the Internet?” should not be answered by calculating
the mean value!
 Recent Studies in mobile environments
| typical performance metrics follows heavy-tailed distributions
• main object sizes
• embedded object size
• number of embedded objects in one request
• embedded object inter-arrival time
• session duration
• interval between two consecutive requests (aka the reading time)
21

22. Some Examples
 Video Systems
| YouTube: The number views can be modelled well by Zipf, Weibull, or Gamma distributions
• Zipf-like distributions fit well this popularity metric in mobile environments
 Intriguing cases of heavy tailedness in the Internet are in the network layer
| strong evidences of heavy tailedness for the sampled IP addressed
| distributions of IP packets per aggregation are all following a Power-law distribution
• the number of packets per flow, unique address, or IP prefixes
 Internet connectivity at several levels of aggregation can be modeled with
heavy tail distributions
 P2P Systems
| video popularity
| session duration
| churn of peers
• user arrival and departure at/from the overlay network
| Different studies have reported different distributions (just be careful with the choice)
22

23. The Tale of Heavy Tails in Computer Networking
Stenio Fernandes
CIn/UFPE, Recife, Brazil
Carleton University - ARS Lab – May 2016