This document discusses using Erlang's queuing models to size systems for concurrency and availability. It defines key terms like utilization, level of service, throughput, and traffic. It explains that utilization alone is not a good metric and that Erlang's models relate blocking probability and resource needs. The document provides examples of using Erlang B and C models to successfully size systems in various industries and discusses assumptions and limitations of the models.

1. 3/5/2019 1
CMG 12 December 5, 2012
Josep Ferrandiz, Alex Gilgur

2. 3/5/2019
A classical Capacity Problem 2

3. 3/5/2019
Imagine… 3
Shared Resource
Exclusive Resource
How many Boa
Constrictors do
we need?
X elephants
per second
R seconds
per elephant

4. Definitions: Loose But True
3/5/2019 4
1.Utilization – number of active units of exclusive resource
divided by total number of units
2.Level of Service – A metric that measures the user
experience.
• We will use the blocking probability as the level of service metric for
exclusive resources (e.g., worker threads).
3.Throughput – number of requests processed per unit of
time – a measure of demand.
4.Duration – total turnaround time of a request. It includes
resource holding time as well as idle time due to contention
and other effects that prevent resource use.
5.Traffic (concurrency) – average number of concurrent
requests in the system.

5. 3/5/2019
How can we size the system?
5
How Many Units
do you need?
75%
utilization
99.9%
availability
Concurrent Sessions:
Peak Traffic

6. How do we size it?
3/5/2019
Utilization is Evil!!! 6
Peak Traffic:
75% of the sessions have U <
24.92 / 265.5 = 9.36%
25% of the time
U = Average / Total
1. Uncertainty: how much to pad?
2. CapEx & OpEx: 50% utilization = 2X the HW
3. It is only a proxy
What is the actual User Experience???

7. History
3/5/2019
How many telephone stations do you need at an exchange???
7

8. History
3/5/2019 8
1909 - Poisson Distribution
applies to random telephone call arrivals
1917 - Erlang Models
Einstein; Rutherford; Bohr; Erlang; Schroedinger; ….

9. 3/5/2019 9
How many telephone stations do you need at an exchange???
So what’s the big deal?
http://www.erlang.com/calculator/erlb/

10. 3/5/2019 10
How many parallel lines do you need ?
How is it relevant ?

11. 3/5/2019 11
How can we use it all?
Are Utilization and
Erlang’s model related?
B = Er (Q, N)
Why is this useful ?
N = Er-1 (B, Q)

12. 3/5/2019 12
It is better to have fewer larger systems
How does it work?
Solid Blue Line:
B = 0.001: Saturation at ~ 400 sessions
Dashed Red Line:
To maintain U = 70%, we grow the system, and
B reduces exponentially

13. 3/5/2019 13
Validation
The model has been repeatedly validated for large and
small systems. Results are conclusively positive.
• The system was sized for 99.9% availability.
• Actual data were collected from system logs during a crisis.
• At ~ 5 sessions, we started getting bulk arrivals (non-Poisson).
For bulk arrivals, we
now have a similar model

14. How do we size it?
1. Find the target value of the BM for which we want to size the circuits.
2. Apply Little’s formula: Q(t) = X(t) * R(t)
3. Build regression Q(t) ~ BM(t)
4. Compute the Q at the target BM
5. Apply Erlang B model for the Q at the level of service determined by the blocking probability
Start
For all servers
For all applications
on each server
Done
It’s all just one big nested loop

15. How do we size it?
Step 1. Forecast the Business Metric
ForecastPro
Autobox
R
SAS ForecastServer
…

16. How do we size it?
Steps 2 & 3. Apply Little’s Law and Build Regression

17. How do we size it?
Steps 3 & 4. Build Quantile Regression;
Compute Q (BM*)
BM*

18. How do we size it?
Step 5. Apply Erlang’s Model to Size the Resource

19. 3/5/2019 19
Mid-Tier DB ProtectionWeb
Success Stories
Sizing Protection-Tier Connections & SPC

20. 3/5/2019 20
Success Stories
LB operation change
• Dozens of Servers
• 90% of soft limit
• Operation Improvement:
• 60% connection reduction
• 2.5X increase in CPU
• More servers
• Overall more
connections
The Problem:
• Switch from RR to LC
• One large system
• “Invisible” servers
• Count doesn’t matter
• Got the 60% connection reduction
The Solution: B = Er (Q, N)

21. When does it work?
 Assumptions:
◦ M/M/c/0:
 No buffer
 Holding Times: Exponential
 Arrivals: Poisson
3/5/2019
There is a sheep inside the box
21
Erlang B model

22. What If? #1
 Queuing:
◦ M/M/c/∞:
 Infinite buffer:
◦ Always room for an arrival
◦ FIFO
 Holding times: Exponential
 Arrivals: Poisson
3/5/2019
Proceed with caution, but it works 22
Erlang C model

23. What if #2
 Assumptions:
◦ M/M/c/0:
 No buffer
 Holding Times:
◦ Non-Exponential
 Arrivals:
◦ Bursty?
3/5/2019
It won’t work on the tails; but for bulk (bursty) arrivals we do have a model
23
Erlang model?
Bursts -> delays &
high concurrency

24. Conclusions
3/5/2019
There is no reason not to use it, but “Proceed with Caution”
24
• Utilization is Evil, but a useful proxy
• Erlang’s models:
• reflect multiplexing
• are corroborated by queuing theory
• have been around for 100+ years
• tools online and in excel
• can be modified for bulk (bursty) arrivals
• can be used to identify bottlenecks
• Be careful when playing in traffic
• If connections are the bottleneck,
• use LC vs. RR load balancing
B = Er (Q, N)

25. Acknowledgments
 A big thank-you goes to
 Erik Ostermueller whose CMG’11 paper inspired this one
 Antoine de Saint-Exupery for the Little Prince
josep_ferrandiz@yahoo.com
alexgilgur@gmail.com
3/5/2019 Thank you! 25
“It is only with the heart that one can see rightly. What is essential is invisible to the eye.”
― Antoine de Saint-Exupéry, The Little Prince