Queuing theory is a branch of mathematics that studies the behavior of waiting lines, or queues, in systems where entities such as customers, jobs, or data packets arrive at a service point and wait for service.

1. CS352 - Introduction
to Queuing Theory
Rutgers University

2. CS352 Fall,2005 2
Queuing theory definitions
 (Bose) “the basic phenomenon of queueing arises
whenever a shared facility needs to be accessed for
service by a large number of jobs or customers.”
 (Wolff) “The primary tool for studying these
problems [of congestions] is known as queueing
theory.”
 (Kleinrock) “We study the phenomena of standing,
waiting, and serving, and we call this study
Queueing Theory." "Any system in which arrivals
place demands upon a finite capacity resource may
be termed a queueing system.”
 (Mathworld) “The study of the waiting times, lengths,
and other properties of queues.”
http://www2.uwindsor.ca/~hlynka/queue.html

3. CS352 Fall,2005 3
Applications of Queuing
Theory
 Telecommunications
 Traffic control
 Determining the sequence of computer
operations
 Predicting computer performance
 Health services (eg. control of hospital bed
assignments)
 Airport traffic, airline ticket sales
 Layout of manufacturing systems.
http://www2.uwindsor.ca/~hlynka/queue.html

4. CS352 Fall,2005 4
Example application of
queuing theory
 In many retail stores and banks
 multiple line/multiple checkout system  a
queuing system where customers wait for the next
available cashier
 We can prove using queuing theory that :
throughput improves increases when queues are
used instead of separate lines
http://www.andrews.edu/~calkins/math/webtexts/prod10.htm#QT

5. CS352 Fall,2005 5
Example application of
queuing theory
http://www.bsbpa.umkc.edu/classes/ashley/Chaptr14/sld006.htm

6. CS352 Fall,2005 6
Queuing theory for studying
networks
 View network as collections of queues
 FIFO data-structures
 Queuing theory provides probabilistic
analysis of these queues
 Examples:
 Average length
 Average waiting time
 Probability queue is at a certain length
 Probability a packet will be lost

7. CS352 Fall,2005 7
Little’s Law
 Little’s Law:
Mean number tasks in system = mean arrival rate x
mean response time
 Observed before, Little was first to prove
 Applies to any system in equilibrium, as long as
nothing in black box is creating or destroying tasks
Arrivals Departures
System

8. CS352 Fall,2005 8
Proving Little’s Law
J = Shaded area = 9
Same in all cases!
1 2 3 4 5 6 7 8
Packet
#
Time
1
2
3
1 2 3 4 5 6 7 8
# in
System
1
2
3
Time
1 2 3
Time in
System
Packet #
1
2
3
Arrivals
Departures

9. CS352 Fall,2005 9
Definitions
 J: “Area” from previous slide
 N: Number of jobs (packets)
 T: Total time
 l: Average arrival rate
 N/T
 W: Average time job is in the system
 = J/N
 L: Average number of jobs in the system
 = J/T

10. CS352 Fall,2005 10
1 2 3 4 5 6 7 8
# in
System
(L) 1
2
3
Proof: Method 1: Definition
Time (T)
1 2 3
Time in
System
(W)
Packet # (N)
1
2
3
=
W
L T
N
)
(

NW
TL
J 

W
L )
(l


11. CS352 Fall,2005 11
Proof: Method 2: Substitution
W
L T
N
)
(

W
L )
(l

)
)(
( N
J
T
N
T
J

T
J
T
J
 Tautology

12. CS352 Fall,2005 12
Model Queuing System
Server System
Queuing System
Queue Server
Queuing System
 Use Queuing models to
 Describe the behavior of queuing systems
 Evaluate system performance

13. CS352 Fall,2005 13
Characteristics of queuing
systems
 Arrival Process
 The distribution that determines how the tasks
arrives in the system.
 Service Process
 The distribution that determines the task
processing time
 Number of Servers
 Total number of servers available to process the
tasks

14. CS352 Fall,2005 14
Kendall Notation 1/2/3(/4/5/6)
 Six parameters in shorthand
 First three typically used, unless specified
1. Arrival Distribution
2. Service Distribution
3. Number of servers
4. Total Capacity (infinite if not specified)
5. Population Size (infinite)
6. Service Discipline (FCFS/FIFO)

15. CS352 Fall,2005 15
Distributions
 M: stands for "Markovian", implying
exponential distribution for service times or
inter-arrival times.
 D: Deterministic (e.g. fixed constant)
 Ek: Erlang with parameter k
 Hk: Hyperexponential with param. k
 G: General (anything)

16. CS352 Fall,2005 16
Kendall Notation Examples
 M/M/1:
 Poisson arrivals and exponential service, 1 server, infinite
capacity and population, FCFS (FIFO)
 the simplest ‘realistic’ queue
 M/M/m
 Same, but M servers
 G/G/3/20/1500/SPF
 General arrival and service distributions, 3 servers, 17
queue slots (20-3), 1500 total jobs, Shortest Packet First

17. CS352 Fall,2005 17
Poisson Process
 For a poisson process with average arrival
rate , the probability of seeing n arrivals in
time interval delta t
l
0
...
)
2
Pr(
)
1
Pr(
)
(
...]
!
2
)
(
1
[
)
1
Pr(
1
)
0
Pr(
)
(
1
...
!
2
)
(
1
)
0
Pr(
)
(
!
)
(
)
Pr(
2
2











































t
t
o
t
t
t
t
te
t
t
o
t
t
t
e
t
n
E
n
t
e
n
t
t
n
t
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l

18. CS352 Fall,2005 18
Poisson process &
exponential distribution
 Inter-arrival time t (time between arrivals) in a
Poisson process follows exponential
distribution with parameter l
l
l l
1
)
(
)
Pr(

 
t
E
e
t t

19. CS352 Fall,2005 19
Analysis of M/M/1 queue
 Given:
• l: Arrival rate of jobs (packets on input link)
• m: Service rate of the server (output link)
 Solve:
 L: average number in queuing system
 Lq average number in the queue
 W: average waiting time in whole system
 Wq average waiting time in the queue

20. CS352 Fall,2005 20
M/M/1 queue model
l
m
m
1
Wq
W
L
Lq

21. CS352 Fall,2005 21
Solving queuing systems
 4 unknowns: L, Lq W, Wq
 Relationships:
 L=lW
 Lq=lWq (steady-state argument)
 W = Wq + (1/m)
 If we know any 1, can find the others
 Finding L is hard or easy depending on the type of
system. In general:
0




n
n
nP
L

22. CS352 Fall,2005 22
Analysis of M/M/1 queue
 Goal: A closed form expression of the
probability of the number of jobs in the queue
(Pi) given only l and m

23. CS352 Fall,2005 23
Equilibrium conditions
n+1
n
n-1
l l l
l
m m
m m
)
(t
Pn
Define to be the probability of having n tasks in the system at time t
0
)
(
)
(
lim
,
)
(
lim
,
when
Stablize
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)]
1
)(
)[(
(
)]
1
)(
)[(
(
]
)
1
)(
1
)[(
(
)
(
)]
1
)(
)[(
(
]
)
1
)(
1
)[(
(
)
(
1
1
1
0
0
0
1
1
1
0
0






























































t
t
P
t
t
P
P
t
P
t
P
t
P
t
P
t
t
P
t
t
P
t
P
t
P
t
t
P
t
t
P
t
t
t
P
t
t
t
P
t
t
t
t
t
P
t
t
P
t
t
t
P
t
t
t
t
t
P
t
t
P
n
n
t
n
n
t
n
n
n
n
n
n
n
n
n
m
l
m
m
l
l
m
l
m
l
l
m
l
m
l
m
l
m
l
m
l
m

24. CS352 Fall,2005 24
Equilibrium conditions
1
1
1
0
)
( 
 



n
n
n P
P
P
P
P
m
l
m
l
m
l
n+1
n
n-1
l l l
l
m m
m m

25. CS352 Fall,2005 25
Solving for P0 and Pn
 Step 1
 Step 2
0
,
0
2
2
0
1 , P
P
P
P
P
P
n
n 


















m
l
m
l
m
l




 























0
0
0
0
0
1
,
1
,
1
n
n
n
n
n
n P
P
then
P
m
l
m
l

26. CS352 Fall,2005 26
Solving for P0 and Pn
 Step 3
 Step 4
 
1
ρ
ρ
1
1
ρ
1
ρ
1
ρ
,
ρ
0
0















 





 n
n
n
n
then
m
l
m
l
 
ρ
1
ρ
and
ρ
1
ρ
1
0
0 







n
n
n
n
P
P

27. CS352 Fall,2005 27
Solving for L
0




n
n
nP
L )
1
(
0





n
n
n 
 )
1
(
1
1






n
n
n


 



 

 1
1
)
1
( d
d






 

0
)
1
(
n
n
d
d


 
 
2
)
1
(
1
)
1
( 

 
 l
m
l



 
 )
1
(

28. CS352 Fall,2005 28
Solving W, Wq and Lq
   l
m
l
l
m
l
l 
 

 1
1
L
W
    )
(
1
1
l
m
m
l
m
l
m
l
m 
 



W
Wq
)
(
)
(
2
l
m
m
l
l
m
m
l
l
l 
 

 q
q W
L

29. CS352 Fall,2005 29
Online M/M/1 animation
 http://www.dcs.ed.ac.uk/home/jeh/Simjava/qu
eueing/mm1_q/mm1_q.html

30. CS352 Fall,2005 30
Response Time vs. Arrivals
l
m
 1
W
Waiting vs. Utilization
0
0.05
0.1
0.15
0.2
0.25
0 0.2 0.4 0.6 0.8 1 1.2
 %
W(sec)

31. CS352 Fall,2005 31
Stable Region
Waiting vs. Utilization
0
0.005
0.01
0.015
0.02
0.025
0 0.2 0.4 0.6 0.8 1
 %
W(sec)
linear region

32. CS352 Fall,2005 32
Example
 On a network gateway, measurements show that the
packets arrive at a mean rate of 125 packets per
second (pps) and the gateway takes about 2
millisecs to forward them. Assuming an M/M/1
model, what is the probability of buffer overflow if the
gateway had only 13 buffers. How many buffers are
needed to keep packet loss below one packet per
million?

33. CS352 Fall,2005 33
Example
 Measurement of a network gateway:
 mean arrival rate (l): 125 Packets/s
 mean response time (m): 2 ms
 Assuming exponential arrivals:
 What is the gateway’s utilization?
 What is the probability of n packets in the gateway?
 mean number of packets in the gateway?
 The number of buffers so P(overflow) is <10-6?

34. CS352 Fall,2005 34
Example
 Arrival rate λ =
 Service rate μ =
 Gateway utilization ρ = λ/μ =
 Prob. of n packets in gateway =
 Mean number of packets in gateway =

35. CS352 Fall,2005 35
Example
 Arrival rate λ = 125 pps
 Service rate μ = 1/0.002 = 500 pps
 Gateway utilization ρ = λ/μ = 0.25
 Prob. of n packets in gateway =
 Mean number of packets in gateway =
n
n
)
25
.
0
(
75
.
0
ρ
)
ρ
1
( 

33
.
0
57
.
0
25
.
0
ρ
1
ρ




36. CS352 Fall,2005 36
Example
 Probability of buffer overflow:
 To limit the probability of loss to less than 10-
6:

37. CS352 Fall,2005 37
Example
 Probability of buffer overflow:
= P(more than 13 packets in gateway)
 To limit the probability of loss to less
than 10-6:

38. CS352 Fall,2005 38
Example
 Probability of buffer overflow:
= P(more than 13 packets in gateway)
= ρ13 = 0.2513 = 1.49x10-8
= 15 packets per billion packets
 To limit the probability of loss to less
than 10-6:

39. CS352 Fall,2005 39
Example
 Probability of buffer overflow:
= P(more than 13 packets in gateway)
= ρ13 = 0.2513 = 1.49x10-8
= 15 packets per billion packets
 To limit the probability of loss to less
than 10-6:
6
10
ρ 

n

40. CS352 Fall,2005 40
Example
 To limit the probability of loss to less
than 10-6:
 or
   
25
.
0
log
/
10
log 6


n
6
10
ρ 

n

41. CS352 Fall,2005 41
Example
 To limit the probability of loss to less
than 10-6:
 or
= 9.96
   
25
.
0
log
/
10
log 6


n
6
10
ρ 

n

42. CS352 Fall,2005 42
Empirical Example
M/M/m
system