Queuing system

1. Lecture 14 – Queuing
Systems
Topics
• Basic structure and components
• Performance measures
• Steady-state analysis and Little’s law
• Birth-death processes
• Single-server and multi-server examples
• Flow balance equations

2. Input
source
Queue Service
mechanism
Arriving
customers
Exiting
customers
Structure of Single Queuing Systems
Note
1. Customers need not be people; other possibilities
include parts, vehicles, machines, jobs.
2. Queue might not be a physical line; other
possibilities include customers on hold, jobs waiting
to be printed, planes circling airport.

3. Input Source The size of the “calling population”
may be modeled as infinite or finite.
Calculations are easier in the infinite case and
in many cases this is a reasonable approximation
(bank, pizza parlor, blood bank).
Queuing Discipline First-come first-served (FCFC or
FIFO) is the most frequent assumption, but
priority ordering is important in some settings.
Components of Model
Service Mechanism One or more servers may be
placed in parallel.

4. System Arrival Process Service Process
Bank Customers Arrive Tellers serve customers
Pizza Orders are phoned Orders are driven to
parlor in customers
Blood Pints of blood arrive Patients use up
bank via donation pints of blood
Shipyard Damaged ships sent Ships are repaired
to shipyard for repair & return to sea
Printers Jobs arrive from Documents are
computers printed
Queuing Applications

5. What is the ...
1. average number of customers in the system?
2. average time a customer spends in the system?
3. probability a customer is rejected?
4. fraction of time a server is idle?
These questions are aimed at
characterizing complex systems.
Analyses used to support decision-making.
Typical Performance Questions
In queuing (and most analyses of complex stochastic
systems), design takes the form of asking “what if ”
questions rather than trying to optimize the design.

6. Multiple Servers, Single Queue
What is average wait in the queue?
What is average time in the system?

7. Multiple Servers, Multiple Queues
What is average wait in the queue?
What is average time in the system?

8. N(t) = # of customers in the system at time t ≥ 0
Pk(t) = probability exactly k customers in system
at time t, given # in system at time 0
s = # of parallel servers
λk = mean arrival rate
(expected # of arrivals per unit time)
µk = mean service rate
(expected # of departures per unit time)
(Both λk and µk assume k customers are in system)
Notation and Terminology

9. If there are s servers, each with the same service rate, then
µk = sµ for k ≥ s and µk = kµ for 0 ≤ k < s.
sµ = customer service capacity per unit time
ρ = λ/sµ = utilization factor (traffic intensity)
The systems we study will have ρ < 1 because otherwise
the # of customers in the system will grow without bound.
We will be interested in the steady-state behavior
of queuing systems (the behavior for t large).
Obtaining analytical results for N(t), Pk(t), . . .
for arbitrary values of t (the transient behavior)
is much more difficult.
If λk does not depend on # of customers in system, λk = λ.

10. Notation for Steady-State Analysis
πk = probability of having exactly k customers in the
system
L = expected number of customers in the system
Lq = expected queue length
(doesn’t include those being served)
W = expected time in system, including service time
Wq = expected waiting time in the queue
(doesn’t include service)

11. For any queuing system that has a steady state
and has an average arrival rate of λ,
L = λW
For example, if the average waiting time is 2 hours
and customers arrive at a rate of 3 per hour then,
on average, there are 6 customers in the system.
Similarly, Lq = λWq
If µk = µ for all k ≥ 1 then W = Wq + 1/µ
(1/µ is the mean
service time here)
Little’s Law

12. These three relationships allow us to calculate all
four quantities L, Lq, W and Wq
(once one of them is known).
L = λW requires no assumptions about arrival or
service time distributions, the size of the
calling population, or limits on the queue.
Benefit of Little’s Law

13. Applications in a variety of areas, but in queuing
“birth” refers to the arrival of a customer while
“death” refers to the departure of a customer.
Recall, N(t) = # of customers in the system at time t.
Assumptions
1. Given that N(t) = k, the pdf governing the remaining
time until the next birth (arrival) is
exp(λk) k = 0, 1, 2, . . .
2. Given that N(t) = k, the pdf governing the remaining
time until the next death
(service completion) is exp(µk) k = 1, 2, . . .
3. All random variables are assumed to be independent.
Birth and Death Processes

14. We will investigate steady-state (not transient) results
for birth-death processes based on the
Expected rate in = Expected rate out principle
µ1
3
µ2 µ3
λ0 λ1 λ2
210
Let πk = steady-state probability of being in state k.
Transition Rate Diagram

15. Flow into 0 → µ1π1 = λ0π0 ← flow out of 0
Flow into 1 → λ0π0 + µ2π2 = (λ1 + µ1)π1 ← flow out of 1
Flow into 2 → λ1π1 + µ3π3 = (λ2 + µ2)π2 ← flow out of 2
•
•
•
Flow into k
λk-1πk-1 + µk+1πk+1 = (λk + µk)πk ← flow out of k
Balance Equations – Section 15.5

16. π1 =
λ0
µ1
π0 π2 =
λ1
µ2
π1 +
1
µ2
(µ1π1 – λ0π0) =
λ1
µ2
π1 =
λ1λ0
µ2µ1
π0
. . .
,
Thus πk = Ckπ0 , k = 1, 2, . . . and Σkπk = 1
so (C0 + C1 + C2 + • • • )π0 = 1 or π0 = 1 / (1 + Σk Ck)
= 0
1 0
0
1 1
k k
k
k k
λ λ λ
π π
µ µ µ
−
−
 
=  ÷
 
L
L
1 0
0
1 1
Let , 1,... and let 1k k
k
k k
C k C
λ λ λ
µ µ µ
−
−
= = =
L
L

17. Once we have calculated the πk’s we can find:
L = Σ kπk = expected # of customers in the system
Lq = Σ (k–s)πk = expected # of customers in queue
Ls = L – Lq = expected # of customers in service
λ = Σ λkπk = average (effective) arrival rate
E = Ls / s = efficiency of system (utilization)
∞
k=1
Some Steady-State Results
∞
k=s+
1
∞
k=0

18. Queuing Example with 3 Servers
Assume
1. Average arrival rate: λ = 5/hr
2. Average service rate: µ = 2/hr
3. Arriving customer balks when 6 are in system.
4. Steady-state probabilities:
State 0 1 2 3 4 5 6
Component π0 π1 π2 π3 π4 π5 π6
Probability 0.068 0.170 0.212 0.177 0.147 0.123 0.102

19. Determining System Characteristics
What is the probability that all servers are idle?
Pr{all servers idle} = π0 = 0.068
What is the probability that a customer will not have to wait?
Pr{no wait} = π0 + π1 + π2 = 0.45
What is the probability that a customer will have to wait?
Pr{wait} = 1 – Pr{no wait} = 0.55
What is the probability that a customer balks?
Pr{customer balks} = π6 = 0.102

20. Steady-State Measures for Example
Expected number in queue:
Lq
= 1π4
+ 2π5
+ 3π6
= 0.700
Expected number in service:
Ls = π1 + 2π2 + 3(1 – π0 – π1 – π2) = 2.244
Expected number in the system:
L = Lq + Ls = 2.944
Efficiency of the servers:
E = Ls /s = 2.244/3 = 0.748 or 74.8%

21. Applying Little’s Law with λ, we can calculate
W = L / λ & Wq = Lq / λ
Expected waiting time in the system Expected waiting time in the queue
Results assume that steady state will be reached.
Little’s Law with Average Arrival Rate

22. Example with 3 Servers (cont.)
To compute average waiting times we must first find
the average arrival rate:
λ = Σ λkπk where λk = λ = 5 (k = 0,1,…,5) and
λk = 0 (k > 5), simplifies to
= λ(π0 + π1 + π2 + π3 + π4 + π5)
= λ(1 – π6) = 5(1 – 0.102) = 4.488 / hour
∞
k=0
Consequently,
Ws = Ls / λ = 0.5 hours
Wq = Lq / λ = 0.156 hours
W = L / λ = 0.656 hours

23. “M ” stands for Markovian, meaning
exponential inter-arrival & exponential service times.
M / M / 1
arrival service # of
process process servers
M/M/1 Queue
210
µ
3
µ µ
λ λ λ λ
. . .
µ
so, λk = λ, k = 0,1,…
µk = µ, k = 1,2,…
where ρ = λ/µ
Utilization factor or
traffic intensity
1 0
1
Thus
k
kk
k
k
C
λ λ λ
ρ
µ µ µ
−  
= = = ÷
 
L
L

24. Thus, π0 = 1 – ρ and πk = ρ k
(1 – ρ).
Derivation of Steady-State Probabilities
provided ρ < 1, i.e., λ < µ
0 0 0
1
1
k
k
k
k k k
C
λ
ρ
µ ρ
∞ ∞ ∞
= = =
 
= = = ÷
− 
∑ ∑ ∑
Recall xk
k =0
∞
∑ =
1
1− x
provided x < 1
0 0 0
Now , where 1/k k kk
C Cπ π π
∞
=
= = ∑

25. L = provided λ < µ
λ
µ − λ
Lq = L − (1 − π0) = λ λ
µ
−
λ2
µ (µ−λ)
=
Performance Measures for M/M/1 Queue
From Little’s law, we know
W =
1
λ L and Wq =
1
λ
Lq or
W =
1
µ−λ
λ
and Wq =
µ (µ−λ)
µ−λ

26. The important thing is not the specific M/M/1 formulas,
but the methodology used to find the results.
• Model the system as a birth-and-death process and
construct the rate diagram.
• Depending on the system, defining the states may
be the first challenge.
• Develop the balance equations.
• Solve the balance equations for πk, k = 0, 1, 2, . . .
• Use the steady-state distribution to derive L and Lq
and, use Little’s law to get W and Wq .
Methodology vs. Formulas

27. • A maintenance worker must keep 2 machines in
working order. The 2 machines operate
simultaneously when both are up.
• The time until a machine breaks has an exponential
distribution with a mean of 10 hours.
• The repair time for the broken machine has an
exponential distribution with a mean of 8 hours.
• The worker can only repair one machine at a time.
Example of M/M/1/2/2 Queue

28. Evaluating Performance
1. Model the system as a birth-and-death process.
2. Develop the balance equations.
3. Calculate the steady-state distribution πk .
4. Calculate and interpret L, Lq, W, Wq .
5. What is the proportion of time the repairman is
busy ?
6. What is the proportion of time that a given
machine, e.g., machine #1, is working ?

29. λ = rate at which a single machine breaks down
= 1/10 hr
µ = rate at which machines are repaired
= 1/8 hr
State of the system = # of broken machines.
State-Transition Diagram
2
1
µ µ
2λ λ
0 2Rate network:

30. µπ1 = 2λπ0
2λπ0 + µπ2 = (λ + µ )π1
λπ1 = µπ2
π0 + π1 + π2 = 1
Balance Equations for Repair Example
Ck =
λk-1… λ0
µk … µ1
πk = Ckπ0 and π0 = 1 / Σ Ck
We can solve these balance equations for π0, π1 and π2,
but in this case, we can simply use the formulas
for general birth-and-death processes:
2
k=0
state 0
state 1
state 2
normalize

31. Here, λ0 = 2λ µ1 = µ
λ1 = λ µ2 = µ
λ2 = 0
L = 0π0 + 1π1 + 2π2 = 1.072 (avg # machines in system)
Lq = 0π1 + 1π2 = 0.33 (avg # waiting for repair)
2
0 1 0
1 2 02
1 2 1
2 2
, , and 1 (by definition). ThusC C C
λ λ λλ λ
µ µ µ µ µ
= = = = =
0 1 02
2
2
2 02
1 2
,0.258 0.412
0.3
2
1
3
2
2
0
λ
π π π
λ λ µ
µ µ
λ
π π
µ
= = = =
+ +
= =

32. Proportion of time that machine #1 is working
= π0 +
1
2
π1 = 0.258 +
1
2
(0.412) = 0.464
Proportion of time repairman is busy = π1 + π2 = 0.742
average arrival rate λ = λkπk = λ0π0 + λ1π1 + λ2π2
= (2λ)π0 + λπ1 = 0.0928
2
Σ
k = 0
= 11.55 hours
Average amount of time
that a machine has to wait
to be repaired, including
the time until the repairman
initiates the work.
1 1
(1.072)
0.0928
W L
λ
= =
= 3.56 hours
Average amount of time that a
machine has to wait until the
repairman initiates the work.
1 1
(0.33)
0.0928
q qW L
λ
= =

33. Excel Add-in for M/M/1/2/2 Queue
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
A B
Queue Station M_M_1_2_2
Entity Arrival Rate 0.100000001
Service Rate/Channel 0.125
Number of Servers 1
Max. Number in System ***
Number in Population 2
Type M/M/1/2/2
Mean Number at Station 1.072164893
Mean Time at Station 11.55555534
Mean Number in Queue 0.329896897
Mean Time in Queue 3.555555582
Mean Number in Service 0.742268026
Mean Time in Service 8
Throughput Rate 0.092783503
Efficiency 0.742268026
Probability All Servers Idle 0.257731944
Prob. All Servers Busy 0.742268085
Prob. System Full 0.329896897
Critical Wait Time 1
P(Wait >= Critical Wait) 0.392220885
P(0) 0.257731944
P(1) 0.412371117
P(2) 0.329896899

34. Multi-Channel Queues – M/M/s
…λ
1
2
s
Additional measures of performance
Efficiency: E = ρ
Pr{ Tq = 0 } = Σk=0,s-1 πk
Pr{ Tq > t } = (1 – Pr{ Tq = 0 })e-sµ(1-ρ)t
for t > 0

35. M/M/s Queue
…λ
1
2
s
Measure Formula
ρ λ
sµ
π0
1
0
( ) ( )
1
! !(1 )
n ss
n
s s
n s
ρ ρ
ρ
−
=
 
+ − 
∑
πn, 1 ≤n ≤s (sρ)n
π0
n!
πn, n ≥s
πsρn−s
=
ssρnπ0
s!
PB
πs/(1–ρ)
Lq
πsρ
(1−ρ)
2
=
ss
ρs+1
π0
s!(1–ρ)
2
Ls
sρ= λ/µ
E ρ
Pr{Tq = 0} πn
n=0
s −1
∑ = 1−
(sρ)s
π0
s!(1−ρ)
Pr{Tq > t} (1 – Pr{Tq = 0})e–sµ(1–ρ)t
for t ≥0

36. Telephone Answering System Example
Situation:
• A utility company wants to determine a staffing plan for
its customer representatives.
• Calls arrive at an average rate of 10 per minute, and it
takes an average of 1 minute to respond to each inquiry.
• Both arrival and service processes are Poisson.
Problem: Determine the number of operators that
would provide a “satisfactory” level of
service to the calling population.
Analysis: λ = 10, 1/µ = 1; ρ = λ/sµ < 1 or s > λ/µ = 10

37. Comparison of Multi-Server Systems
Measure M/M/11 M/M/12 M/M/13
Lq 6.821 2.247 0.951
Wq 0.682 0.225 0.0.95
E 0.909 0.833 0.767
Pr{ Tq = 0 } 0.318 0.551 0.715
Pr{ Tq > 1 } 0.251 0.061 0.014

38. Machine Processing with Limited
Space for Work in Process
• Parts arrive at a machine station at the rate of 1.5/min on
average.
• The mean time for service is 30 seconds.
• Both processes are assumed to be Poisson.
• When the machine is busy, parts queue up until there are
3 waiting. At that point arrivals sent for alternative
processing.
Goal: Analyze the situation under the criteria that no more
than 5% of arriving parts receive alternative
processing and that no more than 10% of the parts
that are serviced directly spend more than 1 minute in
the queue.

39. Parameters for Machine Processing
Queuing System
• Number of servers: s = 1
• Maximum number in system: K = 4
• Size of calling population: N = ∞
• Arrival rate: λ = 1.5/min
• Service rate: µ = 2/min
• Utilization: ρ = λ/µ = 0.75
• Model: M/M/1/4

40. M/M/1/K Queue, ρ ≠ 1
λ 1
λ
λ −λ
K − 1
Measure M /M /1/ K M /M /1
ρ λ/µ λ/µ
π
0
1 − ρ
1 − ρ K + 1
1 – ρ
π
n
, 1 ≤ n ≤ K π
0
ρn π
0
ρ n
π
K
π
0
ρK
0
P
B
ρ (1 − ρ K
)
1 − ρ K + 1
ρ
L
q
ρ 2
[( K − 1)ρ K
− K ρ K −1
+ 1]
(1 − ρ )(1 − ρ K + 1
)
ρ 2
1 − ρ
L s
ρ(1 – π
K
) ρ
L
ρ
1 – ρ
–
( K + 1) ρ K + 1
1 − ρ K + 1
ρ
1 − ρ
λ λ(1 – π
K
) λ
E ρ(1 – π
K
) ρ

41. Excel Add-in for M/M/1/4 Queue
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
A B
Queue Station M_M_1_4
Arrival Rate 1.5
Service Rate/Channel 2
Number of Servers 1
Max. Number in System 4
Number in Population ***
Type M/M/1/4
Mean Number at Station 1.444302
Mean Time at Station 1.074286
Mean Number in Queue 0.772087
Mean Time in Queue 0.574286
Mean Number in Service 0.672215
Mean Time in Service 0.5
Throughput Rate 1.34443
Efficiency 0.672215
Probability All Servers Idle 0.327785
Prob. All Servers Busy 0.672215
Prob. System Full 0.103713
Critical Wait Time 1
P(Wait >= Critical Wait) 0.225043
P(0) 0.327785
P(1) 0.245839
P(2) 0.184379
P(3) 0.138284
P(4) 0.103713

42. Solution for M/M/1/4 Model
Parameters
λ = 1.5/min, µ = 2/min
ρ = λ/µ = 0.75
K = 4
Steady state probabilities (see page 566 in text)
π0 = (1 − ρ) / (1 − ρ K+1
) = (1 – 0.75) / (1 – 0.755
) = 0.25 / 0.7627
= 0.328
πk = π0ρk
= (0.328)(0.75k
); k = 1,…,K
π1 = 0.246, π2 = 0.184, π3 = 0.138, π4 = 0.104

43. Solution for M/M/1/4 Model (cont’d)
Balking probability: PF = π4 = 0.104 or 10.4%
(does not meet 5% goal)
Average arrival rate: λ = λ(1 – PF) = 1.344/min
L = 1.444
W = 1.074 min
E = ρ (1 – PF) = 0.75(1 – 0.104) = 0.672 or 67.2%
Pr{Tq > 1} = 0.225 (see text for computations; does not meet
10% goal)

44. Add Second Machine – M/M/2/5 Model
New results
PF = 0.0068, λ = λ(1 – π5) = 1.49
L = 0.85, W = 0.57 min
E = 37.2%
Pr{Tq > 1} = 0.049 or 4.9%
This solution meets our original goals with the
percentage of balking parts now less than 1% and
the probability of a wait time greater than 1
minute less than 5%

45. What You Should Know About
Queuing Systems
• The major components
• The 5-field notation
• How to use Little’s law
• Components of a Markov queuing
system
• The important performance measures
• How to compute performance
measures