Is Multipath Routing Really a Panacea?

Is Multipath Routing Really a Panacea?
Deep Medhi
Computer Science & Electrical Engineering Department
University of Missouri-Kansas City, USA
dmedhi@umkc.edu
in association with Xuan Liu, Sudhir Mohanraj, and Michał Pi´oro
Supported in part by NSF Grant # CNS-0916505
CNSM Keynote: 11 November 2015
MPR-SPR

Keynote dedicated
in memory of
Karen Medhi
MPR-SPR

Going between Two points in Manhattan, NY
MPR-SPR

Going between Two points in Manhattan, NY: one path
MPR-SPR

Going between Two points in Manhattan: two paths
MPR-SPR

Going between Two points in Manhattan: three paths
MPR-SPR

Going between Several points in Manhattan, NY
MPR-SPR

Going between *ANY* two points in Manhattan, NY
MPR-SPR

Question:
At instant of time, is multipath routing beneﬁcial?
MPR-SPR

What Multipath Routing is NOT
Take one path in the morning, another path in the evening – this
is NOT multipath routing
MPR-SPR

Multipath Routing: A Common Belief
Multipath routing (load sharing): split routing where each
node-to-node trafﬁc can be send among accessible paths.
MPR-SPR

An alternative considered in this context is single-path
routing (non-split routing of the demands).
— Many ISPs prefer (for troubleshooting)
MPR-SPR

Common belief: multipath routing, as compared with
single-path routing, gives a signiﬁcantly better opportunity
to control the link loads and in this way effectively optimize
various trafﬁc objectives.
MPR-SPR

Question: Does it? When? How much?
MPR-SPR

Question: Does it? When? How much?
Taking a Trafﬁc Engineering Perspective...
MPR-SPR

A 3-node Example: Single Demand (Commodity)
1
3
2
10
10
10
Capacity
15
Trafﬁc Volume
between 1 and 2
Easy to see that 15 units of trafﬁc would need to be split
between the paths 1-2 and 1-3-2.
MPR-SPR

3-node example: All pairs with traffic
1
3
2
Capacity
=
10
Capacity = 10
Capacity
=
15
Traffic = 5
Traffic=
7
Traffic=
10
MPR-SPR

Min Cost Routing: (i) Illustration for 3-node network: all pair traffic
Linear Programming Formulation:
Minimize x12 + 2 x132 + x13 + 2 x123 + x23 + 2 x213
subject to
d12: x12 + x132 = 5
d13: x13 + x123 = 10
d23: x23 + x213 = 7
c12: x12 + x123 + x213 <= 10
c13: x132 + x13 + x213 <= 10
c23: x132 + x123 + x23 <= 15
Bounds
x12 >= 0
x132 >= 0
x13 >= 0
x123 >= 0
x23 >= 0
x213 >= 0
End
1
3
2
Capacity
=
10
Capacity = 10
Capacity
=
15
Traffic = 5
Traffic=
7
Traffic=
10
For each pair, the first (direct) path is cheaper than the second path.
Solution: x12 = 5, x13 = 10, x23 = 7 3 positive path-flows
MPR-SPR

3-node example: All pairs with traffic
1
3
2
Capacity
=
10Capacity = 10
Capacity
=
15
Traffic = 5
Traffic=
7
Traffic=
10
Is it possible for each pair to use both the direct and alternate
two-link paths with positive flows at optimality?
MPR-SPR

Min Cost Routing: (ii) 3-node net (swap path cost)
Minimize 2 x12 + x132 + 2 x13 + x123 + 2 x23 + x213
subject to
d12: x12 + x132 = 5
d13: x13 + x123 = 10
d23: x23 + x213 = 7
c12: x12 + x123 + x213 <= 10
c13: x132 + x13 + x213 <= 10
c23: x132 + x123 + x23 <= 15
Bounds
x12 >= 0
x132 >= 0
x13 >= 0
x123 >= 0
x23 >= 0
x213 >= 0
End
1
3
2
Capacity
=
10
Capacity = 10
Capacity
=
15
Traffic = 5
Traffic=
7
Traffic=
10
Note: same traffic demand and link capacity as before; the path cost in objective is
changed. Solution: x12 = 1, x132 = 4, x13 = 3.5, x123 = 6.5, x23 = 4.5, x213 = 2.5
6 positive path-flows
MPR-SPR

Is there a connection?
1) How does the number of positive ﬂows at optimality relate to
the size of the problem?
2) Does the objective function matter?
MPR-SPR

Two Letters to Remember for the Rest of the Talk
D : Number of demands in a Network
L : Number of Links in a Network
MPR-SPR

Two Letters to Remember for the Rest of the Talk
D : Number of demands in a Network
L : Number of Links in a Network
NOTE:
D = N(N − 1)/2 if every pair has trafﬁc (bidirectional) in a
N-node network
MPR-SPR

D + L result: Min Cost Routing Multi-Commodity
Network Flow
min
x≥0
d∈D p∈Pd
ξdp xdp (1a)
p∈Pd
xdp = hd , d ∈ D (demand) (1b)
d∈D p∈Pd
δdp xdp ≤ c , ∈ L (capacity) (1c)
hd : trafﬁc for demand ID d ∈ D (#(D) = D)
c : link capacity (#(L) = L)
ξdp: unit path cost of path p for demand d
δdp : link-path indicator 0/1
MPR-SPR

Min Cost Routing Multi-Commodity Network Flow
min
x≥0
d∈D p∈Pd
ξdp xdp
p∈Pd
xdp = hd , d ∈ D (demand)
d∈D p∈Pd
δdp xdp ≤ c , ∈ L (capacity)
D + L property
In vertex solutions in Linear Program, there are at most D + L positive
path-ﬂows. (Proof skipped)
Corollary: There are at most L demands that require more than one
positive path-ﬂow
MPR-SPR

Quick Review: Feasible Region and vertices for a linear program
Objective
Feasible Region
Optimal Vertex
MPR-SPR

Consider again the following illustration for the 3-node
multi-commodity example:
d12: x12 + x132 = 5
d13: x13 + x123 = 10
d23: x23 + x213 = 7
c12: x12 + x123 + x213 <= 10
c13: x132 + x13 + x213 <= 10
c23: x132 + x123 + x23 <= 15
x non-negative
Adding slack variables:
d12: x12 + x132 = 5
d13: x13 + x123 = 10
d23: x23 + x213 = 7
c12: x12 + x123 + x213 + s12 = 10
c13: x132 + x13 + x213 + s13 = 10
c23: x132 + x123 + x23 + s23 = 15
x, s non-negative
Here, D = 3, L = 3. After adding slack variables, we have six (6) equations with nine
(9) variables. This means that at most 6 variables can be positive at a basic feasible
solution, and consequently, at optimality.
MPR-SPR

Corollary
If the optimization problem (1) is feasible, then at most L trafﬁc
pairs will have more than one path with non-zero ﬂows at
optimality.
MPR-SPR

Corollary
If the optimization problem (1) is feasible, then at most L traffic
pairs will have more than one path with non-zero flows at
optimality.
Proof:
From the theorem, we know that there are D + L non-zero flow
variables. Since there are a total of D pairs, at least one path
for each pair must carry the traffic load. This then leaves us
with at most D + L − D = L pairs that has more than one paths
with non-zero flows.
MPR-SPR

Trafﬁc Engineering Objectives (besides min cost
routing)
Commonly used trafﬁc engineering objectives for IP, MPLS,
SDN networks:
Network Load Balancing (Minimize Maximum utilization),
also known as Congestion Minimization
Average Delay (Minimize Average Network Delay)
MPR-SPR

Load Balancing Optimization: LP Formulation
min
{x≥0,r}
r
subject to
p∈Pd
xdp = hd , d ∈ D
d∈D p∈Pd
δdp xdp ≤ c r, ∈ L
xdp ≥ 0, p = 1, 2, ..., Pd ,
d = 1, 2, ..., D
(3)
Note: introduced a new variable r (load balancing variable)
We again have D + L constraints.
MPR-SPR

LB: load balancing
min
x≥0,r
r
p∈Pd
xdp = hd , d ∈ D
d∈D p∈Pd
δdp xdp ≤ c r, ∈ L
xdp ≥ 0, p = 1, 2, ..., Pd , d = 1, 2, ..., D
D + L − 1 property
In vertex solutions, there are at most D + L − 1 positive path-ﬂows.
There are at most L − 1 demands that require more than one positive
path-ﬂow.
MPR-SPR

3-node Illustration for load balancing
Minimize r
subject to
d12: x12 + x132 = 5
d13: x13 + x123 = 10
d23: x23 + x213 = 7
c12: x12 + x123 + x213 <= 10 r
c13: x132 + x13 + x213 <= 10 r
c23: x132 + x123 + x23 <= 15 r
Bounds
x12 >= 0
x132 >= 0
x13 >= 0
x123 >= 0
x23 >= 0
x213 >= 0
End
1
3
2
Capacity
=
10
Capacity = 10
Capacity
=
15
Traffic = 5
Traffic=
7
Traffic=
10
Solution: x12 = 4.125, x132 = 0.875, x13 = 6.625, x123 = 3.375, x23 = 7, r∗ = 0.75
D + L − 1 = 3 + 3 − 1 = 5
MPR-SPR

3-node Illustration for load balancing
Minimize r
subject to
d12: x12 + x132 = 5
d13: x13 + x123 = 10
d23: x23 + x213 = 7
c12: x12 + x123 + x213 <= 10 r
c13: x132 + x13 + x213 <= 10 r
c23: x132 + x123 + x23 <= 15 r
Bounds
x12 >= 0
x132 >= 0
x13 >= 0
x123 >= 0
x23 >= 0
x213 >= 0
End
1
3
2
Capacity
=
10
Capacity = 10
Capacity
=
15
Traffic = 5
Traffic=
7
Traffic=
10
Solution: x12 = 4.125, x132 = 0.875, x13 = 6.625, x123 = 3.375, x23 = 7, r∗ = 0.75
D + L − 1 = 3 + 3 − 1 = 5 One pair always has single-path routing at optimality
MPR-SPR

Minimize Average Delay (AD)
min
x≥0,y≥0 ∈L
y
c −y
subject to
p∈Pd
xdp = hd , d ∈ D
d∈D p∈Pd
δdp xdp = y , ∈ L
(5)
Note: Objective is non-linear.
MPR-SPR

D + L for non-linear AD
The D + L property holds for the non-linear AD problem!
min
x≥0,y≥0 ∈L
y
c −y
subject to
p∈Pd
xdp = hd , d ∈ D
d∈D p∈Pd
(6)
We’ve a proof!
MPR-SPR

D + L for non-linear AD
The D + L property holds for the non-linear AD problem!
min
x≥0,y≥0 ∈L
y
c −y
subject to
p∈Pd
xdp = hd , d ∈ D
d∈D p∈Pd
(6)
We’ve a proof!
Margins of this slide is too small to ﬁt in the proof :)
MPR-SPR

Two Measures
What percentage of demand pairs have more than one
path at optimality?
– MPM (Multipath Measure)
MPR-SPR

Two Measures
path at optimality?
How far off is the single-path routing compared to multipath
routing?
– Normalized Cost Overhead (COH)
COH =
OPTSinglePath − OPTMultiPath
OPTMultiPath
MPR-SPR

Two Measures
path at optimality?
How far off is the single-path routing compared to multipath
routing?
– Normalized Cost Overhead (COH)
COH =
OPTSinglePath − OPTMultiPath
OPTMultiPath
NOTE: MPM = 0 −→ Single-Path Routing is Optimal
MPR-SPR

MPM: multipath measure
deﬁnition
MPM is equal to the maximum percentage of demands that can
have more than one path with nonzero ﬂow at optimal vertex
solutions.
MCR: MPM = L
D
LB: MPM = L−1
D
AD: MPM = L
D
(max value = 100%)
MPR-SPR

MPM: multipath measure
deﬁnition
MPM is equal to the maximum percentage of demands that can
have more than one path with nonzero ﬂow at optimal vertex
solutions.
MCR: MPM = L
D
LB: MPM = L−1
D
AD: MPM = L
D
(max value = 100%)
MPM computed to optimality: to be denoted by MPM∗
MPM∗ ≤ MPM
MPR-SPR

Average vertex degree for Real-world ISP Networks
Topology Zoo Collection: the highest average vertex degree 4.5
MPR-SPR

RocketFuel Collection (PoP-level topology): the highest average vertex degree
5.24
Use ”3N-Net” as an upper limit for theoretical MPM
– L = 3N means average vertex degree is 6
MPR-SPR

RocketFuel Collection (PoP-level topology): the highest average vertex degree
5.24
Use ”3N-Net” as an upper limit for theoretical MPM
– L = 3N means average vertex degree is 6
Important to note: with L = O(N), D = O(N2)
MPM (=L/D) −→ 0 as N → ∞
MPR-SPR

Theoretical MPM
N = 4 N = 9 N = 16 N = 25 N = 36 N = 49 N = 100 N = 1024
D 6 36 120 300 630 1,176 4,950 523,776
R(MCR/AD) 66.67 25.00 13.33 8.33 5.71 4.17 2.02 0.20
R(LB) 50.00 22.22 12.50 8.00 5.56 4.08 2.00 0.20
G(MCR/AD) 66.67 33.33 20.00 13.33 9.52 7.14 3.64 0.38
G(LB) 50.00 30.56 19.17 13.00 9.37 7.06 3.62 0.38
3N(MCR/AD) 100.00 75.00 40.00 25.00 17.14 12.50 6.06 0.59
3N(LB) 83.33 72.22 39.17 24.67 16.98 12.41 6.04 0.59
MPM (in %) for different network sizes and topologies
D = N/(N-1)/2 (No. of demand pairs)
R(Ring) L = N
G(Grid) L = 2N − 2
√
N
3N(3N-Net) L = 3N
MPR-SPR

Special Results: Ring Network, uniform trafﬁc/capacity
case (”Symmetric Ring”): LB Objective
In a ring network, every pair of nodes have two paths:
clockwise and counter-clockwise.
MPR-SPR

Special Results: Ring Network, uniform trafﬁc/capacity
case (”Symmetric Ring”): LB Objective
In a ring network, every pair of nodes have two paths:
clockwise and counter-clockwise.
If the number of nodes N is odd, then in a ring with uniform
trafﬁc for all node pairs and link capacity (”symmetric ring”),
only one path (minimum-hop shortest path) is used by
each pair at optimality for the LB objective. That is,
MPM∗ = 0
If the number of nodes N is even, then in a symmetric ring
MPM∗ = 1
N−1
Same hold for AD objective too.
MPR-SPR

Symmetric Ring - Odd number of nodes - Illustration
(N = 5)
12
3
4
5
MPR-SPR

Symmetric Ring - Even number of nodes - Illustration
(N = 4)
1 2
34
No split for: 1:2, 2:3, 3:4, 1:4 green
equal split for: 1:3 (blue), 2:4 (brown)
MPM∗ = 2/6 = 1/3
If N = 101, then MPM∗ = 1/100 = 1%
MPR-SPR

A Series of Comprehensive Studies
Topologies: Ring, Grid, Example ISP topologies
Traffic distribution: uniform (U), uniform-perturbed (U-P),
Elephant-mice (EM) traffic, Lognormal (LN) traffic
Network Load: 0.4 to 0.95
For each load, five traffic profiles generated randomly
For Load Balancing Objective, we proved a traffic scaling
property that the optimal solution doesn’t change.
A few representative results presented here.
MPR-SPR

A Representative set of results
Cases
All pair trafﬁc (D = N(N − 1)/2)
Limited pair trafﬁc for Data Center Networks
What happens as we increase D from 1, 2, ...., N(N − 1)/2
MPR-SPR

Ring Topology: MPM∗
and COH
MPR-SPR

Grid Topology: MPM∗
and COH
MPR-SPR

Fully-Mesh Topology
Theoretical MPM: 100%
Symmetric Mesh: Uniform traffic, uniform capacity
– Optimal is direct routing between any two nodes (single-path)
General Traffic
– Use Sprint’s 43-node fully-mesh telephone backbone network
– Traffic for different time of the day
– MPM∗: 19% on average
– MPM∗: as high at 41%
MPR-SPR

Fat-tree Data Center Networks
A special structure: k-pod architecture
NOT all pairs of nodes have trafﬁc
Only Edge Switches have trafﬁc intra-data center case
9 10 11 12 13 14 15 16
1 2 3 4 5 6 7 8
17 18 19 20
Edge
Aggregation
Core
k-pod fat-tree topology
N = 5
4
k2
D = k4
8
− k2
4
L = k3
2
L = O(k3), D = O(k4) MPM = L/D → 0
MPR-SPR

Fat-Tree Topology: MPM and MPM∗
for LB
9 10 11 12 13 14 15 16
1 2 3 4 5 6 7 8
17 18 19 20
Edge
Aggregation
Core
k N D L MPM MPM∗ COH MPM∗ COH
(U) (U) (LN) (LN)
4 20 28 32 100.00 35.71 14.29 15.00 0.53
6 45 153 108 69.93 39.87 5.88 10.85 0.36
8 80 496 256 51.41 28.02 3.23 9.47 0.31
U:=uniform trafﬁc, LN:=lognormal trafﬁc;
COH:=Cost Overhead in (%)
MPR-SPR

What happens when we start from a single demand pair and
continue to add more demand pairs until D = N(N − 1)/2
MPR-SPR

MPM* for grid, lognormal N = 16, 25, 36
0 0.2 0.4 0.6 0.8 1
0
20
40
60
80
100
X: 1
Y: 3.664
Demand Type: Lognormal (µ =16.6, σ=1.04 )
No. of Demand Pairs / Total Pairs
MPM*(%)
Grid: 16 Nodes
Grid: 25 Nodes
Grid: 36 Nodes
[Warning: In some cases, single-path routing was not run long
enough to reach optimality.]
MPR-SPR

MPM* for grid, lognormal N = 49, 64, 100
0 0.1 0.2 0.3 0.4 0.5
0
10
20
30
40
50
60
X: 0.4354
Y: 2.068
Demand Type: Lognormal (µ =16.6, σ=1.04 )
No. of Demand Pairs/Total Pairs
MPM*(%)
Grid: 49 Nodes
Grid: 64 Nodes
Grid: 100 Nodes
[Warning: In some cases, single-path routing was not run long
enough to reach optimality.]
MPR-SPR

MPM* and COH for N = 16, 25, 36 (Grid, lognormal)
# of Demand PairsMPM* COH(%) # of Demand PairsMPM* COH(%) # of Demand PairsMPM* COH(%)
1 100.00 100.00 1 80 69.79 1 100.00 113.89
2 70.00 117.86 2 60 43.04 2 60.00 100.00
4 65.00 52.83 4 50 42.82 4 35.00 22.89
8 37.50 30.34 8 35 35.06 8 27.50 55.17
16 25.00 24.16 16 18.75 12.92 16 25.00 30.63
32 18.75 15.94 32 16.25 20.66 32 13.13 16.07
64 9.06 3.36 64 13.438 10.62 64 12.81 4.09
120 4.83 0.00 128 7.966 1.11 128 7.81 1.01
256 4.532 0.00 256 5.08 0.00
300 3.664 0.00 512 3.75 0.00
630 2.32 0.00
16 Nodes 36 Nodes25 Nodes
MPR-SPR

MPM* and COH for N = 49, 64, 100 (Grid, lognormal)
# of Demand PairsMPM*(%) Overhead (%)# of Demand PairsMPM*(%) Overhead (%)# of Demand PairsMPM*(%) Overhead (%)
1 60.00 23.33 1 40.00 14.29 1 20.00 6.67
2 50.00 46.51 2 20.00 24.24 2 10.00 21.74
4 35.00 57.89 4 35.00 40.00 4 30.00 4.17
8 20.00 20.35 8 15.00 15.79 8 12.50 0.00
16 15.00 6.95 16 8.75 4.30 16 13.75 5.71
32 11.25 9.27 32 13.75 13.27 32 14.37 8.00
64 9.06 22.49 64 9.69 14.75 64 8.75 18.52
128 7.66 7.66 128 4.84 0.68 128 5.00 0.00
256 5.31 0.10 256 3.59 0.16 256 5.31 1.40
512 2.07 0.00 512 3.40 0.63 512 2.42 0.00
1024 1.78 0.00 1024 1.11 0.50 1024 1.62 2.15
1176 1.26 0.00 2016 0.92 0.00 2048 0.98 0.11
4096 0.40 0.00
4950 0.34 0.00
100 Nodes49 Nodes 64 Nodes
MPR-SPR

So, what went wrong in our thought process about the
beneﬁt of multipath routing?
Our minds do play trick:)
— We forget that others are using the network too
— Remember the Manhattan Street Network
MPR-SPR

Often, smaller topologies were studied where multipath is
certainly beneﬁcial
MPR-SPR

For large problems, heuristic algorithms were developed to
show the “beneﬁt” of multipath routing
MPR-SPR

For large problems, heuristic algorithms were developed to
show the “beneﬁt” of multipath routing
– Problem is ....
– Heuristic gives a false sense of beneﬁt of multipath
routing since the solution is near optimal, but not an
optimal vertex solution!
MPR-SPR

3-node Load Balancing example - revisit: optimal vs. near optimal solution
Minimize r
subject to
d12: x12 + x132 = 5
d13: x13 + x123 = 10
d23: x23 + x213 = 7
c12: x12 + x123 + x213 <= 10
c13: x132 + x13 + x213 <= 10
c23: x132 + x123 + x23 <= 15
End
1
3
2
Capacity
=
10
Capacity = 10
Capacity
=
15
Traffic = 5
Traffic=
7
Traffic=
10
Optimal r∗ = 0.75
One demand is always single-path at optimality; In this case, pair 2:3 has single path
routing, x23=7. Forcing this demand pair to take two paths will result in multipath for
every pair, but the solution is NOT optimal.
Let’s say, x23 = 7 − ε, and x213 = ε > 0. Then, the best r becomes 0.75 + ε, which is
NOT optimal.
Objective
Feasible Region
Optimal Vertex
MPR-SPR

Science and Engineering in Network Management
Are we always swayed by our drive to get a “better”
approach?
– Engineering
MPR-SPR

Science and Engineering in Network Management
Are we always swayed by our drive to get a “better”
approach?
– Engineering
Are we forgetting to study a system as it is?
– Science
MPR-SPR

Summary
Back to the original question: Is it a Panacea?
It depends:)
For networks N ≤ 25 with all pair traffic, it’s reasonably
beneficial
The benefit of multipath routing diminishes as N increases
and L = O(N) [realistic ISP topologies]
If N ≈ 100, the benefit is quite minimal.
MPM∗ observed is much lower than theoretical MPM
The objective function is not an impacting factor
D + L result is traffic/capacity invariant.
Science of a Network Management problem is important to
investigate!
MPR-SPR

What about TCP Throughput?
TCP throughput problem is modeled as a utility
maximization problem:
max
x≥0,X≥0 d∈D
wd log Xd
subject to
p∈Pd
xdp = Xd , d ∈ D
d∈D p∈Pd
δdp xdp ≤ c , ∈ L
(7)
There are D + L constraints
We can see that the objective is non-linear concave; we
can use the piece-linear approximation trick.
That is, the D + L property holds
MPR-SPR

X. Liu, S. Mohanraj, M. Pioro, and D. Medhi, “Multipath
Routing from a Trafﬁc Engineering Perspective: How
Beneﬁcial is It?”, Proc. of 22nd IEEE International
Conference on Network Protocols (ICNP), The Research
Triangle, North Carolina, October 2014.
http://sce2.umkc.edu/csee/dmedhi/papers/
lmpm-icnp-2014.pdf
MPR-SPR

Is Multipath Routing Really a Panacea?

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Is Multipath Routing Really a Panacea?

Similar to Is Multipath Routing Really a Panacea? (20)

Recently uploaded

Recently uploaded (20)

Is Multipath Routing Really a Panacea?