We consider a distributed Software Defined Net- working (SDN) architecture adopting a cluster of multiple controllers to improve network performance and reliability. Differently from previous work, we focus on the control traffic exchanged among the controllers, in addition to the Openflow control traffic exchanged between controllers and switches. We develop an analytical model to estimate the reaction time per- ceived at the switches due to the inter-controller communications, based on the data-ownership model adopted in the cluster. We advocate a careful placement of the controllers, taking into account the two above kinds of control traffic. We evaluate, for some real ISP network topologies, the possible delay tradeoffs for the controllers placement problem.

1. The Role of Inter-Controller Traffic in
SDN Controllers Placement
Tianzhu Zhang, Andrea Bianco, Paolo Giaccone
IEEE NFV-SDN: Palo Alto, CA
November 8th, 2016 1

2. Software Defined Networking
• programmability
• enabled by centralized view of the
network state
Advantages
• scalability for large networks
One relevant challenge
2

3. Distributed controllers
• fault-tolerance and resilience
• load balancing, thus higher scalability
Motivation
• how the network state is distributed across
the controllers to allow a centralized logical
view
Critical issue
3

4. Control-plane in distributed controllers
• in-band for large networks (e.g. SD-WAN)
• switch-controller (Sw-Ctr) traffic
– standard protocols (e.g. OpenFlow)
• controller-controller (Ctr-Ctr) traffic
4
Switch-controller
traffic
Inter-controller traffic

5. Inter-controller (Crt-Ctr) traffic
• needed to synchronize the controllers’ states
– shared data structures
• generated by consistency protocols
– ad-hoc protocols
– different models for consistency
5

6. Consistency models
• assume a shared table=(key,value)
• strong consistency
– any read(key) returns always the same value
• eventual consistency
– any read(key) returns eventually the same value
• adopted model affects heavily
– the mechanisms to distribute and update the data
– the reactivity of the SDN controllers perceived by the
network devices
– the correct behavior of the network
• e.g. inconsistent view of the network graph may lead to
routing loops
6

7. Consistency in SDN controllers
• eventual consistency
• anti-entropy algorithm
• network topology
• flow rules
• flow statistics
• strong consistency
• RAFT consensus algorithm
• switch-controller mapping
• distributed locks
ONOS
• strong consistency
• RAFT consensus algorithm
• all shared data structures
OpenDaylight
7

8. Data-ownership model
• Consistency model affects that controller owner of the
shared data
8
• Read/write operations always forwarded by the
local controller to the data-owner controller
• distributed architecture only for high availability
• e.g. strong consistent data structures
Single data-ownership (SDO) model
• Read/write operations are local and then
forwarded (asynchronously) to the other
controllers
• e.g. eventual consistent data structures
Multiple data-ownership (MDO) model

9. Our main contributions
• we provide formulas to estimate the reactivity
perceived at the switch due to the Sw-Ctr and
Ctr-Ctr interactions
• we show the performance tradeoffs for the
controller placement problem
9

10. Reactivity for Multi Data-Ownership
Switch S1
ResponseUpdate
data
Flood
update
Data owner
controller
TR
TR = Sw-Ctr RTT
Data owner
controller
Data owner
controller

11. Reactivity for Single Data-Ownership
Data owner
controller
Switch S1
Raft
request
Log
replicatiom
ResponseUpdate
data
Log
reply
Log commit
Controller
Controller
Hp: RAFT algorithm for
strong consistency
TR
TR = Sw-Ctr RTT+2 Ctr-Ctr RTT

12. Optimal controller placements
13
0
10
20
30
40
50
60
70
050100150200250300350400
Reaction time for MDO [ms]
ReactiontimeforSDO[ms]
OptimalSDOplacement
OptimalMDOplacement
3 controllers in large ISP
For Single Data-Ownership (SDO), 3 possible data owners

13. The role of data owner in SDO
Choosing carefully the data owner is very important for the
reactivity 14
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0100200300400500600700800900
Reduction factor
Placementid
Maximum
Minimum
Minimum reduction factor =
Reactivity (2nd best data owner choice)
Reactivity (best data owner choice)
Maximum reduction factor =
Reactivity (worst data owner choice)
Reactivity (best data owner choice)

14. Conclusions
• single/multiple data-ownership models adopted
by real SDN controllers affect strongly the
performance perceived at the network devices
• reactivity formulas allow to devise the optimal
placement of the controllers
– the latencies among controllers are crucial in SDO
model
• in the SDO model the choice of the data owner is
crucial
• accurate validation of the given formulas in a
large SD-WAN controlled by OpenDaylight
– available on extended version [1] on arxiv
15