Hardware in clouds in commonly connected by Ethernet LAN. There are alternatives but all are much more expensive by comparison. The problem in Ethernet used to be its CSMA/CD protocol which helps detect and resolve collisions. Even though modern switches are 100\% collision-free, high-rate multiparty chatter over the shared medium creates congestion and reduced overall utilization efficiency. One way to drastically improve efficiency is to revert to the old technology of circuit switching. This paper looks into possibilities of implementing a distributed scheduler which would emulate a circuit switching environment for multiparty communications. The immediate practical application in mind is access to local shared storage in clouds, or specifically, data centers.

2. .
Setting the Mood
• "It's time to get rid of TCP/UDP protocols in DCs"
• DCs are closed worlds, new technologies are welcome
• with bulk transfers (BigData, ...), the business value of TCP/UDP
alternative is high
• circuits are an alternative to packets
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 2/24
...
2/24

3. .
Ethernet is the Best
.
Ethernet...
..
.
... is the cheapest and most available technology with e2e
support
• Fiber Channel (FC), SATA, etc. require expensive hardware, low
compatibility, no e2e support
• FCoE = Ethernet, same problems, expensive hardware, no e2e support
• network virtualization is best fit for Ethernet
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 3/24
...
3/24

4. .
Ethernet is the Worst
.
Ethernet...
..
.... is the worst technology in terms of throughput
• CSMA/CD is the biggest throughput limitation
◦ not in modern switches, but still major problem in wireless
• apart from CSMA/CD, contention cannot be easily resolved
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 4/24
...
4/24

5. .
Ethernet Contention
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 5/24
...
5/24

6. .
Ethernet and Contention
• whaterver you do, Ethernet L2 domains cannot avoid contention
Switch Switch
Qualitatively
Identical
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 6/24
...
6/24

7. .
Some Contention Control
• you can turn off your NIC, no TX, but some RX
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 7/24
...
7/24

8. .
Contention in Practice (2 flows)
20 24 28 32 36 40
Transfer time in contention (s)
20
24
28
32
36
40
Transfertimebyexclusivecircuits(s)
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 8/24
...
8/24

9. .
Switches
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 9/24
...
9/24

10. .
Switches : Basic Facts
• cut-through versus store-and-forward
• cut-through is 10 15x better
• Cisco has advanced cut-through : +bytes versus routing decision tradeoff
• store-and-forward is subjected to QoS classes
◦ L3 DSCP versus L2 CoS, AF, EF, BE, SBE models
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 10/24
...
10/24

11. .
Switches : Modeling
C: Cut Through
Check,
etc. Q: Queue
D: Drop
QoS
classes
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 11/24
...
11/24

12. .
Switches : Modeling (2)
.
Model Reading
..
.
Shorter paths have lower e2e delay
and higher throughput
C: Cut Through
Check,
etc. Q: Queue
D: Drop
QoS
classes
• Ta means throughput of path a
• TC > TSF
• TSF > TSQF
• TSQF > TSQD
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 12/24
...
12/24

13. .
Proposal
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 13/24
...
13/24

14. .
Proposal : Circuits
.
Circuits
..
.
... are emulations which allow for exclusive access to L2 domain by
individual parties
• circuits-over-packets emulation
• cut-through mode for each circuit is guaranteed
• highest possible throughput
• NOTE: will work with cheepest switches
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 14/24
...
14/24

15. .
Basic Math
• circuits
R = C
• packets
R = f
(wC
kV
)
• C capacity, R throughput/rate, w
weight, k parallel count, V bulk in bytes,
d e2e delay, pL loss rate
• packets: w = V case is the
upper bound for throughput
• packets: in reality, upper bound is
never achieved
• circuits: R = C is achievable locally
(DC intranet)
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 15/24
...
15/24

16. .
Trivial Optimization
• special case: traffic engineering + scheduling
• time shifts are optimized, bulk is known in advance
• implementation: GA, works well in small (DC-level) networks
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 16/24
...
16/24

17. .
Non-Trivial Optimization
Bulk size per transmission
Goodput
2 potential distributions in practice
Response
curve(s)
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 17/24
...
17/24

18. .
Implementation
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 18/24
...
18/24

19. .
Implementation : 2 cases
• left: book-then-send, right: separate control layer
SWITCH
NOC
Storage
Node A
Storage
Node B
Step 1:
Book
session
Step 2:
Transfer
bulk
SWITCH
Storage
Node A
Storage
Node B
SWITCH
Booking
segment
Bulk
Segment
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 19/24
...
19/24

20. .
Impl.: Centralized Case
SWITCH
NOC
Storage
Node A
Storage
Node B
Step 1:
Book
session
Step 2:
Transfer
bulk
• same network for booking and
circuits
• inefficient but still valid/practical
• legacy-compatible,
partial implementation, etc.
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 20/24
...
20/24

21. .
Impl.: Distributed Case
SWITCH
Storage
Node A
Storage
Node B
SWITCH
Booking
segment
Bulk
Segment
• book on one network, send on another
• legacy-incompatible
• contention-sensing possible →
fully distributed models
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 21/24
...
21/24

22. .
Implementation : Details
• working prototype of legacy-compatible system is ready
• using for backup, transfers of VM images, etc.
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 22/24
...
22/24

23. .
Future FollowUps
• Goal 1: non-legacy, sensing implementation
• Goal 2: optimizations in DC-size 6-10 hop networks
• Goal 3: SDN package with PICO8 switches
• Goal 4: Advanced/Curve Tradeoff Optimization models
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 23/24
...
23/24

24. .
That’s all, thank you ...
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 24/24
...
24/24

25. .
[01] myself (2014)
High Availability Cloud Storage...
NS研
[02] Cisco (2014)
LAN Switching and Wireless, CCNA Exploration Companion Guide
Cisco Press
[03] Cisco (2014)
Cut-Through and Store-and-Forward Ethernet Switching for Low-Latency....
Cisco Press
[04] NetOptics (2014)
Cut-Through Ethernet Switching: A Versatile Resource for Low Latency...
White Paper
[05] Cisco (2006)
QoS: DSCP Classification Guidelines
RFC4594
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 24/24
...
24/24

26. .
[06] Cisco (2010)
A Differentiated Services Code Point (DSCP)...
RFC5865
[07] open source (current)
PICA8 Project for Low Latency Virtual Networking
http://www.pica8.com/
M.Zhanikeev -- maratishe@gmail.com -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 24/24
...
24/24