This document summarizes a presentation on data center network architectures. It begins with an analysis of different data center architectures including definitions, classifications of architectures as electronic/optical and switch-centric/server-centric. Example architectures like Fat-Tree, BCube and DCell are described. Metrics for evaluating energy consumption and communication latency are presented. Results show different architectures impact energy costs and communication differently. Current scheduling approaches are discussed and directions for communication-aware scheduling criteria are proposed.

1. University of Luxembourg
Data center architectures and their relevance in
cloud processes
Claudio Fiandrino
claudio.fiandrino@uni.lu
December 18, 2014

2. Agenda
Analysis of data center architectures
Communication-aware scheduling criteria
Claudio Fiandrino | Agenda - 1 of 17

3. Outline
Analysis of data center architectures
Definition and Classifications
Architectures Analysis
Energy Consumption and Communication Latency Evaluation
Communication-aware scheduling criteria
Claudio Fiandrino | Analysis of data center architectures - Definition and Classifications 1 of 17

4. Data Center: Definitions
Data Center Infrastructure
The data center is home to the computational power, storage,
management and dissemination of data and information necessary to a
particular body of knowledge or pertaining to a particular business.
Data Center Network Architecture
The data center network architecture is the set of network nodes and
links that characterize the interconnectivity among the computing
servers and to the external world.
Claudio Fiandrino | Analysis of data center architectures - Definition and Classifications 2 of 17

5. A brief list of Architectures
Fat-Tree
“Al-Fares”
Portland
Hedera
Clos (VL2)
Hypercube
BCube
DCell
DPillar
FiConn
FlatNet
Helios
C-Through
Petabit
GreenCloud (Not the Simulator)
[1] Ali Hammadi, Lotfi Mhamdi, A survey on architectures and energy efficiency in Data Center
Networks, Computer Communications, Volume 40, 1 March 2014, Pages 1-21, ISSN 0140-3664,
http://dx.doi.org/10.1016/j.comcom.2013.11.005.
(http://www.sciencedirect.com/science/article/pii/S0140366413002727)
[2] Bari, M.F.; Boutaba, R.; Esteves, R.; Granville, L.Z.; Podlesny, M.; Rabbani, M.G.; Qi Zhang; Zhani,
M.F., “Data Center Network Virtualization: A Survey," IEEE Communications Surveys & Tutorials, vol.15,
no.2, pp.909,928, Second Quarter 2013 doi: 10.1109/SURV.2012.090512.00043
Claudio Fiandrino | Analysis of data center architectures - Definition and Classifications 3 of 17

6. Classifications
Electronic/Optical
Which technology is used in the forwarding?
Switch/Server Centric
Who performs the forwarding?
Ali Hammadi, Lotfi Mhamdi, A survey on architectures and energy efficiency in Data Center Net-
works, Computer Communications, Volume 40, 1 March 2014, Pages 1-21, ISSN 0140-3664,
http://dx.doi.org/10.1016/j.comcom.2013.11.005.
(http://www.sciencedirect.com/science/article/pii/S0140366413002727)
Claudio Fiandrino | Analysis of data center architectures - Definition and Classifications 4 of 17

7. Electronic vs Optical
Fully electronic
Fully optical
Hybrid
Claudio Fiandrino | Analysis of data center architectures - Definition and Classifications 5 of 17

8. Swich Centric vs Server Centric
Switch Centric: switches are the key components in forwarding
Switches are complex
Servers no forwarding functionalities
Bandwidth oversubscription
Server Centric: servers are the key components in forwarding
Switches are very simple
Servers with forwarding functionalities
No bandwidth oversubscription
Trade off
Servers devoted to computational purposes only
Exploiting full link capacity
Claudio Fiandrino | Analysis of data center architectures - Definition and Classifications 6 of 17

9. Outline
Analysis of data center architectures
Definition and Classifications
Architectures Analysis
Energy Consumption and Communication Latency Evaluation
Communication-aware scheduling criteria
Claudio Fiandrino | Analysis of data center architectures - Architectures Analysis 6 of 17

10. Three-tier
Hierarchical structure
Bandwidth oversubscription
Computing Servers
Access Layer
Aggregation Layer
Core Layer
Gateway Router
Internet
Claudio Fiandrino | Analysis of data center architectures - Architectures Analysis 7 of 17

11. Bandwidth oversubscription
Typical example:
Computing Servers
Access Layer
Aggregation Layer
Core Layer
48 1 Gbps links
2 10 Gbps links
12 10 Gbps links
8 10 Gbps links
Access level:
2.4 : 1
B = 416 Mbps
Aggregation level:
1.5 : 1
B = 277 Mbps
Claudio Fiandrino | Analysis of data center architectures - Architectures Analysis 8 of 17

12. BCube
Main parameters:
n: number of ports per switch
k + 1: number of ports per server
Number of servers: nk+1
Number of switches: (k + 1) · nk
Example with n = 4 and k = 1
Internet
Computing Servers
Commodity Switches
Level 0
Commodity Switches
Level k + 1
Load Balancers
Gateway Router
Claudio Fiandrino | Analysis of data center architectures - Architectures Analysis 9 of 17

13. DCell
Main parameters:
n: number of servers per DCell0
k: number of DCell levels, server
degree of connectivity (k + 1)
Number of servers:
tj+1 = tj · (tj + 1), t0 = n
Number of switches (#DCell0):
gj = tj−1 + 1 or #servers/n
Example with n = 3 and k = 1
Internet
Computing Servers
Commodity Switches
Load Balancers
Gateway Router
Claudio Fiandrino | Analysis of data center architectures - Architectures Analysis 10 of 17

14. Outline
Analysis of data center architectures
Definition and Classifications
Architectures Analysis
Energy Consumption and Communication Latency Evaluation
Communication-aware scheduling criteria
Claudio Fiandrino | Analysis of data center architectures - Energy Consumption and Communication Latency Evaluation 10 of 17

15. Evaluation Criteria: Computing Servers
Power consumption (DVFS)
P(l) = Pidle +
Ppeak − Pidle
2
· (1 + l − e−( l
τ ))
l server load
τ in the range [0.5, 0.8]
Key idea:
Load= 0
Servers
Access Layer
Aggregation Layer
Core Layer
→ →
Increasing Load
→ →
Load= 1
Idle link; Active link; Idle device; Active device;
Number of computing servers: 4096
Idle and peak power from Dell PowerEdge R720, Huawei Tecal
RH2288H V2 and IBM System x3500 M4
Claudio Fiandrino | Analysis of data center architectures - Energy Consumption and Communication Latency Evaluation 11 of 17

16. Energy: Results
0 0.2 0.4 0.6 0.8 1
0
1
2
3
4
·105
Load (l)
PowerConsumption(W)
Dcell Three-tier BCube
Claudio Fiandrino | Analysis of data center architectures - Energy Consumption and Communication Latency Evaluation 12 of 17

17. Evaluation Criteria: Network
Three-tier:
128 racks
16 aggregation switches
8 core switches
BCube:
n = 8, k = 3
2048 switches
DCell:
n = 8
2 < k < 3
Other parameters:
1 Gbits link (Three-Tier, BCube and
DCell)
10 Gbits link (Three-Tier)
Test packets of 40 B and 1500 B
Methodology
One way delay
Transmission and Propagation
Claudio Fiandrino | Analysis of data center architectures - Energy Consumption and Communication Latency Evaluation 13 of 17

18. Communication: Results
PERFORMANCE INDEX
ARCHITECTURES
Three-tier BCube DCell
Latency (40 B) 1.98 µs 3.93 µs 4.73 µs
Latency (1500 B) 28.34 µs 73.72 µs 93.92 µs
Hop distance 5.78 7.00 8.94
Server Degree Connectivity 1 4 2.79
Beware: we counted as 2 the number of links between any pair of servers within a DCell0 and in BCube.
Claudio Fiandrino | Analysis of data center architectures - Energy Consumption and Communication Latency Evaluation 14 of 17

19. Take-home messages
Data Center Network Architectures
Different Architectures have:
Different impact on energy costs
Different impact on communication processes
Claudio Fiandrino | Analysis of data center architectures - Energy Consumption and Communication Latency Evaluation 15 of 17

20. Outline
Analysis of data center architectures
Communication-aware scheduling criteria
Current state of the art
Design directions
Claudio Fiandrino | Communication-aware scheduling criteria - Current state of the art 15 of 17

21. Objectives of existing solutions
Communication-unaware
policies
Job distribution
Energy efficiency
Thermal efficiency
Communication-aware
policies
Load balancing
Energy efficiency
[1] D. Kliazovich, P. Bouvry, and Samee U. Khan, DENS: Data Center Energy-Efficient Network-Aware
Scheduling Cluster Computing, special issue on Green Networks, vol. 16, no. 1, pp. 65-75, 2013
Claudio Fiandrino | Communication-aware scheduling criteria - Current state of the art 16 of 17

22. Outline
Analysis of data center architectures
Communication-aware scheduling criteria
Current state of the art
Design directions
Claudio Fiandrino | Communication-aware scheduling criteria - Design directions 16 of 17

23. What to consider?
Claudio Fiandrino | Communication-aware scheduling criteria - Design directions 17 of 17

24. Thank You!Thank You!Thank You!