10G Ethernet Overview & Use Cases

3,850 views

Published on

Overview of 10Gb Ethernet for connecting servers. Includes several use cases.

Published in: Technology, Sports
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,850
On SlideShare
0
From Embeds
0
Number of Embeds
214
Actions
Shares
0
Downloads
12
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide
  • Jun 25, 2009 BLADE Network Technologies
  • Ibm 14 servers/chassis x 18 chassis = 252 servers per row HP 16 servers/chassis x 18 chassis = 288 servers per row
  • 10G Ethernet Overview & Use Cases

    1. 1. BLADE 10G Ethernet and Use Cases
    2. 2. Why 10G Ethernet? <ul><li>Trends </li></ul><ul><ul><li>Multi-core processor technology </li></ul></ul><ul><ul><li>Converged Ethernet and FCoE </li></ul></ul><ul><ul><li>Departmental clusters/HPC </li></ul></ul><ul><ul><li>IP Storage </li></ul></ul><ul><li>10G Ethernet price points are dropping </li></ul><ul><ul><li>10G NICs </li></ul></ul><ul><ul><li>Switch is below $500 per port </li></ul></ul><ul><ul><li>Very attractive price performance </li></ul></ul><ul><ul><li>Lower OPEX and implementation costs </li></ul></ul><ul><li>Leverage existing staff and management tools </li></ul><ul><li>10G Ethernet technology has matured </li></ul><ul><ul><li>SFP+ and CX4 are now well established </li></ul></ul><ul><ul><li>Benchmark data available </li></ul></ul><ul><ul><li>Volume deployments—2009/2010 </li></ul></ul>Jun 25, 2009 www.bladenetwork.net Gigabit Ethernet 282 56.4% 10G Ethernet 0 0% Infiniband 141 28.2% Myrinet 10 2.0% Other 67 13.4% Interconnect Count Share
    3. 3. 10 Gigabit Ethernet Technology Overview <ul><li>Ratified in 2002, IEEE 802.3ae </li></ul><ul><li>Same principles of operation as 1GbE; but 10x faster and 8x improvement in latency </li></ul><ul><li>Ease of use, debug and management </li></ul><ul><li>Path to 40G and 100G </li></ul><ul><li>Operates over both copper & fiber </li></ul>10 GE Optical 10 GE Copper CX4 SFP+ DAC Leading Technologies Today Jun 25, 2009 www.bladenetwork.net XENPAK X2 XPAK XFP SFP+
    4. 4. Basic Elements of a 10 GE Network Jun 25, 2009 www.bladenetwork.net 10 GE Adapter 10 GE ‘Blade’ & Rack Switches 10 GE ‘Blade’ & Rack Servers 10G Link Server-to-Switch 10G NIC 10G Up-Link Switch-to-Switch 10/100/1000 Mbps Links Clients 1-GE Switch 10-GE Switch Server Typical 10 GE Network
    5. 5. BLADE’s RackSwitch: Purposely Designed for Data Centers Share I/O ports across any number of Blade Chassis’ or Racks, and automatically migrate VMs RackSwitch consumes 57% less power than competing solutions *Source: Tolly Group http://www.bladenetwork.net/media/PDFs/Tolly_RackS_G8100_v_C_4900M.pdf RackSwitch provides plug-n-play networking over a single converged fabric for both data and storage traffic Jun 25, 2009 www.bladenetwork.net Virtual Cooler Easier
    6. 6. SFP+ Direct Attached Cable (Twinax Copper) BLADE RackSwitch G8124 Jun 25, 2009 www.bladenetwork.net Redundant AC Power Supplies ~ 165W consumption 24-ports 10GbE SFP+ (supports 1GbE SFPs) Front-to-Rear Cooling or Rear-to-Front Redundant FANs Wire-speed ALL ports ~600 nanoseconds
    7. 7. RackSwitch Use Case 1: 1G Ethernet Rack Server Aggregation Jun 25, 2009 www.bladenetwork.net Top of Rack G8000s 1 GE Rack Servers ~2:1 Oversubscription Line-Rate 10G 10G 10G G8124 G8124 G8000s G8000s . . . . . . . . . 1 GE Rack Servers 1 GE Rack Servers Item Description Qty Spine Switches G8124 2 Leaf Switches G8000 6 Interconnect cabling SFP+ DAC 48
    8. 8. RackSwitch Use Case 2: 10G Ethernet Server Aggregation <ul><li>Most cost-effective 10GbE Top of Rack solution in the entire industry </li></ul><ul><li>20 Rackable Servers </li></ul><ul><li>Each with dual 10GbE interface </li></ul><ul><li>High uplink capacity </li></ul><ul><li>8 SFP+ 10Gb Ethernet uplink ports </li></ul><ul><li>High Availability </li></ul><ul><li>Uplink Failure detection </li></ul>Jun 25, 2009 www.bladenetwork.net Top of Rack G8124s 10 GE Rack Servers 5:1 Oversubscription Line-Rate 10G 10G 10G G8124 G8124 G8124s G8124s . . . . . . . . . 10 GE Rack Servers 10 GE Rack Servers Spine & Leaf Switches G8124 6 Interconnect Cabling SFP+ DAC 48 Item Description Qty.
    9. 9. RackSwitch Use Case 3: 10G Non-Blocking HPC Cluster (48 Nodes) <ul><li>Less than 600ns port-to-port latency </li></ul><ul><li>Logical aggregation of trunk connections for scalability and performance </li></ul><ul><li>No STP required </li></ul><ul><li>Uplink Failure Detection (UFD) enables traffic load-sharing over redundant paths </li></ul>Jun 25, 2009 www.bladenetwork.net Six 10G Links Spine Switches G8100 or 8124 Six 10G Links Leaf Switches G8100 or 8124 (12) 10GbE Ports Per Leaf Switch Spine & Leaf Switches G8124 6 Interconnect Cabling SFP+ DAC 48 Item Description Qty.
    10. 10. RackSwitch Use Case 3a: 10G Non-Blocking HPC Cluster (72+ Nodes) Spine & Leaf Switches G8100/G8124 9 Interconnect Cabling CX-4 or SFP+ DAC 72 Item Description 72-Node Qty 12 96 96-Node Qty Jun 25, 2009 www.bladenetwork.net 72-Port Fat Tree CLOS Design 96-Port Fat Tree CLOS Design G8100/ G8124 G8100/ G8124 Leaf Spine Spine Leaf (72) 10GbE Ports (96) 10GbE Ports
    11. 11. 10G Top of Rack and Blade Server Usage Scenario—Row Design www.bladenetwork.net <ul><li>Financial Data Center Row Design </li></ul><ul><li>Low Latency </li></ul><ul><ul><li>Chassis to chassis L2 connectivity </li></ul></ul><ul><li>18 Blade Server Chassis </li></ul><ul><ul><li>In a single row </li></ul></ul><ul><ul><li>2 x 1:10G Blade Switches per chassis </li></ul></ul>Jun 25, 2009
    12. 12. Blade Server Switch Aggregation <ul><li>Most cost-effective 10GbE Top of Rack Blade Server Switch aggregator </li></ul><ul><li>5 Blade Server Chassis (70-80 servers) </li></ul><ul><li>Each with 8 10GbE interfaces </li></ul><ul><li>High uplink capacity </li></ul><ul><li>8 SFP+ 10Gb Ethernet uplink ports </li></ul>Jun 25, 2009 www.bladenetwork.net
    13. 13. Low Latency HPC Solution <ul><li>6 Blade Server Systems </li></ul><ul><ul><li>16 servers/chassis </li></ul></ul><ul><ul><li>96 servers </li></ul></ul><ul><ul><li>16 10G Adapters (NICs) </li></ul></ul><ul><ul><li>2 10G Ethernet blade switches </li></ul></ul><ul><ul><li>1 Gigabit switch for management </li></ul></ul><ul><li>12 Low-Latency 10Gb 24 port Switches </li></ul><ul><li>Fits in two 19” racks </li></ul><ul><li>Non-blocking 10Gb throughput </li></ul><ul><li>End to end latency <3  s </li></ul>Jun 25, 2009 www.bladenetwork.net

    ×