Backplane Technology OverviewB k l     T h l      O    i      for AdvancedTCA  Hui Chen                       Theresienstr...
OutlineIntroduction to AdvancedTCAInterconnect ProtocolsI t        tP t     l① Ethernet② InfiniBand③ StarFabric④ PCI Expre...
What is AdvancedTCA ?• AdvancedTCA (Advanced Telecom Computing Architecture)    Standard chassis/ backplane solution for h...
Base Spec of AdvancedTCA• Board dimension     1.2 inches in width     280 mm deep, ~322 mm high• Power supply & Cooling   ...
Data Transport Interface• Base Interface: Dual Star      Supports only Ethernet• Fabric Interface: Dual Star/ Full Mesh   ...
OutlineIntroduction to AdvancedTCAInterconnect Protocols①   Ethernet②   InfiniBand③   StarFabric④   PCI Express⑤   RapidIO...
Protocol ① ― Ethernet• Application:     Initial in Local Area Network     Successful in backplane     but defective in con...
Protocol ② ― InfiniBand ⑴• Application                                                       Processor Node     Initial in...
Protocol ② ― InfiniBand ⑵• Three   link widths                                       x1  Links                x1       x4 ...
Protocol ② ― InfiniBand ⑶• Supports Quality of Service     Thru 16 Virtual Lanes (VLs)         Multiplex independent data ...
Protocol ③ ― StarFabric ⑴• Application     Backplane, chassis-to-chassis     Embedded distributed processing     (e.g. mul...
Protocol ③ ― StarFabric ⑵• Packet format• Supports Quality of Service     7 classes of traffic        asynchronous traffic...
Protocol ④ ― PCI Express ⑴• Successor to PCI     Backplane, chip-to-chip interconnect     PC (e.g. Graphics cards)• Topolo...
Protocol ④ ― PCI Express ⑵• Packet format• Supports Quality of Service     Thru     Thr 8 Traffic Classes (TCs) & 8 Virt a...
Protocol ⑤ ― RapidIO ⑴• Application     Backplane, chip-to-chip interconnect     Embedded systems (e.g. DSPs)     Networki...
Protocol ⑤ ― RapidIO ⑵• Packet format     Header: smaller than Ethernet:     1-byte source address ~ 28 Endpoints → suffic...
OutlineIntroduction to AdvancedTCAInterconnect Protocols① Ethernet② InfiniBand③ StarFabric④ PCI Express⑤ RapidIOComparison...
Comparison – FeatureFeature         GigE            InfiniBand       StarFabric    PCI Expr                               ...
Comparison – Protocol Efficiency• Protocol Efficiency = Payload / Packet size• Payload < ~500B : RapidIO, highest efficien...
Comparison – Effective Bandwidth• Effective Bandwidth = Protocol efficiency × Bandwidth• InfiniBand x4; PCIe x4; RapidIO 4...
Comparison – System Level• Quality of Service (QoS)                          GigE                          Gi E           ...
OutlineIntroduction to AdvancedTCAInterconnect Protocols① Ethernet② InfiniBand③ StarFabric④ PCI Express⑤ RapidIOComparison...
Conclusion• Low bandwidth backplane applications      Ethernet                +) well understood, low risk                ...
Thank you for your attention !               Any questions ?                                                    Special th...
Upcoming SlideShare
Loading in...5
×

Backplane Technology Overview for AdvancedTCA

1,370
-1

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,370
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Backplane Technology Overview for AdvancedTCA

  1. 1. Backplane Technology OverviewB k l T h l O i for AdvancedTCA Hui Chen Theresienstr. 90 Supervisor: Rainer Ohlendorf 80290 Munich Germany July, 13th 2007 www.lis.ei.tum.de © Institute for Integrated Systems
  2. 2. OutlineIntroduction to AdvancedTCAInterconnect ProtocolsI t tP t l① Ethernet② InfiniBand③ StarFabric④ PCI Express⑤ RapidIOComparison of Interconnect Protocols① Feature② Protocol Efficiency③ Effective Bandwidth④ System Level ConsiderationConclusionC l i Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 2 Hui Chen, July 13th 2007 Integrated Systems
  3. 3. What is AdvancedTCA ?• AdvancedTCA (Advanced Telecom Computing Architecture) Standard chassis/ backplane solution for high-availability communications equipment Adopted in 2003• AdvancedTCA Chassis Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 3 Hui Chen, July 13th 2007 Integrated Systems
  4. 4. Base Spec of AdvancedTCA• Board dimension 1.2 inches in width 280 mm deep, ~322 mm high• Power supply & Cooling -48 V power feeds 200 Watts per board Board of AdvancedTCA • Backplane architecture Zone 1: P Z 1 Power, MManagement t Zone 2: Data Transport Interface Zone 3: User-defined I/O Backplane of AdvancedTCA Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 4 Hui Chen, July 13th 2007 Integrated Systems
  5. 5. Data Transport Interface• Base Interface: Dual Star Supports only Ethernet• Fabric Interface: Dual Star/ Full Mesh Supports various protocols Max. 4 channels (a) Dual Star configuration• Dual Star Redundant switch: eliminates single point of failure Effective for large redundant systems• F ll M h Full Mesh +) Higher data throughput -) Higher pins count & cost Effective for Eff ti f small systems ll t (b) Full Mesh configuration Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 5 Hui Chen, July 13th 2007 Integrated Systems
  6. 6. OutlineIntroduction to AdvancedTCAInterconnect Protocols① Ethernet② InfiniBand③ StarFabric④ PCI Express⑤ RapidIOComparison of Interconnect Protocols① Feature② Protocol Efficiency③ Effective Bandwidth④ System Level ConsiderationConclusion Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 6 Hui Chen, July 13th 2007 Integrated Systems
  7. 7. Protocol ① ― Ethernet• Application: Initial in Local Area Network Successful in backplane but defective in congestion management• Congestion management PAUSE function: all data flows are paused when only one flow is congested → congestion spreading, when complex data p g p g, p paths• Topology P2P connections: Ethernet Switch ↔ Endpoints Ethernet Topology• Frame Header: large Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 7 Hui Chen, July 13th 2007 Integrated Systems
  8. 8. Protocol ② ― InfiniBand ⑴• Application Processor Node Initial in System Area Network now also i b k l l in backplane, chassis-to-chassis h i t h i Supercomputer, Linux cluster• Topology InfiniBand Switches + Channel Adapters HCA: Host Channel Adapter in processor node TCA: Target Channel Adapter in I/O node I/O Node HCA talks TCAs to enable remote InfiniBand Topology storage and connectivity• Packet format Larger payload size than Ethernet Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 8 Hui Chen, July 13th 2007 Integrated Systems
  9. 9. Protocol ② ― InfiniBand ⑵• Three link widths x1 Links x1 x4 x12 x4 Data Rate (Gb/s) 2 8 24 x12• Supports Quality of Service (QoS) QoS: a method to prioritize network traffic Ensures most important data gets thru the network as quickly as possible Defines traffic to many streams, streams to differentiated classes; reserves bandwidth to streams and classes Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 9 Hui Chen, July 13th 2007 Integrated Systems
  10. 10. Protocol ② ― InfiniBand ⑶• Supports Quality of Service Thru 16 Virtual Lanes (VLs) Multiplex independent data streams → the same physical link Represent a set of send & receive p buffers in a port Each port: one management VL (VL15) & up to 15 data VLs Priorities VL15 (highest) VL14 … VL0 Virtual Lanes Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 10 Hui Chen, July 13th 2007 Integrated Systems
  11. 11. Protocol ③ ― StarFabric ⑴• Application Backplane, chassis-to-chassis Embedded distributed processing (e.g. multiprocessor systems)• Topology StarFabric Switch: six-port Connects other switches/ bridges Cascadable StarFabric Bridge StarFabric Topology Translates other protocols (e.g. PCI) into StarFabric traffic St F b i t ffi• Not scalable Operates only at 2 Gb/s O t l t Gb/ Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 11 Hui Chen, July 13th 2007 Integrated Systems
  12. 12. Protocol ③ ― StarFabric ⑵• Packet format• Supports Quality of Service 7 classes of traffic asynchronous traffic (data) multicast … isochronous traffic (voice & video)• Backwards compatible to PCI Converting a serial fabric to parallel PCI Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 12 Hui Chen, July 13th 2007 Integrated Systems
  13. 13. Protocol ④ ― PCI Express ⑴• Successor to PCI Backplane, chip-to-chip interconnect PC (e.g. Graphics cards)• Topology Switches Multiple Endpoints: I/O devices Host Bridge To create an I/O hierarchy PCI Express Topology• Six link widths: Links x1 x2 x4 x8 x16 x32 Data Rate (Gb/s) 2 4 8 16 32 64 Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 13 Hui Chen, July 13th 2007 Integrated Systems
  14. 14. Protocol ④ ― PCI Express ⑵• Packet format• Supports Quality of Service Thru Thr 8 Traffic Classes (TCs) & 8 Virt al Channels (VCs) Virtual Each port : up to 8 VCs. VC0 (lowest priority), VC7 (highest) TCs: differentiated services Each E h packet a TC t determine which VC b ff t d k t TC: to d t i hi h buffer to drop packet i t k t into TC/VC mapping: ☺ TC[0:6]→VC0 TC7→VC[0:1] An Example of TC/VC mapping Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 14 Hui Chen, July 13th 2007 Integrated Systems
  15. 15. Protocol ⑤ ― RapidIO ⑴• Application Backplane, chip-to-chip interconnect Embedded systems (e.g. DSPs) Networking (e.g. Carrier Cards)•T Topology l A Switch, multiple Endpoints Large systems: one / more host processors For system exploration, initialization exploration Monitors system-level activity RapidIO Topology• Two link widths (x1 x4) (x1, Links x1 x4 Data Rate (Gb/s) 1 2 2.5 4 8 10 Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 15 Hui Chen, July 13th 2007 Integrated Systems
  16. 16. Protocol ⑤ ― RapidIO ⑵• Packet format Header: smaller than Ethernet: 1-byte source address ~ 28 Endpoints → sufficient for backplane applications Payload size: relatively small• Supports Quality of Service Six logical flows: as prioritized classes Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 16 Hui Chen, July 13th 2007 Integrated Systems
  17. 17. OutlineIntroduction to AdvancedTCAInterconnect Protocols① Ethernet② InfiniBand③ StarFabric④ PCI Express⑤ RapidIOComparison of Interconnect Protocols① Feature② Protocol Efficiency③ Effective Bandwidth④ System Level ConsiderationConclusion Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 17 Hui Chen, July 13th 2007 Integrated Systems
  18. 18. Comparison – FeatureFeature GigE InfiniBand StarFabric PCI Expr Expr. RapidIOTransport PCB, Copper & PCB, Copper & PCB & PCB, Copper &Media Fiber Fiber Optical PCB only CAT5 cableScalable Link None Yes None Yes YesWidths Encapsulated, Yes, Yes, Yes,PCI Migration None requires SW q transparent p transparent p requires SWMax Signal 1 Gbps 2 Gbps 2 Gbps 2 Gbps 2.5 GbpsChannel Rate yMax Payload 1500 B tes Bytes 4096 B t Bytes 128 B t Bytes 4096 B t Bytes 256 B tes BytesSize Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 18 Hui Chen, July 13th 2007 Integrated Systems
  19. 19. Comparison – Protocol Efficiency• Protocol Efficiency = Payload / Packet size• Payload < ~500B : RapidIO, highest efficiency smallest overhead Payload > ~500B : PCI Express, highest Express Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 19 Hui Chen, July 13th 2007 Integrated Systems
  20. 20. Comparison – Effective Bandwidth• Effective Bandwidth = Protocol efficiency × Bandwidth• InfiniBand x4; PCIe x4; RapidIO 4x 2Gbps• O Overall: Gi E l ll GigE, lowest t Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 20 Hui Chen, July 13th 2007 Integrated Systems
  21. 21. Comparison – System Level• Quality of Service (QoS) GigE Gi E InfiniBand I fi iB d StarFabric St F b i PCI E Expr. RapidIO R idIO Prioritized Traffic - 16 7 8 6 Classes•P f Performance GigE InfiniBand StarFabric PCI Expr. RapidIO Max Bandwidth for M B d idth f 1 Gbps 8 Gbps 2 Gbps 8 Gbps 10 Gbps AdvancedTCA, x4 Latency High Medium Medium Low Low (~100 us) (1.29-2.6 us) (< 3 us) (0.3-0.7 us) (< 100 ns) Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 21 Hui Chen, July 13th 2007 Integrated Systems
  22. 22. OutlineIntroduction to AdvancedTCAInterconnect Protocols① Ethernet② InfiniBand③ StarFabric④ PCI Express⑤ RapidIOComparison of Interconnect Protocols① Feature② Protocol Efficiency③ Effective Bandwidth④ System Level ConsiderationConclusion Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 22 Hui Chen, July 13th 2007 Integrated Systems
  23. 23. Conclusion• Low bandwidth backplane applications Ethernet +) well understood, low risk -) high overhead, hi h l ) hi h h d high latency -) not scalable• High bandwidth (tradeoffs of flexibility vs. latency, overhead) g ( y y, ) InfiniBand and StarFabric Complex header: +) for complex routing -) reduces effective bandwidth RapidIO Smaller packets: +) to lower latency -) overhead: a higher percentage of the packet -) reduces effective b d idth ) d ff ti bandwidth PCI Express Middle: +) design goal: software transparency -) constrains the protocol raises latency protocol, Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 23 Hui Chen, July 13th 2007 Integrated Systems
  24. 24. Thank you for your attention ! Any questions ? Special thanks to: Rainer OhlendorfHauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 24 Hui Chen, July 13th 2007 Integrated Systems
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×