Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Backplane Technology Overview for AdvancedTCA

1,799 views

Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

Backplane Technology Overview for AdvancedTCA

  1. 1. Backplane Technology OverviewB k l T h l O i for AdvancedTCA Hui Chen Theresienstr. 90 Supervisor: Rainer Ohlendorf 80290 Munich Germany July, 13th 2007 www.lis.ei.tum.de © Institute for Integrated Systems
  2. 2. OutlineIntroduction to AdvancedTCAInterconnect ProtocolsI t tP t l① Ethernet② InfiniBand③ StarFabric④ PCI Express⑤ RapidIOComparison of Interconnect Protocols① Feature② Protocol Efficiency③ Effective Bandwidth④ System Level ConsiderationConclusionC l i Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 2 Hui Chen, July 13th 2007 Integrated Systems
  3. 3. What is AdvancedTCA ?• AdvancedTCA (Advanced Telecom Computing Architecture) Standard chassis/ backplane solution for high-availability communications equipment Adopted in 2003• AdvancedTCA Chassis Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 3 Hui Chen, July 13th 2007 Integrated Systems
  4. 4. Base Spec of AdvancedTCA• Board dimension 1.2 inches in width 280 mm deep, ~322 mm high• Power supply & Cooling -48 V power feeds 200 Watts per board Board of AdvancedTCA • Backplane architecture Zone 1: P Z 1 Power, MManagement t Zone 2: Data Transport Interface Zone 3: User-defined I/O Backplane of AdvancedTCA Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 4 Hui Chen, July 13th 2007 Integrated Systems
  5. 5. Data Transport Interface• Base Interface: Dual Star Supports only Ethernet• Fabric Interface: Dual Star/ Full Mesh Supports various protocols Max. 4 channels (a) Dual Star configuration• Dual Star Redundant switch: eliminates single point of failure Effective for large redundant systems• F ll M h Full Mesh +) Higher data throughput -) Higher pins count & cost Effective for Eff ti f small systems ll t (b) Full Mesh configuration Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 5 Hui Chen, July 13th 2007 Integrated Systems
  6. 6. OutlineIntroduction to AdvancedTCAInterconnect Protocols① Ethernet② InfiniBand③ StarFabric④ PCI Express⑤ RapidIOComparison of Interconnect Protocols① Feature② Protocol Efficiency③ Effective Bandwidth④ System Level ConsiderationConclusion Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 6 Hui Chen, July 13th 2007 Integrated Systems
  7. 7. Protocol ① ― Ethernet• Application: Initial in Local Area Network Successful in backplane but defective in congestion management• Congestion management PAUSE function: all data flows are paused when only one flow is congested → congestion spreading, when complex data p g p g, p paths• Topology P2P connections: Ethernet Switch ↔ Endpoints Ethernet Topology• Frame Header: large Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 7 Hui Chen, July 13th 2007 Integrated Systems
  8. 8. Protocol ② ― InfiniBand ⑴• Application Processor Node Initial in System Area Network now also i b k l l in backplane, chassis-to-chassis h i t h i Supercomputer, Linux cluster• Topology InfiniBand Switches + Channel Adapters HCA: Host Channel Adapter in processor node TCA: Target Channel Adapter in I/O node I/O Node HCA talks TCAs to enable remote InfiniBand Topology storage and connectivity• Packet format Larger payload size than Ethernet Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 8 Hui Chen, July 13th 2007 Integrated Systems
  9. 9. Protocol ② ― InfiniBand ⑵• Three link widths x1 Links x1 x4 x12 x4 Data Rate (Gb/s) 2 8 24 x12• Supports Quality of Service (QoS) QoS: a method to prioritize network traffic Ensures most important data gets thru the network as quickly as possible Defines traffic to many streams, streams to differentiated classes; reserves bandwidth to streams and classes Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 9 Hui Chen, July 13th 2007 Integrated Systems
  10. 10. Protocol ② ― InfiniBand ⑶• Supports Quality of Service Thru 16 Virtual Lanes (VLs) Multiplex independent data streams → the same physical link Represent a set of send & receive p buffers in a port Each port: one management VL (VL15) & up to 15 data VLs Priorities VL15 (highest) VL14 … VL0 Virtual Lanes Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 10 Hui Chen, July 13th 2007 Integrated Systems
  11. 11. Protocol ③ ― StarFabric ⑴• Application Backplane, chassis-to-chassis Embedded distributed processing (e.g. multiprocessor systems)• Topology StarFabric Switch: six-port Connects other switches/ bridges Cascadable StarFabric Bridge StarFabric Topology Translates other protocols (e.g. PCI) into StarFabric traffic St F b i t ffi• Not scalable Operates only at 2 Gb/s O t l t Gb/ Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 11 Hui Chen, July 13th 2007 Integrated Systems
  12. 12. Protocol ③ ― StarFabric ⑵• Packet format• Supports Quality of Service 7 classes of traffic asynchronous traffic (data) multicast … isochronous traffic (voice & video)• Backwards compatible to PCI Converting a serial fabric to parallel PCI Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 12 Hui Chen, July 13th 2007 Integrated Systems
  13. 13. Protocol ④ ― PCI Express ⑴• Successor to PCI Backplane, chip-to-chip interconnect PC (e.g. Graphics cards)• Topology Switches Multiple Endpoints: I/O devices Host Bridge To create an I/O hierarchy PCI Express Topology• Six link widths: Links x1 x2 x4 x8 x16 x32 Data Rate (Gb/s) 2 4 8 16 32 64 Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 13 Hui Chen, July 13th 2007 Integrated Systems
  14. 14. Protocol ④ ― PCI Express ⑵• Packet format• Supports Quality of Service Thru Thr 8 Traffic Classes (TCs) & 8 Virt al Channels (VCs) Virtual Each port : up to 8 VCs. VC0 (lowest priority), VC7 (highest) TCs: differentiated services Each E h packet a TC t determine which VC b ff t d k t TC: to d t i hi h buffer to drop packet i t k t into TC/VC mapping: ☺ TC[0:6]→VC0 TC7→VC[0:1] An Example of TC/VC mapping Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 14 Hui Chen, July 13th 2007 Integrated Systems
  15. 15. Protocol ⑤ ― RapidIO ⑴• Application Backplane, chip-to-chip interconnect Embedded systems (e.g. DSPs) Networking (e.g. Carrier Cards)•T Topology l A Switch, multiple Endpoints Large systems: one / more host processors For system exploration, initialization exploration Monitors system-level activity RapidIO Topology• Two link widths (x1 x4) (x1, Links x1 x4 Data Rate (Gb/s) 1 2 2.5 4 8 10 Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 15 Hui Chen, July 13th 2007 Integrated Systems
  16. 16. Protocol ⑤ ― RapidIO ⑵• Packet format Header: smaller than Ethernet: 1-byte source address ~ 28 Endpoints → sufficient for backplane applications Payload size: relatively small• Supports Quality of Service Six logical flows: as prioritized classes Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 16 Hui Chen, July 13th 2007 Integrated Systems
  17. 17. OutlineIntroduction to AdvancedTCAInterconnect Protocols① Ethernet② InfiniBand③ StarFabric④ PCI Express⑤ RapidIOComparison of Interconnect Protocols① Feature② Protocol Efficiency③ Effective Bandwidth④ System Level ConsiderationConclusion Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 17 Hui Chen, July 13th 2007 Integrated Systems
  18. 18. Comparison – FeatureFeature GigE InfiniBand StarFabric PCI Expr Expr. RapidIOTransport PCB, Copper & PCB, Copper & PCB & PCB, Copper &Media Fiber Fiber Optical PCB only CAT5 cableScalable Link None Yes None Yes YesWidths Encapsulated, Yes, Yes, Yes,PCI Migration None requires SW q transparent p transparent p requires SWMax Signal 1 Gbps 2 Gbps 2 Gbps 2 Gbps 2.5 GbpsChannel Rate yMax Payload 1500 B tes Bytes 4096 B t Bytes 128 B t Bytes 4096 B t Bytes 256 B tes BytesSize Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 18 Hui Chen, July 13th 2007 Integrated Systems
  19. 19. Comparison – Protocol Efficiency• Protocol Efficiency = Payload / Packet size• Payload < ~500B : RapidIO, highest efficiency smallest overhead Payload > ~500B : PCI Express, highest Express Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 19 Hui Chen, July 13th 2007 Integrated Systems
  20. 20. Comparison – Effective Bandwidth• Effective Bandwidth = Protocol efficiency × Bandwidth• InfiniBand x4; PCIe x4; RapidIO 4x 2Gbps• O Overall: Gi E l ll GigE, lowest t Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 20 Hui Chen, July 13th 2007 Integrated Systems
  21. 21. Comparison – System Level• Quality of Service (QoS) GigE Gi E InfiniBand I fi iB d StarFabric St F b i PCI E Expr. RapidIO R idIO Prioritized Traffic - 16 7 8 6 Classes•P f Performance GigE InfiniBand StarFabric PCI Expr. RapidIO Max Bandwidth for M B d idth f 1 Gbps 8 Gbps 2 Gbps 8 Gbps 10 Gbps AdvancedTCA, x4 Latency High Medium Medium Low Low (~100 us) (1.29-2.6 us) (< 3 us) (0.3-0.7 us) (< 100 ns) Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 21 Hui Chen, July 13th 2007 Integrated Systems
  22. 22. OutlineIntroduction to AdvancedTCAInterconnect Protocols① Ethernet② InfiniBand③ StarFabric④ PCI Express⑤ RapidIOComparison of Interconnect Protocols① Feature② Protocol Efficiency③ Effective Bandwidth④ System Level ConsiderationConclusion Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 22 Hui Chen, July 13th 2007 Integrated Systems
  23. 23. Conclusion• Low bandwidth backplane applications Ethernet +) well understood, low risk -) high overhead, hi h l ) hi h h d high latency -) not scalable• High bandwidth (tradeoffs of flexibility vs. latency, overhead) g ( y y, ) InfiniBand and StarFabric Complex header: +) for complex routing -) reduces effective bandwidth RapidIO Smaller packets: +) to lower latency -) overhead: a higher percentage of the packet -) reduces effective b d idth ) d ff ti bandwidth PCI Express Middle: +) design goal: software transparency -) constrains the protocol raises latency protocol, Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 23 Hui Chen, July 13th 2007 Integrated Systems
  24. 24. Thank you for your attention ! Any questions ? Special thanks to: Rainer OhlendorfHauptseminar am Lehrstuhl für Integrierte Systeme © Institute for 24 Hui Chen, July 13th 2007 Integrated Systems

×