The document presents an overview of InfiniBand, a switch-based serial I/O interconnect architecture that operates at varying speeds of 2.5, 10, and 30 Gb/s while providing features like QoS and RAS. It details the architecture's layered protocol design, including the physical, link, network, and transport layers, as well as its advantages like low-latency and high efficiency, and drawbacks such as complexity and limited support. InfiniBand is positioned for markets such as application clustering, inter-processor communication, and storage area networks.

1. INFINIBAND“Bandwidth Out Of The Box”Presented By- Shekhar Kumar074, CS-’B’Guided By- Latha S. Nair

2. ContentsIntroductionI/O Architectures- Fabric vs BusTechnical OverviewInfiniBand LayersInfiniBand ArchitectureAdvantages & DrawbacksMarketsConclusionReferences

3. IntroductionA switch-based serial I/O interconnect architecture

4. Operates at a base speed of 2.5 Gb/s or 10 Gb/s in each direction (per port)

5. Provides both QoS (Quality of Service) and RAS (Reliability, Availability and Serviceability)

6. Superset of the Virtual Interface Architecture

7. Used in high performance computing and data centresI/O Architectures- Fabric vs Bus

8. I/O Architectures Contd…Shared Bus ArchitectureCPUSystem MemorySystem ControllerI/O ControllerPCI to PCI BridgePCI to PCI BridgeI/O ControllerI/O ControllerI/O ControllerI/O Controller

9. I/O Architectures Contd…Switched Fabric Architecture

10. Technical OverviewSwitch-based point-to-point interconnect architectureEach individual link is based on a four-wire 2.5Gb/s bidirectional connection.Architecture defines a layered hardware protocol (Physical, Link, Network, Transport Layers) as well as a software layerEach link can support multiple transport services for reliability

11. InfiniBand Feature SetLayered Protocol- Physical, Link, Network, Transport, Upper LayersPacket based communicationQuality Of ServiceThree link speeds1X- 2.5 Gb/s, 4 Wire4X- 10 Gb/s, 16 Wire12X- 30 Gb/s, 48 WirePCB, Copper and Fiber cable interconnectRemote DMA supportMulticast and Unicast support

12. InfiniBand Layers

13. InfiniBand LayersContd…Physical LayerDefines both electrical and mechanical characteristics for the systemIncludes cables and receptables for fibre and copper media, backplane connectorsDefines three link speeds, 1X,4X,12XEach individual link is a 4-wire differential connection that provides a full duplex connection at 2.5 Gb/s

14. InfiniBand Layers Contd…2) Link LayerEncompasses packet layout, point-to-point operations, and switching within a local subnetPackets- Two types: Management and Data packetsManagement Packets- Used for link configuration and maintenanceData Packets- Carry up to 4k bytes of a transaction payloadSwitching- Devices within a subnet have a 16-bit Local ID assigned by the subnet manager. This LID is used for addressing.

15. InfiniBand LayersContd…Link LayerQoS- supported through virtual lanes(VL) VLs are separate logical communication links which share a single physical linkData IntegrityTwo CRCs per packet, Variant CRC and Invariant CRC

16. 16-bit VCRC includes all fields in the packet and is recalculated at each hop

17. 32-bit ICRC covers only the fields that do not change from hop to hopInfiniBand LayersContd…Network LayerHandles routing of packets from one subnet to another(within a subnet a network layer is not required)Packets contain a global route header(GRH)GRH contains the 128-bit IPv6 addressTransport LayerResponsible for in-order packet delivery, channel multiplexing and transport servicesAlso handles transaction data segmentation when sending, and reassembly when receiving

18. InfiniBand Architecture

19. InfiniBand ArchitectureContd…Defines multiple devices for system communication: channel adapter, switch, router, and a subnet managerAt least one channel adapter is required for each end nodeSubnet Manager sets up and maintains linkChannel adapters and switches contains Subnet Management Agent(SMA) for handling communication with the subnet manager

20. Channel AdaptersConnects InfiniBand to other devicesTwo types: Host Channel adapter(HCA) and Target Channel Adapter(TCA)HCA- Provides interface to a host device and supports all software verbs defined by InfiniBandTCA- Provides connection to an I/O device from InfiniBand with a subset of HCA features necessary for specific operations of each device

21. Switches and Routers

22. Switches and RoutersContd…SwitchContains more than one InfiniBand port and forwards packets from one of its port to another based on the LID

23. Can be configured to forward either unicast packets or multicast packets

24. Can consume or generate management packetsRouterForwards packets from one subnet to another without consuming or generating packets

25. Reads the Global Route Header to forward the packet based on IPv6 addressAdvantagesSuperior performanceLow-latencyHigh-efficiencyFabric consolidation and low energy usageReliable, stable connectionsData IntegrityHighly interoperable environment