Seminar on Infiniband

Uploaded on

InfiniBand is a type of communications link for data flow between processors and I/O devices that offers throughput of up to 2.5 gigabytes per second and support for up to 64,000 addressable devices. …

InfiniBand is a type of communications link for data flow between processors and I/O devices that offers throughput of up to 2.5 gigabytes per second and support for up to 64,000 addressable devices.

Presented by Shekhar Kumar of SOE,CUSAT, 200

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


    Presented By-
    Shekhar Kumar
    074, CS-’B’
    Guided By- Latha R. Nair
  • 2. Contents
    I/O Architectures- Fabric vs Bus
    Technical Overview
    InfiniBand Layers
    InfiniBand Architecture
    Advantages & Drawbacks
  • 3. Introduction
    • A switch-based serial I/O interconnect architecture
    • 4. Operates at a base speed of 2.5 Gb/s or 10 Gb/s in each direction (per port)
    • 5. Provides both QoS (Quality of Service) and RAS (Reliability, Availability and Serviceability)
    • 6. Used in high performance computing and data centres
  • I/O Architectures- Fabric vs Bus
  • 7. I/O Architectures Contd…
    Shared Bus Architecture
    System Memory
    System Controller
    I/O Controller
    PCI to PCI Bridge
    PCI to PCI Bridge
    I/O Controller
    I/O Controller
    I/O Controller
    I/O Controller
  • 8. I/O Architectures Contd…
    Switched Fabric Architecture
  • 9. Technical Overview
    Switch-based point-to-point interconnect architecture
    Each individual link is based on a four-wire 2.5Gb/s bidirectional connection.
    Architecture defines a layered hardware protocol (Physical, Link, Network, Transport Layers) as well as a software layer
    Each link can support multiple transport services for reliability
  • 10. InfiniBand Feature Set
    Layered Protocol- Physical, Link, Network, Transport, Upper Layers
    Packet based communication
    Quality Of Service
    Three link speeds
    1X- 2.5 Gb/s, 4 Wire
    4X- 10 Gb/s, 16 Wire
    12X- 30 Gb/s, 48 Wire
    PCB, Copper and Fiber cable interconnect
    Multicast and Unicast support
  • 11. InfiniBand Layers
  • 12. InfiniBand LayersContd…
    Physical Layer
    Defines both electrical and mechanical characteristics for the system
    Includes cables and receptables for fibre and copper media, backplane connectors
    Defines three link speeds, 1X,4X,12X
    Each individual link is a 4-wire differential connection that provides a full duplex connection at 2.5 Gb/s
  • 13. InfiniBand Layers Contd…
    2) Link Layer
    Encompasses packet layout, point-to-point operations, and switching within a local subnet
    Packets- Two types: Management and Data packets
    Management Packets- Used for link configuration and maintenance
    Data Packets- Carry up to 4k bytes of a transaction payload
    Switching- Devices within a subnet have a 16-bit Local ID assigned by the subnet manager. This LID is used for addressing.
  • 14. InfiniBand LayersContd…
    Link Layer
    QoS- supported through virtual lanes(VL)
    • VLs are separate logical communication links which share a single physical link
    Data Integrity
    • Two CRCs per packet, Variant CRC and Invariant CRC
    • 15. 16-bit VCRC includes all fields in the packet and is recalculated at each hop
    • 16. 32-bit ICRC covers only the fields that do not change from hop to hop
  • InfiniBand LayersContd…
    Network Layer
    Handles routing of packets from one subnet to another(within a subnet a network layer is not required)
    Packets contain a global route header(GRH)
    GRH contains the 128-bit IPv6 address
    Transport Layer
    Responsible for in-order packet delivery, channel multiplexing and transport services
    Also handles transaction data segmentation when sending, and reassembly when receiving
  • 17. InfiniBand Architecture
  • 18. InfiniBand ArchitectureContd…
    Defines multiple devices for system communication: channel adapter, switch, router, and a subnet manager
    At least one channel adapter is required for each end node
    Subnet Manager sets up and maintains link
    Channel adapters and switches contains Subnet Management Agent(SMA) for handling communication with the subnet manager
  • 19. Channel Adapters
    Connects InfiniBand to other devices
    Two types: Host Channel adapter(HCA) and Target Channel Adapter(TCA)
    HCA- Provides interface to a host device and supports all software verbs defined by InfiniBand
    TCA- Provides connection to an I/O device from InfiniBand with a subset of HCA features necessary for specific operations of each device
  • 20. Switches and Routers
  • 21. Switches and RoutersContd…
    • Contains more than one InfiniBand port and forwards packets from one of its port to another based on the LID
    • 22. Can be configured to forward either unicast packets or multicast packets
    • 23. Can consume or generate management packets
    • Forwards packets from one subnet to another without consuming or generating packets
    • 24. Reads the Global Route Header to forward the packet based on IPv6 address
  • Advantages
    Superior performance
    Fabric consolidation and low energy usage
    Reliable, stable connections
    Data Integrity
    Highly interoperable environment
  • 25. Drawbacks
    Complex in design
    Few platforms supports it as yet
    Bleeding edge, for now, so users will need to perform extensive testing
  • 26. Markets
    Application Clustering
    Inter-Processor Communication(IPC)
    Storage Area Networks
    Greater processor performance
    Supports RAS
    Works both in-the-box and enables Bandwidth Out of the Box
    Supports scalability
    InfiniBandTrade Association
    InfiniBandNetwork Architecture, MindShare, Inc. By Tom Shanley
    InfiniBand Architecture, Development and Deployment: A strategic guide to server I/O solutions By William T. Futral
    Qos in InfiniBandSubnetworks:
    Alfaro,F.J.; Sanchez, J.L.; Duato, J.; Department de Informatica, Univ. de Castilla-La Mancha, Albacete, Spain
  • 29. THANK YOU
  • 30. Queries