In this deck from the Switzerland HPC Conference, Jeffrey Stuecheli from IBM presents: Open CAPI, A New Standard for High Performance Attachment of Memory, Acceleration, and Networks. "OpenCAPI sets a new standard for the industry, providing a high bandwidth, low latency open interface design specification. This session will introduce the new standard and it's goals. This includes details on how the interface protocol provides unprecedented latency and bandwidth to attached devices." Watch the video: http://wp.me/p3RLHQ-gDZ Learn more: http://opencapi.org/ and http://hpcadvisorycouncil.com/events/2017/swiss-workshop/agenda.php Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

1. OpenCAPI
TM
- A New Standard for High Performance
Memory, Acceleration and Networks
Jeff Stuecheli
April 10, 2017
Open Coherent Accelerator Processor Interface

2. What is Open CAPI
• Device attach
• Memory
• Accelerators
• Network
• Storage
• Etc.
• Latency: 10s of ns interface overhead
• Bandwidth: 25G+ differential signaling
• Flexibility: One interface scaling low latency memory to sophisticated
accelerators

3. Use Cases – A truly heterogeneous architecture built upon OpenCAPI

4. Asymmetric Design Philosophy
• Motivation
• Simplify accelerator to enable
1. Host ISA agnostic
2. Contain coherence complexity to host silicon
3. Higher performance as logic in host silicon higher perf than accelerator
4. Contain accelerator in sandbox to enable fault tolerance and security

5. Importance of Latency
• Server memory latency is critical TOC factor
• Differential solution must provide ~equivalent effective latency of DDR
standards
• POWER8 DMI round trip latency à 10ns
• Typical PCIe round trip latency à ~100s ns
• Why is DMI so low?
• DMI designed from ground up for minimum latency due to ld/str
requirements
• Open CAPI key concept
• Provide DMI like latency, but with enhanced command set of CAPI

6. Comparison of Acceleration Paradigms
7
Examples: Encryption, Compression, Erasure prior to
network or storage
Processor Chip
Acc
Data
Egress Transform
DLx/TLx
Processor Chip
Acc
Data
Bi-Directional Transform
Acc
TLx/DLx
Examples: NoSQL such as Neo4J with Graph Node Traversals
etc
Needle-in-a-haystack Engine
Examples: Machine Learning, Deep Learning potentially
using OpenCAPI attached memory
Memory Transform
Processor Chip
Acc
DataDLx/TLx
Example: Basic offload
Processor Chip
Acc
NeedlesDLx/TLx
Examples: Database searches, joins,
intersections,merges
Ingress Transform
Processor Chip
Acc
DataDLx/TLx
Examples: Video Analytics, HFT, VPN/IPsec/SSL, Deep Packet
Inspection (DPI), Data Plane Accelerator (DPA), Video Encoding (H.265) etc
Needle-In-A-Haystack Engine
Haystack
Data

7. Comparison of Memory Paradigms
8
Needle-in-a-haystack Engine
Main Memory
Processor Chip
DDR4/5
DataDLx/TLx
Example: Basic DDR attach
Processor Chip
DLx/TLx
Emerging Memory
Processor Chip DataDLx/TLx
Tiered Memory
SCMDDR4/5
DataDLx/TLx
SCM

8. ‘zero’ Cycle DDR4/5 buffer chip strawman
serdes
DDR
serdes
DDR
25.6 GHz
1.6 GHz
3.2 GHz
Serdes 8:1
DDR phy
Bypass
Activate
decode
Bypass
Data
DDR

9. Simplified High Performance Accelerator
• Explicit command templates enable extremely simple command
decode
• For best latency CRC bypass
• Virtual address based cache enables simplified parallel accelerator
caching structures based around data structure semantics (rather
than typical Von Neumann bottleneck)

10. Host scalability
• Asymmetric design isolated host coherence throughput from
accelerator (avoid scenario of plugging a card or flashing an FPGA
image slowing entire system).
• Ground up high throughput design with explicit memory barriers
enables efficient host implementation (rather than overly strict PCIe).
• Open CAPI optimized serdes significant power+area advantage over
PCIe enables higher IO bw

11. Virtual Addressing
ØAn OpenCAPI device operates in the virtual address spaces of the applications that it supports
Ø Eliminates kernel and device driver software overhead
Ø Improves accelerator performance
Ø Allows device to operate directly on application memory without kernel-level data copies or pinned pages
Ø Simplifies programming effort to integrate accelerators into applications
ØThe Virtual-to-Physical Address Translation occurs in the host CPU
Ø Reduces design complexity of OpenCAPI-attached devices
Ø Makes it easier to ensure interoperability between an OpenCAPI device and multiple CPU architectures
Ø Since the OpenCAPI device never has access to a physical address, this eliminates the possibility of a
defective or malicious device accessing memory locations belonging to the kernel or other applications that
it is not authorized to access

12. OpenCAPI Protocol Stack
Ø The OpenCAPI transaction layer specifies the control and response packets between a host and an endpoint
OpenCAPI device
Ø The transaction layer on the host is referred to as the TL
Ø The transaction layer on the endpoint OpenCAPI device is referred to as the TLx
Ø On the host side the transaction layer converts:
Ø Host specific protocol requests into transaction layer defined commands
Ø TLx commands into host specific protocol requests.
Ø When the host protocol completes it provides responses to the TLx commands (if required)
Ø TLx responses into responses for host initiated requests
Ø On the endpoint OpenCAPI device, the transaction layer converts:
Ø AFU protocol requests into transaction layer commands
Ø TL commands into AFU protocol requests.
Ø When the AFU protocol completes it provides responses to the TL commands (if required)
Ø TL responses into responses for AFU initiated requests
Ø The full TL specification can be obtained by simply going to opencapi.org and registering under the technical
-> specifications pull down menu
Ø The OpenCAPI data link layer supports a 25Gbps serial data rate per lane connecting a processor to an FPGA
or an ASIC that contains an endpoint accelerator or device
Ø The basic configuration supports 8 lanes running at 25.78125 GHz for a 25 GB/s data rate.
Ø The data link layer implemented on the host is referred to as the DL
Ø The data link layer implemented on the endpoint OpenCAPI device is referred to as the DLx.
Ø The full DL specification can be obtained by simply going to opencapi.org and registering under the technical
-> specifications pull down menu

13. DL Flits and TL Frame Format
Ø Transmission order from right to left, top to bottom
Ø CRC in DL content covers flits of same color
Ø Control flit may be followed by another control flit or 0 to 8 data flits
Ø Data descriptor in TL command / response tells how many data flits follow
Ø Optimized for low latency (FPGA friendly)
Ø Data packet alignment to avoid any byte rotation at receiver
Ø CRC alignment enables lowest latency control packet processing
time
64B control flit
64B data flit
September IBM + Google Technical Meeting

14. 25 Gbit PHY
Ø Open CAPI is agnostic to processor architecture and as such the electrical interface is not being defined by the OpenCAPI
consortium or any of its workgroups
Ø However if a partner wishes to connect with IBM’s Power9 microprocessor the electrical interface is defined as follows
Ø Definition is being driven by the 25G workgroup within the OpenPower Foundation
Ø Based on the OIF CEI 28G SR specification
Ø 25Gbit/sec signaling and protocol built to enable very low latency interface on CPU and attached device
Ø Allows for future looking media improvements such as 32 Gb/s and 56 Gb/s signaling

15. Accelerated Function
OpenCAPI Coherence Programming Model
Host Process
Threads with Fast
Notification
Atomics
Host Memory
Open CAPI Architecture
offers advancements for
the Host<->Accelerator
programming model
Accelerator Memory
Shared Host-Accelerator
memory local to Host
Shared Host-Accelerator
memory local to Accelerator
Threads can be local to
Host or Accelerator

16. FPGA versus ASIC/Structured Array
Ø The OpenCAPI architecture is truly agnostic to a specific vendor technology
Ø The TLx and DLx reference RTL is written for the Xilinx FPGA Vivado toolchain and statistics are provided using
that flow in this deck
Ø BRAM(s), Distributed RAM(s) are specific Xilinx constructs
Ø To convert to an ASIC vendor or a structured array technology would be a very minimal exercise to port
Ø Discussions have been held to take our reference RTL and harden it into a structured array technology
Ø Master Definition is underway between IBM and Toshiba
Ø Economy of scale and NRE are considerations that partners need to make in deciding whether to go the FPGA,
ASIC, or structured array route to market

17. Reference Card Design
Ø Definition of FPGA reference card is being driven as part of the 25G workgroup within the
OpenPower consortium
Ø Definition of the cable(s) are also driven as part of the 25G workgroup within the OpenPower
consortium
Ø Currently IBM and Xilinx are driving the initial definition of a PCIE based form factor card
Ø Representative Diagram is articulated below
18
FPGA
Sideband signals (low freq)
25G

18. Thank-you!
Any questions?