The document discusses security issues with remote direct memory access (RDMA) and potential attacks. It proposes a secure RDMA system design where a smartNIC protects packets by authenticating, encrypting, and validating them before performing RDMA operations between endpoints. The smartNIC offloads cryptographic operations to improve performance and security compared to relying only on the host CPU. Future plans include programming InfiniBand RDMA and offloading specific cipher suites to the smartNIC.

1. 1
RDMA Intro.
with bluefield smartnic

2. 2
Introduction
▪ State-of-the-art remote direct memory access(RDMA) technologies such as
• InfiniBand (IB)
• RDMA over Converged Ethernet (RoCE)
are becoming widely used in data center applications.
▪ Hence, the security of RDMA architectures is crucial, yet potential security
implications of using RDMA communication remain largely unstudied.
ReDMArk: Bypassing RDMA Security Mechanisms (ETH Zurich)

3. 3
Introduction
➨ RFC 5042 analyzes basic security issues, but lack in-depth analysis.
ReDMArk: Bypassing RDMA Security Mechanisms (ETH Zurich)

4. 4
Introduction
▪ Unfortunately, encryption and authentication of RDMA packets is not part of
current RDMA specifications.
• IPsec standard does not support InfiniBand traffic (only RoCE)
• Application-level encryption (e.g., TLS) is not possible since RDMA operations
can be handled without involvement of the CPU.
ReDMArk: Bypassing RDMA Security Mechanisms (ETH Zurich)

5. 5
RDMA Background
RDMA has been key ingredient of HPC and supercomputing environments for years.
At a high level RDMA is a networking approach consisting of two basic concepts:
Design Guidelines for High Performance RDMA Systems (USENIX ATC’ 16)
1. Operating system “stack bypass”: In many
applications, the overhead of going through the
kernel networking layers is the bottleneck to
processing speed.
2. Full CPU bypass: For certain, more-specialized
applications, RDMA hardware can allow one
computer to read and write directly to/from the
memory of another node in the cluster, without
the remote node’s CPU or OS being involved at all.

6. 6
Large RDMA design space
Design Guidelines for High Performance RDMA Systems (USENIX ATC’ 16)
Operations
Transports
Optimizations
READ WRITE ATOMIC
Remote bypass (one-sided)
SEND, RECV
Two-sided
Reliable Unreliable Connected Datagram
Inlined Unsignaled Doorbell batching WQE shrinking
0B-RECVs

7. 7
Large RDMA design space
Design Guidelines for High Performance RDMA Systems (USENIX ATC’ 16)
PCI Express messages are expensive
Reduce CPU-to-NIC messages (MMIOs)
Reduce NIC-to-CPU messages (DMAs)

8. 8
Security Gaps in RDMA
10 discovered vulnerabilities in ReDMArk
V1 Memory Protection Key Randomness
V2 Static Initialization State for Key Generation
V3 Shared Key Generator
V4 Consecutive Allocation of Memory Regions
V5 Linearly Increasing QP Numbers
V6 Fixed Starting Packet Sequence Number
V7 Limited Attack Detection Capabilities
V8 Missing Encryption and Authentication in RDMA Protocols
V9 Single Protection Domain for all QPs
V10 Implicit On-Demand Paging (ODP)
Packet Injection using Impersonation
Dos Attack by Transiting QPs to an Error State
Unauthorized Memory Access
DoS Attack based on QP Allocation Resource Exhaustion
Performance Degradation using Resource Exhaustion
Facilitating Attacks using RDMA

9. 1.Host A sends data to SNIC A
2.SNIC A protects the packet
3.SNIC A sends the protected packet to SNIC B
4.SNIC B validates the packet
5.SNIC B performs RDMA write to the requested
memory
9
Secure RDMA System Design
sRDMA – Efficient NIC-based Authentication and Encryption for Remote Direct Memory Access
▪ The current IBA protection mechanism do not suffice to ensure secure
communication between endpoints, allowing adversaries numerous attacks.
▪ Thus, the primary goal of our work is to secure RDMA protocols against attacks by
providing source and data authentication along with data secrecy and data freshness.
QP connection
Endpoint A
Endpoint B
Host A Host B

10. Intelligent Offloads – Higher Performance, Efficiency, Scalability
ConnectX-4 / Lx
ConnectX-5/6/6-Dx
BlueField
© 2018 Mellanox Technologies
10

11. 111
FUTURE PLANS
▪ Programming Infiniband RDMA
▪ OFFLOAD a specific CipherSuite (e.g., TLS1.2-ECDHE-ECDSA-AES128-GCM-SHA256)
to SmartNIC.