Recommended
More Related Content
What's hot
What's hot (20)
Similar to Q1 Memory Fabric Forum: Compute Express Link (CXL) 3.1 Update
Similar to Q1 Memory Fabric Forum: Compute Express Link (CXL) 3.1 Update (20)
More from Memory Fabric Forum
More from Memory Fabric Forum (20)
Recently uploaded
Recently uploaded (20)
Q1 Memory Fabric Forum: Compute Express Link (CXL) 3.1 Update
- 1. Siamak Tavallaei CXL Advisor to the Board, CXL Consortium Steering Committee, OCP Feb 08, 2024 CXL-relatedActivitieswithinOCP MemoryFabricForum(Feb2024)
- 2. An Update: CXL™ 3.1 Specification Released! (Nov 14, 2023) Presented by Siamak Tavallaei CXL Advisor to the Board, CXL Consortium
- 3. 11 CXL 3.1 Featureenhancements ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium.
- 4. The CXL specification continues to evolve to meet new usage models • New features introduced in the CXL 3.1 specification: • CXL Fabric Improvements/Extensions • Scale-outof CXL fabrics using PBR (Port Based Routing) • Trusted-Execution-Environment Security Protocol (TSP) • Allows for Virtualization-basedTrustedExecutionEnvironments (TEEs) to host Confidential Computing Workloads • Memory Expander Improvements • Up to 32-bit of metadata and RAS capability enhancements CXL 3.1 Feature Enhancements 12 ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium.
- 5. 13 CXL Fabric Improvements/ Extensions ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium.
- 6. • Fabric Decode/Routing requirements • Host-to-Host communication with Global Integrated Memory (GIM) concept (with .UIO) • Direct P2P .mem support through PBR Switches • Adds a form of symmetric Link Layer definition • Enables direct caching of CXL.mem for an accelerator (caching is not possible with .UIO) • Fabric Manager (FM) API definition for PBR Switch CXL Fabric Improvements/Extensions ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium. 14
- 7. CXL 3.1: Fabric Enhancement Features Port-Based Routing (PBR) compared to Hierarchy-Based Routing (HBR) Support fabric topologiesother thantreetopologiesthatHBR switches offer • Address-based,non-prescriptiveroutingfor large memory fabrics • Supports tree,mesh, ring,star,butterfly,and multi-dimensionaltopologies ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium. Host Host CXL CXLHBRSwitch CXL HBR Switch CXL HBR Switch CXL HBR Switch Type-3 CXLMem Type-3 CXLMem Type-3 CXLMem Type-3 CXLMem Type-3 CXLMem Type-3 CXLMem CXL PBR Switc h Host Host Host Host Host Host Host Host CXL PBR Switch CXL PBR Switch CXL PBR Switch CXL PBR Switch Type-1/2/3 CXLDevice Type-1/2/3 CXLDevice Type-1/2/3 CXLDevice Type-1/2/3 CXLDevice Type-1/2/3 CXLDevice Type-1/2/3 CXLDevice Type-1/2/3 CXLDevice Type-1/2/3 CXLDevice
- 8. ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium.
- 9. ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium.
- 10. ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium. CXL 3.1: Fabric Enhancement Features Fabric Manager (FM) API definition for PBR Switch
- 11. 19 Trusted Execution Environment (TEE) & TEE Security Protocol (TSP) ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium.
- 12. RECAP: CXL 2.0 Security Benefits CXL 2.0 provides Integrity and Data Encryption (IDE) of traffic across all entities (Root Complex, Switch, Device) at the Link Layer CXL 2.0 Switch CPU/SoC Root Complex CXL Device Home Agent MC Host Memory CoherentBridge IO Bridge IOMMU CXL.memory CXL.io Area of Protection Coherent Cache (Optional) DTLB MC Device Memory CXL.cache ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium.
- 13. CXL 3.1 Trusted Security Protocol (TSP) Allows for Virtualization-based, Trusted Execution Environments (TEEs) to host Confidential Computing Workloads (CC WL) Benefits: • Freedom tomigratesensitiveWLs to TSP-enabledClouds • Collaborationwithmultiplepartiesfor sharingdata • Conform to Compliance & Data-sovereigntyprograms • StrengthenApplicationsecurity& SoftwareIP protection KeyCapabilities: • Separationbetween TVM*& CSP’s infrastructure(VMM) • Configurationof CXL device • Encryptionof sensitivedatain both Host and Devicememory • Cryptographicallyverify correctconfigurationof trusted computingenvironment *TVM = Trusted VM ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium. CXL Device CXL Link MC TVM* VM VM VM VM CXL.io CXL.mem CXL IDE (Defined in CXL v2.0) TEE Capable Host MC Device Memory Device Memory Host Mem Host Mem CXL.io CXL.mem
- 14. Elements of TSP / TSP Overview 22 ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium. TSP Components for Confidential Computing • TrustedExecutionState& Access Control • How access to memoryis controlled • Configuration • Ability to determinethe supported security featureson the device, enable required features,and lock the configuration • Attestation& Authentication • Trusting who you are talking to • Memory Encryption • Encrypting data-at-rest • Transport Security • Encrypting the link to protect data-in-flight and detect/preventphysical attacks Confidential Computing (initiator) CXL TrustedExecutionEnvironment SecurityProtocol (TSP) Attestation Authentication Trusted Execution State & Access Control Memory Encryption (Data-at-rest) Transport Security (Data-in-flight) Configuration CXL memoryexpander (target) HDM-H (CXL 3.1), HDM-DB (CXL 3.1 ECN)
- 15. 23 Memory Expander Improvements ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium.
- 17. CXL Specification Feature Summary Features CXL 1.0 / 1.1 CXL 2.0 CXL 3.0 CXL 3.1 Releasedate 2019 2020 August 2022 November2023 Max link rate 32GT/s 32GT/s 64GT/s 64GT/s Flit 68 byte (up to 32 GT/s) Type 1, Type 2 and Type 3 Devices MemoryPooling w/ MLDs Global PersistentFlush CXL IDE Switching (Single-level) Switching (Multi-level) Multiple Type 1/Type 2 devicesper rootport Direct memoryaccess for peer-to-peer 256-byte Flit (up to 64 GT/s PAM4) 256-byte Flit (Enhanced coherency) 256-byte Flit (Memorysharing) 256-byte Flit (Fabric capabilities) Fabric Manager API definition for PBR Switch Host-to-Hostcommunication with Global Integrated Memory(GIM) concept Trusted-Execution-Environment(TEE) SecurityProtocol Memoryexpander enhancements (up to 34-bit ofmeta data, RAS capability enhancements) ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium. Supported Not Supported
- 18. • The CXL specification continues to evolve to meet the usage models • New features introduced in the CXL 3.1 specification: • CXL Fabric Improvements/Extensions • Scale-outof CXL fabrics using PBR (Port Based Routing) • Trusted-Execution-Environment Security Protocol (TSP) • Allows for Virtualization-based,TrustedExecutionEnvironments(TEEs) to host ConfidentialComputing Workloads • Memory Expander Improvements • Up to 34-bit of meta-dataand RAS capability enhancements CXL 3.1 Summary 26 ComputeExpressLink™ andCXL™aretrademarksof the ComputeExpressLinkConsortium. www.ComputeExpressLink.org Call to Action Supportfuture specification development by joining the CXL Consortium Download the CXL 3.1 Specification Followus on social media for updates!
- 20. Siamak Tavallaei has recently served as the CXL Consortium President, Chief Systems Architect at Google Cloud, and the Incubation Committee (IC) Representative for the Server Project. He is currently the CXL Advisor to the Board at CXL Consortium and actively participates in OCP Steering Committee. His current focus is the system optimization for large-scale, mega-datacenters for general-purpose and tightly-connected, accelerated machines built on co-designed hardware, software, security, and management. He continues to drive the architecture and productization of CXL-enabled solutions for AI/ML, HPC, and large memory-footprint Databases. In 2016, he joined OCP as a co-lead of Server Project where he drove open-sourced modular design concepts for integrated hardware/software solutions (OAI, DC-SCM, CMS, DC-MHS, and DC-Stack). His experiences as Chief Systems Architect at Google, Principal Architect at Microsoft Azure, Distinguished Technologist at HP, and Principal Member of Technical Staff at Compaq along with his contributions to industry collaborations such as EISA, PCI, InfiniBand, and CXL give Siamak a broad understanding of requirements and solutions for the Enterprise, Hyperscale, and Edge datacenters and industry-wide initiatives. Bio
Editor's Notes
- CXL 2.0 = scale out CXL 3.0 = scale up
- HBR is classic switch routing – where the host(s) communicates to all the devices underneath the tree. However, the host(s) cannot talk to each other and neither can the devices. While, PBR allows for communication between the ports (hosts or end-point). PRB (Port Base Routed) switch allow for switch topologies beyond 2.0 single layer HBR switches. Global Fabric Device (G-FAM device ) Global Fabric Attached Memory GIM Global Integrated Memory address space allows for multiple Host and Multiple Fabric Attached Memory to share address spaces.
- CXL 3.1: Fabric Manager API for PBR switches Multi-host CXL Cluster with Memory on Host and Device Exposed as GIM CXL System components configured, allocated and managed by CXL System Fabric Manager Enabled by CXL 3.1 management API All CXL devices, Hosts, Accelerators, Switches and Memory are managed by CXL Fabric Manager DMTF Redfish based /SNIA Swordfish and OFA Sunfish compliant for Industry Ecosystem interoperability
- Fabric manager API is now able to support PBR switches compared to previous generations where it only supported HBR switches
- Speaker notes: Integrity and Data Encryption (CXL IDE) – data at flight encryption expand re data encryption Security extends to CXL Switch as well Protect platforms against physical attacks and sophisticated hardware attacks on platform interconnects Provides Confidentiality, Integrity and Replay protection for data transiting the CXL link The cryptographic schemes aligned with current industry best practices Supports a variety of use models while providing for broad interoperability. CXL IDE can be used to secure traffic within a Trusted Execution Environment (TEE) that is composed of multiple components. Maintains high performance while maintaining flexibly for security
- Speaker notes: Integrity and Data Encryption (CXL IDE) – data at flight encryption expand re data encryption Security extends to CXL Switch as well Protect platforms against physical attacks and sophisticated hardware attacks on platform interconnects Provides Confidentiality, Integrity and Replay protection for data transiting the CXL link The cryptographic schemes aligned with current industry best practices Supports a variety of use models while providing for broad interoperability. CXL IDE can be used to secure traffic within a Trusted Execution Environment (TEE) that is composed of multiple components. Maintains high performance while maintaining flexibly for security