4 p9 architecture overview japan meetup
•
1 like•248 views
Document provided by Haman Yu (IBM Taiwan)
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to 4 p9 architecture overview japan meetup
Similar to 4 p9 architecture overview japan meetup (20)
More from Yutaka Kawai
More from Yutaka Kawai (20)
Recently uploaded
Recently uploaded (20)
4 p9 architecture overview japan meetup
- 1. P9 Architecture Overview © 2018 OpenPOWER Foundation Haman Yu IBM OpenPOWER Technical Enablement
- 2. Agenda Ø POWER Processor Roadmap Ø POWER9 Scale-out Processor Overview Ø POWER9 Systems Block Diagram Ø Mihawk (P9 La Grange) - Wistron Ø Zaius & Barreleye G2 (P9 La Grange) - Google/Rackspace 2 © 2018 OpenPOWER Foundation
- 3. POWER8 Architecture POWER9 Architecture 2014 POWER8 12 cores 22nm New Micro- Architecture New Process Technology 2016 POWER8 w/ NVLink 12 cores 22nm Enhanced Micro- Architecture With NVLink 2017 P9 SO 12/24 cores 14nm New Micro- Architecture Direct attach memory New Process Technology 2018 P9 SU 12/24 cores 14nm Enhanced Micro- Architecture Buffered Memory POWER7 Architecture 2010 POWER7 8 cores 45nm New Micro- Architecture New Process Technology 2012 POWER7+ 8 cores 32nm Enhanced Micro- Architecture New Process Technology 2020+ P10 TBD cores New Micro- Architecture New Technology POWER10 2019 P9 w/ Adv.I/O 12/24 cores 14nm Enhanced Micro- Architecture New Memory Subsystem Up To 150 GB/s PCIe Gen4 x48 25 GT/s 300GB/s CAPI 2.0, OpenCAPI3.0, NVLink2.0 Sustained Memory Bandwidth Standard I/O Interconnect Advanced I/O Signaling Advanced I/O Architecture Up To 210 GB/s PCIe Gen4 x48 25 GT/s 300GB/s CAPI 2.0, OpenCAPI3.0, NVLink2.0 Up To 350 GB/s PCIe Gen4 x48 25 GT/s 300GB/s CAPI 2.0, OpenCAPI4.0, NVLink3.0 Up To 435 GB/s PCIe Gen5 32 & 50 GT/s TBD Up To 210 GB/s PCIe Gen3 N/A CAPI 1.0 Up To 210 GB/s PCIe Gen3 20 GT/s 160GB/s CAPI 1.0 , NVLink 1.0 Up To 65 GB/s PCIe Gen2 N/A N/A Up To 65 GB/s PCIe Gen2 N/A N/A POWER Processor Roadmap © 2018 OpenPOWER Foundation
- 4. POWER9 Scale-out Processor (Nimbus) New POWER9 Cores optimized for Analytics, HPC and Cloud Two Socket Support Direct Drive DDR4 Memory • 8 DDR4 Channels • 1866-2666 MHz DIMM Support New Core Microarchitecture • Stronger thread performance • Efficient agile pipeline • POWER ISA v3.0 Enhanced Cache Hierarchy • 120MB NUCA L3 architecture • 12 x 20-way associative regions • Advanced replacement policies • Fed by 7 TB/s on-chip bandwidth Cloud + Virtualization Innovation • Quality of service assists • New interrupt architecture • Workload optimized frequency • Hardware enforced trusted execution Leadership Hardware Acceleration Platform • Enhanced on-chip acceleration • Nvidia NVLink 2.0: High bandwidth and advanced new features (25G Link) • CAPI 2.0: Coherent accelerator and storage attach (PCIe G4) • OpenCAPI: Improved latency and bandwidth, open interface (25G Link) State of the Art I/O Subsystem • PCIe Gen4 – 48 lanes High Bandwidth Signaling Technology • 16 Gb/s interface – Local SMP • 25 Gb/s Link interface – Accelerator 14nm finFET Semiconductor Process • Improved device performance and reduced energy • 17 layer metal stack and eDRAM • 8.0 billion transistors SMPInterconnect& Off-ChipAcceleratorEnablement On-Chip Accel DDR4 InterfaceAccelerator Signaling Accelerator Signaling DDR4 Interface PCIeSignaling SMPSignaling Core L2 L3 Region Core Core L2 L3 Region Core Core L2 L3 Region Core Core L2 L3 Region Core PCIe Core L2 L3 Region Core Core L2 L3 Region Core Core L2 L3 Region Core Core L2 L3 Region Core Core L2 L3 Region Core Core L2 L3 Region Core Core L2 L3 Region Core Core L2 L3 Region Cor e © 2018 OpenPOWER Foundation
- 5. POWER9 – Ideal for Acceleration Seamless CPU/Accelerator Interaction § Coherent memory sharing § Enhanced virtual address translation § Data interaction with reduced SW & HW overhead 2 x 1x CPU GPU Accelerator CPU 5x 7-10x PCIe Gen3 x16 GPU Accelerator GPU Accelerator GPU PCIe Gen4 x16 POWER8 with NVLink 1.0 POWER9 with 25G Increased Performance / Features / Acceleration Opportunity Extreme CPU/Accelerator Bandwidth Broader Application of Heterogeneous Compute § Designed for efficient programming models § Accelerate complex analytic / cognitive applications © 2018 OpenPOWER Foundation
- 6. POWER9 - Premier Acceleration Platform § Extreme Processor / Accelerator Bandwidth and Reduced Latency § Coherent Memory and Virtual Addressing Capability for all Accelerators § OpenPOWER Community Enablement – Robust Accelerated Compute Options 6 § State of the Art I/O and Acceleration Attachment Signaling – PCIe Gen 4 x 48 lanes – 192 GB/s duplex bandwidth – – IBM 25G Link - 25Gb/s x 48 lanes – 300 GB/s duplex bandwidth § Robust Accelerated Compute Options with OPEN standards – On-Chip Acceleration - Gzip x1, 842 Compression x2, AES/SHA x2 – – CAPI 2.0 - 4x bandwidth of POWER8 using PCIe Gen 4 – – NVLink 2.0 – Next generation of GPU/CPU bandwidth and integration using 25G Link – Open CAPI 3.0 – High bandwidth, low latency and open interface using 25G Link POWER9
- 7. POWER9 Scale Out Packages 50x50 P9 48 lanes PCIe Gen4 - PCIE devices and CAPI 2.0 X Bus 4B4 DDR4 Memory ports 68 x 68 P9 34 lanes PCIe Gen4 PCIE devices and CAPI 2.0 X Bus 4B8 DDR4 Memory Ports NV Link (6 bricks) OR Up to 4 CAPI 25Gb ports (50 GB/s each) Sforza Package 50 x 50 mm Monza Package 68.5 x 68.5 mm PCI-E PHBs E0 - x16 CAPI E1 - x8,x8 E2 - x8, x4,x4 CAPI 68 x 68 P9 42 lanes PCIe Gen4 PCIE devices and CAPI 2.0 X Bus 4B 8 DDR4 Memory Ports NV Link (2 bricks) OR Up to 2 CAPI 25Gb port (50 GB/s) LaGrange Package 68.5 x 68.5 mm X Bus 4B ~ 60GB/s x 2 ~ 60GB/s x 2 ~ 120GB/s x 2 © 2018 OpenPOWER Foundation
- 8. Raptor Block Diagram © 2018 OpenPOWER Foundation Sforza (CPU0) x8 x16 E0 E1 E2 LPC Sforza (CPU1) X-Bus DDR4 DDR4 DDR4 DDR4 DDR4 DDR4 DDR4 DDR4 E0 E1 E2 x16 x8 x16 PCIe Slot PCIeSlot PCIeSlot PCIeSlot PCIeSlot LSISAS3008 TI TUSB7340 Aspeed AST2500 Broadcom BCM5719 NCSI SATA x6 x8 x2 x1 x2 VGA USB-Internal USB USB RJ-45 RJ-45 (PCIe Lanes) (PCIe Lanes) COM Port Header COM Port Header USB2.0 BOM x2 USB Front Panel Header MicroSemi PM8068
- 9. Zaius block diagram © 2018 OpenPOWER Foundation
- 10. Sierra - IBM S922LC block diagram (2 GPUs / socket) 8 DIMMs 8 DIMMsX Bus 4B 68 x 68 P9 68 x 68 P9 NV Links (3 Bricks ea) 25G Link NV Links (3 Bricks ea) 25G Link GPU GPU GPU GPU Mellanox IB EDR NIC Shared Slot x8x8PCIe Gen4 x16 PCIe Gen4 x16 USB x4 Storage Ctlr x4 x1 x1 PEX 9733 Four x2 PCIe Buses One per GPU x4 PCIe Gen4 x4 BMC CAPI CAPI E0 E2E2E1E1 E0E2 E2 E2 © 2018 OpenPOWER Foundation
- 12. System Overview D: 820 mm H: 86 mm W: 435 mm 6x 6056 FANs HDD Bay (12x LFF or 24x SFF) 2x 2KW PSUs 2x CPUs (P9-La Grange) (Up to 44 Cores, 176 Threads) 32x DDR4 RDIMMs (4TB Max) 10x PCIe Adapters (4x Gen4, 6x Gen3)4x OpenCAPI 3.0 Connectors © 2018 OpenPOWER Foundation
- 13. Configurations © 2018 OpenPOWER Foundation
- 14. Front StorageSKU 3.5” HDD Cage (Support 4x NVMe) NVMe NVMe NVMe NVMe 2.5” HDD Cage (24x SAS/SATA or 16x SAS/SATA + 8x NVMe or 24x NVMe) NVMe x8 © 2018 OpenPOWER Foundation
- 15. I/O Bandwidth – LFF and SFF SKU © 2018 OpenPOWER Foundation
- 16. I/O Bandwidth – HybridSKU © 2018 OpenPOWER Foundation
- 17. PCIeSlot2 • Gen3x16LP PCIeSlot7 • Gen3x16LP PSU1 PSU2 PCIeSlot6 • Gen4x16 LP • CAPI2.0enabled PCIeSlot1 • Gen4x16 LP • CAPI2.0enabled VGA 2xUSB3.0 COM 2x1GRJ45 * SharedBMC 1GRJ45 * Dedicated BMC GPUGV100 To4xNVMe Rear I/O – HybridSKU GPUGV100 To4xNVMe Support PCIe baseGV100 © 2018 OpenPOWER Foundation
- 18. 2xCPUs(P9-LaGrange) (Up to 44 Cores,176 Threads) 32xDDR4RDIMMs (4TBMax) 4xPCIeAdapters (2x Gen4,2xGen3) 6x6056 FANs HDDBay (Max 16 NVMeSSD) 2x2KWPSUs D:820 mm H:86 mm W: 435mm 2xGV100(Volta) SXM2formfactor System Overview – TurboSKU © 2018 OpenPOWER Foundation
- 19. PCIeGen4 x16 Mellanox CX5 PCIeGen3 x16 Formula 16Gb/s * 16 100Gb/s * 2 8Gb/s * 16 Bandwidth 256 Gb/s 200Gb/s 128 Gb/s Performance Test – PCIeGen4 With Mellanox CX5 100G dual-port Infinibandadapter System WistronMihawk SugonI620-G30 CPU P9LaGrange20core (190W) * 2 Intel 815316core (125W) * 2 Memory 256GB OS RHEL7.5 OFED MLNX_OFED_LINUX-4.3-3.0.2.1 © 2018 OpenPOWER Foundation
- 20. Performance Test – OpenCAPI With Alpha Data ADM-9V3 - High-Performance Network Accelerator l 4x OpenCAPI adapters l Signal Integrity: 25.78125 Gbps l Bandwidth Test: 88 GB/s (~700Gbs)
- 21. Performance Test – SPECCPU2K17
- 22. Zaiu s & Barre leye G2 Accelerator Ecosystem around Google / Rackspace 48VOpenPOWERPlatform Farther | Faster | Together
- 23. Family Tree ZAIUS Motherboard • 2 x POWER9 LaGrange • 48V input • Front IO & service access • 80 Lanes of PCIe Gen4 • 32 Lanes of OpenCAPI / NVLink 2.0 • Open Source BMC & Host Firmware Za iu s + Barreley e G2 BARRELEYE G2 Chassis • Full-depth 48V open rack v2 • 2 OU chassis supports FHFL cards • Hot swap storage bay(24 Drives) • Hot swap fans and VGA access • Wattage plan to support accelerators via Power board ZAIUS Shelf & Sled • Compact enclosure of Zaius MB • 1.5OU height • Shorter than HH cards • Compatible with 48V open rack v2 with deployment shelf
- 25. • 6 PCIe G4 slots • 2 x8 slots • 3 x16 slots • 1 x16 / x8 OCP Mez 2.0 slot SLOT TYPE QTY TOTAL THROUGHPUT (UNI-DIR) x16 Gen4 3 31.5 Gbytes / sec (252 Gbps) x8 Gen4 2 15.75 Gbytes / sec (126 Gbps) x16 Mez A/B Gen4 Or x8 Mez A Gen4 1 31.5 Gbytes / sec (252 Gbps) 15.75 Gbytes / sec (126 Gbps) PCIe G4/Mezz 2.0
- 26. • GEN4 X8 STORAGE ADAPTER Eideticom NVM Express Offload • Gen3 Limitation: ~6.8 GB/s PCIe G4/Mezz 2.0 Demo Network & Storage Adapters Result: 187.7 Gb/s Result: 13.5 GB/s • GEN4 X16 NETWORK CARD • Mellanox CX5 Mezz 2.0 • Gen3 Limitation: ~94 Gb/s
- 27. Barreleye G2 Storage Storage Expander Board Storage Canister Universal Backplane (SAS / SATA / NVMe) Broadcom Host Bus Adapter (Tri-mode) • 24 Drives • SAS / SATA / NVMe • Hardware RAID on NVMe • Tri-mode Hot-Swap Interested? Join me at 1:00pm at the OCP Storage Workshop for more in-depth information.
- 28. OpenCAPI at Scale: Update Demo with 4 x OpenCAPI Cards
- 29. OpenCAPI at Scale: Update Demo De tai l s • 4x OpenCAPI Cards • Signal Integrity: 25.78125 Gbps (3x) • Round Trip Latency < 80ns (~5x ) • Bandwidth Test: 88 GB/s (~700 Gbps) • Coherent: No kernel Overhead • Upstream Driver OpenCAPI: Faster / Cooler PCIe
- 30. Accelerator Cards and Cable Mount Options* Parallel Mount w/ Mez** Co-Proc 25G Cable PCIeslot Accelerator Card w/PCIe card edge & 25G Connector 25G Cable PCIeto Native mez adapter PCIeslot Accelerator card (e.g. GPU,OpenCAPI,etc) Supplier native connector * Possible approach to co-processor card, mount, and cable designs. * * Does not refer to OCPMezstandard. Co-Proc PCIe slot Perpendicular Mount Generic PCIe Accelerator Card w/PCIe card edge