SlideShare a Scribd company logo
Impact of Intel® Optane™
Technology on HPC
Andrey Kudryavtsev
HPC Storage Architect
Intel Corporation
2
Re-architecting the Memory/Storage Hierarchy
Intel® Optane™ persistent memory (PMem) 200 series fills the DRAM gap
10s GB <0.1 microseconds
1s TB <10 microseconds
10s TB <100 microseconds
10s TB <10–100 milliseconds
DRAM
HOT TIER
HDD/TAPE
COLD TIER
100s GB <1 microsecond
MEMORY
STORAGE
CAPACITY LATENCY
IMPROVING
Capacity and persistence
IMPROVING
SSD storage
IMPROVING
Efficient storage NAND SSD
3
Accelerate Insight and Innovation
Intel® Optane™ persistent memory (PMem) 200 series fills the DRAM gap
3 Supports for more than 1.2x more VMs than the previous generation. Based on testing by Intel as of April 27, 2020 (Baseline) and March 31, 2020 (New). For details, see Endnotes slides.
High performance. High capacity. Hardware-enhanced security.
Lower overall total cost of ownership (TCO).3 Do more
with each server—increase CPU utilization, in-memory
database capacity, throughput, virtual machine (VM) density,
and services for users on a consolidated footprint.
Protect data automatically. Help secure all data at
rest in persistent memory with application-transparent
AES-256 encryption, which enhances security without
code changes.
Extract more from larger datasets. Expand your memory
pool in persistent memory and support near-real-time data
analysis; deliver deep insights, improve operations, or
create new revenue streams.
4
persistent memory: in a league of its own
HIGHER BANDWIDTH
Up to 3.7X read/write
bandwidth vs NVMe SSDs,
with one module; more with
multiple modules
LOWER LATENCY
Orders of magnitude lower
latency than NVMe SSDs
• 1000X lower latency than
NAND NVMe SSD at 1GB/s
Performance results are based on testing as of Feb 22, 2019 and may not reflect all publicly available security updates. No product or component can be absolutely secure.
Results have been estimated based on tests conducted on pre-production systems, and provided to you for informational purposes. Software and workloads used in performance tests may have been optimized for performance only on Intel
microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You
should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to
www.intel.com/benchmarks.
2, 3 Based on internal testing by Intel . Configuration: see slides 38 and 39.
3.7
X
HIGHER
bandwidth
up
to
10
00
X
LOWER
latency
up
to
Intel DC P4610
NVMe SSD
Intel Optane
DC SSD
P4800X
5
Intel® Optane™ Persistent Memory (PMem)
200 Series Operating Modes
Memory Mode (MM)
Intel Optane PMem–aware OS/VMM
Intel Optane PMem–aware application
Intel Optane PMem DRAM
Persistent performance and
high capacity
Unmodified OS/VMM
Unmodified application
Volatile memory pool
Affordable memory capacity for
many applications
DRAM as cache
Intel Optane PMem
Total memory: Intel Optane PMem size Total memory: Intel Optane PMem size + DRAM size
App Direct Mode (AD) Storage over AD
OS with NVDIMM driver
Unmodified application
Intel Optane PMem
DRAM
Acts as fast storage on the memory bus
Avoids the latency of the I/O bus
STORAGE ON MEMORY BUS
Persistent VolatilePersistent Volatile
6
Intel® Optane™ Persistent Memory (PMem)
at a Glance
See backup for configuration details. For more complete information about performance and benchmark results, visit www.intel.com/benchmarks.
3 Based on testing by Intel as of April 27, 2020 (Baseline) and March 31, 2020 (New). For details, see Endnotes slides.
* 3 TB = 6 x 512 GB Intel Optane PMem per socket
** 4 TB = 8 x 512 GB Intel Optane PMem per socket
Intel Optane PMem
100 series
2nd Generation Intel® Xeon® Scalable processors
SP on 2S/4S/8S platform
6 channels
memory
2,666 MT/s
DDR4
3 TB
Intel Optane PMem
per socket*
18 W
top sustained
Intel Optane PMem
200 series
3rd Generation Intel Xeon Scalable processors
SP on 4S platform
2,666 MT/s
DDR4
3 TB
Intel Optane PMem
per socket*
25%
UP TO
higher memory bandwidth
over the previous generation3
256-BIT
AES
encryption
Secure
Erase
512 GB
UP TO
modules
12–15 W
thermal
design power
16–28 cores
6 channels
memory
7
A Strong Memory and Storage Future
For each processor above, Intel Optane PMem will be supported on select SKUs.
3rd Generation
Intel Xeon
Scalable processors
(2 socket platform)
Intel Optane
persistent memory
200 series
Intel® Optane™
persistent memory
100 series
2nd Generation
Intel® Xeon® Scalable
processors
3rd Generation
Intel Xeon
Scalable processors
(4 socket platform)
3rd Gen
Persistent
Memory​
4th Gen
Persistent
Memory​
Future2019 2020 2020H2
8
Intel Optane SSD
Intel 3D NAND SSD
accelerati
ng
caching Tiering
Intel
Optane
SSD
Intel 3D NAND SSD
intelligent data placement
storage
compute
intel® optane™ technology breaks memory
and storage bottlenecks
store data
about data
temporarily copy or
hold hottest data
Intel Optane
Persistent Memory
DRAM
Intel Optane
Persistent Memory
DRAM
caching Tiering
copy of less frequently
used data from DRAM
app-aware data placement
accelerati
ngkeep more data
in memory
Intel® Optane™
Persistent Memory
DRAM
Intel®
Optane™
SSD
Intel® 3D NAND SSD
Intel CorporationCloud & Enterprise Solutions Group 9
The Problem with POSIX & Blocks (& Objects)
…
Data Metadata
Scientific
Application
I/O Middleware
HDF5
MPI-IO
SCR/VeloC
Dataset Serialized into a POSIX File
Block 1
Block 2
Block 3
Block 4
Dataset Serialized into a POSIX File
Block 1 header size key1 size val1
val1 cont’d size sizekey2
val2
val2 cont’d
val2 cont’d size size val3key3
val3
Block 2
Block 3
Block 4
Block 5
Block 6
AI/Analytics Application
key1
val1
key2
val2
key3
val3
@
@
@
POSIX
Serialization
POSIX
Serialization
Block 1
Block 2
Block 3
Block 4
Block 5
Block 6
Intel CorporationCloud & Enterprise Solutions Group 10
What is DAOS?
 A new, innovative scale-out storage-as-a-service stack based on Intel
Optane Persistent Memory and NVMe SSDs
 Globally accessible from many nodes
 Delivers exceptionally high bandwidth and IOPS on commodity servers
 Can be utilized either as a standalone file system, or as a performance
tier integrated with existing storage systems
10
More IOPs and bandwidth per dollar
Network / Can Leverage RDMA
DAOS Storage Engine
Intel® Optane PMem
NVMe
Interface
Metadata, low-latency I/Os &
indexing/query
Bulk data
NVMe SSD
Memory
Interface
Storage Instance
AI/Analytics/Scientific Workflow
DAOS library
Files KVoF Spark…
Compute Instance
Objects (No)SQL Python
Intel CorporationCloud & Enterprise Solutions Group 11
Application Interface
Dataset
Mover
Capacity Tier DAOS Storage Tier
POSIX I/O
AI/Analytics/Scientific Workflow
HDF5 MPI-IO Python
Native
API
Legac
y
Apache
Spark
Apache
Hadoop
HDFS Connector
Intel CorporationCloud & Enterprise Solutions Group 12
DAOS in the overall cluster architecture
DAOS
node #1
Optane DC
Persistent Memory
P4510/
P4610
SSDs
Optane DC
Persistent Memory
P4510/
P4610
SSDs
Optane DC
Persistent Memory
P4510/
P4610
SSDs
DAOS
node #2
DAOS
node #Y
DataPool1
Data
Pool2
Application data
Checkpoint
Performance/$ optimized storage
Capacity
Storage
All Flash or Hybrid HPC
storage options:
Lustre*
IBM Spectrum Scale
CEPH, NFS, etc
HDD
NVMe TLC,
QLC SSDs
DataPool3
DataPoolN
DAOS Cluster
Compute
Node #1
Node #2
Node #N
Remote memory
Temp / Scratch
Data movers
Name space integration
Data proxies
Cloud Storage
HDD
AWS S3
Microsoft Azure
etc
Data proxies
Intel CorporationCloud & Enterprise Solutions Group 13
DAOS and AI Best Play Together
13
Compute AI training workloads dominate more reads than writes
Parallel throughput becomes critical at training stage for a shared
data set
Traditional parallel filesystems are not able to keep up with
performance demand
DAOS has a native capability to set the replication factor per object
This can be used to bring higher bandwidth for the given object and
increase parallelism and proximity to the data.
• In addition to replication, DAOS has optimized object placement
planned in the roadmap for version 2.2.
DAOS further accelerates All Flash Arrays already been used with
GPU accelerated compute nodes.
proximity
DAOS
node #1
Optane DC
Persistent MemoryP4510/
P4610
SSDs
Optane DC
Persistent MemoryP4510/
P4610
SSDs
Optane DC
Persistent MemoryP4510/
P4610
SSDs
DAOS
node #2
DAOS
node #N
Node #1
Node #2
Node #N
Data Pool
DatasetDatasetDataset
copy
N-copy
proximity
proximity
DAOS
Intel CorporationCloud & Enterprise Solutions Group 14
DAOS Community Roadmap
1Q20 2Q20 3Q20 4Q20 1Q21 2Q21 3Q21 4Q21 1Q22 2Q22 3Q22
1.0 1.2 2.0 2.2 2.4
Released on June 18
DAOS:
- NVMe & DCPMM support
- Per-pool ACL
- UNS in DAOS via dfuse
- Replication & self-healing (Preview)
Application Interface:
- MPI-IODriver
- HDF5 DAOS Connector
- Basic POSIX I/O support
DAOS:
- End-to-end data integrity
- Per-container ACL
- Improved control plane
- Replication & self-healing
- Conditional updates
- Erasure Code (Preview)
- Online server addition
- Lustre/UNS integration
Application Interface:
- HDF5 vol w/o async
- POSIX I/O with conditional
update support
- POSIX data mover
- Async HDF5 operations
over DAOS
- Spark
DAOS:
- Erasure code
- Telemetry& per-job statistics
- Advanced control plane
- Distributedtransactions
- Multi OFI provider support
Application Interface:
- POSIX I/O with distributed
transaction support
- HDF5 data mover
- Container parking/serialization
DAOS:
- Progressivelayout
- Placement optimizations
- Checksum scrubbing
DAOS:
- Catastrophic recovery tools
NOTE: All information provided in this roadmap is subject to change without notice.
Intel CorporationCloud & Enterprise Solutions Group 15
Resources
 Github
• https://github.com/daos-stack/daos
 DAOS online documentation
• http://daos.io
 Community mailing list
• https://daos.groups.io
 Annual DAOS User Group in November
• https://wiki.hpdd.intel.com/display/DC/DUG20

More Related Content

What's hot

IBM SAN Volume Controller Performance Analysis
IBM SAN Volume Controller Performance AnalysisIBM SAN Volume Controller Performance Analysis
IBM SAN Volume Controller Performance Analysis
brettallison
 
Moving to PCI Express based SSD with NVM Express
Moving to PCI Express based SSD with NVM ExpressMoving to PCI Express based SSD with NVM Express
Moving to PCI Express based SSD with NVM Express
Odinot Stanislas
 
Introduction to NVMe Over Fabrics-V3R
Introduction to NVMe Over Fabrics-V3RIntroduction to NVMe Over Fabrics-V3R
Introduction to NVMe Over Fabrics-V3R
Simon Huang
 
NGENSTOR_ODA_P2V_V5
NGENSTOR_ODA_P2V_V5NGENSTOR_ODA_P2V_V5
NGENSTOR_ODA_P2V_V5
UniFabric
 
UniPlex vScaleDB pat. pending
UniPlex vScaleDB pat. pendingUniPlex vScaleDB pat. pending
UniPlex vScaleDB pat. pending
UniFabric
 
EMC Data domain advanced features and functions
EMC Data domain advanced features and functionsEMC Data domain advanced features and functions
EMC Data domain advanced features and functions
solarisyougood
 
VIO LPAR Introduction | Basics | Demo
VIO LPAR Introduction | Basics | DemoVIO LPAR Introduction | Basics | Demo
VIO LPAR Introduction | Basics | Demo
Kernel Training
 
FalconStor NSS Presentation
FalconStor NSS PresentationFalconStor NSS Presentation
FalconStor NSS Presentation
rpsprowl
 
Big Data Uses with Distributed Asynchronous Object Storage
Big Data Uses with Distributed Asynchronous Object StorageBig Data Uses with Distributed Asynchronous Object Storage
Big Data Uses with Distributed Asynchronous Object Storage
Intel® Software
 
Virtual Private Server Documentation
Virtual Private Server DocumentationVirtual Private Server Documentation
Virtual Private Server Documentation
webhostingguy
 
XPDS16: High-Performance Virtualization for HPC Cloud on Xen - Jun Nakajima &...
XPDS16: High-Performance Virtualization for HPC Cloud on Xen - Jun Nakajima &...XPDS16: High-Performance Virtualization for HPC Cloud on Xen - Jun Nakajima &...
XPDS16: High-Performance Virtualization for HPC Cloud on Xen - Jun Nakajima &...
The Linux Foundation
 
PCI Express* based Storage: Data Center NVM Express* Platform Topologies
PCI Express* based Storage: Data Center NVM Express* Platform TopologiesPCI Express* based Storage: Data Center NVM Express* Platform Topologies
PCI Express* based Storage: Data Center NVM Express* Platform Topologies
Odinot Stanislas
 
Xiv overview
Xiv overviewXiv overview
Xiv overview
Jinesh Shah
 
VM-aware Adaptive Storage Cache Prefetching
VM-aware Adaptive Storage Cache PrefetchingVM-aware Adaptive Storage Cache Prefetching
VM-aware Adaptive Storage Cache Prefetching
Shinagawa Laboratory, The University of Tokyo
 
Unified Hardware Abstraction Layer with Device Masquerade
Unified Hardware Abstraction Layer with Device MasqueradeUnified Hardware Abstraction Layer with Device Masquerade
Unified Hardware Abstraction Layer with Device Masquerade
Shinagawa Laboratory, The University of Tokyo
 
Presentation oracle on power power advantages and license optimization
Presentation   oracle on power power advantages and license optimizationPresentation   oracle on power power advantages and license optimization
Presentation oracle on power power advantages and license optimization
solarisyougood
 
Avoiding Chaos: Methodology for Managing Performance in a Shared Storage A...
Avoiding Chaos:  Methodology for Managing Performance in a Shared Storage A...Avoiding Chaos:  Methodology for Managing Performance in a Shared Storage A...
Avoiding Chaos: Methodology for Managing Performance in a Shared Storage A...
brettallison
 
Raid 1 3
Raid 1 3Raid 1 3
Raid 1 3
Muhammad Ishaq
 
Ibm aix technical deep dive workshop advanced administration and problem dete...
Ibm aix technical deep dive workshop advanced administration and problem dete...Ibm aix technical deep dive workshop advanced administration and problem dete...
Ibm aix technical deep dive workshop advanced administration and problem dete...
solarisyougood
 
Simple Virtualization Overview
Simple Virtualization OverviewSimple Virtualization Overview
Simple Virtualization Overview
bassemir
 

What's hot (20)

IBM SAN Volume Controller Performance Analysis
IBM SAN Volume Controller Performance AnalysisIBM SAN Volume Controller Performance Analysis
IBM SAN Volume Controller Performance Analysis
 
Moving to PCI Express based SSD with NVM Express
Moving to PCI Express based SSD with NVM ExpressMoving to PCI Express based SSD with NVM Express
Moving to PCI Express based SSD with NVM Express
 
Introduction to NVMe Over Fabrics-V3R
Introduction to NVMe Over Fabrics-V3RIntroduction to NVMe Over Fabrics-V3R
Introduction to NVMe Over Fabrics-V3R
 
NGENSTOR_ODA_P2V_V5
NGENSTOR_ODA_P2V_V5NGENSTOR_ODA_P2V_V5
NGENSTOR_ODA_P2V_V5
 
UniPlex vScaleDB pat. pending
UniPlex vScaleDB pat. pendingUniPlex vScaleDB pat. pending
UniPlex vScaleDB pat. pending
 
EMC Data domain advanced features and functions
EMC Data domain advanced features and functionsEMC Data domain advanced features and functions
EMC Data domain advanced features and functions
 
VIO LPAR Introduction | Basics | Demo
VIO LPAR Introduction | Basics | DemoVIO LPAR Introduction | Basics | Demo
VIO LPAR Introduction | Basics | Demo
 
FalconStor NSS Presentation
FalconStor NSS PresentationFalconStor NSS Presentation
FalconStor NSS Presentation
 
Big Data Uses with Distributed Asynchronous Object Storage
Big Data Uses with Distributed Asynchronous Object StorageBig Data Uses with Distributed Asynchronous Object Storage
Big Data Uses with Distributed Asynchronous Object Storage
 
Virtual Private Server Documentation
Virtual Private Server DocumentationVirtual Private Server Documentation
Virtual Private Server Documentation
 
XPDS16: High-Performance Virtualization for HPC Cloud on Xen - Jun Nakajima &...
XPDS16: High-Performance Virtualization for HPC Cloud on Xen - Jun Nakajima &...XPDS16: High-Performance Virtualization for HPC Cloud on Xen - Jun Nakajima &...
XPDS16: High-Performance Virtualization for HPC Cloud on Xen - Jun Nakajima &...
 
PCI Express* based Storage: Data Center NVM Express* Platform Topologies
PCI Express* based Storage: Data Center NVM Express* Platform TopologiesPCI Express* based Storage: Data Center NVM Express* Platform Topologies
PCI Express* based Storage: Data Center NVM Express* Platform Topologies
 
Xiv overview
Xiv overviewXiv overview
Xiv overview
 
VM-aware Adaptive Storage Cache Prefetching
VM-aware Adaptive Storage Cache PrefetchingVM-aware Adaptive Storage Cache Prefetching
VM-aware Adaptive Storage Cache Prefetching
 
Unified Hardware Abstraction Layer with Device Masquerade
Unified Hardware Abstraction Layer with Device MasqueradeUnified Hardware Abstraction Layer with Device Masquerade
Unified Hardware Abstraction Layer with Device Masquerade
 
Presentation oracle on power power advantages and license optimization
Presentation   oracle on power power advantages and license optimizationPresentation   oracle on power power advantages and license optimization
Presentation oracle on power power advantages and license optimization
 
Avoiding Chaos: Methodology for Managing Performance in a Shared Storage A...
Avoiding Chaos:  Methodology for Managing Performance in a Shared Storage A...Avoiding Chaos:  Methodology for Managing Performance in a Shared Storage A...
Avoiding Chaos: Methodology for Managing Performance in a Shared Storage A...
 
Raid 1 3
Raid 1 3Raid 1 3
Raid 1 3
 
Ibm aix technical deep dive workshop advanced administration and problem dete...
Ibm aix technical deep dive workshop advanced administration and problem dete...Ibm aix technical deep dive workshop advanced administration and problem dete...
Ibm aix technical deep dive workshop advanced administration and problem dete...
 
Simple Virtualization Overview
Simple Virtualization OverviewSimple Virtualization Overview
Simple Virtualization Overview
 

Similar to Impact of Intel Optane Technology on HPC

Reimagining HPC Compute and Storage Architecture with Intel Optane Technology
Reimagining HPC Compute and Storage Architecture with Intel Optane TechnologyReimagining HPC Compute and Storage Architecture with Intel Optane Technology
Reimagining HPC Compute and Storage Architecture with Intel Optane Technology
inside-BigData.com
 
Yashi dealer meeting settembre 2016 tecnologie xeon intel italia
Yashi dealer meeting settembre 2016 tecnologie xeon intel italiaYashi dealer meeting settembre 2016 tecnologie xeon intel italia
Yashi dealer meeting settembre 2016 tecnologie xeon intel italia
Yashi Italia
 
NVMe_Infrastructure_final1.pdf
NVMe_Infrastructure_final1.pdfNVMe_Infrastructure_final1.pdf
NVMe_Infrastructure_final1.pdf
IrfanBroadband
 
DAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI Convergence
DAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI ConvergenceDAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI Convergence
DAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI Convergence
inside-BigData.com
 
Deep Dive On Intel Optane SSDs And New Server Platforms
Deep Dive On Intel Optane SSDs And New Server PlatformsDeep Dive On Intel Optane SSDs And New Server Platforms
Deep Dive On Intel Optane SSDs And New Server Platforms
NEXTtour
 
Intel and DataStax: 3D XPoint and NVME Technology Cassandra Storage Comparison
Intel and DataStax: 3D XPoint and NVME Technology Cassandra Storage ComparisonIntel and DataStax: 3D XPoint and NVME Technology Cassandra Storage Comparison
Intel and DataStax: 3D XPoint and NVME Technology Cassandra Storage Comparison
DataStax Academy
 
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based Hardware
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based HardwareRed hat Storage Day LA - Designing Ceph Clusters Using Intel-Based Hardware
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based Hardware
Red_Hat_Storage
 
Accelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDS
Accelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDSAccelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDS
Accelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDS
Ceph Community
 
Intel ssd dc data center family for PCIe
Intel ssd dc data center family for PCIeIntel ssd dc data center family for PCIe
Intel ssd dc data center family for PCIe
Low Hong Chuan
 
Red Hat Storage Day Atlanta - Designing Ceph Clusters Using Intel-Based Hardw...
Red Hat Storage Day Atlanta - Designing Ceph Clusters Using Intel-Based Hardw...Red Hat Storage Day Atlanta - Designing Ceph Clusters Using Intel-Based Hardw...
Red Hat Storage Day Atlanta - Designing Ceph Clusters Using Intel-Based Hardw...
Red_Hat_Storage
 
Ceph Day Shanghai - SSD/NVM Technology Boosting Ceph Performance
Ceph Day Shanghai - SSD/NVM Technology Boosting Ceph Performance Ceph Day Shanghai - SSD/NVM Technology Boosting Ceph Performance
Ceph Day Shanghai - SSD/NVM Technology Boosting Ceph Performance
Ceph Community
 
Increasing Throughput per Node for Content Delivery Networks
Increasing Throughput per Node for Content Delivery NetworksIncreasing Throughput per Node for Content Delivery Networks
Increasing Throughput per Node for Content Delivery Networks
DESMOND YUEN
 
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and ContributionsCeph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
Colleen Corrice
 
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and ContributionsCeph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
Red_Hat_Storage
 
Accelerate Your Apache Spark with Intel Optane DC Persistent Memory
Accelerate Your Apache Spark with Intel Optane DC Persistent MemoryAccelerate Your Apache Spark with Intel Optane DC Persistent Memory
Accelerate Your Apache Spark with Intel Optane DC Persistent Memory
Databricks
 
Hands-on Lab: How to Unleash Your Storage Performance by Using NVM Express™ B...
Hands-on Lab: How to Unleash Your Storage Performance by Using NVM Express™ B...Hands-on Lab: How to Unleash Your Storage Performance by Using NVM Express™ B...
Hands-on Lab: How to Unleash Your Storage Performance by Using NVM Express™ B...
Odinot Stanislas
 
Cерверы Depo storm 3400 на базе новейших процессоров intel xeon e5 2600v3 fin
Cерверы Depo storm 3400 на базе новейших процессоров intel xeon e5 2600v3 finCерверы Depo storm 3400 на базе новейших процессоров intel xeon e5 2600v3 fin
Cерверы Depo storm 3400 на базе новейших процессоров intel xeon e5 2600v3 fin
DEPO Computers
 
Building an open memory-centric computing architecture using intel optane
Building an open memory-centric computing architecture using intel optaneBuilding an open memory-centric computing architecture using intel optane
Building an open memory-centric computing architecture using intel optane
UniFabric
 
April 2014 IBM announcement webcast
April 2014 IBM announcement webcastApril 2014 IBM announcement webcast
April 2014 IBM announcement webcast
HELP400
 
Accelerating Virtual Machine Access with the Storage Performance Development ...
Accelerating Virtual Machine Access with the Storage Performance Development ...Accelerating Virtual Machine Access with the Storage Performance Development ...
Accelerating Virtual Machine Access with the Storage Performance Development ...
Michelle Holley
 

Similar to Impact of Intel Optane Technology on HPC (20)

Reimagining HPC Compute and Storage Architecture with Intel Optane Technology
Reimagining HPC Compute and Storage Architecture with Intel Optane TechnologyReimagining HPC Compute and Storage Architecture with Intel Optane Technology
Reimagining HPC Compute and Storage Architecture with Intel Optane Technology
 
Yashi dealer meeting settembre 2016 tecnologie xeon intel italia
Yashi dealer meeting settembre 2016 tecnologie xeon intel italiaYashi dealer meeting settembre 2016 tecnologie xeon intel italia
Yashi dealer meeting settembre 2016 tecnologie xeon intel italia
 
NVMe_Infrastructure_final1.pdf
NVMe_Infrastructure_final1.pdfNVMe_Infrastructure_final1.pdf
NVMe_Infrastructure_final1.pdf
 
DAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI Convergence
DAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI ConvergenceDAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI Convergence
DAOS - Scale-Out Software-Defined Storage for HPC/Big Data/AI Convergence
 
Deep Dive On Intel Optane SSDs And New Server Platforms
Deep Dive On Intel Optane SSDs And New Server PlatformsDeep Dive On Intel Optane SSDs And New Server Platforms
Deep Dive On Intel Optane SSDs And New Server Platforms
 
Intel and DataStax: 3D XPoint and NVME Technology Cassandra Storage Comparison
Intel and DataStax: 3D XPoint and NVME Technology Cassandra Storage ComparisonIntel and DataStax: 3D XPoint and NVME Technology Cassandra Storage Comparison
Intel and DataStax: 3D XPoint and NVME Technology Cassandra Storage Comparison
 
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based Hardware
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based HardwareRed hat Storage Day LA - Designing Ceph Clusters Using Intel-Based Hardware
Red hat Storage Day LA - Designing Ceph Clusters Using Intel-Based Hardware
 
Accelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDS
Accelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDSAccelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDS
Accelerating Cassandra Workloads on Ceph with All-Flash PCIE SSDS
 
Intel ssd dc data center family for PCIe
Intel ssd dc data center family for PCIeIntel ssd dc data center family for PCIe
Intel ssd dc data center family for PCIe
 
Red Hat Storage Day Atlanta - Designing Ceph Clusters Using Intel-Based Hardw...
Red Hat Storage Day Atlanta - Designing Ceph Clusters Using Intel-Based Hardw...Red Hat Storage Day Atlanta - Designing Ceph Clusters Using Intel-Based Hardw...
Red Hat Storage Day Atlanta - Designing Ceph Clusters Using Intel-Based Hardw...
 
Ceph Day Shanghai - SSD/NVM Technology Boosting Ceph Performance
Ceph Day Shanghai - SSD/NVM Technology Boosting Ceph Performance Ceph Day Shanghai - SSD/NVM Technology Boosting Ceph Performance
Ceph Day Shanghai - SSD/NVM Technology Boosting Ceph Performance
 
Increasing Throughput per Node for Content Delivery Networks
Increasing Throughput per Node for Content Delivery NetworksIncreasing Throughput per Node for Content Delivery Networks
Increasing Throughput per Node for Content Delivery Networks
 
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and ContributionsCeph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
 
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and ContributionsCeph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
 
Accelerate Your Apache Spark with Intel Optane DC Persistent Memory
Accelerate Your Apache Spark with Intel Optane DC Persistent MemoryAccelerate Your Apache Spark with Intel Optane DC Persistent Memory
Accelerate Your Apache Spark with Intel Optane DC Persistent Memory
 
Hands-on Lab: How to Unleash Your Storage Performance by Using NVM Express™ B...
Hands-on Lab: How to Unleash Your Storage Performance by Using NVM Express™ B...Hands-on Lab: How to Unleash Your Storage Performance by Using NVM Express™ B...
Hands-on Lab: How to Unleash Your Storage Performance by Using NVM Express™ B...
 
Cерверы Depo storm 3400 на базе новейших процессоров intel xeon e5 2600v3 fin
Cерверы Depo storm 3400 на базе новейших процессоров intel xeon e5 2600v3 finCерверы Depo storm 3400 на базе новейших процессоров intel xeon e5 2600v3 fin
Cерверы Depo storm 3400 на базе новейших процессоров intel xeon e5 2600v3 fin
 
Building an open memory-centric computing architecture using intel optane
Building an open memory-centric computing architecture using intel optaneBuilding an open memory-centric computing architecture using intel optane
Building an open memory-centric computing architecture using intel optane
 
April 2014 IBM announcement webcast
April 2014 IBM announcement webcastApril 2014 IBM announcement webcast
April 2014 IBM announcement webcast
 
Accelerating Virtual Machine Access with the Storage Performance Development ...
Accelerating Virtual Machine Access with the Storage Performance Development ...Accelerating Virtual Machine Access with the Storage Performance Development ...
Accelerating Virtual Machine Access with the Storage Performance Development ...
 

More from MemVerge

Analytical Biosciences Accelerates Single Cell Sequencing with Big Memory
Analytical Biosciences Accelerates Single Cell Sequencing with Big MemoryAnalytical Biosciences Accelerates Single Cell Sequencing with Big Memory
Analytical Biosciences Accelerates Single Cell Sequencing with Big Memory
MemVerge
 
Checkpointing the Uncheckpointable
Checkpointing the UncheckpointableCheckpointing the Uncheckpointable
Checkpointing the Uncheckpointable
MemVerge
 
HPC Market Update and Observations on Big Memory
HPC Market Update and Observations on Big MemoryHPC Market Update and Observations on Big Memory
HPC Market Update and Observations on Big Memory
MemVerge
 
Live Data: For When Data is Greater than Memory
Live Data: For When Data is Greater than MemoryLive Data: For When Data is Greater than Memory
Live Data: For When Data is Greater than Memory
MemVerge
 
Big Memory for HPC
Big Memory for HPCBig Memory for HPC
Big Memory for HPC
MemVerge
 
Tech Talk: Moneyball - Hitting real-time apps out of the park with Big Memory
Tech Talk: Moneyball - Hitting real-time apps out of the park with Big MemoryTech Talk: Moneyball - Hitting real-time apps out of the park with Big Memory
Tech Talk: Moneyball - Hitting real-time apps out of the park with Big Memory
MemVerge
 
MemVerge Company Overview
MemVerge Company OverviewMemVerge Company Overview
MemVerge Company Overview
MemVerge
 
IDC Technology Spotlight: Big Memory Computing Emerges to Better Enable Dat...
IDC Technology Spotlight:   Big Memory Computing Emerges to Better Enable Dat...IDC Technology Spotlight:   Big Memory Computing Emerges to Better Enable Dat...
IDC Technology Spotlight: Big Memory Computing Emerges to Better Enable Dat...
MemVerge
 
Big Memory Webcast
Big Memory WebcastBig Memory Webcast
Big Memory Webcast
MemVerge
 

More from MemVerge (9)

Analytical Biosciences Accelerates Single Cell Sequencing with Big Memory
Analytical Biosciences Accelerates Single Cell Sequencing with Big MemoryAnalytical Biosciences Accelerates Single Cell Sequencing with Big Memory
Analytical Biosciences Accelerates Single Cell Sequencing with Big Memory
 
Checkpointing the Uncheckpointable
Checkpointing the UncheckpointableCheckpointing the Uncheckpointable
Checkpointing the Uncheckpointable
 
HPC Market Update and Observations on Big Memory
HPC Market Update and Observations on Big MemoryHPC Market Update and Observations on Big Memory
HPC Market Update and Observations on Big Memory
 
Live Data: For When Data is Greater than Memory
Live Data: For When Data is Greater than MemoryLive Data: For When Data is Greater than Memory
Live Data: For When Data is Greater than Memory
 
Big Memory for HPC
Big Memory for HPCBig Memory for HPC
Big Memory for HPC
 
Tech Talk: Moneyball - Hitting real-time apps out of the park with Big Memory
Tech Talk: Moneyball - Hitting real-time apps out of the park with Big MemoryTech Talk: Moneyball - Hitting real-time apps out of the park with Big Memory
Tech Talk: Moneyball - Hitting real-time apps out of the park with Big Memory
 
MemVerge Company Overview
MemVerge Company OverviewMemVerge Company Overview
MemVerge Company Overview
 
IDC Technology Spotlight: Big Memory Computing Emerges to Better Enable Dat...
IDC Technology Spotlight:   Big Memory Computing Emerges to Better Enable Dat...IDC Technology Spotlight:   Big Memory Computing Emerges to Better Enable Dat...
IDC Technology Spotlight: Big Memory Computing Emerges to Better Enable Dat...
 
Big Memory Webcast
Big Memory WebcastBig Memory Webcast
Big Memory Webcast
 

Recently uploaded

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
SOFTTECHHUB
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Neo4j
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
KAMESHS29
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Adtran
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 

Recently uploaded (20)

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 

Impact of Intel Optane Technology on HPC

  • 1. Impact of Intel® Optane™ Technology on HPC Andrey Kudryavtsev HPC Storage Architect Intel Corporation
  • 2. 2 Re-architecting the Memory/Storage Hierarchy Intel® Optane™ persistent memory (PMem) 200 series fills the DRAM gap 10s GB <0.1 microseconds 1s TB <10 microseconds 10s TB <100 microseconds 10s TB <10–100 milliseconds DRAM HOT TIER HDD/TAPE COLD TIER 100s GB <1 microsecond MEMORY STORAGE CAPACITY LATENCY IMPROVING Capacity and persistence IMPROVING SSD storage IMPROVING Efficient storage NAND SSD
  • 3. 3 Accelerate Insight and Innovation Intel® Optane™ persistent memory (PMem) 200 series fills the DRAM gap 3 Supports for more than 1.2x more VMs than the previous generation. Based on testing by Intel as of April 27, 2020 (Baseline) and March 31, 2020 (New). For details, see Endnotes slides. High performance. High capacity. Hardware-enhanced security. Lower overall total cost of ownership (TCO).3 Do more with each server—increase CPU utilization, in-memory database capacity, throughput, virtual machine (VM) density, and services for users on a consolidated footprint. Protect data automatically. Help secure all data at rest in persistent memory with application-transparent AES-256 encryption, which enhances security without code changes. Extract more from larger datasets. Expand your memory pool in persistent memory and support near-real-time data analysis; deliver deep insights, improve operations, or create new revenue streams.
  • 4. 4 persistent memory: in a league of its own HIGHER BANDWIDTH Up to 3.7X read/write bandwidth vs NVMe SSDs, with one module; more with multiple modules LOWER LATENCY Orders of magnitude lower latency than NVMe SSDs • 1000X lower latency than NAND NVMe SSD at 1GB/s Performance results are based on testing as of Feb 22, 2019 and may not reflect all publicly available security updates. No product or component can be absolutely secure. Results have been estimated based on tests conducted on pre-production systems, and provided to you for informational purposes. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to www.intel.com/benchmarks. 2, 3 Based on internal testing by Intel . Configuration: see slides 38 and 39. 3.7 X HIGHER bandwidth up to 10 00 X LOWER latency up to Intel DC P4610 NVMe SSD Intel Optane DC SSD P4800X
  • 5. 5 Intel® Optane™ Persistent Memory (PMem) 200 Series Operating Modes Memory Mode (MM) Intel Optane PMem–aware OS/VMM Intel Optane PMem–aware application Intel Optane PMem DRAM Persistent performance and high capacity Unmodified OS/VMM Unmodified application Volatile memory pool Affordable memory capacity for many applications DRAM as cache Intel Optane PMem Total memory: Intel Optane PMem size Total memory: Intel Optane PMem size + DRAM size App Direct Mode (AD) Storage over AD OS with NVDIMM driver Unmodified application Intel Optane PMem DRAM Acts as fast storage on the memory bus Avoids the latency of the I/O bus STORAGE ON MEMORY BUS Persistent VolatilePersistent Volatile
  • 6. 6 Intel® Optane™ Persistent Memory (PMem) at a Glance See backup for configuration details. For more complete information about performance and benchmark results, visit www.intel.com/benchmarks. 3 Based on testing by Intel as of April 27, 2020 (Baseline) and March 31, 2020 (New). For details, see Endnotes slides. * 3 TB = 6 x 512 GB Intel Optane PMem per socket ** 4 TB = 8 x 512 GB Intel Optane PMem per socket Intel Optane PMem 100 series 2nd Generation Intel® Xeon® Scalable processors SP on 2S/4S/8S platform 6 channels memory 2,666 MT/s DDR4 3 TB Intel Optane PMem per socket* 18 W top sustained Intel Optane PMem 200 series 3rd Generation Intel Xeon Scalable processors SP on 4S platform 2,666 MT/s DDR4 3 TB Intel Optane PMem per socket* 25% UP TO higher memory bandwidth over the previous generation3 256-BIT AES encryption Secure Erase 512 GB UP TO modules 12–15 W thermal design power 16–28 cores 6 channels memory
  • 7. 7 A Strong Memory and Storage Future For each processor above, Intel Optane PMem will be supported on select SKUs. 3rd Generation Intel Xeon Scalable processors (2 socket platform) Intel Optane persistent memory 200 series Intel® Optane™ persistent memory 100 series 2nd Generation Intel® Xeon® Scalable processors 3rd Generation Intel Xeon Scalable processors (4 socket platform) 3rd Gen Persistent Memory​ 4th Gen Persistent Memory​ Future2019 2020 2020H2
  • 8. 8 Intel Optane SSD Intel 3D NAND SSD accelerati ng caching Tiering Intel Optane SSD Intel 3D NAND SSD intelligent data placement storage compute intel® optane™ technology breaks memory and storage bottlenecks store data about data temporarily copy or hold hottest data Intel Optane Persistent Memory DRAM Intel Optane Persistent Memory DRAM caching Tiering copy of less frequently used data from DRAM app-aware data placement accelerati ngkeep more data in memory Intel® Optane™ Persistent Memory DRAM Intel® Optane™ SSD Intel® 3D NAND SSD
  • 9. Intel CorporationCloud & Enterprise Solutions Group 9 The Problem with POSIX & Blocks (& Objects) … Data Metadata Scientific Application I/O Middleware HDF5 MPI-IO SCR/VeloC Dataset Serialized into a POSIX File Block 1 Block 2 Block 3 Block 4 Dataset Serialized into a POSIX File Block 1 header size key1 size val1 val1 cont’d size sizekey2 val2 val2 cont’d val2 cont’d size size val3key3 val3 Block 2 Block 3 Block 4 Block 5 Block 6 AI/Analytics Application key1 val1 key2 val2 key3 val3 @ @ @ POSIX Serialization POSIX Serialization Block 1 Block 2 Block 3 Block 4 Block 5 Block 6
  • 10. Intel CorporationCloud & Enterprise Solutions Group 10 What is DAOS?  A new, innovative scale-out storage-as-a-service stack based on Intel Optane Persistent Memory and NVMe SSDs  Globally accessible from many nodes  Delivers exceptionally high bandwidth and IOPS on commodity servers  Can be utilized either as a standalone file system, or as a performance tier integrated with existing storage systems 10 More IOPs and bandwidth per dollar Network / Can Leverage RDMA DAOS Storage Engine Intel® Optane PMem NVMe Interface Metadata, low-latency I/Os & indexing/query Bulk data NVMe SSD Memory Interface Storage Instance AI/Analytics/Scientific Workflow DAOS library Files KVoF Spark… Compute Instance Objects (No)SQL Python
  • 11. Intel CorporationCloud & Enterprise Solutions Group 11 Application Interface Dataset Mover Capacity Tier DAOS Storage Tier POSIX I/O AI/Analytics/Scientific Workflow HDF5 MPI-IO Python Native API Legac y Apache Spark Apache Hadoop HDFS Connector
  • 12. Intel CorporationCloud & Enterprise Solutions Group 12 DAOS in the overall cluster architecture DAOS node #1 Optane DC Persistent Memory P4510/ P4610 SSDs Optane DC Persistent Memory P4510/ P4610 SSDs Optane DC Persistent Memory P4510/ P4610 SSDs DAOS node #2 DAOS node #Y DataPool1 Data Pool2 Application data Checkpoint Performance/$ optimized storage Capacity Storage All Flash or Hybrid HPC storage options: Lustre* IBM Spectrum Scale CEPH, NFS, etc HDD NVMe TLC, QLC SSDs DataPool3 DataPoolN DAOS Cluster Compute Node #1 Node #2 Node #N Remote memory Temp / Scratch Data movers Name space integration Data proxies Cloud Storage HDD AWS S3 Microsoft Azure etc Data proxies
  • 13. Intel CorporationCloud & Enterprise Solutions Group 13 DAOS and AI Best Play Together 13 Compute AI training workloads dominate more reads than writes Parallel throughput becomes critical at training stage for a shared data set Traditional parallel filesystems are not able to keep up with performance demand DAOS has a native capability to set the replication factor per object This can be used to bring higher bandwidth for the given object and increase parallelism and proximity to the data. • In addition to replication, DAOS has optimized object placement planned in the roadmap for version 2.2. DAOS further accelerates All Flash Arrays already been used with GPU accelerated compute nodes. proximity DAOS node #1 Optane DC Persistent MemoryP4510/ P4610 SSDs Optane DC Persistent MemoryP4510/ P4610 SSDs Optane DC Persistent MemoryP4510/ P4610 SSDs DAOS node #2 DAOS node #N Node #1 Node #2 Node #N Data Pool DatasetDatasetDataset copy N-copy proximity proximity DAOS
  • 14. Intel CorporationCloud & Enterprise Solutions Group 14 DAOS Community Roadmap 1Q20 2Q20 3Q20 4Q20 1Q21 2Q21 3Q21 4Q21 1Q22 2Q22 3Q22 1.0 1.2 2.0 2.2 2.4 Released on June 18 DAOS: - NVMe & DCPMM support - Per-pool ACL - UNS in DAOS via dfuse - Replication & self-healing (Preview) Application Interface: - MPI-IODriver - HDF5 DAOS Connector - Basic POSIX I/O support DAOS: - End-to-end data integrity - Per-container ACL - Improved control plane - Replication & self-healing - Conditional updates - Erasure Code (Preview) - Online server addition - Lustre/UNS integration Application Interface: - HDF5 vol w/o async - POSIX I/O with conditional update support - POSIX data mover - Async HDF5 operations over DAOS - Spark DAOS: - Erasure code - Telemetry& per-job statistics - Advanced control plane - Distributedtransactions - Multi OFI provider support Application Interface: - POSIX I/O with distributed transaction support - HDF5 data mover - Container parking/serialization DAOS: - Progressivelayout - Placement optimizations - Checksum scrubbing DAOS: - Catastrophic recovery tools NOTE: All information provided in this roadmap is subject to change without notice.
  • 15. Intel CorporationCloud & Enterprise Solutions Group 15 Resources  Github • https://github.com/daos-stack/daos  DAOS online documentation • http://daos.io  Community mailing list • https://daos.groups.io  Annual DAOS User Group in November • https://wiki.hpdd.intel.com/display/DC/DUG20