Databricks, profile picture
Uploaded byDatabricks
PDF, PPTX2,772 views

Accelerate Your Apache Spark with Intel Optane DC Persistent Memory

The document outlines Intel's offerings related to analytics solutions and persistent memory technologies, highlighting features like Intel Optane DC persistent memory and its compatibility with big data frameworks such as Spark. It discusses the performance of different memory configurations, emphasizing benchmarks and scenarios where persistent memory can significantly enhance workloads in data-intensive environments. Furthermore, it provides a technical overview of cache design, optimization strategies for Spark, and operational modes to improve data handling efficiency in various applications.

Embed presentation

Download as PDF, PPTX
WIFI SSID:SparkAISummit | Password: UnifiedAnalytics
Cheng Xu, Intel Balcer Piotr, Intel Accelerate your Spark with Intel Optane DC Persistent Memory #UnifiedAnalytics #SparkAISummit
Notices and Disclaimers 3 © 2018 Intel Corporation. Intel, the Intel logo, 3D XPoint, Optane, Xeon, Xeon logos, and Intel Optane logo are trademarks of Intel Corporation in the U.S. and/or other countries. All products, computer systems, dates, and figures specified are preliminary based on current expectations, and are subject to change without notice. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at intel.com. The cost reduction scenarios described are intended to enable you to get a better understanding of how the purchase of a given Intel based product, combined with a number of situation-specific variables, might affect future costs and savings. Circumstances will vary and there may be unaccounted-for costs related to the use and deployment of a given product. Nothing in this document should be interpreted as either a promise of or contract for a given level of costs or cost reduction. The benchmark results reported above may need to be revised as additional testing is conducted. The results depend on the specific platform configurations and workloads utilized in the testing, and may not be applicable to any particular user’s components, computer system or workloads. The results are not necessarily representative of other benchmarks and other benchmark results may show greater or lesser impact from mitigations. Results have been estimated based on tests conducted on pre-production systems, and provided to you for informational purposes. Any differences in your system hardware, software or configuration may affect your actual performance. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance results are based on testing as of 03-14-2019 and may not reflect all publicly available security updates. See configuration disclosure for details. No product can be absolutely secure. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to www.intel.com/benchmarks. Intel processors of the same SKU may vary in frequency or power as a result of natural variability in the production process. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Optimization Notice: Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice Revision #20110804. Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit www.intel.com/benchmarks. *Other names and brands may be claimed as the property of others.
About US Intel SSP/DAT – Data Analytics Technology ● Active open source contributions: About 30 committers from our team with significant contributions to a lot of big data key projects including Spark, Hadoop, HBase, Hive... ● A global engineering team (China, US, India) - Major team is based in Shanghai ● Mission: To fully optimize analytics solutions on IA and make Intel the best and easiest platform for big data and analytics customers Intel NVMS – Non-Volatile Memory Solutions Group • Tasked with preparing the ground for Intel® Optane™ DC Persistent Memory • Enabling new and existing software through Persistent Memory Development Kit (PMDK) • pmem.io 4
Out of memory Large Spill Some Challenges in Spark: Memory???
Re-architecting the Memory/Storage Hierarchy
7 Intel®Optane™DCPersistentMemory-ProductOverview (Optane™ based Memory Module for the Data Center) IMC Cascade Lake Server IMC • 128, 256, 512GB DIMM Capacity • 2666 MT/sec Speed • 3TB (not including DRAM) Capacity per CPU • DDR4 electrical & physical • Close to DRAM latency • Cache line size access * DIMM population shown as an example only.
PersistentmemoryOperatingModes Memory Mode & AppDirect 8
9 Intel®Optane™DCPersistentMemory-OperationalModes AppDirectMode memoryMode Persistent High availability / less downtime High capacity Affordable Significantly faster storage Ease of adoption† Note that a BIOS update will be required before using Intel persistent memory
10 INTEL®OPTANE™DCPERSISTENTMEMORYSupportforBreadthofapplications PERSISTENT PERFORMANCE & MAXIMUM CAPACITY APPLICATION VOLATILE MEMORY POOL O P T A N E P E R S I S T E N T M E M O R Y D R A M A S C A C H E AFFORDABLE MEMORY CAPACITY FOR MANY APPLICATIONS APPLICATION OPTANE PERSISTENT MEMORY DRAM
11 AppDirectModeOptions Legacy Storage APIs Block Atomicity Storage APIs with DAX (AppDirect) PersistentMemory USERSPACEKERNELSPACE Standard File API GenericNVDIMMDriver Application FileSystem Standard Raw Device Access mmap Load/ Store Standard File API pmem-Aware FileSystem MMU Mappings “DAX” BTT DevDAX PMDK mmap HARDWARE • No Code Changes Required • Operates in Blocks like SSD/HDD • Traditional read/write • Works with Existing File Systems • Atomicity at block level • Block size configurable • 4K, 512B* • NVDIMM Driver required • Support starting Kernel 4.2 • Configured as Boot Device • Higher Endurance than Enterprise SSDs • High Performance Block Storage • Low Latency, higher BW, High IOPs *Requires Linux • Code changes may be required* • Bypasses file system page cache • Requires DAX enabled file system • XFS, EXT4, NTFS • No Kernel Code or interrupts • No interrupts • Fastest IO path possible * Code changes required for load/store direct access if the application does not already support this.
Low Bandwidth IO storage (e.g. HDD, S3) RDD Cache SparkDCPMMoptimizationOverview OAP Cache DRAM IO Layer Compute Layer Spark Input Data Cache Hit Cache Miss DRAM Tied Storage RDD cache DCPMM optimization (Against DRAM): ● Reduce DRAM footprint ● Higher Capacity to cache more data ● Status: To Be Added OAP DCPMM optimization (Against DRAM) ● High Capacity, Less Cache Miss ● Avoid heavy cost disk read ● Status: To Be Added 9 I/O intensive Workload (from Decision Support Queries) K-means Workload
Use Case 1: Spark SQL OAP I/O Cache
OAP (Optimized Analytics Package) Goal • IO cache is critical for I/O intensive workload especially on low bandwidth environment (e.g. Cloud, On-Prem HDD based system) • Make full use of the advantage from DCPMM to speed up Spark SQL • High capacity and high throughput • No latency and reduced DRAM footprint • Better performance per TCO Feature • Fine grain cache (e.g. column chunk for Parquet) columnar based cache • Cache aware scheduler (V2 API via preferred location) • Self managed DCPMM pool (no extra GC overhead) • Easy to use (easily turn on/off), transparent to user (no changes for their queries) 14 https://github.com/Intel-bigdata/OAP
OAP 15 SparkDCPMMFullSoftwareStack SQL workload Spark SQL VMEMCACHE PMEM-AWARE File SYSTEM Intel® Optane™ DC Persistent Memory Module Unmodified SQL Unchanged Spark Provide scheduler and fine gain cache based on Data Source API Abstract away hardware details and cache implementation Expose persistent memory as memory-mapped files (DAX) Native library to access DCPMM
Deployment Overview Server 1 Local Storage (HDD) Spark Executor Spark Gateway (e.g. ThriftServer, Spark shell)SQL Cached Data Source (v1/v2) Task scheduled Intel Optane DC Persistent Memory Cache Hit Cache Miss Server 2 Native library (vmemcache) Cache Aware Scheduler
Cache Design - Problem statement • Local LRU cache • Support for large capacities available with persistent memory (many terabytes per server) • Lightweight, efficient and embeddable • In-memory • Scalable
Cache Design - Existing solutions • In-memory databases tend to rely on malloc() in some form for allocating memory for entries – Which means allocating anonymous memory • Persistent Memory is exposed by the operating system through normal file- system operations – Which means allocating byte-addressable PMEM needs to use file memory mapping (fsdax). • We could modify the allocator of an existing in-memory database and be done with it, right? J
Cache Design - Fragmentation • Manual dynamic memory management a’la dlmalloc/jemalloc/tcmalloc/palloc causes fragmentation • Applications with substantial expected runtime durations need a way to combat this problem – Compacting GC (Java, .NET) – Defragmentation (Redis, Apache Ignite) – Slab allocation (memcached) • Especially so if there’s substantial expected variety in allocated sizes
Cache Design - Extent allocation • If fragmentation is unavoidable, and defragmentation/compacting is CPU and memory bandwidth intensive, let’s embrace it! • Usually only done in relatively large blocks in file-systems. • But on PMEM, we are no longer restricted by large transfer units (sectors, pages etc)
Cache Design - Scalable replacement policy • Performance of libvmemcache was bottlenecked by naïve implementation of LRU based on a doubly-linked list. • With 100st of threads, most of the time of any request was spent waiting on a list lock… • Locking per-node doesn’t solve the problem…
Cache Design - Buffered LRU • Our solution was quite simple. • We’ve added a wait-free ringbuffer which buffers the list-move operations • This way, the list only needs to get locked during eviction or when the ringbuffer is full.
Cache Design - Lightweight, embeddable, in-memory caching https://github.com/pmem/vmemcache VMEMcache *cache = vmemcache_new("/tmp", VMEMCACHE_MIN_POOL, VMEMCACHE_MIN_EXTENT, VMEMCACHE_REPLACEMENT_LRU); const char *key = "foo"; vmemcache_put(cache, key, strlen(key), "bar", sizeof("bar")); char buf[128]; ssize_t len = vmemcache_get(cache, key, strlen(key), buf, sizeof(buf), 0, NULL); vmemcache_delete(cache); libvmemcache has normal get/put APIs, optional replacement policy, and configurable extent size Works with terabyte-sized in-memory workloads without a sweat, with very high space utilization. Also works on regular DRAM.
24 Parquet File Footer RowGroup #1 Column Chunk #1 Column Chunk #2 Column Chunk #N RowGroup #2 RowGroup #N Guava Based Cache Vmemcache Fine Grain Cache Cache Status & Report Cache Design – Status and Fine Grain
25 Experiments and Configurations DCPMM DRAM Hardware DRAM 192GB (12x 16GB DDR4) 768GB (24x 32GB DDR4) Intel Optane DC Persistent Memory 1TB (QS: 8 x 128GB) N/A DCPMM Mode App Direct (Memkind) N/A SSD N/A N/A CPU 2 * Cascadelake 8280M (Thread(s) per core: 2, Core(s) per socket: 28, Socket(s): 2 CPU max MHz: 4000.0000 CPU min MHz: 1000.0000 L1d cache: 32K, L1i cache: 32K, L2 cache: 1024K, L3 cache: 39424) OS 4.20.4-200.fc29.x86_64 (BKC: WW06'19, BIOS: SE5C620.86B.0D.01.0134.100420181737) Software OAP 1TB DCPMM based OAP cache 610GB DRAM based OAP cache Hadoop 8 * HDD disk (ST1000NX0313, 1-replica uncompressed & plain encoded data on Hadoop) Spark 1 * Driver (5GB) + 2 * Executor (62 cores, 74GB), spark.sql.oap.rowgroup.size=1MB JDK Oracle JDK 1.8.0_161 Workload Data Scale 2TB, 3TB, 4TB Decision Making Queries 9 I/O intensive queries Multi-Tenants 9 threads (Fair scheduled)
Test Scenario 1: Both Fit In DRAM And DCPMM - 2TB *The performance gain can be much lower if IO is improved (e.g. compression & encoding enabled) or some hacking codes fixed (e.g. NUMA scheduler) With 2TB Data Scale, both DCPMM and DRAM based OAP cache can hold the full dataset (613GB). DCPMM is 24.6% (=1- 100.1/132.81) performance lower than DRAM as measured on 9 I/O intensive decision making queries. On Premise Performance Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks. Configurations: See page20.Test by Intel on 24/02/2019.
Test Scenario 2: Fit In DCPMM Not For DRAM - 3TB *The performance gain can be much lower if IO is improved (e.g. compression & encoding enabled) or some hacking codes fixed (e.g. NUMA scheduler) With 3TB Data Scale, only DCPMM based OAP cache can hold the full dataset (920GB). DCPMM shows 8X* performance gain over DRAM as measured on 9 I/O intensive decision making queries. On Premise Performance Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks. Configurations: See page20.Test by Intel on 24/02/2019.
Performance Analysis - System Metrics ● Input data is all cached in DCPMM while partially for DRAM ● DCPMM reaches up to 18GB/sbandwidth while DRAM case it’s bounded by disk IO which is only about 250MB/s ~ 450MB/s ● OAP cache doesn’t apply for shuffle data (intermediate data) that extra IO pressure put onto DRAM case ● DCPMM reads about 27.5GBwhile DRAM reads about 395.6GB OAP Cache works avoiding disk read (blue) and only disk write for shuffle (red) Disk read (blue) comes from shuffle data Disk read (blue) happens from time to time DCPMM
Test Scenario 3: None Of DRAM &DCPMM Fit - 4TB *The performance gain can be much lower if IO is improved (e.g. compression & encoding enabled) or some hacking codes fixed (e.g. NUMA scheduler) With 4TB Data Scale, none of DCPMM and DRAM based OAP cache can hold the full dataset (1226.7GB). DCPMM shows 1.66X* (=2252.80/1353.67) performance gain over DRAM as measured on 9 I/O intensive decision making queries. On Premise Performance Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks. Configurations: See page20.Test by Intel on 24/02/2019.
Use Case 2: Machine Learning Spark K-means
DCPMM Storage Level DRAM Disk (SSD, HDD) Object Byte buffer Byte buffer Spark Core (Compute/Storage Memory) DRAM (on heap) DRAM (off heap) Disk K-means workload GraphX workload Decompression Deseriliazation • Spark Storage Level • A few storage levels serving for different purposes including memory and disk • Off-heap memory is supported to avoid GC overhead in Java • Large capacity storage level (disk) is widely used for iterative computation workload (e.g. K-means, GraphX workloads) by caching hot data in storage level • DCPMM Storage Level • Extend memory layer • Using Pmem library to access DCPMM avoiding the overhead of decompression from disk • Large capacity and high I/O performance of DCPMM shows better performance than tied solution (original DRAM + Disk solution)
32 K-means in Spark M0 - Mj Mj+1 - Mk Mk+1 - Mm … Mx+1 - Mn HDFS Cache #1 (DRAM + DISK) Cache #2 (DRAM + DISK) Cache #3 (DRAM + DISK) … Cache #N (DRAM + DISK) Spark Executor Processes Loading Normalization Caching C1, C2, C3, … CK For Each Record in the Cache For Each Record in the Cache For Each Record in the Cache For Each Record in the Cache For Each Record in the Cache Step 1: For each record in the cache, Sum the vectors with the same closet centroid Update the new Centroids Step 2: Sum the vectors according to the centroids and find out the new centroids globally Sync N iterations Load TrainRandom Initial Centroids 2 Iterations for all of the cache data to get the K centroids in a random mode Load the data from HDFS to DRAM (and DCPMM / SSD if DRAM cannot hold all of the data)(Load), after that, the data will not be changed, and will be iterated repeatedly in Initialization and Train stages.
Compute Node 33 K-means Basic Data Flow • Load data from HDFS to memory. • Spill over to local storage Load • Compute using initial centroid based on data in memory or local storage Initialization • Compute iterations based on local data Train CPU Memory Storage Local Shared Compute Node CPU Memory Storage Local Shared Compute Node CPU Memory Storage Local Shared HDFS … … K centroids C1, C2, C3, … CK 1 2 3 1 2 3
34 Experiments and Configurations #1 AD #2 DRAM Hardware DRAM 192GB (12x 16GB DDR4 2666) 768GB (24× 32GB DDR4 2666) Intel Optane DC Persistent Memory 1TB (8x 128GB QS) N/A DCPMM Mode AppDirect N/A CPU Intel(R) Xeon(R) Platinum 8280L CPU @ 2.70GHz Disk Drive 8x Intel DC S4510 SSD (1.8T) + 2 * P4500 (1.8T) Software CPU / Memory Allocation Driver (5GB + 10 cores) + 2 * Executor (80GB + 32 Cores) Software Stack Spark (2.2.2-snapshot: bb3e6ed9216f98f3a3b96c8c52f20042d65e2181) + Hadoop (2.7.5) OS & Kernel & BIOS Linux-4.18.8-100.fc27.x86_64-x86_64-with-fedora-27-Twenty_Seven BIOS: SE5C620.86B.0D.01.0299.122420180146 Mitigation variants (1,2,3,3a,4, L1TF) 1,2,3,3a,4, L1TF Cache Spill Disk N/A 8 * SSD NUMA Bind 2 NUMA Nodes bind 2 Spark Executors respectively Data Cache Size (OffHeap) DCPMM (no spill) 680GB DRAM (partially spill) Workload Record Count / Data Size / Memory Footprint Row size 6.1 Billion Records / data size 1.1TB / cache data 954GB Cache Spill Size 0 235GB Kmeans (Iteration times) 10 Kmeans (K value) 5 1 Modified to support DCPMM
35 Performance Comparison(DCPMM AD V.S. DRAM) • DCPMM provides larger “memory” than DRAM and avoid the data spill in this case, hence got up to 4.98X performance in Training(READ), and 2.85X end to end performance; • The DCPMM AppDirect mode is about ~19% faster than DRAM in Load(WRITE cache into DCPMM / DRAM); • Training(READ) execution gap of DRAM case caused by storage media, as well as the data decompression and de-serialization from the local spill file; 4.98X1.19X 2.85X Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks. Configurations: See page11.Test by Intel on 14/03/2019. DCPMM DCPMM
DRAM System Analysis DCPMM load training trainingload DRAM ● SSD read bandwidth is 2.86GB/s ● It’s I/O bound leading to low CPU utilization load training load training DCPMM ● Much high CPU utilization than DRAM only system
• Intel Optane DC Persistent Memory brings high capacity, low latency and high throughput • Two operation modes for DCPMM: 1) App-Direct 2) 2LM • Good for Spark in: 1) memory bound workload 2) I/O intensive workload Summary
Want to know more? See you at Intel Booth 100 Download source code from Github: https://github.com/intel-bigdata/OAP https://github.com/pmem/vmemcache
DON’T FORGET TO RATE AND REVIEW THE SESSIONS SEARCH SPARK + AI SUMMIT

More Related Content

PDF
Supporting Over a Thousand Custom Hive User Defined Functions
byDatabricks
 
PDF
Best Practice of Compression/Decompression Codes in Apache Spark with Sophia...
byDatabricks
 
PDF
Data Security at Scale through Spark and Parquet Encryption
byDatabricks
 
PDF
FPGA-Based Acceleration Architecture for Spark SQL Qi Xie and Quanfu Wang
bySpark Summit
 
PDF
Elastify Cloud-Native Spark Application with Persistent Memory
byDatabricks
 
PDF
Apache Arrow-Based Unified Data Sharing and Transferring Format Among CPU and...
byDatabricks
 
PDF
Improving Apache Spark by Taking Advantage of Disaggregated Architecture
byDatabricks
 
PDF
Spark Summit EU talk by Mike Percy
bySpark Summit
 
Supporting Over a Thousand Custom Hive User Defined Functions
byDatabricks
 
Best Practice of Compression/Decompression Codes in Apache Spark with Sophia...
byDatabricks
 
Data Security at Scale through Spark and Parquet Encryption
byDatabricks
 
FPGA-Based Acceleration Architecture for Spark SQL Qi Xie and Quanfu Wang
bySpark Summit
 
Elastify Cloud-Native Spark Application with Persistent Memory
byDatabricks
 
Apache Arrow-Based Unified Data Sharing and Transferring Format Among CPU and...
byDatabricks
 
Improving Apache Spark by Taking Advantage of Disaggregated Architecture
byDatabricks
 
Spark Summit EU talk by Mike Percy
bySpark Summit
 

What's hot

PDF
Next CERN Accelerator Logging Service with Jakub Wozniak
bySpark Summit
 
PPTX
Node Labels in YARN
byDataWorks Summit
 
PDF
Hardware Acceleration of Apache Spark on Energy-Efficient FPGAs with Christof...
bySpark Summit
 
PDF
Unleashing Data Intelligence with Intel and Apache Spark with Michael Greene
byDatabricks
 
PDF
Sherlock: an anomaly detection service on top of Druid
byDataWorks Summit
 
PDF
Speed up UDFs with GPUs using the RAPIDS Accelerator
byDatabricks
 
PDF
Downscaling: The Achilles heel of Autoscaling Apache Spark Clusters
byDatabricks
 
PPTX
The columnar roadmap: Apache Parquet and Apache Arrow
byDataWorks Summit
 
PDF
An Adaptive Execution Engine for Apache Spark with Carson Wang and Yucai Yu
byDatabricks
 
PDF
Accelerating Apache Spark-based Analytics on Intel Architecture-(Michael Gree...
bySpark Summit
 
PDF
Fast and Reliable Apache Spark SQL Releases
byDataWorks Summit
 
PDF
Deep Dive of ADBMS Migration to Apache Spark—Use Cases Sharing
byDatabricks
 
PDF
Accelerating Apache Spark Shuffle for Data Analytics on the Cloud with Remote...
byDatabricks
 
PDF
Magnet Shuffle Service: Push-based Shuffle at LinkedIn
byDatabricks
 
PDF
Near Data Computing Architectures: Opportunities and Challenges for Apache Spark
byAhsan Javed Awan
 
PDF
Designing and Building Next Generation Data Pipelines at Scale with Structure...
byDatabricks
 
PDF
Migrating ETL Workflow to Apache Spark at Scale in Pinterest
byDatabricks
 
PDF
Cooperative Task Execution for Apache Spark
byDatabricks
 
PDF
Creating an 86,000 Hour Speech Dataset with Apache Spark and TPUs
byDatabricks
 
PDF
Spark SQL Catalyst Code Optimization using Function Outlining with Kavana Bha...
byDatabricks
 
Next CERN Accelerator Logging Service with Jakub Wozniak
bySpark Summit
 
Node Labels in YARN
byDataWorks Summit
 
Hardware Acceleration of Apache Spark on Energy-Efficient FPGAs with Christof...
bySpark Summit
 
Unleashing Data Intelligence with Intel and Apache Spark with Michael Greene
byDatabricks
 
Sherlock: an anomaly detection service on top of Druid
byDataWorks Summit
 
Speed up UDFs with GPUs using the RAPIDS Accelerator
byDatabricks
 
Downscaling: The Achilles heel of Autoscaling Apache Spark Clusters
byDatabricks
 
The columnar roadmap: Apache Parquet and Apache Arrow
byDataWorks Summit
 
An Adaptive Execution Engine for Apache Spark with Carson Wang and Yucai Yu
byDatabricks
 
Accelerating Apache Spark-based Analytics on Intel Architecture-(Michael Gree...
bySpark Summit
 
Fast and Reliable Apache Spark SQL Releases
byDataWorks Summit
 
Deep Dive of ADBMS Migration to Apache Spark—Use Cases Sharing
byDatabricks
 
Accelerating Apache Spark Shuffle for Data Analytics on the Cloud with Remote...
byDatabricks
 
Magnet Shuffle Service: Push-based Shuffle at LinkedIn
byDatabricks
 
Near Data Computing Architectures: Opportunities and Challenges for Apache Spark
byAhsan Javed Awan
 
Designing and Building Next Generation Data Pipelines at Scale with Structure...
byDatabricks
 
Migrating ETL Workflow to Apache Spark at Scale in Pinterest
byDatabricks
 
Cooperative Task Execution for Apache Spark
byDatabricks
 
Creating an 86,000 Hour Speech Dataset with Apache Spark and TPUs
byDatabricks
 
Spark SQL Catalyst Code Optimization using Function Outlining with Kavana Bha...
byDatabricks
 

Similar to Accelerate Your Apache Spark with Intel Optane DC Persistent Memory

PPTX
Impact of Intel Optane Technology on HPC
byMemVerge
 
PDF
Intel Optane Data Center Persistent Memory
byinside-BigData.com
 
PPTX
Reimagining HPC Compute and Storage Architecture with Intel Optane Technology
byinside-BigData.com
 
PDF
Optimizing Apache Spark Throughput Using Intel Optane and Intel Memory Drive...
byDatabricks
 
PDF
Disrupt the Storage & Memory Hierarchy
byIntelÂŽ Software
 
PPTX
Developing Software for Persistent Memory / Willhalm Thomas (Intel)
byOntico
 
PDF
Intel and DataStax: 3D XPoint and NVME Technology Cassandra Storage Comparison
byDataStax Academy
 
PDF
Intel: How to Use Alluxio to Accelerate BigData Analytics on the Cloud and Ne...
byAlluxio, Inc.
 
PDF
Watch your transactional database performance climb with Intel Optane DC pers...
byPrincipled Technologies
 
PDF
Overcoming Scaling Challenges in MongoDB Deployments with SSD
byMongoDB
 
PDF
Persistent Memory Development Kit (PMDK) Essentials: Part 1
byIntelÂŽ Software
 
PDF
Persistent Memory Development Kit (PMDK) Essentials: Part 2
byIntelÂŽ Software
 
PDF
hbaseconasia2019 HBase Bucket Cache on Persistent Memory
byMichael Stack
 
PDF
Accelerate and Scale Big Data Analytics with Disaggregated Compute and Storage
byAlluxio, Inc.
 
PDF
Crooke CWF Keynote FINAL final platinum
byAlan Frost
 
PDF
Give your VDI users the memory they need with server technology from Dell EMC...
byPrincipled Technologies
 
PDF
Deep Dive On Intel Optane SSDs And New Server Platforms
byNEXTtour
 
PPTX
Training - HPE and Intel Optane SSD Solution.PPTX
byali2k2sec
 
PDF
Big data intel platform commenting
byIntel IT Center
 
PDF
Tech Talk: Moneyball - Hitting real-time apps out of the park with Big Memory
byMemVerge
 
Impact of Intel Optane Technology on HPC
byMemVerge
 
Intel Optane Data Center Persistent Memory
byinside-BigData.com
 
Reimagining HPC Compute and Storage Architecture with Intel Optane Technology
byinside-BigData.com
 
Optimizing Apache Spark Throughput Using Intel Optane and Intel Memory Drive...
byDatabricks
 
Disrupt the Storage & Memory Hierarchy
byIntelÂŽ Software
 
Developing Software for Persistent Memory / Willhalm Thomas (Intel)
byOntico
 
Intel and DataStax: 3D XPoint and NVME Technology Cassandra Storage Comparison
byDataStax Academy
 
Intel: How to Use Alluxio to Accelerate BigData Analytics on the Cloud and Ne...
byAlluxio, Inc.
 
Watch your transactional database performance climb with Intel Optane DC pers...
byPrincipled Technologies
 
Overcoming Scaling Challenges in MongoDB Deployments with SSD
byMongoDB
 
Persistent Memory Development Kit (PMDK) Essentials: Part 1
byIntelÂŽ Software
 
Persistent Memory Development Kit (PMDK) Essentials: Part 2
byIntelÂŽ Software
 
hbaseconasia2019 HBase Bucket Cache on Persistent Memory
byMichael Stack
 
Accelerate and Scale Big Data Analytics with Disaggregated Compute and Storage
byAlluxio, Inc.
 
Crooke CWF Keynote FINAL final platinum
byAlan Frost
 
Give your VDI users the memory they need with server technology from Dell EMC...
byPrincipled Technologies
 
Deep Dive On Intel Optane SSDs And New Server Platforms
byNEXTtour
 
Training - HPE and Intel Optane SSD Solution.PPTX
byali2k2sec
 
Big data intel platform commenting
byIntel IT Center
 
Tech Talk: Moneyball - Hitting real-time apps out of the park with Big Memory
byMemVerge
 

More from Databricks

PPTX
DW Migration Webinar-March 2022.pptx
byDatabricks
 
PPTX
Data Lakehouse Symposium | Day 1 | Part 1
byDatabricks
 
PPT
Data Lakehouse Symposium | Day 1 | Part 2
byDatabricks
 
PPTX
Data Lakehouse Symposium | Day 2
byDatabricks
 
PPTX
Data Lakehouse Symposium | Day 4
byDatabricks
 
PDF
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
byDatabricks
 
PDF
Democratizing Data Quality Through a Centralized Platform
byDatabricks
 
PDF
Learn to Use Databricks for Data Science
byDatabricks
 
PDF
Why APM Is Not the Same As ML Monitoring
byDatabricks
 
PDF
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
byDatabricks
 
PDF
Stage Level Scheduling Improving Big Data and AI Integration
byDatabricks
 
PDF
Simplify Data Conversion from Spark to TensorFlow and PyTorch
byDatabricks
 
PDF
Scaling your Data Pipelines with Apache Spark on Kubernetes
byDatabricks
 
PDF
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
byDatabricks
 
PDF
Sawtooth Windows for Feature Aggregations
byDatabricks
 
PDF
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
byDatabricks
 
PDF
Re-imagine Data Monitoring with whylogs and Spark
byDatabricks
 
PDF
Raven: End-to-end Optimization of ML Prediction Queries
byDatabricks
 
PDF
Processing Large Datasets for ADAS Applications using Apache Spark
byDatabricks
 
PDF
Massive Data Processing in Adobe Using Delta Lake
byDatabricks
 
DW Migration Webinar-March 2022.pptx
byDatabricks
 
Data Lakehouse Symposium | Day 1 | Part 1
byDatabricks
 
Data Lakehouse Symposium | Day 1 | Part 2
byDatabricks
 
Data Lakehouse Symposium | Day 2
byDatabricks
 
Data Lakehouse Symposium | Day 4
byDatabricks
 
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
byDatabricks
 
Democratizing Data Quality Through a Centralized Platform
byDatabricks
 
Learn to Use Databricks for Data Science
byDatabricks
 
Why APM Is Not the Same As ML Monitoring
byDatabricks
 
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
byDatabricks
 
Stage Level Scheduling Improving Big Data and AI Integration
byDatabricks
 
Simplify Data Conversion from Spark to TensorFlow and PyTorch
byDatabricks
 
Scaling your Data Pipelines with Apache Spark on Kubernetes
byDatabricks
 
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
byDatabricks
 
Sawtooth Windows for Feature Aggregations
byDatabricks
 
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
byDatabricks
 
Re-imagine Data Monitoring with whylogs and Spark
byDatabricks
 
Raven: End-to-end Optimization of ML Prediction Queries
byDatabricks
 
Processing Large Datasets for ADAS Applications using Apache Spark
byDatabricks
 
Massive Data Processing in Adobe Using Delta Lake
byDatabricks
 

Recently uploaded

PPTX
Query Parsing: The SEO Gatekeeper That Decides Rankings
bySemantic SEO BD
 
PDF
Step by Step Guide to Buying a Old or Aged Verified Paxful Accounts in US.pdf
byTopSelleriTDotCom
 
PDF
PathRAG rag pipeline PathRAG rag pipeline .pdf
byhotslix023
 
PPTX
250106204341-c238c4fadonebeforelastyear.pptx
byAbduRouf3
 
PDF
Analysis of Union Budget 2026-27: Major Announcements and their Impact
byDeeshiPavecha
 
PDF
LECTURE - Overcoming the AI Failure Rate - AIG, DQM.pdf
bySami Laine
 
PDF
Understanding Data Analytics: Concepts, Types, and Use Cases
byThe IoT Academy
 
PPTX
Organizational Structure and Design Topic 4
bydavenieralayaay
 
PPTX
inbound4709748695955476507 Deformable bodies.pptx
byaltheabautistam
 
PPTX
GENPHYSICS-2.pptx this about general physics. Grade 11 lesson
byGwynMorata
 
PPTX
End to End Supply Chain Management- Analytics Case Study.
byNandhuNandha1
 
PPTX
Lecture 14- Introduction to Hypothesis Testing.pptx
bygbadamosiayomide
 
PDF
chapter one introduction to Probability theory.pdf
bySeid42
 
PPTX
Top Institute for Data Analyst Course in Navi Mumbai
byPatil sagar
 
PDF
Columbus Data and Analytics Wednesday - People Analytics 101: Making Sense of...
byTim Wilson
 
PDF
Trustworthy AI : Governance of AI through Ethics
byKarim BaĂŻna
 
PDF
Khan Traders Fish Meal – MSDS Material Safety Data Sheet
byKhan Traders Karachi
 
PPTX
I need all your support to get the data for the sections like photo gallery o...
byhsharsha1
 
PPTX
Social Media Audit of Starbucks Coffee Company.pptx
bydiviap21
 
PDF
US Digital Fan Engagement Index 2026: Benchmarking Online Demand for Sports T...
byHyperset Group Ltd
 
Query Parsing: The SEO Gatekeeper That Decides Rankings
bySemantic SEO BD
 
Step by Step Guide to Buying a Old or Aged Verified Paxful Accounts in US.pdf
byTopSelleriTDotCom
 
PathRAG rag pipeline PathRAG rag pipeline .pdf
byhotslix023
 
250106204341-c238c4fadonebeforelastyear.pptx
byAbduRouf3
 
Analysis of Union Budget 2026-27: Major Announcements and their Impact
byDeeshiPavecha
 
LECTURE - Overcoming the AI Failure Rate - AIG, DQM.pdf
bySami Laine
 
Understanding Data Analytics: Concepts, Types, and Use Cases
byThe IoT Academy
 
Organizational Structure and Design Topic 4
bydavenieralayaay
 
inbound4709748695955476507 Deformable bodies.pptx
byaltheabautistam
 
GENPHYSICS-2.pptx this about general physics. Grade 11 lesson
byGwynMorata
 
End to End Supply Chain Management- Analytics Case Study.
byNandhuNandha1
 
Lecture 14- Introduction to Hypothesis Testing.pptx
bygbadamosiayomide
 
chapter one introduction to Probability theory.pdf
bySeid42
 
Top Institute for Data Analyst Course in Navi Mumbai
byPatil sagar
 
Columbus Data and Analytics Wednesday - People Analytics 101: Making Sense of...
byTim Wilson
 
Trustworthy AI : Governance of AI through Ethics
byKarim BaĂŻna
 
Khan Traders Fish Meal – MSDS Material Safety Data Sheet
byKhan Traders Karachi
 
I need all your support to get the data for the sections like photo gallery o...
byhsharsha1
 
Social Media Audit of Starbucks Coffee Company.pptx
bydiviap21
 
US Digital Fan Engagement Index 2026: Benchmarking Online Demand for Sports T...
byHyperset Group Ltd
 

Accelerate Your Apache Spark with Intel Optane DC Persistent Memory

  • 1.
    WIFI SSID:SparkAISummit |Password: UnifiedAnalytics
  • 2.
    Cheng Xu, Intel BalcerPiotr, Intel Accelerate your Spark with Intel Optane DC Persistent Memory #UnifiedAnalytics #SparkAISummit
  • 3.
    Notices and Disclaimers 3 ©2018 Intel Corporation. Intel, the Intel logo, 3D XPoint, Optane, Xeon, Xeon logos, and Intel Optane logo are trademarks of Intel Corporation in the U.S. and/or other countries. All products, computer systems, dates, and figures specified are preliminary based on current expectations, and are subject to change without notice. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at intel.com. The cost reduction scenarios described are intended to enable you to get a better understanding of how the purchase of a given Intel based product, combined with a number of situation-specific variables, might affect future costs and savings. Circumstances will vary and there may be unaccounted-for costs related to the use and deployment of a given product. Nothing in this document should be interpreted as either a promise of or contract for a given level of costs or cost reduction. The benchmark results reported above may need to be revised as additional testing is conducted. The results depend on the specific platform configurations and workloads utilized in the testing, and may not be applicable to any particular user’s components, computer system or workloads. The results are not necessarily representative of other benchmarks and other benchmark results may show greater or lesser impact from mitigations. Results have been estimated based on tests conducted on pre-production systems, and provided to you for informational purposes. Any differences in your system hardware, software or configuration may affect your actual performance. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance results are based on testing as of 03-14-2019 and may not reflect all publicly available security updates. See configuration disclosure for details. No product can be absolutely secure. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to www.intel.com/benchmarks. Intel processors of the same SKU may vary in frequency or power as a result of natural variability in the production process. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Optimization Notice: Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice Revision #20110804. Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit www.intel.com/benchmarks. *Other names and brands may be claimed as the property of others.
  • 4.
    About US Intel SSP/DAT– Data Analytics Technology ● Active open source contributions: About 30 committers from our team with significant contributions to a lot of big data key projects including Spark, Hadoop, HBase, Hive... ● A global engineering team (China, US, India) - Major team is based in Shanghai ● Mission: To fully optimize analytics solutions on IA and make Intel the best and easiest platform for big data and analytics customers Intel NVMS – Non-Volatile Memory Solutions Group • Tasked with preparing the ground for Intel® Optane™ DC Persistent Memory • Enabling new and existing software through Persistent Memory Development Kit (PMDK) • pmem.io 4
  • 5.
    Out of memoryLarge Spill Some Challenges in Spark: Memory???
  • 6.
  • 7.
    7 Intel®Optane™DCPersistentMemory-ProductOverview (Optane™ based MemoryModule for the Data Center) IMC Cascade Lake Server IMC • 128, 256, 512GB DIMM Capacity • 2666 MT/sec Speed • 3TB (not including DRAM) Capacity per CPU • DDR4 electrical & physical • Close to DRAM latency • Cache line size access * DIMM population shown as an example only.
  • 8.
  • 9.
    9 Intel®Optane™DCPersistentMemory-OperationalModes AppDirectMode memoryMode Persistent High availability/ less downtime High capacity Affordable Significantly faster storage Ease of adoption† Note that a BIOS update will be required before using Intel persistent memory
  • 10.
    10 INTEL®OPTANE™DCPERSISTENTMEMORYSupportforBreadthofapplications PERSISTENT PERFORMANCE & MAXIMUMCAPACITY APPLICATION VOLATILE MEMORY POOL O P T A N E P E R S I S T E N T M E M O R Y D R A M A S C A C H E AFFORDABLE MEMORY CAPACITY FOR MANY APPLICATIONS APPLICATION OPTANE PERSISTENT MEMORY DRAM
  • 11.
    11 AppDirectModeOptions Legacy Storage APIs BlockAtomicity Storage APIs with DAX (AppDirect) PersistentMemory USERSPACEKERNELSPACE Standard File API GenericNVDIMMDriver Application FileSystem Standard Raw Device Access mmap Load/ Store Standard File API pmem-Aware FileSystem MMU Mappings “DAX” BTT DevDAX PMDK mmap HARDWARE • No Code Changes Required • Operates in Blocks like SSD/HDD • Traditional read/write • Works with Existing File Systems • Atomicity at block level • Block size configurable • 4K, 512B* • NVDIMM Driver required • Support starting Kernel 4.2 • Configured as Boot Device • Higher Endurance than Enterprise SSDs • High Performance Block Storage • Low Latency, higher BW, High IOPs *Requires Linux • Code changes may be required* • Bypasses file system page cache • Requires DAX enabled file system • XFS, EXT4, NTFS • No Kernel Code or interrupts • No interrupts • Fastest IO path possible * Code changes required for load/store direct access if the application does not already support this.
  • 12.
    Low Bandwidth IOstorage (e.g. HDD, S3) RDD Cache SparkDCPMMoptimizationOverview OAP Cache DRAM IO Layer Compute Layer Spark Input Data Cache Hit Cache Miss DRAM Tied Storage RDD cache DCPMM optimization (Against DRAM): ● Reduce DRAM footprint ● Higher Capacity to cache more data ● Status: To Be Added OAP DCPMM optimization (Against DRAM) ● High Capacity, Less Cache Miss ● Avoid heavy cost disk read ● Status: To Be Added 9 I/O intensive Workload (from Decision Support Queries) K-means Workload
  • 13.
    Use Case 1:Spark SQL OAP I/O Cache
  • 14.
    OAP (Optimized AnalyticsPackage) Goal • IO cache is critical for I/O intensive workload especially on low bandwidth environment (e.g. Cloud, On-Prem HDD based system) • Make full use of the advantage from DCPMM to speed up Spark SQL • High capacity and high throughput • No latency and reduced DRAM footprint • Better performance per TCO Feature • Fine grain cache (e.g. column chunk for Parquet) columnar based cache • Cache aware scheduler (V2 API via preferred location) • Self managed DCPMM pool (no extra GC overhead) • Easy to use (easily turn on/off), transparent to user (no changes for their queries) 14 https://github.com/Intel-bigdata/OAP
  • 15.
    OAP 15 SparkDCPMMFullSoftwareStack SQL workload Spark SQL VMEMCACHE PMEM-AWARE FileSYSTEM Intel® Optane™ DC Persistent Memory Module Unmodified SQL Unchanged Spark Provide scheduler and fine gain cache based on Data Source API Abstract away hardware details and cache implementation Expose persistent memory as memory-mapped files (DAX) Native library to access DCPMM
  • 16.
    Deployment Overview Server 1 LocalStorage (HDD) Spark Executor Spark Gateway (e.g. ThriftServer, Spark shell)SQL Cached Data Source (v1/v2) Task scheduled Intel Optane DC Persistent Memory Cache Hit Cache Miss Server 2 Native library (vmemcache) Cache Aware Scheduler
  • 17.
    Cache Design -Problem statement • Local LRU cache • Support for large capacities available with persistent memory (many terabytes per server) • Lightweight, efficient and embeddable • In-memory • Scalable
  • 18.
    Cache Design -Existing solutions • In-memory databases tend to rely on malloc() in some form for allocating memory for entries – Which means allocating anonymous memory • Persistent Memory is exposed by the operating system through normal file- system operations – Which means allocating byte-addressable PMEM needs to use file memory mapping (fsdax). • We could modify the allocator of an existing in-memory database and be done with it, right? J
  • 19.
    Cache Design -Fragmentation • Manual dynamic memory management a’la dlmalloc/jemalloc/tcmalloc/palloc causes fragmentation • Applications with substantial expected runtime durations need a way to combat this problem – Compacting GC (Java, .NET) – Defragmentation (Redis, Apache Ignite) – Slab allocation (memcached) • Especially so if there’s substantial expected variety in allocated sizes
  • 20.
    Cache Design -Extent allocation • If fragmentation is unavoidable, and defragmentation/compacting is CPU and memory bandwidth intensive, let’s embrace it! • Usually only done in relatively large blocks in file-systems. • But on PMEM, we are no longer restricted by large transfer units (sectors, pages etc)
  • 21.
    Cache Design -Scalable replacement policy • Performance of libvmemcache was bottlenecked by naïve implementation of LRU based on a doubly-linked list. • With 100st of threads, most of the time of any request was spent waiting on a list lock… • Locking per-node doesn’t solve the problem…
  • 22.
    Cache Design -Buffered LRU • Our solution was quite simple. • We’ve added a wait-free ringbuffer which buffers the list-move operations • This way, the list only needs to get locked during eviction or when the ringbuffer is full.
  • 23.
    Cache Design -Lightweight, embeddable, in-memory caching https://github.com/pmem/vmemcache VMEMcache *cache = vmemcache_new("/tmp", VMEMCACHE_MIN_POOL, VMEMCACHE_MIN_EXTENT, VMEMCACHE_REPLACEMENT_LRU); const char *key = "foo"; vmemcache_put(cache, key, strlen(key), "bar", sizeof("bar")); char buf[128]; ssize_t len = vmemcache_get(cache, key, strlen(key), buf, sizeof(buf), 0, NULL); vmemcache_delete(cache); libvmemcache has normal get/put APIs, optional replacement policy, and configurable extent size Works with terabyte-sized in-memory workloads without a sweat, with very high space utilization. Also works on regular DRAM.
  • 24.
    24 Parquet File Footer RowGroup #1 ColumnChunk #1 Column Chunk #2 Column Chunk #N RowGroup #2 RowGroup #N Guava Based Cache Vmemcache Fine Grain Cache Cache Status & Report Cache Design – Status and Fine Grain
  • 25.
    25 Experiments and Configurations DCPMMDRAM Hardware DRAM 192GB (12x 16GB DDR4) 768GB (24x 32GB DDR4) Intel Optane DC Persistent Memory 1TB (QS: 8 x 128GB) N/A DCPMM Mode App Direct (Memkind) N/A SSD N/A N/A CPU 2 * Cascadelake 8280M (Thread(s) per core: 2, Core(s) per socket: 28, Socket(s): 2 CPU max MHz: 4000.0000 CPU min MHz: 1000.0000 L1d cache: 32K, L1i cache: 32K, L2 cache: 1024K, L3 cache: 39424) OS 4.20.4-200.fc29.x86_64 (BKC: WW06'19, BIOS: SE5C620.86B.0D.01.0134.100420181737) Software OAP 1TB DCPMM based OAP cache 610GB DRAM based OAP cache Hadoop 8 * HDD disk (ST1000NX0313, 1-replica uncompressed & plain encoded data on Hadoop) Spark 1 * Driver (5GB) + 2 * Executor (62 cores, 74GB), spark.sql.oap.rowgroup.size=1MB JDK Oracle JDK 1.8.0_161 Workload Data Scale 2TB, 3TB, 4TB Decision Making Queries 9 I/O intensive queries Multi-Tenants 9 threads (Fair scheduled)
  • 26.
    Test Scenario 1:Both Fit In DRAM And DCPMM - 2TB *The performance gain can be much lower if IO is improved (e.g. compression & encoding enabled) or some hacking codes fixed (e.g. NUMA scheduler) With 2TB Data Scale, both DCPMM and DRAM based OAP cache can hold the full dataset (613GB). DCPMM is 24.6% (=1- 100.1/132.81) performance lower than DRAM as measured on 9 I/O intensive decision making queries. On Premise Performance Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks. Configurations: See page20.Test by Intel on 24/02/2019.
  • 27.
    Test Scenario 2:Fit In DCPMM Not For DRAM - 3TB *The performance gain can be much lower if IO is improved (e.g. compression & encoding enabled) or some hacking codes fixed (e.g. NUMA scheduler) With 3TB Data Scale, only DCPMM based OAP cache can hold the full dataset (920GB). DCPMM shows 8X* performance gain over DRAM as measured on 9 I/O intensive decision making queries. On Premise Performance Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks. Configurations: See page20.Test by Intel on 24/02/2019.
  • 28.
    Performance Analysis -System Metrics ● Input data is all cached in DCPMM while partially for DRAM ● DCPMM reaches up to 18GB/sbandwidth while DRAM case it’s bounded by disk IO which is only about 250MB/s ~ 450MB/s ● OAP cache doesn’t apply for shuffle data (intermediate data) that extra IO pressure put onto DRAM case ● DCPMM reads about 27.5GBwhile DRAM reads about 395.6GB OAP Cache works avoiding disk read (blue) and only disk write for shuffle (red) Disk read (blue) comes from shuffle data Disk read (blue) happens from time to time DCPMM
  • 29.
    Test Scenario 3:None Of DRAM &DCPMM Fit - 4TB *The performance gain can be much lower if IO is improved (e.g. compression & encoding enabled) or some hacking codes fixed (e.g. NUMA scheduler) With 4TB Data Scale, none of DCPMM and DRAM based OAP cache can hold the full dataset (1226.7GB). DCPMM shows 1.66X* (=2252.80/1353.67) performance gain over DRAM as measured on 9 I/O intensive decision making queries. On Premise Performance Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks. Configurations: See page20.Test by Intel on 24/02/2019.
  • 30.
    Use Case 2:Machine Learning Spark K-means
  • 31.
    DCPMM Storage Level DRAM Disk(SSD, HDD) Object Byte buffer Byte buffer Spark Core (Compute/Storage Memory) DRAM (on heap) DRAM (off heap) Disk K-means workload GraphX workload Decompression Deseriliazation • Spark Storage Level • A few storage levels serving for different purposes including memory and disk • Off-heap memory is supported to avoid GC overhead in Java • Large capacity storage level (disk) is widely used for iterative computation workload (e.g. K-means, GraphX workloads) by caching hot data in storage level • DCPMM Storage Level • Extend memory layer • Using Pmem library to access DCPMM avoiding the overhead of decompression from disk • Large capacity and high I/O performance of DCPMM shows better performance than tied solution (original DRAM + Disk solution)
  • 32.
    32 K-means in Spark M0- Mj Mj+1 - Mk Mk+1 - Mm … Mx+1 - Mn HDFS Cache #1 (DRAM + DISK) Cache #2 (DRAM + DISK) Cache #3 (DRAM + DISK) … Cache #N (DRAM + DISK) Spark Executor Processes Loading Normalization Caching C1, C2, C3, … CK For Each Record in the Cache For Each Record in the Cache For Each Record in the Cache For Each Record in the Cache For Each Record in the Cache Step 1: For each record in the cache, Sum the vectors with the same closet centroid Update the new Centroids Step 2: Sum the vectors according to the centroids and find out the new centroids globally Sync N iterations Load TrainRandom Initial Centroids 2 Iterations for all of the cache data to get the K centroids in a random mode Load the data from HDFS to DRAM (and DCPMM / SSD if DRAM cannot hold all of the data)(Load), after that, the data will not be changed, and will be iterated repeatedly in Initialization and Train stages.
  • 33.
    Compute Node 33 K-means Basic DataFlow • Load data from HDFS to memory. • Spill over to local storage Load • Compute using initial centroid based on data in memory or local storage Initialization • Compute iterations based on local data Train CPU Memory Storage Local Shared Compute Node CPU Memory Storage Local Shared Compute Node CPU Memory Storage Local Shared HDFS … … K centroids C1, C2, C3, … CK 1 2 3 1 2 3
  • 34.
    34 Experiments and Configurations #1AD #2 DRAM Hardware DRAM 192GB (12x 16GB DDR4 2666) 768GB (24× 32GB DDR4 2666) Intel Optane DC Persistent Memory 1TB (8x 128GB QS) N/A DCPMM Mode AppDirect N/A CPU Intel(R) Xeon(R) Platinum 8280L CPU @ 2.70GHz Disk Drive 8x Intel DC S4510 SSD (1.8T) + 2 * P4500 (1.8T) Software CPU / Memory Allocation Driver (5GB + 10 cores) + 2 * Executor (80GB + 32 Cores) Software Stack Spark (2.2.2-snapshot: bb3e6ed9216f98f3a3b96c8c52f20042d65e2181) + Hadoop (2.7.5) OS & Kernel & BIOS Linux-4.18.8-100.fc27.x86_64-x86_64-with-fedora-27-Twenty_Seven BIOS: SE5C620.86B.0D.01.0299.122420180146 Mitigation variants (1,2,3,3a,4, L1TF) 1,2,3,3a,4, L1TF Cache Spill Disk N/A 8 * SSD NUMA Bind 2 NUMA Nodes bind 2 Spark Executors respectively Data Cache Size (OffHeap) DCPMM (no spill) 680GB DRAM (partially spill) Workload Record Count / Data Size / Memory Footprint Row size 6.1 Billion Records / data size 1.1TB / cache data 954GB Cache Spill Size 0 235GB Kmeans (Iteration times) 10 Kmeans (K value) 5 1 Modified to support DCPMM
  • 35.
    35 Performance Comparison(DCPMM ADV.S. DRAM) • DCPMM provides larger “memory” than DRAM and avoid the data spill in this case, hence got up to 4.98X performance in Training(READ), and 2.85X end to end performance; • The DCPMM AppDirect mode is about ~19% faster than DRAM in Load(WRITE cache into DCPMM / DRAM); • Training(READ) execution gap of DRAM case caused by storage media, as well as the data decompression and de-serialization from the local spill file; 4.98X1.19X 2.85X Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks. Configurations: See page11.Test by Intel on 14/03/2019. DCPMM DCPMM
  • 36.
    DRAM System Analysis DCPMM load training trainingload DRAM ●SSD read bandwidth is 2.86GB/s ● It’s I/O bound leading to low CPU utilization load training load training DCPMM ● Much high CPU utilization than DRAM only system
  • 37.
    • Intel OptaneDC Persistent Memory brings high capacity, low latency and high throughput • Two operation modes for DCPMM: 1) App-Direct 2) 2LM • Good for Spark in: 1) memory bound workload 2) I/O intensive workload Summary
  • 38.
    Want to knowmore? See you at Intel Booth 100 Download source code from Github: https://github.com/intel-bigdata/OAP https://github.com/pmem/vmemcache
  • 39.
    DON’T FORGET TORATE AND REVIEW THE SESSIONS SEARCH SPARK + AI SUMMIT