The document discusses opportunities for improving Apache Spark performance using near data computing architectures. It proposes exploiting in-storage processing and 2D integrated processing-in-memory to reduce data movement between CPUs and memory. Certain Spark workloads like joins and aggregations that are I/O bound would benefit from in-storage processing, while iterative workloads are more suited for 2D integrated processing-in-memory. The document outlines a system design using FPGAs to emulate these architectures for evaluating Spark machine learning workloads like k-means clustering.

1. Ahsan J. Awan (KTH)
Near Data Computing Architectures:
Opportunities and Challenges for
Apache Spark
#EUres10
1

2. 2#EUres10
About me ?
2
1988 20112010 20132012 2014 2015 2016 2017
B.E. MTS
NUST,
Pakistan
EMECS,
TUKL,
Germany
EMECS,
UoS,
UK
Lecturer,
NUST,
Pakistan
EMJD-DC,
KTH/SICS,
Sweden
PhD Intern,
Recore
Netherlands
EMJD-DC,
UPC/BSC,
Spain
PhD Intern,
IBM Research,
Japan
Born in
Pakistan

3. 3#EUres10
A Quick Recap from last year ?
3
Problems Identified Solutions Proposed
https://spark-summit.org/eu-2016/events/performance-characterization-of-apache-spark-on-scale-up-servers/
Work Time Inflation
Poor Multi-core Scalability of
data analytics with Spark
Thread Level Load
Imbalance
Wait Time on I/O
GC overhead
DRAM Bound
Latency
NUMA Awareness
Hyper Threaded
Cores
No next-line
prefetchers
Lower DRAM
speed
Exploiting Near Data Processing
Choice of GC
algorithm
Multiple Small
executors

4. 4#EUres10
Further Reading ?
4
● Project Night-King: Improving the single performance for Apache Spark using Near
Data Processing Architectures.
● Identifying the potential of Near Data Computing Architectures for Apache Spark in
Memory Systems Symposium, 2017.
● Node Architecture Implications for In-Memory Data Analytics in Scale-in Clusters in
IEEE/ACM Conference in Big Data Computing, Applications and Technologies, 2016.
● Micro-architectural Characterization of Apache Spark on Batch and Stream
Processing Workloads, in IEEE Conference on Big Data and Cloud Computing, 2016.
●
How Data Volume Affects Spark Based Data Analytics on a Scale-up Server in 6th
Workshop on Big Data Benchmarks, Performance Optimization and Emerging
Hardware (BpoE), held in conjunction with VLDB 2015, Hawaii, USA .
● Performance characterization of in-memory data analytics on a modern cloud server,
in IEEE Conference on Big Data and Cloud Computing, 2015 (Best Paper Award).

5. 5#EUres10
Exploiting NDP/Moving compute closer to data ?
5
Loh et al. A processing in memory taxonomy and a case for studying fixedfunction pim. In Workshop on Near-Data Processing (WoNDP), 2013.
1. Processing in Memory
2. In-Storage Processing
Improve the performance
by reducing costly data
movements back and forth
between the CPUs and
Memories

6. 6#EUres10
Trends of Integrating NVM in the System Architecture ?
6
Chang et al. A limits study of benefits from nanostore-based future data-centric system architectures. In Computing Frontiers 2012

7. 7#EUres10
Can Spark workloads benefit from Near data processing ?
7
Host
CPU
PIM
device
ISP
device
Project: NightKing

8. 8#EUres10
The case for in-storage processing ?
8
Grep (Gp)
K-means (Km)Windowed Word Count (Wwc)

9. 9#EUres10
The case for 2D integrated PIM instead of 3D Stacked PIM ?
9
M. Radulovic et al. Another Trip to the Wall: How
Much Will Stacked DRAM Benefit HPC?

10. 10#EUres10
A refined hypothesis based on workload characterization ?
10
● Spark workloads, which are not iterative and have high ratio of I/O wait time / CPU time
like join, aggregation, filter, word count and sort are ideal candidates for ISP.
● Spark workloads, which have low ratio of I/O wait time / CPU time like stream
processing and iterative graph processing workloads are bound by latency of frequent
accesses to DRAM and are ideal candidates for 2D integrated PIM.
● Spark workloads, which are iterative and have moderate ratio of I/O wait time / CPU
time like K-means, have both I/O bound and memory bound phases and hence will
benefit from hybrid 2D integrated PIM and ISP.
● In order to satisfy the varying compute demands of Spark workloads, we envision an NDC
architecture with programmable logic based hybrid ISP and 2D integrated PIM.

11. 11#EUres10
How to test the refined hypothesis ?
11
● Simulation Approach
● Very slow for big data applications :(
● Modeling Approach
● Overly estimated numbers :(
● Emulation Approach
● A lot of development :(
How about a combination of Modeling and partial Emulation ?

12. 12#EUres10
Can existing tightly coupled servers be used as emulators ?
12

13. 13#EUres10
Our System Design ?
13

14. 14#EUres10
Which programming model ?
14
Iterative MapReduce

15. 15#EUres10
Which workloads ?
15
K-means and SGD
Mahan et al. TABLA: A unified template-based framework for accelerating statistical machine learning

16. 16#EUres10
Our programmable accelerators ?
16

17. 17#EUres10
Advantages of the design ?
17
● Template based design to support generality.
● No of mappers and reducers can be instantiated based on the FPGA
card.
● General Sequencer is a Finite State Machine whose states can be varied
to meet the diverse set of workloads
● Mappers and Reducers can be programmed in C/C++ and can be
synthesized using High Level Synthesis.
● Support hardware acceleration of Diverse set of workloads

18. 18#EUres10
How about using a roof-line model ?
18
Used to estimate Arithmetic
Intensities
Used to generate roof-line model

19. 19#EUres10
Let's show some numbers ?
19
Poor multi-core scalability of Apache Spark

20. 20#EUres10
What are the opportunities ?
20
K-means (Km)

21. 21#EUres10
What are the challenges ?
21
● How to design the best hybrid CPU + FPGA ML workloads ?
● How to attain peak performance on CPU side ?
● How to attain peak performance on FPGA side ?
● How to balance load between CPU and FPGA ?
● How hide communication between JVM and FPGA ?
● How to attain peak CAPI bandwidth consumption ?
● How to design the clever ML workload accelerators using HLS tools ?

22. 22#EUres10
What High Level Synthesis (Xilinx SDSoC Tool Chain) can do ?
22
20x 10x

23. 23#EUres10
Things to remember from this talk ?
23
● 3D Stacked Memories are the over-kill for Apache Spark Workloads.
● Project Night-King aims at improving the single node performance of Apache Spark
using programmable accelerators near DRAM and NVRAM.
● Conservatively, Near-data accelerators augmented Scale-up Servers can improve the
performance of Spark MLlib by 5x.
● Never Trust the 20x speed-up claims being made in the industry. Most of the time, the
reference points are wrong!
● Xilinx SDSoC Tool Chain needs to support pragmas for map-reduce programming model.
● Performance Characterization and Optimization of In-Memory Data Analytics on a Scale-
up Server, PhD thesis, Ahsan Javed Awan (ISBN: 978-91-7729-584-6)

24. 24#EUres10
That's all for now ?
24
Email: ajawan@kth.seEmail: ajawan@kth.se
Profile:Profile: www.kth.se/profile/ajawan/www.kth.se/profile/ajawan/
https://se.linkedin.com/in/ahsanjavedawanhttps://se.linkedin.com/in/ahsanjavedawan
THANK YOU