The term Massively Parallel BigData Processor names a recent advance in bigdata processing technology which has advanced from the traditional distributed processing represented by Hadoop to modern parallel processing represented by various multicore and manycore architectures. The problem in massively parallel systems is the irregularity which is when a single item -- can be referred to as a blackswan -- incurs much more processing cost than the majority of items. This paper discusses the respective countermeasures divided into (1) distribution and (2) runtime models and optimizations. The differences between multicore and manycore cases is also discussed, where the latter is represented by the currently default design with tiles and n-neighbor switching.

2. Risk Management for BigData
• hardware moving towards regular designs -- manycore
• but execution environments are becoming more irregular
◦ even physics 14, modeling and other HPC is becoming irrsegular
• the question: what is the risk from running irregular apps on regular
structures?
◦ other questions: which hardware platforms are better and which can be
improved?
• this talk presents the concept of massively multicore as a possible
answer
14 S.Jarp+2 "The future of commodity computing and many-core versus the interests of HEP software" Journal of Physics (2012)
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 2/22
2/22

3. The Story of Massively Multicore
….
Time
Now
(buffer head)
Manager
Job
Job
Buffer
tail
pos
pos
Controller
Kill
2 Report
Manage
in realtime
One Replay Batch
One
Buffer
One
Buffer
One
BufferJobs
Jobs
Jobs
Replay at
a scale
1
• traditional Hadoop has reached
its limits 0708
• massively multicore: many cores
but connected in the multicore (not manycore)
design
• many uses, but specifically for
BigData Replay on Multicore 01
• merits: can pack jobs in batches,
optimize batches at runtime, etc.
• irregularity is in the variance in
playback positions across jobs
01 myself+0 "Streaming Algorithms for Big Data Processing on Multicore" Big Data: Algorithms, ...CRC (2015)
07 K.Shvachko+0 "HDFS Scalability: the Limits to Growth" the Magazine of USENIX, vol.35, no.2 (2012)
08 A.Rowstron+4 "Nobody ever got fired for using Hadoop on a cluster" 1st Int.Work. on Hot Topics in Cloud Data Processing (2012)
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 3/22
3/22

4. Implementing Massively Parallel
DRAM
shmap
shmap
…
…
Manager
Jobs
• shmap is the best
• lockfree shmap is even
better 02
• recently discussed in MPI as
the true one-sided
communication method 15
• on traditional hardware all
shmaps share the same DRAM
• ... but there is hope for
non-traditional hardware in
near future (working on it)
02 myself+0 "A lock-free shared memory design for high-throughput multicore packet traffic capture" IJNM (2014)
15 S.Potluri+4 "Optimizing MPI One Sided Communication on Multi-core InfiniBand Clusters..." 18th EuroMPI (2011)
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 4/22
4/22

5. Performance Benchmarking
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 5/22
5/22

6. Benchmark : Parameter Space
• shmap size in bytes
• batchcount : number of batches = shmap regions
• batchsize : number of jobs in each batch
• experimental setup : commodity 4-core hardware
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 6/22
6/22

7. Benchmark : Results (1)
size#100000batchcount#1
size#100000batchcount#2
size#100000batchcount#5
size#100000batchcount#10
size#100000batchcount#25
size#100000batchcount#50
size#1000000batchcount#1
size#1000000batchcount#2
size#1000000batchcount#5
size#1000000batchcount#10
size#1000000batchcount#25
size#1000000batchcount#50
size#10000000batchcount#1
size#10000000batchcount#2
size#10000000batchcount#5
size#10000000batchcount#10
size#10000000batchcount#25
size#10000000batchcount#50
0
500000
1000000
1500000
2000000
2500000
3000000
3500000
4000000
Executiontime(us)
Order: size batchcount batchsize
• visualizing multidim
space via
permutation
sequence on X
• the order is key
-- first parameter is
outer loop, and so on
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 7/22
7/22

8. Benchmark : Results (2)
size#100000batchsize#1
size#100000batchsize#2
size#100000batchsize#5
size#100000batchsize#10
size#1000000batchsize#1
size#1000000batchsize#2
size#1000000batchsize#5
size#1000000batchsize#10
size#10000000batchsize#1
size#10000000batchsize#2
size#10000000batchsize#5
size#10000000batchsize#10
0
500000
1000000
1500000
2000000
2500000
3000000
3500000
4000000
Executiontime(us)
Order: size batchsize batchcount
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 8/22
8/22

9. Benchmark : Results (3)
batchcount#1batchsize#1
batchcount#1batchsize#2
batchcount#1batchsize#5
batchcount#1batchsize#10
batchcount#2batchsize#1
batchcount#2batchsize#2
batchcount#2batchsize#5
batchcount#2batchsize#10
batchcount#5batchsize#1
batchcount#5batchsize#2
batchcount#5batchsize#5
batchcount#5batchsize#10
batchcount#10batchsize#1
batchcount#10batchsize#2
batchcount#10batchsize#5
batchcount#10batchsize#10
batchcount#25batchsize#1
batchcount#25batchsize#2
batchcount#25batchsize#5
batchcount#25batchsize#10
batchcount#50batchsize#1
batchcount#50batchsize#2
batchcount#50batchsize#5
batchcount#50batchsize#10
0
500000
1000000
1500000
2000000
2500000
3000000
3500000
4000000
Executiontime(us)
Order: batchcount batchsize size
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 9/22
9/22

10. Benchmark : Results (4)
batchcount#1size#100000
batchcount#1size#1000000
batchcount#1size#10000000
batchcount#2size#100000
batchcount#2size#1000000
batchcount#2size#10000000
batchcount#5size#100000
batchcount#5size#1000000
batchcount#5size#10000000
batchcount#10size#100000
batchcount#10size#1000000
batchcount#10size#10000000
batchcount#25size#100000
batchcount#25size#1000000
batchcount#25size#10000000
batchcount#50size#100000
batchcount#50size#1000000
batchcount#50size#10000000
0
500000
1000000
1500000
2000000
2500000
3000000
3500000
4000000
Executiontime(us)
Order: batchcount size batchsize
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 10/22
10/22

11. Irregularity Countermeasures
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 11/22
11/22

12. Grow vs Drop Models
….
Time
Now
(buffer head)
Manager
Job
Job
Buffer
tail
pos
pos
Controller
Kill
2 Report
Manage
in realtime
One Replay Batch
One
Buffer
One
Buffer
One
BufferJobs
Jobs
Jobs
Replay at
a scale
1
• practical problem: how to
manage batches with high
variance across jobs?
• grow: let the batch grow in
size, no need to kill/remap jobs
• drop: size is fixed, lagging
jobs are killed and possibly
remapped to other batches
• ... these are basic models,
other variants are possible
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 12/22
12/22

13. Analysis Setup
• start with 100 cores, one job per core, run for some time collecting statistics
• use hotspot distribution to describe processing time per data unit
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 13/22
13/22

14. Hotspot Distribution
Hotspot Disribution 06...
...consists of normal, popular, and hot/
flash sets
0 20 40 60 80 100
Decreasing order
0
0.35
0.7
1.05
1.4
1.75
2.1
2.45
2.8
log(value)
Class A Class B Class C Class D Class E
• CDN example: normal are almost
never watched videos, popular are
watches sometimes, and only hot/
flash are the videos which are hot
normally but also experience Flash
Crowds (go viral)
• additional classification: assign a
letter to the curve based on the
fatness of its tail (size of head)
06 myself+1 "Popularity-Based Modeling of Flash Events in Synthetic Packet Traces" IEICE CQ研 (2012)
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 14/22
14/22

15. Elasticity of the two Models
0 0.15 0.3 0.45 0.6 0.75 0.9 1.05
Norm. drop count
0
0.15
0.3
0.45
0.6
0.75
0.9
1.05
Norm.dragwindow
size#100000 batchcount#5
0 0.15 0.3 0.45 0.6 0.75 0.9 1.05
Norm. drop count
0
0.15
0.3
0.45
0.6
0.75
0.9
1.05
Norm.dragwindow
size#100000 batchcount#10
0 0.15 0.3 0.45 0.6 0.75 0.9 1.05
Norm. drop count
0
0.15
0.3
0.45
0.6
0.75
0.9
1.05
Norm.dragwindow
size#100000 batchcount#25
0 0.15 0.3 0.45 0.6 0.75 0.9 1.05
Norm. drop count
0
0.15
0.3
0.45
0.6
0.75
0.9
1.05
Norm.dragwindow
size#1000000 batchcount#5
0 0.15 0.3 0.45 0.6 0.75 0.9 1.05
Norm. drop count
0
0.15
0.3
0.45
0.6
0.75
0.9
1.05
Norm.dragwindow
size#1000000 batchcount#10
0 0.15 0.3 0.45 0.6 0.75 0.9 1.05
Norm. drop count
0
0.15
0.3
0.45
0.6
0.75
0.9
1.05
Norm.dragwindow
size#1000000 batchcount#25
• same setups run for each model
• metrics: shmap size for grow vs drop
count for drop
• figure: plot normalized distributions of
outcomes
• figure: drop model is much more flexible
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 15/22
15/22

16. The ManyCore Design
ManyCore is about Tiles
...where each tile has CPU + L1/L2 cache
+ switch
Manycore Device
Manager
…
…
…
… …
…
I/O
One Batch
Tile
Jobsshmap
• wormhole routing is common
1311
• against intuition, wormhole method
is low-latency and
high-throughput but not
contention-free
• hardware makers offer various
tricks 13 in this area, but do not
resolve the key problem
11 J.Duato+3 "...Router Architectures for Virtual Cut-Through and Wormhole Switching in a NOW Environment" 13th IPPS/SPDP (1999)
13 D.Wentzlaff+9 "On-chip interconnection architecture of the tile processor" IEEE Micro, vol.27, issue 5 (2007)
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 16/22
16/22

17. ManyCore Parameters
• each batch is a spatial area on the chip, areas compete for space
• heterometric: a measure of irregularity = variance in batchsize, hotspots,
etc.
• performance metric: failure to map a new job to an existing batch
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 17/22
17/22

18. ManyCore Example Run
heterometric#1
batches#5
batchsize#10
classrange#A
failed grows#20
4
1
0 3
2
epoch#0
3
3
3
2
2
3
3
3
3
3
3
2
2
2
3
2
4
4
4
4
4
4
4
4
1
4
4
4
1
1
1
1
1
0
0
0
2
0
0
0
0
2
2
0
0
0
0
3
2
2
epoch#10
3
3
3
2
2
3
3
3
3
3
3
2
2
2
3
2
4
4
4
4
4
4
4
4
4
4
4
1
1
1
1
1
0
0
0
2
0
0
0
0
2
2
0
0
0
0
3
1
2
2
epoch#20
3
3
3
2
2
3
3
3
3
3
3
2
2
2
3
2
4
4
4
4
4
4
1
4
4
4
4
4
1
1
1
1
1
0
0
0
2
0
0
0
0
2
2
0
0
0
0
3
2
2
epoch#30
3
3
3
2
2
3
3
3
3
3
3
2
2
2
3
2
4
4
4
4
4
4
4
4
4
4
4
1
1
1
1
1
1
0
0
0
2
0
0
0
0
2
2
0
0
0
0
3
2
2
epoch#40
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 18/22
18/22

19. MassiveMulti- vs Many-Core (1)
• different yet comparable response to irregularity
• MassivelyMulti judged by processing time, ManyCore by failed
mappings
• elasticity : ∆output/∆configuration
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 19/22
19/22

20. MassiveMulti- vs Many-Core (2)
0.552
0.763
8.333
manycore / heterometric
2.282
23.077
1470
manycore / batches
0.792
1.155
1000
manycore / classrange
0.622
0.794
52.408
shmap / size
0.867
1.265
30.795
shmap / batchsize
0.739
1.016
19.728
shmap / batchcount
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 20/22
20/22

21. Wrapup
• ManyCore may remain for regular apps
◦ scientific modeling, Earth simulator, etc.
• irregular apps perform better on Massively Multicore
• towards new platforms : virtualization techniques for DRAM on standard
multicore 16?
16 R.Brightwell+0 "Lightweight Kernel Support for Direct Shared Memory Access..." W. on Managed Many-Core Systems (2008)
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 21/22
21/22

22. That’s all, thank you ...
M.Zhanikeev -- maratishe@gmail.com Irregularity Countermeasures in Massively Parallel BigData Processors -- bit.do/151016 22/22
22/22