1) Lucata's Emu architecture features cacheless multicore processors on DRAM with hardware that migrates threads between cores to access data, providing high memory bandwidth. 2) Benchmark results show the Emu architecture achieves near-peak memory bandwidth and flat performance for a pointer-chasing workload, unlike x86 which varies with data access patterns. 3) Implementing BFS on a dynamic graph data structure achieved higher performance and bandwidth on Emu compared to x86, demonstrating Emu's suitability for streaming workloads.

1. GraphBLAS and Emus
Jason Riedy (all opinions my own, no planning guarantees)
GraphBLAS BoF at IEEE HPEC, 22 September 2020
Lucata/ Emu Technology

2. Lucata’s PGAS Architecture
1 nodelet
Gossamer
Core 1
Memory-Side Processor
Gossamer
Core 4
...
Migration Engine
RapidIODisk I/O
8 nodelets
per node
64 nodelets
per Chick
RapidIO
Stationary
Core
• Cacheless multithreaded
multicore
• Memory-side “processor” at
narrow-channel DRAM
• Stationary core for OS
• Physically distributed
memory
• Threads migrate in
hardware on reads!
GraphBLAS and Emus — 22 Sep 2020 2/8

3. Pointer-Chasing Benchmark
Data-dependent loads, fine-grained access1
Ordered
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Intra-block shuffle: weak locality
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Full block shuffle: weak locality
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
1
Eric Hein, Young, Srinivas Eswar, Jiajia Li, Patrick Lavin, Vuduc, Riedy. “An Initial Characterization of the Emu
Chick,” Workshop on Accelerators and Hybrid Exascale Systems (AsHES) 2018.
GraphBLAS and Emus — 22 Sep 2020 3/8

4. Selected Results: x86 Pointer-Chasing Benchmark
1
4
16
64
256
1K
4K
16K
64K
256K
1M
4M
Block size (number of 16B elements)
0
20
40
60
80
100Memorybandwidth(GBs) peak STREAM bandwidth
56 threads
1
4
16
64
256
1K
4K
16K
64K
256K
1M
4M
Block size (number of 16B elements)
peak STREAM bandwidth
112 threads
block_shuffle intra_block_shuffle full_block_shuffle
Haswell results, every pattern is different.2
2
Eric Hein, Young, Srinivas Eswar, Jiajia Li, Patrick Lavin, Riedy, Vuduc, Conte. “A Microbenchmark Characterization
of the Emu Chick.” Parallel Computing, 10.1016/j.parco.2019.04.012
GraphBLAS and Emus — 22 Sep 2020 4/8

5. Selected Results: Emu Pointer-Chasing Benchmark
1
4
16
64
256
1K
4K
16K
64K
256K
1M
4M
Block size (number of 16B elements)
0
2
4
6
8
10
12
Memorybandwidth(GBs)
peak STREAM bandwidth
2048 threads
1
4
16
64
256
1K
4K
16K
64K
256K
1M
4M
Block size (number of 16B elements)
peak STREAM bandwidth
4096 threads
block_shuffle intra_block_shuffle full_block_shuffle
Mostly flat performance, high utilization.2
GraphBLAS and Emus — 22 Sep 2020 5/8

6. Selected Results: BFS on a Dynamic Data Structure
15 16 17 18 19 20 21
scale
0
20
40
60
80
100
MTEPS
Emu single node - Cilk
Emu multi-node - Cilk
x86 Haswell - STINGER
x86 Haswell - Cilk
0
500
1000
1500
EdgeBandwidth(MB/s)
Note: Streaming data structure, not statically optimized. 3
3
Hein, Eswar, Abdurrahman Yasar, Prasanth Chatarasi, Li, Young, Conte, Ümit Çatalyürek, Vuduc, Riedy, Bora Uçar.
“Programming Strategies for Irregular Algorithms on the Emu Chick.” ACM ToPC, to appear.
https://arxiv.org/abs/1901.02775
GraphBLAS and Emus — 22 Sep 2020 6/8

7. Implications for a GraphBLAS Implementation
• We can be more flexible in data organization.
• Not tied to CSR / CSC / COO.
• NCDIMM: No cache line issues
• Stride between vertices, values can be arbitrary.
• Can incorporate more semantic information.
• Targeting “streaming” use.
• High rate of change in a massive graph.
• Linked list of blocks... (STINGER, HORNET)
• But must remember graphs live in a separate
memory space.
• Gossamer side calls stay there.
• Stationary core calls must transfer input and output.
GraphBLAS and Emus — 22 Sep 2020 7/8

8. Experiences “Porting” Existing Apps & Bindings
Capabilities nice to have:
• Allocating memory to hold k entries w/o knowing the type
• Converting the support to a bool GxB_Matrix (T →bool)
• Eases operating on masks of different types
• Execution context: SC-GC, GC-GC, SC-SC
• A sized blob type that is not a UDT
• Sometimes used to hold keys with no GB meaning
• Selects and ops with bools still useful
• Users want “iterators”
• Some uses are horrible to replace without a relational
join-type operation
• Still coming up with more...
GraphBLAS and Emus — 22 Sep 2020 8/8