Martin Děcký presented on microkernels in the era of data-centric computing. He discussed how emerging memory technologies and near-data processing can break away from the von Neumann architecture. Near-data processing provides benefits like reduced latency, increased throughput, and lower energy consumption. This leads to more distributed and heterogeneous systems that are well-matched to multi-microkernel architectures, with microkernels providing isolation and message passing between cores. The talk outlined an incremental approach from initial workload offloading to developing a full multi-microkernel system.

1. Microkernels in the Era of Data-Centric Computing
Martin Děcký
martin.decky@huawei.com
February 2018

2. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 2
Who Am I
Passionate programmer and operating systems enthusiast
With a specific inclination towards multiserver microkernels
HelenOS developer since 2004
Research Scientist since 2006
Charles University (Prague), Distributed Systems Research Group
Senior Research Engineer since 2017
Huawei Technologies (Munich), Central Software Institute

3. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 3
Motivation3

4. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 4
Memory Barrier

5. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 5
LaMarca, Ladner (1996)
Quick Sort
O(n×log n) operations
Radix Sort
O(n) operations
4 8 16 32 64 128 256 512 1024 2048 4096
0
200
400
600
800
1000
1200
Quick Sort Radix Sort
Thousands of items
Instructions/item

6. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 6
LaMarca, Ladner (1996)
Quick Sort
O(n×log n) operations
Radix Sort
O(n) operations
4 8 16 32 64 128 256 512 1024 2048 4096
0
200
400
600
800
1000
1200
1400
1600
1800
2000
Quick Sort Radix Sort
Thousands of items
Clockcycles/item

7. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 7
LaMarca, Ladner (1996)
Quick Sort
O(n×log n) operations
Radix Sort
O(n) operations
4 8 16 32 64 128 256 512 1024 2048 4096
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Quick Sort Radix Sort
Thousands of items
Cachemisses/item

8. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 8
The Myth of RAM
Accessing a random memory location requires
O(1)
operations
Accessing a random memory location takes
O(1)
time units

9. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 9
The Myth of RAM
Accessing a random memory location requires
O(1)
operations
Accessing a random memory location takes
O(1)
time units

10. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 10
The Myth of RAM
Accessing a random memory location requires
O(1)
operations
Accessing a random memory location takes
O(√n)
time units

11. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 11
Breaking the Barrier

12. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 12
Von Neumann Forever?
Data
Control
Status
ALU
Input
peripheral
Output
peripheral
RAM
Controller

13. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 13
Von Neumann Forever?
New emerging memory technologies
Bridging the gap between volatile and non-volatile memory
No longer necessary to keep the distinction between RAM and storage
(peripherals)
Single-level memory (universal memory)
See also the talk by Liam Proven (The circuit less traveled), Janson, Sat 13:00
Many technologies in development
Magnetoresistive random-access memory (MRAM)
Racetrack memory
Ferroelectric random-access memory (FRAM)
Phase-change memory (PCM)
Nano-RAM (Nanotube RAM)

14. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 14
Less Radical Solution
Near-Data Processing
Moving the computation closer to the place where the data is
Not a completely new idea at all
Spatial locality in general
GPUs processing graphics data locally
Breaking the monopoly of CPU on data processing even more
CPUs are fast, but also power-hungry
CPUs can only process the data already fetched from the memory/storage
The more data we avoid moving from the memory/storage to the CPU,
the more efficiently the CPU runs

15. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 15
Near-Data Processing
Benefits
Decreased latency, increased throughput
Not necessarily on an unloaded system, but improvement under load
[1] Gu B., Yoon A. S., Bae D.-H., Jo I., Lee
J., Yoon J., Kang J.-U., Kwon M., Yoon C.,
Cho S., Jeong J., Chang D.: Biscuit: A
Framework for Near-Data Processing of
Big Data Workloads, in Proceedings of
43rd Annual International Symposium
on Computer Architecture, ACM/IEEE,
2016

16. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 16
Near-Data Processing
Benefits (2)
Decreased energy consumption
[2] Kim S., Oh H., Park C., Cho S., Lee S.-
W.: Fast, Energy Efficient Scan inside
Flash Memory SSDs, in Proceedings of
37th International Conference on Very
Large Data Bases (VLDB), VLDB
Endowment, 2011

17. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 17
In-Memory Near-Data Processing
Adding computational capability to DRAM cells
Simple logical comparators/operators that could be computed in parallel
on individual words
Filtering based on bitwise pattern
Bitwise operations
Making use of the inherent parallelism
Avoiding moving unnecessary data out of DRAM
Avoiding linear processing of independent words of the data in CPU

18. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 18
Dynamic RAM
memory
matrix
rowdecoder
⁞
controllogic Sense amps
.........
address
data
Y-gating
.........
Y
RAS
CAS
WE

19. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 19
Dynamic RAM with NDP
memory
matrix
rowdecoder
⁞
controllogic Sense amps
.........
address
data
Filtering / Computing
.........
opcode
RAS
CAS
WE

20. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 20
In-Storage Near-Data Processing
Adding computational capability to SSD controllers
Again, making use of inherent parallelism of flash memory
But SSD controllers also contain powerful embedded cores
Flash Translation Layer, garbage collection, wear leveling
Thus computation is not limited to simple bitwise filtering and operations
Complex trade-offs between computing on the primary CPU and
offloading to the SSD controller

21. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 21
Our Prototype
Based on OpenSSD
http://openssd.io/
Open source (GPL) NVMe SSD controller
Hanyang University (Seoul), Embedded and Network Computing Lab
FPGA design for Xilinx Xynq-7000
ONFI NAND flash controller with ECC engine
PCI-e NVMe host interface with scatter-gather DMA engine
Controller firmware
ARMv7
Flash Translation Layer, page caching, greedy garbage collection

22. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 22
Our Prototype (2)
NVMe NDP extensions
NDP module deployment
Static native code so far, moving to safe byte-code (eBPF)
NDP datasets
Safety boundaries for the NDP modules (for multitenancy, etc.)
NDP Read / Write
Extensions of the standard NVMe Read / Write commands
NDP module executed on each block, transforms/filters data
NDP Transform
Arbitrary data transformations (in-place copying, etc.)
Flow-based computational model

23. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 23
Our Prototype (3)
Fast prototyping
QEMU model of the OpenSSD hardware (for running the OpenSSD firmware on ARMv7)
Connected to a second host QEMU/KVM (as a regular PCI-e NVMe storage device)
Planned evaluation
Real-world application proof-of-concept
Custom storage engine for MySQL with operator push-down
Ceph operator push-down
Key-value store
Generic file system acceleration
Still a very much work-in-progress
Preliminary results quite promising

24. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 24
How Microkernels Fit
into this?

25. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 25
Future Vision
Not just near-data processing, but data-centric computing
As opposed to CPU-centric computing
Running the computation dynamically where it is the most efficient
Not necessarily moving the data to the central processing unit
The CPU is the orchestrator
Massively distributed systems
Within the box of your machine (desktop, laptop, smartphone)
Outside your machine (edge cloud, fog, cloud, data center)
Massively heterogeneous systems
Different ISAs
Fully programmable, partially programmable, fixed-function

26. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 26
Future Vision & Microkernels
file system
driver server
CPU
microkernel
application application application
privileged mode
unprivileged mode
memory
mgmt
scheduler IPC
naming
server
location
server
device driver
server
device driver
server
device driver
server
file system
driver server
file system
driver server
device
multiplexer
file system
multiplexer
network
stack
security
server
...

27. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 27
Future Vision & Multi-Microkernels
file system
driver server
Primary
CPU
microkernel
distributed
application
distributed
application
distributed
application
privileged mode
unprivileged mode
memory
mgmt
scheduler IPC
naming
server
location
server
device driver
server
device driver
server
device driver
server
file system
driver server
file system
driver server
device
multiplexer
file system
multiplexer
network
stack
security
server
...
SSD
CPU
NIC
CPU
GPU
microkernelmemory
mgmt
scheduler IPC microkernelmemory
mgmt
scheduler IPC microkernelmemory
mgmt
scheduler IPC
... ... compositor
distributed
application

28. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 28
Microkernels for Data-Centric Computing
Multiserver microkernels
Naturally support distribution and heterogeneity
IPC is a well-defined mechanism to communicate across isolation
boundaries
Communicating between multiple (heterogeneous) computing cores is just a
different transport
Message passing as the least common denominator
– But also memory coherency emulation is possible
Multiserver designers are used to think in terms of fine-grained components
Composing an application from multiple components
Missing piece: Framework for automatic workload migration

29. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 29
Microkernels for Data-Centric Computing (2)
Why not multikernels?
Yes, why not!
Many multikernels (i.e. Barrelfish) are essentially microkernels to the
extreme
Each core running its independent microkernel
No memory coherency assumed
Why not unikernels?
OK, if you insist
A microkernel running a single fixed payload (e.g. a virtual machine)
could be considered a unikernel

30. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 30
Road Forward
As usual, incremental steps
Start with just the basic workload off-loading to the SSD, NIC, etc.
Gradually transform the run-time for the workload off-loading into a
proto-microkernel
Consider the dynamic workload migration framework
Gradually move to a full-fledged multi-microkernel

31. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 31

32. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 32
Summary
There are potential ways of abandoning the von Neumann
architecture
Revolutionary: Single-level (universal) memory
Evolutionary: Near-Data Processing
Near-Data Processing is beneficial in terms of performance and
energy consumption
Data-centric computing leads to massively distributed and
heterogeneous systems
Multi-microkernels are a natural match for such architectures

33. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 33
Acknowledgements
Huawei Technologies in-storage NDP team
Antonio Barbalace
Martin Decky
Anthony Iliopoulos
Javier Picorel
Dmitry Voytik
Hangjun Zhao
References & Inspiration
Mark Silberstein (Technion)
Biscuit framework (Samsung Electronics)

34. Martin Děcký, FOSDEM 2018, February 3rd
2018 Microkernels in the Era of Data-Centric Computing 34
Q&A

35. Thank You!