Mach was an early microkernel-based operating system from the 1980s that served as the basis for several later systems. It employed asynchronous inter-process communication (IPC) which led to significant performance penalties compared to monolithic kernels. The L3 and L4 microkernels from the 1990s improved IPC performance by an order of magnitude through techniques like synchronous IPC and architecture-specific optimizations. This helped validate that microkernels could achieve adequate performance. L4Linux brought the Linux kernel to run as a user process on top of L4 microkernels like Fiasco.OC. It required paravirtualizing portions of Linux to interface with the microkernel but typically incurred only 5-10% performance penalties compared to native

1. μ-Kernel Evolution
Based on
The Performance of μ-Kernel-Based Systems - Hermann Hartig - Michael Hohmuth - Jochen Liedtke - Sebastian
Schonberg - Jean Wolter
Dresden University of Technology Department of Computer Science D-01062 Dresden, Germany,
Oct 97’

2. I am Sergio Shevchenko
You can find me in Telegram with @sshevchenko
Hello!
2

3. 3

4. Agenda
◉ Kernel structures
◉ Mach
◉ L3 and L4
◉ Bring Linux to L4
4

5. Kernel structures
Let’s start from the beginning
1
5

6. Kernel design
◉ Monolithic kernel
◉ Microkernel
◉ Hybrid Kernel
◉ Exokernel
◉ Virtual machine or Hypervisor
6

7. Concept diagram of Monolithic kernel
7
App 1 App 2 App n
OS services and
device drivers
Hardware
...
Each app in is own
hardware address space
OS in its own hardware
address space
Managed by OS

8. Monolithic kernel
is like a
Stonehenge
8
MULTICS, Unix, FreeBSD, Linux

9. Concept diagram of DOS-like kernel
9
App 1 App 2 App n
OS services and
device drivers
Hardware
...
No protection between the
application and OS
Managed by OS

10. Concept diagram of Micro-kernel
1010
App 1 App 2 App n
Hardware
...
Each app in is own
hardware address space
Address space of kernel
Memory
manager
CPU
Scheduler
Filesystem
Micro-kernel
Each service in its own
address space

11. 11

12. Torvalds - Tanenbaum Debate
VS
12
Author of Minix, Amoeba, Globe
Many books on OS and distributed system
One of the most influential fellow in OS research
Chief architect of the Linux kernel
Most influential fellow in open source

13. Torvalds - Tanenbaum Debate
Historical context early 1990s
◉ Windows 3.1
◉ GNU kernel was not out yet
Andrew:
◉ “Linux is obsolete”
◉ “Microkernels have won.” and “Linux is a giant step back into the 1970”
◉ “Designing a monolithic kernel in 1991 is a fundamental error. Be thankful you are
not my student. You would not get a high grade for sucha design :-)”
◉ “The limitations of Minix are partly due to my being a prof.”
13

14. Torvalds - Tanenbaum Debate
Torvalds:
◉ On ”for me MINIX is a hobby”: “You use this as an excuse for the limitations of minix? Sorry, but you
loose: I've got more excuses than you have, and linux still beats the pants of minix in almost all areas.
◉ On “Minix is a micro-kernel design, Linux is a monolithic style sys”: “If this was the only criterion for
the "goodness" of a kernel, you'd be right. What you don't mention is that minix doesn't do the
micro-kernel thing very well, and has problems with real multitasking (in the kernel). If I had made an
OS that had problems with a multithreading filesystem, I wouldn't be so fast to condemn others: in
fact, I'd do my damndest to make others forget about the fiasco.”
◉ "Portability is for people who cannot write new programs”: “I agree that portability is a good thing: but
only where it actually has some meaning. There is no idea in trying to make an operating system overly
portable: adhering to a portable API is good enough. The very idea of an operating system is to use the
hardware features, and hide them behind
14

15. Potential performance loss
15
App 1
OS services and
device drivers
Monolithic
App 1
Microkernel
Filesystem
Microkernel

16. Hybrid kernel
◉ Combine the best of both worlds
◉ Speed and simple design of a monolithic kernel
◉ Modularity and stability of a microkernel
◉ Still similar to a monolithic kernel
◉ Disadvantages still apply here
○ Examples: Windows NT, NetWare, BeOS, Darwin (MacOS X),
DragonFly BSD
16

17. Exokernel
◉ Follows end-to-end principle ("Library" OS)
○ Extremely minimal
○ Fewest hardware abstractions as possible
○ Just allocates physical resources to apps
◉ Disadvantages
○ More work for application developers
Examples: MIT Exokernel, Nemesis
17

18. Kernel structures
Monolithic
The entire operating
system is working in
kernel space, it alone
defines a high-level
virtual interface over
computer hardware.
DOS-like
There is no “kernel”
space, all system is
running in user mode.
MicroKernel
Minimal set of
primitives such as
thread management,
address spaces and
interprocess
communication are
running in kernel
mode.
18

19. Based on discussion thus for...
19
Feature Monolithic OS DOS-like OS MicroKernel OS
Performance
Extensibility
Protection

20. What do we want?
20
Extensibility
Protection Performance
Microkernel
Dos-like
Monolithic

21. ◉ Mach (1985-1994)
○ Replace pipes with RPC
○ Improves stability (vs monolithic)
○ Poor performance
◉ L3 & L4 (1990-2001)
○ order of magnitude improvement in IPC performance
■ L3 written in assembly, sacrificed CPU portability
■ only synchronous IPC
■ very small
Generations of Microkernel
21

22. Mach
MicroKernel 1st generation
2
22

23. 23
Year: 1987● OpenStep / Apple Darwin
(XNU), GNU Hurd, IBM
Workplace OS, OSF/1
● Derived from CMU Accent
operating system
○ Accent has no ability to
execute UNIX
applications (1981)
○ Mach = BSD Unix
system + Accent concepts
○ Carnegie Mellon
University

24. Mach
◉ Simple, extensible communication kernel
◉ Multiprocessor support
◉ POSIX-compatibility
◉ Drivers still in the kernel
24

25. Mach failure
◉ Mach were notoriously noted for suffering from excessive
performance limitations
◉ Mach on a DEC-Station 5200/200 was found to endure peak
degradations of up to 66% when compared to Ultrix running on the
same hardware
◉ Mach-based OSF/1 is cited to perform on average at only half the
performance level of monolithic OSF/1. [1]
25
[1] Michael Condict, Don Bolinger, Dave Mitchell, and Eamonn McManus. Microkernel modularity with integrated kernel performance. Technical report, OSF Research Institute,
Cambridge, 1994.

26. Mach failure
◉ Checking resource access permissions on system calls
○ Single user machines do not need to do this
◉ Asynchronous IPC
○ Most calls only need synchronous IPC
○ Synchronous IPC can be faster than asynchronous
○ Asynchronous IPC can be built on top of synchronous
◉ Virtual memory
○ How to prevent key processes from being paged out?
26

27. L3 and L4
MicroKernel 2st generation
2
27

28. 28
Year: 1993
1953 - 2001

29. “
Microkernels can have adequate
performance if they are
architecture-dependent
29
Jochen Liedtke 1995

30. 30
Jochen Liedtke

31. L3 Abstractions
31
Task
Threads: each has global uid
Own address space
Shared data spaces
Message
From: thread ID
To: thread ID
Direct/Indirect String
Data (optional)
Microkernel
Manages tasks
Sends messages between
tasks
◉ How handle a hardware
interrupt?

32. Minimal IPC
32
Kernel
Thread A1
Task A
Thread B1
Task B
Message
There is no a secure way sending message directly

33. Minimal IPC
33
Kernel
Task A
Thread B1
Task B
Thread A1
1. load B1 ID
2. load message
3. call kernel
9. message receive
4. access Thread B1
5. switch stack pointer
6. switch address space
7. load A1 ID
8. return to user mode

34. Implementation
◉ 1993 L3
◉ 5 Mhz CPU
◉ call kernel is the lowest operation
◉ Synchronous
○ Receiving thread is waiting
○ Sender waits until reply
There is no timeouts: all IPC must
guarantee termination

35. Minimal IPC with large message
35
Kernel
Task A
Thread B1
Task B
Thread A1
1. load B1 ID
2. load message
3. call kernel
9. message receive
4. access Thread B1
5. switch stack pointer
6. switch address space
7. load A1 ID
8. return to user mode
0. setup receive buffBufferData

36. Implementation Year 2013

37. L4
Linux
Bring Linux to L4 Microkernel
37
3

38. L4
Linux and L4
◉ Differences from L3
○ Platform independence
○ Isolation
○ Robustness
○ Security
38

39. L4
Linux
◉ Some portions of Linux are architecture dependent:
○ Interrupts
○ Low-level device drivers (DMA)
○ Methods for interaction with use processes
○ Switching between kernel contexts
○ Copyin/copyout between kernel and user space
○ Mapping/unmapping for address space
○ System calls
39

40. L4
Linux History Context
◉ Is a paravirtualized Linux first presented at SOSP’97 running on the
original L4
kernel
○ L4
Linux predates the x86 virtualization hype
○ L4
Linux 2.4 first version to run on L4
Env
○ L4
Linux 2.6 uses Fiasco.OC’s paravirtualization features
◉ Current status
○ based on Linux 3.8
○ x86 and ARM support
○ SMP (Symmetric multiprocessing)
40

41. Linux Architecture
41

42. L4
Linux
◉ Linux kernel as L4 user service
○ Runs as an L4 thread in a single L4 address space
○ Creates L4 threads for its user processes
○ Maps parts of its address space to user process threads (using L4
primitives)
○ Acts as pager thread for its user threads
○ Has its own logical page table
42

43. L4
Linux
43

44. L4
Linux Interrupt handling
◉ All interrupt handlers are mapped to messages
◉ The Linux server contains threads that do nothing but wait for
interrupt messages
◉ Interrupt threads have a higher priority than the main thread
44

45. L4
Linux User Process
◉ Each different user process is implemented as a different L4 task: Has
its own address space and threads
◉ Linux Server is the pager for these processes. Any fault by the
user-level processes is sent by RPC from the L4 kernel to the Server
45

46. L4
Linux System Calls
◉ The statically linked and shared C libraries are modified
○ Systems calls in the library call the Linux kernel using IPC
◉ For unmodified Linux programs, use “trampoline”
○ The application traps
○ Control bounces to a user-level exception handler
○ The handler calls the modified shared library
Binary compatible
46

47. L4
Linux Signal
◉ Each user-level process has an additional thread for signal handling
◉ Main server thread cannot directly manipulate user process stack, so
it sends a message for the signal handling thread, telling the user
thread to save its state and enter Linux
47

48. L4
Linux Scheduling
◉ All thread scheduling is down by the L4 kernel
◉ Linux server’s schedule() routine is only used for multiplexing it’s
single thread
◉ After each system call, if no other system call is pending, it simply
resumes the user process thread and sleeps
48

49. 49
◉ What is the penalty of using L4
Linux instead of
native Linux?
◉ Does the performance of the underlying
microkernel matter?

50. L4
Linux Microbenchmarks
50

51. L4
Linux Macrobenchmarks
51

52. L4
Linux Macrobenchmarks
52

53. 53
◉ What is the penalty of using L4
Linux instead of
native Linux?
○ The current implementation of L4Linux comes reasonably
close to the behavior of native Linux, even under high
load. Typical penalties range from 5% to 10%.
◉ Does the performance of the underlying
microkernel matter?
○ Yes, absolutely

54. L4
Linux Conclusions
◉ Native Linux
○ L4Linux 8.3% slower; Only 6.8% slower at maximum load.
◉ MkLinux (in-kernel)
○ Linux ported to run inside Mach
○ 29% slower
◉ MkLinux (user)
○ Linux ported to run as a user process on top of Mach
○ 49% slower
54

55. 8,500
Lines of C code for architecture dependent part
55

56. Any questions ?
You can find me at
◉ @sshevchenko (Telegram)
◉ s.shevchenko@studenti.unisa.it
Thanks!
56