A whirlwind tour of what has been happening in the kernel development community and what can be expected in the near future. The Linux kernel is at the core of any Linux system; the performance and capabilities of the kernel will, in the end, place an upper bound on what the system as a whole can do. This talk will review recent events in the kernel development community, discuss the current state of the kernel and the challenges it faces, and look forward to how the kernel may address those challenges. Attendees of any technical ability should gain a better understanding of how the kernel got to its current state and what can be expected in the near future.

1. The kernel report
(LinuxLab 2018 edition)
Jonathan Corbet
LWN.net
corbet@lwn.net

2. Recent releases
Version Date Days Devs Changesets
4.14 Nov 12 70 1,753 13,452
4.15 Jan 28 77 1,802 14,686
4.16 Apr 1 63 1,670 13,630
4.17 Jun 3 63 1,696 13,541
4.18 Aug 12 70 1,668 13,283
4.19 Oct 21 70 1,749 14,029

3. Recent releases
Version Date Days Devs Changesets
4.14 Nov 12 70 1,753 13,452
4.15 Jan 28 77 1,802 14,686
4.16 Apr 1 63 1,670 13,630
4.17 Jun 3 63 1,696 13,541
4.18 Aug 12 70 1,668 13,283
4.19 Oct 21 70 1,749 14,029
Boring!

4. Recent releases
Version Date Days Devs Changesets
4.14 Nov 12 70 1,753 13,452
4.15 Jan 28 77 1,802 14,686
4.16 Apr 1 63 1,670 13,630
4.17 Jun 3 63 1,696 13,541
4.18 Aug 12 70 1,668 13,283
4.19 Oct 21 70 1,749 14,029

6. Handling of hardware
vulnerabilities
They look like software vulnerabilities
We have a process for those!

7. Handling of hardware
vulnerabilities
They look like software vulnerabilities
We have a process for those!
That process was not followed

8. What happened?
Secrecy
Photo: Emily Kinnaird

9. What happened?
Silos

10. Consequences
Distributor fragmentation
...and some rather poor solutions

11. Let’s compare
Meltdown
Fix developed in public
In good shape at disclosure
Relative uniformity

12. Let’s compare
Meltdown
Fix developed in public
In good shape at disclosure
Relative uniformity
Spectre
Fixes done in private
Not ready at disclosure
Did not survive into mainline

13. The fact that this patch made it through the
maintainers is probably a late effect of the
secrecy that covered all the Spectre/Meltdown
work one year ago.
— Paolo Bonzini, November 29

14. Consequences
Distributor fragmentation
...and some rather poor solutions
Developer burnout and frustration

15. What part of this whole [bleep] mess isn't
entirely batshit insane to start with?
— David Woodhouse

16. Consequences
Distributor fragmentation
...and some rather poor solutions
Developer burnout and frustration
Many left out in the cold

17. The good news
The lessons have (hopefully)
been learned

18. The bad news

19. The bad news

20. What’s to be done?

21. What’s to be done?
Running on your own hardware?
No untrusted software?
Relax.

22. What’s to be done?
Disable SMT

23. What’s to be done?
Coscheduling

24. What’s to be done?
Run updated kernels!

25. Stable kernels

26. Current stable kernels
Release Original Changes
4.19.4 Oct 2018 636
4.18.19 Aug 2018 2,284
4.14.82 Nov 2017 8,051
4.9.139 Dec 2016 10,431
4.4.163 Jan 2016 9,924
3.18.125 Dec 2014 7,626

27. -stable initiatives
Longer-term support
4.4 until 2022, 4.9 to 2023
CIP looking at 10-20 years of support!

28. -stable initiatives
Longer-term support
4.4 until 2022, 4.9 to 2023
CIP looking at 10-20 years of support!
Catching more fixes
4.9→4.9.139: 10,431 changes
4.9→4.19: 149,000 changes

29. -stable problems
Regressions

30. -stable problems
Regressions
Huge invasive fixes
(Meltdown/Spectre)

31. Is the “longterm stable kernel”
model broken?

32. The illusion to support a product for 20 years with
software from 20 years ago has been destroyed
long ago, but still people cling to it for any price.
— Thomas Gleixner

33. The solution?
Move older systems to newer kernels
(current long-term stable)
All the latest fixes – and features too
The best kernel we know how to make

34. One little problem...
Ancient hardware

35. Testing

36. Kernel testing initiatives
0day robot
KernelCI
kselftest
Syzbot
LKFT
...
(+ vendor internal efforts)

37. One has to start somewhere!

38. BPF

39. BPF
An in-kernel virtual machine

40. BPF
An in-kernel virtual machine
Built-in verifier

41. BPF
An in-kernel virtual machine
Built-in verifier
In-kernel JIT compiler

42. Where BPF shows up
Security policy decisions
seccomp
Landlock security module

43. Where BPF shows up
Security policy decisions
seccomp
Landlock security module
Protocol implementations
IR remote control

44. Where BPF shows up
Security policy decisions
seccomp
Landlock security module
Protocol implementations
IR remote control
Instrumentation
Kernel tracing

45. Where BPF shows up
Packet filtering
bpfilter
Network
processing
AF_XDP

46. Where BPF shows up
Packet filtering
bpfilter
Network
processing
AF_XDP

47. In short
BPF is used...
To supplement existing kernel functionality

48. In short
BPF is used...
To supplement existing kernel functionality
To replace existing kernel functionality

49. In short
BPF is used...
To supplement existing kernel functionality
To replace existing kernel functionality
...to allow us to push code into the kernel

50. Pushing code to user space
AF_XDP
Seccomp trap to user space
ELF modules
userfaultfd()

51. Kernel
User space

52. Kernel
User space

53. Out-of-tree code

54. Does your phone run Linux?

55. Does your phone run Linux?
...are you sure...?

57. Why this hurts
Vendors get stuck on old kernels
No mainline kernels on Android devices
Hobbyists are locked out
Updates aren’t possible
Lots of wasted effort

58. Signs of progress
Android Oreo
3.18, 4.4, or 4.9 required

59. Signs of progress
Android Oreo
3.18, 4.4, or 4.9 required
Android Pie
4.4.107+, 4.9.84+, or 4.14.42+

60. Signs of progress
Android Oreo
3.18, 4.4, or 4.9 required
Android Pie
4.4.107+, 4.9.84+, or 4.14.42+
Future:
Stable updates must be shipped

61. Signs of progress
The Android Common Kernel
Only 30 patches
Generic System Image
An AOSP build that must boot

62. The goal
The core kernel is in the GSI
Vendor code exists as kernel modules

63. Mainline kernels on Android may
yet happen

64. Conduct

67. The wild west

68. The Good Olde Days
No source-code management
No change tracking
No release discipline
No rules on regressions
No automated testing
…

69. The Good Olde Days
No source-code management
No change tracking
No release discipline
No rules on regressions
No automated testing
…
No code of conduct

70. The Good Olde Days
✔ source-code management
✔ change tracking
✔ release discipline
✔ rules on regressions
✔ automated testing
…
No code of conduct

71. The Good Olde Days
✔ source-code management
✔ change tracking
✔ release discipline
✔ rules on regressions
✔ automated testing
…
✔ code of conduct

73. Thank you