kernelci.org is an open source project that aims to validate Linux kernel code by building, booting, and testing kernel configurations on various hardware platforms. It consists of five automated hardware labs around the world. For each change to the kernel tree, it builds over 300,000 configurations, boots over 400,000 platforms, and executes over 5,000 tests. The goals are to ensure regressions are caught, provide a platform reference for the community, and fix bugs reported from results.

1. kernelci.org
The upstream kernel validation project

2. Who is this guy?
● Tyler Baker
● Director of Engineering at Linaro
○ Automation and CI
● Maintainer of Linaro’s Automated Validation
Architecture (LAVA)
● Upstream Linux Kernel contributor
● Embedded Software Engineer

3. Goals
Build every configuration for each architecture.
Boot these configurations.
Execute tests on these configurations.
Do ALL of this each time a kernel tree changes.
Report the results in a sensible way.
Profit.
#exactsteps

4. Wait, but why?
To ensure the hard work being done upstream
is gross regression free.
In addition, to provide a platform reference for
the community.

5. Prologue
The ARM SoC maintainers developed scripts to
build and boot test various boards for sanity
purposes. This work would eventually become
the basis for kernelci.org.

6. Current Status
kernelci.org consists of five automated
hardware labs distributed all over the world.
The hardware ranges from x86 servers to very
small embedded platforms. This cluster of labs
produce ~450 platform results for each tree.

7. 0-Day
Does a really good job, it is extremely fast, but
does have some issues.
Source code not available
Only x86 VMs
Does not publish artifacts

8. How is this system different?
kernelci.org is…
distributed
bootstrapping from metal
open source
community driven
publishing artifacts

9. Metrics
436,599 platforms booted ~1000 boot per day
298,343 kernel trees built (arm, arm64, x86)
5,182 test cases executed
43 kernel bugs fixed upstream in 2015

10. Lessons Learned

11. Enough Talk Already
SHOW ME SOME
DEMOS

12. EMAIL REPORT
DEMO

13. FRONTEND
DEMO

14. KSELFTEST
DEMO

15. LAVA
DEMO

16. AUTOMATED
BISECTION
DEMO

17. API
DEMO

18. Future
Test result reporting and visualization
Result deltas
Automagic bisection
Toolchain testing
Moar hardware

19. Big Data
We have been archiving all data since v3.12.
Elastic Search?
Trends?

20. What is missing?

21. How can I halp?
Donations!
● Upstream supported platforms
● Build machines
Debug!
● Report issues
● Confirm fixes
● Send patches

22. Get involved
#kernelci on freenode
https://github.com/kernelci
kernel-build-reports mailing list
http://wiki.kernelci.org
http://api.kernelci.org
info@kernelci.org

23. Thanks!
Any Questions?