Codethink Ltd, an expert in open source software, offers long-term Linux kernel maintenance through its participation in the Civil Infrastructure Project (CIP), which aims to support industrial-grade Linux systems. The CIP, formed by major companies like Hitachi and Siemens, focuses on creating a stable, maintainable Linux core with specific strategies for product maintenance, opting for a long-term support (LTS) approach. The presentation outlines the benefits of collaboration and open-source methodologies in maintaining complex systems while addressing the challenges and strategies associated with the CIP kernel.

1. Systems Software Experts
www.codethink.co.uk

2. In Need Of A Linux Kernel
Maintained For A Very Long
Time?
CIP kernel maintenance overview
Agustín Benito Bethencourt
Principal Consultant - Codethink Ltd
OSSJ, Tokyo, 20th June 2018
CC BY-SA

3. About the speaker

4. The speaker: @toscalix
● Principal Consultant at Codethink Ltd
● Agustín represents Codethink Ltd at CIP.
● Experienced managing the development, delivery and maintenance
of complex Linux-based systems, platforms and distributions.
● Contributor in several FOSS communities (KDE, openSUSE…).
● More about Agustín at his professional site.

5. About Codethink

6. About Codethink
● Founded 2007 as an expert provider of Open Source, Linux and
embedded software services.
● Headquartered in Manchester, UK.
● 90+ staff, mostly systems software engineers and technical
consultants.
● Supporting 10 customers today - including global brands in
automotive, finance, medical devices, consumer electronics,
semiconductors.
● Owner-managed, independent, unbiased and ethical.

7. ● We deliver in reliable, transparent, honest relationships.
● We provide high-calibre people, and continuity.
● We are experts in Open Source and embedded software.
● We can slot into existing projects, adopt existing procedures.
● We hand over all IP and documentation.
● You can stand us down if demand reduces.
Services Value Metrics

8. ● Founded by FOSS engineers.
● Long and successful track record in creating, supporting and
exploiting Open Source initiatives.
● Our people have been heavily involved in the development and
integration of FOSS components e.g. within Ubuntu, Debian,
GNOME, KDE, the Linux Kernel, etc.
● Support corporate customers in making the most of FOSS.
● Affiliated to OIN, GENIVI, The Linux Foundation (CIP and AGL).
Open Source experts

9. ● Scale Microcontrollers to servers
Operating systems
System components
Software tooling
● Technologies Linux, Android, RTOS, bare-metal
Qt, GTK+, Wayland, WebKit, GStreamer, OpenStack, OpenGL
C, Python, Assembly, C++, C#, Java, Haskell, Lua, Ruby
● Specialist skills Graphics
Speech recognition
Augmented reality
Algorithm optimisation
Signal processing
Virtualisation and containerisation
Systems Software Engineering

10. ● Open Source Integration with proprietary technology
Licensing
Upstreaming
Long-term maintenance strategy
● Process and Tooling Continuous delivery
Build and integration pipelines
Chain of custody from design to delivery
Automation
Metrics
● Technical architecture and system design review
Consultancy Services

11. R&D
● BuildStream - framework for modelling build and CI pipelines.
● MUSTARD - FOSS requirements and architecture tracking tool.
● Trustable Software - forum for the discussion and development of
software for safety and security critical applications.
● Baserock - tooling for traceable, reproducible building of software
systems.

12. Why Codethink?
Open Source company.
+
Embedded experience.
+
Passionate about developing, delivering and maintaining
complex Linux systems.

13. Let’s start!

14. Session structure
1. About CIP: 5 minutes.
2. CIP problem statement: 5 minutes.
3. CIP kernel: 5 minutes
4. Development, delivery and maintenance process: 10 minutes
5. Takeaways: 5 minutes
6. Q&A: 10 minutes
Total: 40 minutes

15. About CIP

16. About CIP
● Linux FoundationTM
Initiative formed in 2016 by Hitachi, Siemens,
Toshiba, Codethink and Plat’Home.
● Open Source project with code first approach.
● Governance: Board of Directors and TSC.
● For further information, check the Links slide.

17. About CIP
● Linux FoundationTM
Initiative formed in 2016.
○ Initially driven by Hitachi, Siemens and Toshiba. Founded together
with Codethink and Plat’Home.
● Focused on sharing effort towards creating in the open a
common industrial-grade Linux-based core system and
maintaining it for a very long time.
● For up to date information, check our CIP blog.

18. Provide a super long-term maintained industrial-grade
embedded Linux platform.
Platinum Members
Silver Members

19. CIP at OSSJ 2018
● Visit our booth! There are cool demos including the CIP kernel.
● Talk: Civil Infrastructure Platform: 2 Years Experience of
Industrial-grade Open Source Base Layer Development and its
Future.
○ Friday June 22, 2018 16:20 - 17:00 Room 1.
● Talk: Embedded Linux Quality Assurance: How to not lie with
Statistics.
○ Thursday June 21, 2018 16:00 - 16:40 Room 6.

20. CIP problem statement

21. Product maintenance strategies
Open Source based systems delivery + maintenance models:
● Fire and forget: product release and no software maintenance.
● LTS approach: product release and time-boxed service packs
including at least bug and security fixes.
● Rolling model: frequent updates. Stay as close as possible to
upstream.

22. Product maintenance strategies
CIP chose as initial strategy the one that is expected to better match
“CIP users” business requirements in the coming years: LTS approach.

23. Maintenance in the Linux
world

24. CIP: Railway Control System
● 3 – 5 years development time.
● 2 – 4 years customer specific extensions.
● 1 year initial safety certifications / authorization.
● 3 – 6 months safety certifications / authorization for follow-up
releases (depending on amount of changes).
● 25 – 50 years product lifetime.

25. CIP: Power Plant Control System
● 3 – 5 years development time.
● 0.5 – 4 years customer specific extensions.
● 6 - 8 years supply time.
● 15 years hardware maintenance after latest shipment.
● 20 – 60 years product lifetime.

26. CIP vs other industries
3 to 4 times
longer

27. CIP challenge
● Can the LTS approach be extended: Super Long Term Support
(SLTS)?
● Would it be possible to create SLTS to SLTS transitions?
● Can we have an -rt version of the Linux Kernel as SLTS?

28. CIP kernel

29. Kernel maintainers
● 4.4-cip kernel: Ben Hutchings
○ Debian kernel maintainer.
○ Software engineer at Codethink.
● 4.4-cip-rt kernel: Daniel Wagner
○ 4.4 -rt kernel and ConnMan maintainer
○ Software Engineer at Siemens AG.

30. Current CIP kernel
● Linux kernel 4.4-stable maintained by Greg K-H until Feb. 2022
(6 years).
● CIP kernel based on 4.4-stable.
● Extra patches applied:
○ Platform support changes (from mainline)
○ Security features (from mainline)
○ Fixes for stable regressions (also sent to stable list)

31. Current CIP kernel
● -cip-rt kernel: -rt patches applied to CIP 4.4 kernel.
● Members kernel configs evaluated.
● Latest version released yesterday: 4.4.138-cip25
○ The -rt version to be released soon.

32. CIP kernel development, delivery
and maintenance process

33. CIP kernel overview

34. CIP kernel maintenance
policies
● Very conservative maintenance approach. Simpler process than
LTSI.
● Upstream first, including platform support.
● Involvement in -stable review process.
● Synergies: LTS, LTSI, AGL, Debian LTS, -rt, EdgeX, Y2038, etc.
● CIP kernel maintenance policies.

35. CIP -rt kernel overview

36. CIP -rt kernel main policies
● Upstream first.
○ Fixes should go through the linux kernel -rt stable process before
landing in CIP-rt.
○ No platform specific fixes unless in mainline-rt
● Take ownership of 4.4-rt kernel upstream maintenance.

37. Next CIP kernel
● The next major CIP kernel version is planned to be selected
around the end of 2018 or beginning of 2019.
● The ideal situation would be LTS = LTSI = CIP = Debian LTS

38. CIP kernel limits to lifetime
● Y2038: 2038 as hard maintenance deadline.
● Hardware: limitations in backports and forward-ports for
hardware support.
● Software:
○ Internal API changes over time so backport effort increases over
time.
○ -rt: no specific unsolvable limitations detected for now.

39. Strategies to increase lifetime
● Driven by members’ requirements: CIP kernel is a downstream
kernel.
● Scope reduction.
● -rt as first class citizen of the Linux kernel.
● Lower rate of fixes required to backport over time.
● Collaboration.

40. Takeaways

41. Takeaways
● CIP users have been maintaining HW/SW platforms for years.
● CIP is about doing it the “Open Source way”.
● As with any downstream project, the closer to upstream, the
better.

42. Takeaways
● Not every limitation to the SLTS is known today, but the model
might represent a significant improvement.
● CIP is already adding value to the Linux world. More is yet to
come.

43. Thank you
For further information contact:
● General information about the presentation: Agustín Benito
Bethencourt.
● CIP 4.4 kernel specifics: Ben Hutchings.
● CIP 4.4-rt kernel specifics: Daniel Wagner.

44. Q&A

45. Systems Software Experts

46. Further content

47. ● CIP website.
● CIP whitepaper (2017). An
update to be published very soon.
● CIP wiki.
● CIP blog.
Links
● cip-dev mailing list.
● #cip at irc.freenode.net
● 4.4-cip kernel.
● 4.4-cip-rt kernel.
● Board at Desk: B@D.
● CIP Core.

48. ● This presentation is licensed under the Creative Commons
Attribution-ShareAlike 4.0 International license: CC BY-SA.
● The license does not apply to Trademark affected content like:
○ Codethink:
○ Check the Linux Foundation TM
trademark policy.
○ CIP Member and third party organizations commercial brands.
© Codethink 2018
License