SZ Lin, profile picture
Uploaded bySZ Lin
PPTX, PDF1,212 views

Design, Build,and Maintain the Embedded Linux Platform

The document outlines the key considerations and best practices for using embedded Linux platforms, including aspects such as hardware survey, software lifecycle management, and open source compliance. It also emphasizes the importance of addressing cybersecurity, seeking longevity and stability, as well as providing effective software and system update approaches. Additionally, it compares various components and tools necessary for embedded Linux development and highlights collaboration within the community for diverse target applications.

Embed presentation

Downloaded 24 times
Embedded Linux Platform 林上智 (SZ LIN) szlin@debian.org Date: 2020/7/15
2 林上智 (SZ LIN) • Software Engineer • Embedded Linux Design and Development • Kernel Contributor • Civil Infrastructure Platform TSC Member • Open Source Development and Governance • Debian Developer • OpenChain Project Governing Board Member • Cybersecurity • CISSP – ISSAP, CSSLP • ISA/ IEC 62443 Cybersecurity Expert About Me 178F 8338 B314 01E3 04FC 44BA A959 B38A 9561 F3F9
3 RoutersEthernet Switches Wireless LAN Network Infrastructure Embedded Computers Computing Serial Connectivity I/O Connectivity Video Connectivity Edge Connectivity Device
Before Using Embedded Linux Something You Should Know 4
5 Copyright Patent A patent gives its owner the right to exclude others from making, using, selling, and importing an invention for a limited period of time, usually twenty years. src: https://en.wikipedia.org/wiki/Patent Copyright is a legal right, that grants the creator of an original work exclusive rights to determine whether, and under what conditions, this original work may be used by others src: https://en.wikipedia.org/wiki/Copyright
6 Copyright Patent Identify key recommended processes for effective open source management [1]. It is a shared defensive patent pool with the mission to protect Linux [2].
Target Application Align with business 7
8 Longevity + Stability + Security Performance Real-time SafetyResource Limited
9 Longevity + Stability + Security Performance Real-time SafetyResource Limited Performance Real-time SafetyResource Limited SoftwareHardware
10 Image: https://upload.wikimedia.org/wikipedia/commons/5/5b/Linux_kernel_map.png
11 Hardware Survey – Key Components Short LifecycleLong Lifecycle (> 10 years) SoC e.g., Intel, ARM, MIPS Non-volatile memory e.g., NAND flash, eMMC, SD, SSD DRAMRTC Watchdog PMIC Cellular module Wi-Fi module PHY Switch module UART module WPAN module External LAN controller
12 Lifecycle of Embedded Linux Platform Maintenance PhaseDevelopment Phase System Preparation Regular UpdateBuild and Testing 4321 Long-term Test
13 13 Maintenance PhaseDevelopment Phase System Preparation Regular UpdateBuild and Testing 4321 Long-term Test More info: Manage Vulnerabilities in Open Source Components in ICS Industrial Control Systems (ICS) Cyber Security Conference Singapore, 2020 [3]
Development Phase Platform design and customization 14
15 GNU/ Linux User Space Kernel Space Toolchain
Choose Proper Bootloader 16
17
18 Category License Supported Platforms Supported UEFI Maintainer Das U-Boot [4] GPL-2+ 68k, ARM, Blackfin, MicroBlaze, MIPS, Nios, SuperH, PPC, RISC-V, x86 (on top of Coreboot) Y DENX Software Engineering Coreboot [5] GPL-2 IA-32, x86-64, ARMv7, ARMv8, MIPS, RISC-V, POWER8 Y coreboot.org GRUB GPL-3 IA-32, x86-64, IA-64, ARM, PowerPC, MIPS and SPARC Y GNU Project rEFInd [6][7] GNU GPLv3, Modified BSD License (original program), additional components released under various licenses x86, x86-64, or ARM64 Y Roderick W. Smith
19 Kernel Space
Choose Proper Kernel Based on the application requirement 20
21 Linux Kernel Comparison Table Category Latest version Target Application Maintainer Linux kernel 5.7 • Performance • Resource Limited [8] [9] Kernel.org Preempt RT kernel 5.6 • Real-time • Functional safety • Resource Limited Real Time Linux collaborative project
22 SoC Board Support Package Kernel • Kernel version depends on SoC vendors – Well made but not well maintained • Contain lots of in-house patches – Errata patches – Specific feature patches – … • Different SoC might use different versions of kernel • The lifetime is unsure
23 LTS: Long Term Stable Kernel [10] Extend software uptime for stable kernel • Only accept bug fixes and security fixes img: https://www.kernel.org/category/releases.html
24 LTSI: Long Term Support Initiative [11] • Linux Foundation collaborative project – Based on LTS – Add another chance to include further patches on top of LTS – Auto Test framework – Same lifetime with LTS (yearly release and 2 years life time)
25 CIP (Civil Infrastructure Platform) [12] • Linux Foundation collaborative project – Support kernel and core package – Auto Test framework – Maintenance period • 10 years and more (10-20 years) More info: Activities of Super Long Term Support Kernel Workgroup in Civil Infrastructure Platform Project Embedded Linux Conference Europe, 2019[51]
26 Linux Kernel Source Comparison Table Version Maintenance Period (years) Features Latest Version Supported Real- time kernel Maintainer SoC BSP kernel ? Bug fixes ? N SoC vendor kernel team LTS kernel 2 ~ ? • Bug fixes • Security fixes 5.4 N Kernel.org LTSI kernel 2 ~ ? • Bug fixes • Security fixes • Specific features • New features 4.14 N LTSI (Linux Foundation Projects) CIP kernel 10 + • Bug fixes • Security fixes • Specific features • New features 4.19 Y CIP (Linux Foundation Projects)
27 ELISA: Safety-Critical Systems [13] • Linux Foundation collaborative project – Build and certify Linux-based safety-critical applications – Define and maintain a common set of tools and processes • SIL2LinuxMP [14] project and the Linux Foundation’s Real-Time Linux project – IEC 61508
28 Longevity + Stability + Security Performance Real-time SafetyResource Limited Mutually Exclusive ?
Multiple Kernel In Single Platform To fulfill multiple user scenarios 29
30 FIT (Flattened Image Tree) • Tree data structure • Handle multiple types of image – kernel : kernel image – fdt : dtb file – ramdisk : root file system • Image hashing – md5 – sha1 • Image signing • Each node in configurations has their image configuration in booting stage
31 More info.: http://git.denx.de/?p=u-boot.git;a=blob_plain;f=doc/uImage.FIT/source_file_format.txt;hb=HEAD
32 User Space
Choose Proper C Library and Toolchain 33
34 C Library and Toolchain Comparison Table Category License Features Target Application Maintainer User glibc [15] LGPL 2.1 • Stable ABI • Backward compatibility • Fully symbol versioning • Stack smashing protection/ heap corruption detection • Profiling • Performance • Security GNU uClibc-ng [16] LGPL 2.1 • No-MMU architecture support • Tiny size • Resource Limited uclibc-ng.org Musl [17] MIT • Stable ABI • Backward compatibility • Stack smashing protection/ heap corruption detection • Resource Limited • Security musl-libc.org * Be aware of year 2038 problem [19]Other option [72]
35 Year 2038 Problem [20] • The time_t datatype is a data type in the ISO C library and kernel structure defined for storing system time values. • 32-bit system can represent dates from  Dec 13 1901  Jan 19th 2038 • It causes integer overflowing on 03:14:08 UTC 19 January 2038
Init System 36
37 Init System Comparison Table Category License C Library User Note busybox GPL 2.0 uClinux-ng Glibc musl ProteanOS PiBox Resource-limited application sysvinit GPL 2.0+ uClinux-ng glibc musl Devuan systemd LGPL 2.1+ glibc Arch, CentOS, CoreOS, Debian, Fedora, Mint, OpenSUSE, Redhat, Ubuntu Linux only openrc 2-clause BSD musl glibc Gentoo Alpine Linux upstart GPL 2.0 glibc Chromium OS Linux only
Choose proper RFS (Root filesystem) 38
39 Root Filesystem Comparison Table Category Maintenance Period (years) Number of packages C Library Security Tracker CI Busybox ? 300 ~ 400 applets • uClibc • glibc ? ? Yocto Latest release the previous two releases It depends on meta- * • glibc • musl Y Y Buildroot 1 2000+ [42] • glibc • musl • uClibc-ng Y Y Debian 3 + 2 (i386, amd64, armel, armhf and arm64) 51000+ • glibc • musl Y Y
System Development Tools 40
41 System Development Tools Comparison Table Root filesystem System Development Tools Toolchain System Development Tools License Busybox Yocto OE-Core MIT Yocto Yocto OE-Core MIT Buildroot Buildroot Buildroot GPL 2.0+ Debian ISAR Debian toolchain Metadata: MIT Others: GPL 2.0 ELBE Debian toolchain GPL 3.0+ Yocto Deby (meta-debian) OE-Core MIT Live-build Debian toolchain GPL 3.0+
42 Why to Choose Debian [40] Good system security [41] Everything is open Usually, fixed packages are uploaded within a few days Stability unstable → testing → stable Scalability Server, Desktop, Laptop, Embedded devices Long term support 5 more years by Debian-LTS project (i386, amd64, armel, armhf and arm64) Multiple architectures alpha, amd64, armel, armhf, aarch64, hppa, i386, ia64, mips, mipsel, powerpc, s390, and sparc Open source compliance All packages in the Debian archive must to comply with the Debian Free Software Guidelines (DFSG)
Upstream First 43
v1.0 Security fixes New features v1.1 v2.0 Project inside the organization v1.0 v1.1 v2.0 In-house security or bug patches Bug fixes Upstream 44
•The project shares its results with the upstream •The project fulfills longer time maintenance and security fixes •The project develops their code very quickly •The project faces difficulties to backport upstream patches due to conflicts as time goes by 45
46 46 Maintenance PhaseDevelopment Phase System Preparation Regular UpdateBuild and Testing 4321 Long-term Test More info: Building, Deploying and Testing an Industrial Linux Platform Open Source Summit Japan, 2017 [42]
47
48 CI/ CD Automatic Release Pipeline Building Testing Deploying 432 1 Release Continuous Delivery Continuous DeploymentCode Continuous Integration
49 CI/ CD Automatic Release Pipeline Building Testing Deploying 432 1 Release Code Continuous Integration
50
51 Static Testing Cases Management - Jenkins Static analysis #1 Static analysis #2 … Static analysis #n
52 Image: https://c1.staticflickr.com/5/4030/4438139050_04604b4908.jpg
53 Distributed Compiler • Software – Icecream/ IceCC was created by SUSE based on distcc [43][44] • Improve performance of compile jobs in parallel • Add dynamic scheduler of the compilation jobs • Support multiple platform • Support cross compiling • Hardware - for each node – SSD – Large capacity memory – Gigabit LAN
54 CI/ CD Automatic Release Pipeline Building Testing Deploying 432 1 Release Continuous Delivery Code Continuous Integration
55
56 Continuous Delivery – LAVA [45][46]
57
58 Fuego [47][48] • Test framework for testing embedded Linux – Official automated test framework for the LTSI project. • BSD 3-Clause license in default • Over 100 pre-packaged tests – Ability for 3rd parties to initiate or schedule tests on our hardware, and the ability to share our test results with others.
59
Maintenance Phase Long-term Testing and Regular Update 60
More info: Long-term Maintenance Model of Embedded Industrial Linux Distribution Open Source Summit China 2017 61 61 Maintenance PhaseDevelopment Phase System Preparation Regular UpdateBuild and Testing 4321 Long-term Test
62 Notification Test Results * Test cases are managed by test framework
63 For Stable Kernel Maintenance • Automated Linux Kernel Testing [49][50] – Detect, bisect, report and fix regressions on upstream Kernel trees before release – Short tests on many configurations KernelCI
64 Reproducible Builds [51] • Create an independently-verifiable path from source to binary – Ensure builds have identical results – Act as part of a chain of trust – Prove the source code has not been tampered/modified
65 Open Source Testing Tools Continuous Integration • Jenkins [55] • Jenkins X [56] Continuous Delivery/ Deployment • LAVA 2 [45] Distributed compiler service • icecc • GOMA [57][58] • distcc [51] Test Case Management • Jenkins • LAVA 2 • Fuego [47][48] Version Control • Git with gitlab [64] Static Program Analysis • Coding style • OWASP • Infer [53] • Sonarqube [54] Dynamic Program Analysis • Gcov [59] • Valgrind [60] • Profiling tools [61] Vulnerability Scanning • OpenVAS [63] • Vuls [65] Fuzzing Testing • Syzkaller [66] • Trinity [68] • OSS-fuzz [69] • The Fuzzing Project [70]
66 CI/ CD/ LT are concepts of software engineering instead of tools or procedures
67 67 Maintenance PhaseDevelopment Phase System Preparation Regular UpdateBuild and Testing 4321 Long-term Test More info: Software Update for Embedded Systems COSCUP Taiwan 2019
68 Why We Need Software Update? Bug fixes Security fixes New userspace program New kernel features
69 The Components Might Be Updated Components Size Update frequency Risk Peripheral devices firmware < 10 MB Rarely Mid Bootloader (including SPL) < 1 MB Rarely High Device tree <100 kB Rarely High Linux kernel < 10 MB Regularly High Root file system Variant Regularly High System configuration < 1 MB Rarely Low Application Variant Often Low
70 Software Update Requirements Basic Features Fail-safe Roll-back Size reduction Signatures Multiple storage type support (e.g., NOR/NAND flash, eMMC) Build system integration Remote access (e.g., OTA)
71 Software Update Requirements Additional Features Online and offline updates Encryption Delta-updates Successful update detection Proactive updating
72 Update Approaches Components Size Complexity Time Cost Image/ block based Large Low Very High File based Variant Low Variant Package based (e.g., deb, rpm) Variant Low Variant Delta based Low Very High Low
73 Partition Architecture
74 Asymmetric/ Symmetric Firmware Updates [71]
75 Comparison - Features Category Fail-Safe Roll-Back Delta- Updates Signatures Multiple Storage Type Support Build System Integration SWUpdate Y Y librsync Y •NOR NAND flashes •UBI volumes •SD / eMMC Yocto/ Buildroot RAUC Y Y casync Y •NOR NAND flashes •UBI volumes •SD / eMMC Yocto/ Buildroot OSTree N Y archive- z2 Y ? Yocto
76 Comparison - Others Method Asymmetric/ Symmetric Image Updates Type Language License SWUpdate Both Image-based File-based C99 GPLv2 With openssl exception RAUC Both Image-based File-based C LGPLv2.1 OSTree Asymmetric File-based C/C++ MPL 2.0 /LGPLv2+
77 Conclusion Community Collaboration Different approach for multiple target applications Preparedness Planning Longevity, stability and security
78 Thank you
79 References [1] https://www.openchainproject.org [2] https://www.openinventionnetwork.com/ [3] https://icss20.sched.com/event/ZjMw/managing-vulnerabilities-in-open-source- components-in-ics [4] https://en.wikipedia.org/wiki/Das_U-Boot [5] https://en.wikipedia.org/wiki/Coreboot [6] http://www.rodsbooks.com/refind/ [7] https://en.wikipedia.org/wiki/REFInd [8] https://tiny.wiki.kernel.org/start [9] https://bootlin.com/pub/conferences/2017/jdll/opdenacker-embedded-linux-in-less- than-4mb-of-ram/opdenacker-embedded-linux-in-less-than-4mb-of-ram.pdf [10] https://www.kernel.org/category/releases.html [11] https://ltsi.linuxfoundation.org/ [12] https://www.cip-project.org/ [13] https://elisa.tech/
80 References [14] http://www.osadl.org/SIL2LinuxMP.sil2-linux-project.0.html [15] https://www.gnu.org/software/libc/ [16] https://uclibc-ng.org/ [17] http://events.linuxfoundation.org/sites/events/files/slides/uclibc-still-makes-sense-brodkin- elce2017.pdf [18] https://www.musl-libc.org/ [19] https://en.wikipedia.org/wiki/Year_2038_problem [20] http://elinux.org/images/6/6e/End_of_Time_--_Embedded_Linux_Conference_2015.pdf [21] https://en.wikipedia.org/wiki/Linux_startup_process [22] http://upstart.ubuntu.com/faq.html [23] https://en.wikipedia.org/wiki/Systemd [24] https://sysdfree.wordpress.com/2019/03/09/135/ [25] https://wiki.gentoo.org/wiki/Comparison_of_init_systems [26] https://elinux.org/images/6/69/Demystifying_Systemd.pdf [27] http://proteanos.com/ [28] https://www.piboxproject.com/ [29] https://lists.debian.org/debian-devel/2016/02/msg00122.html [30] https://busybox.net/FAQ.html#libc [31] https://wiki.yoctoproject.org/wiki/Stable_branch_maintenance
81 References [32] https://www.debian.org/social_contract#guidelines [33] https://bootlin.com/pub/conferences/2018/elc/petazzoni-buildroot-whats-new/petazzoni-buildroot- whats-new.pdf [34] https://events.static.linuxfound.org/sites/events/files/slides/libc-talk.pdf [35] http://events17.linuxfoundation.org/sites/events/files/slides/ELC%202016%20- %20Designing%20a%20distro%20from%20scratch%20using%20OpenEmbedded.pdf [36] https://github.com/meta-debian/meta-debian [37] https://events.linuxfoundation.org/wp-content/uploads/2017/12/ELCE2018_Debian-Yocto-State-of- the-Art_r6_Kazuhiro-Hayashi.pdf [38] https://events.linuxfoundation.org/wp-content/uploads/2017/12/Buildroot-vs-Yocto-Differences-for- Your-Daily-Job-Luca-Ceresoli-AIM-Sportline.pdf [39] https://events.static.linuxfound.org/sites/events/files/slides/belloni-petazzoni-buildroot-oe_0.pdf [40] https://www.debian.org/intro/why_debian.en.html [41] https://www.debian.org/security/index.en.html [42] http://events.linuxfoundation.org/sites/events/files/slides/Building%2C%20Deploying%20and%20Testing% 20an%20Industrial%20Linux%20Platform.pdf [43] https://github.com/icecc [44] https://www.slideshare.net/szlin/distributed-compiler-icecc
82 References [45] https://validation.linaro.org/static/docs/v2/# [46] http://elinux.org/images/3/35/LAVA_Project_Update.pdf [47] https://elinux.org/Fuego [48] http://fuegotest.org/ [49] https://kernelci.org/ [50] https://reproducible-builds.org/ https://fosdem.org/2019/schedule/event/kernelci_a_new_dawn/attachments/slides/3300/export/events/attachments/ker nelci_a_new_dawn/slides/3300/gtucker_kernelci_fosdem_2019_v2_3_1024x768.pdf [51] https://github.com/distcc/distcc [52] https://wiki.jenkins-ci.org/display/JENKINS/Plugins [53] http://fbinfer.com/ [54] https://www.sonarqube.org/ [55] https://jenkins.io [56] https://jenkins.io/projects/jenkins-x/ [57] https://chromium.googlesource.com/infra/goma/server/ [58] https://chromium.googlesource.com/infra/goma/client [59] https://gcc.gnu.org/onlinedocs/gcc/Gcov.html [60] http://valgrind.org/ [61] https://perf.wiki.kernel.org/index.php/Main_Page [62] http://linux-test-project.github.io/ [63] http://www.openvas.org/
83 References [64] https://about.gitlab.com/ [65] https://vuls.io/ [66] https://github.com/google/syzkaller [67] https://kselftest.wiki.kernel.org [68] http://codemonkey.org.uk/projects/trinity/ [69] https://github.com/google/oss-fuzz [70] https://fuzzing-project.org/ [71] https://mkrak.org/wp-content/uploads/2018/04/FOSS-NORTH_2018_Software_Updates.pdf [72] https://android.googlesource.com/platform/bionic [73] https://events.linuxfoundation.org/wp-content/uploads/2017/12/Strategies-for-Developing-and- Deploying-your-Embedded-Applications-and-Images-Mirza-Krak-Mender.io_.pdf [74] http://events17.linuxfoundation.org/sites/events/files/slides/ELC2017_SWUpdate.pdf [75] https://events.linuxfoundation.org/wp-content/uploads/2017/12/ELCE-2018-Update-Tools-BoF_Jan- Lubbe.pdf [76] https://events.linuxfoundation.org/wp-content/uploads/2017/12/ELCE-2018-Update-Tools-BoF_Jan- Lubbe.pdf [77] https://elinux.org/images/f/f5/Embedded_Systems_Software_Update_for_IoT.pdf [78] https://rauc.readthedocs.io/en/latest/ [79] http://elinux.org/images/6/6e/End_of_Time_--_Embedded_Linux_Conference_2015.pdf

More Related Content

PDF
Using open source software to build an industrial grade embedded linux platfo...
bySZ Lin
 
PPTX
Manage kernel vulnerabilities in the software development lifecycle
bySZ Lin
 
PPTX
Software update for embedded systems
bySZ Lin
 
PDF
[ELCE] Activities of super long term support kernel workgroup in civil infras...
bySZ Lin
 
PDF
Introduction to Civil Infrastructure Platform
bySZ Lin
 
PDF
Select, manage, and backport the long term stable kernels
bySZ Lin
 
PDF
Learning notes on Open Source License
bySZ Lin
 
PDF
Take a step forward from user to maintainer or developer in open source secur...
bySZ Lin
 
Using open source software to build an industrial grade embedded linux platfo...
bySZ Lin
 
Manage kernel vulnerabilities in the software development lifecycle
bySZ Lin
 
Software update for embedded systems
bySZ Lin
 
[ELCE] Activities of super long term support kernel workgroup in civil infras...
bySZ Lin
 
Introduction to Civil Infrastructure Platform
bySZ Lin
 
Select, manage, and backport the long term stable kernels
bySZ Lin
 
Learning notes on Open Source License
bySZ Lin
 
Take a step forward from user to maintainer or developer in open source secur...
bySZ Lin
 

What's hot

PPTX
Open source Android 10 on Orange Pi: Meth or Reality?
byGlobalLogic Ukraine
 
PDF
2009-09-24 Get the Hype on System z Webinar with IBM, Current & Future Linux ...
byShawn Wells
 
PPTX
Embedded Linux/ Debian with ARM64 Platform
bySZ Lin
 
PPTX
Fast boot
bySZ Lin
 
PPTX
What should you know about Net Core?
byDamir Dobric
 
PPTX
Distributed Compiler Icecc
bySZ Lin
 
PDF
Userspace drivers-2016
byChris Simmonds
 
PPTX
LAS16-300K2: Geoff Thorpe - IoT Zephyr
byShovan Sargunam
 
PDF
Software, Over the Air (SOTA) for Automotive Grade Linux (AGL)
byLeon Anavi
 
PPTX
y2038 issue
bySZ Lin
 
PDF
Long-term Maintenance Model of Embedded Industrial Linux Distribution
bySZ Lin
 
PDF
2008-09-09 IBM Interaction Conference, Red Hat Update for System z
byShawn Wells
 
PDF
Using Embedded Linux for Infrastructure Systems
byYoshitake Kobayashi
 
PDF
2008-07-30 IBM Teach the Teacher (IBM T3), Red Hat Update for System z
byShawn Wells
 
PDF
Building, deploying and testing an industrial linux platform @ Open source su...
bySZ Lin
 
PDF
Kernel Recipes 2013 - ARM support in the Linux kernel
byAnne Nicolas
 
PDF
Linux Kernel Participation HowTo
bybenavrhm
 
PDF
淺談 Live patching technology
bySZ Lin
 
PDF
Distro Recipes 2013: What&rsquo;s new in gcc 4.8?
byAnne Nicolas
 
PPT
Configuration Management
byelliando dias
 
Open source Android 10 on Orange Pi: Meth or Reality?
byGlobalLogic Ukraine
 
2009-09-24 Get the Hype on System z Webinar with IBM, Current & Future Linux ...
byShawn Wells
 
Embedded Linux/ Debian with ARM64 Platform
bySZ Lin
 
Fast boot
bySZ Lin
 
What should you know about Net Core?
byDamir Dobric
 
Distributed Compiler Icecc
bySZ Lin
 
Userspace drivers-2016
byChris Simmonds
 
LAS16-300K2: Geoff Thorpe - IoT Zephyr
byShovan Sargunam
 
Software, Over the Air (SOTA) for Automotive Grade Linux (AGL)
byLeon Anavi
 
y2038 issue
bySZ Lin
 
Long-term Maintenance Model of Embedded Industrial Linux Distribution
bySZ Lin
 
2008-09-09 IBM Interaction Conference, Red Hat Update for System z
byShawn Wells
 
Using Embedded Linux for Infrastructure Systems
byYoshitake Kobayashi
 
2008-07-30 IBM Teach the Teacher (IBM T3), Red Hat Update for System z
byShawn Wells
 
Building, deploying and testing an industrial linux platform @ Open source su...
bySZ Lin
 
Kernel Recipes 2013 - ARM support in the Linux kernel
byAnne Nicolas
 
Linux Kernel Participation HowTo
bybenavrhm
 
淺談 Live patching technology
bySZ Lin
 
Distro Recipes 2013: What&rsquo;s new in gcc 4.8?
byAnne Nicolas
 
Configuration Management
byelliando dias
 

Similar to Design, Build,and Maintain the Embedded Linux Platform

PDF
Building Embedded Linux Full Tutorial for ARM
bySherif Mousa
 
PDF
SLTS kernel and base-layer development in the Civil Infrastructure Platform
byYoshitake Kobayashi
 
PPTX
Linux internals v4
byLiran Ben Haim
 
PDF
Module 4 Embedded Linux
byTushar B Kute
 
PDF
Embedded Linux on ARM
byEmertxe Information Technologies Pvt Ltd
 
PDF
Linux for embedded_systems
byVandana Salve
 
PDF
Embedded Operating System - Linux
byEmertxe Information Technologies Pvt Ltd
 
PDF
NetBSD and Linux for Embedded Systems
byMahendra M
 
PPT
Enea Linux and LWRT FTF China 2012
byEneaSoftware
 
PDF
Linux Presentation_SSD.pdf
byssuser37b0e0
 
PPTX
Linux Introduction , Unix vs Linux , Linux Advantages
byminzo1988minzawoo
 
PDF
Yocto Project Linux as a platform for embedded system design
byAlex Gonzalez
 
PPTX
Linux introduction, class 1
byAdrian Mikeliunas
 
ODP
ERTS 2008 - Using Linux for industrial projects
byChristian Charreyre
 
PDF
Embedded Linux primer
byDrew Fustini
 
PDF
Introducing the Civil Infrastructure Platform
byYoshitake Kobayashi
 
PPTX
UNIT 3 HISTORY OF EMBEDDED LINUXEMBEDDED LINUX.pptx
byEEEDept2
 
PDF
Hillel kobrovski Linux security overview for ciso
byHillel Kobrovski
 
PDF
Time is ready for the Civil Infrastructure Platform
byYoshitake Kobayashi
 
PDF
00000001_Embedded_Linux_RevisionFile.pdf
byDiana Atef
 
Building Embedded Linux Full Tutorial for ARM
bySherif Mousa
 
SLTS kernel and base-layer development in the Civil Infrastructure Platform
byYoshitake Kobayashi
 
Linux internals v4
byLiran Ben Haim
 
Module 4 Embedded Linux
byTushar B Kute
 
Embedded Linux on ARM
byEmertxe Information Technologies Pvt Ltd
 
Linux for embedded_systems
byVandana Salve
 
Embedded Operating System - Linux
byEmertxe Information Technologies Pvt Ltd
 
NetBSD and Linux for Embedded Systems
byMahendra M
 
Enea Linux and LWRT FTF China 2012
byEneaSoftware
 
Linux Presentation_SSD.pdf
byssuser37b0e0
 
Linux Introduction , Unix vs Linux , Linux Advantages
byminzo1988minzawoo
 
Yocto Project Linux as a platform for embedded system design
byAlex Gonzalez
 
Linux introduction, class 1
byAdrian Mikeliunas
 
ERTS 2008 - Using Linux for industrial projects
byChristian Charreyre
 
Embedded Linux primer
byDrew Fustini
 
Introducing the Civil Infrastructure Platform
byYoshitake Kobayashi
 
UNIT 3 HISTORY OF EMBEDDED LINUXEMBEDDED LINUX.pptx
byEEEDept2
 
Hillel kobrovski Linux security overview for ciso
byHillel Kobrovski
 
Time is ready for the Civil Infrastructure Platform
byYoshitake Kobayashi
 
00000001_Embedded_Linux_RevisionFile.pdf
byDiana Atef
 

More from SZ Lin

PDF
Cellular technology with Embedded Linux - COSCUP 2016
bySZ Lin
 
PDF
Industry Insights Common Pitfalls and Key Considerations in Using Software Bi...
bySZ Lin
 
PPTX
OpenChain - The Industry Standard for Open Source Compliance
bySZ Lin
 
PDF
OpenChain 電信 SBOM 指引 1.0：強化軟體供應鏈透明度與安全性
bySZ Lin
 
PDF
Debian 套件打包教學指南 - 繁體中文翻譯
bySZ Lin
 
PDF
Debian 套件打包教學指南 v0.19 - 繁體中文翻譯
bySZ Lin
 
PPTX
OpenChain 2.0 specification in a nutshell
bySZ Lin
 
DOCX
OpenChain spec 2.0 繁體中文
bySZ Lin
 
Cellular technology with Embedded Linux - COSCUP 2016
bySZ Lin
 
Industry Insights Common Pitfalls and Key Considerations in Using Software Bi...
bySZ Lin
 
OpenChain - The Industry Standard for Open Source Compliance
bySZ Lin
 
OpenChain 電信 SBOM 指引 1.0：強化軟體供應鏈透明度與安全性
bySZ Lin
 
Debian 套件打包教學指南 - 繁體中文翻譯
bySZ Lin
 
Debian 套件打包教學指南 v0.19 - 繁體中文翻譯
bySZ Lin
 
OpenChain 2.0 specification in a nutshell
bySZ Lin
 
OpenChain spec 2.0 繁體中文
bySZ Lin
 

Recently uploaded

PDF
Day 3 - Data and Application Security - 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
PDF
How to Evaluate a High Performance Database
byScyllaDB
 
PDF
100 Insights After the 200th Issue of NewMind AI Journal
byNewMind AI
 
PPTX
Building Agents in Microsoft Agent Framework.pptx
byUdaiappa Ramachandran
 
PPTX
Gemini vs ChatGPT : Comparing Top AI Models
byDigicarrom
 
PPTX
Emancipatory Information Retrieval: Radically Reorienting Information Retriev...
byBhaskar Mitra
 
PPTX
How to make Ai agents using Google Gemini AI.pptx
bypkluffy111
 
PDF
WCAG Analyzer Tools and Techniques for User-Friendly Websites
byAccessibility Analyzer
 
PDF
Security Forum Sessions from Houston 2025 Event
byMark Simos
 
PDF
Day 5 - Red Team + Blue Team in the Cloud - 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
PPTX
MGw_MRS Benfits seu beficios de redes 4g
byJoaquimBarros18
 
PDF
What Is a Private LLM and Why Enterprises Need It
byAivada
 
PDF
CompTIA Cybersecurity Analyst (CySA+) CS0-003: Unit 5
byVICTOR MAESTRE RAMIREZ
 
PPTX
UiPath Autonomous Agents | Building and Orchestrating Agents End-to-End
byUiPathCommunity
 
PDF
Lab 1.5 - Sharing Secrets with GPG ~ 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
PDF
Access Control 2025: From Security Silo to Software-Defined Ecosystem
byMemoori
 
PPTX
From Backup to Resilience: How MSPs Are Preparing for 2026
byMSP360
 
PDF
WHITE PAPER_ Ransomware Payment Policy and Resilience Strategy_ A Framework f...
byssuser0d171c
 
PPTX
wob-report.pptxwob-report.pptxwob-report.pptx
byssuser0d171c
 
DOCX
iRobot Post‑Mortem and Alternative Paths - Discussion Document for Boards and...
byDave Litwiller
 
Day 3 - Data and Application Security - 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
How to Evaluate a High Performance Database
byScyllaDB
 
100 Insights After the 200th Issue of NewMind AI Journal
byNewMind AI
 
Building Agents in Microsoft Agent Framework.pptx
byUdaiappa Ramachandran
 
Gemini vs ChatGPT : Comparing Top AI Models
byDigicarrom
 
Emancipatory Information Retrieval: Radically Reorienting Information Retriev...
byBhaskar Mitra
 
How to make Ai agents using Google Gemini AI.pptx
bypkluffy111
 
WCAG Analyzer Tools and Techniques for User-Friendly Websites
byAccessibility Analyzer
 
Security Forum Sessions from Houston 2025 Event
byMark Simos
 
Day 5 - Red Team + Blue Team in the Cloud - 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
MGw_MRS Benfits seu beficios de redes 4g
byJoaquimBarros18
 
What Is a Private LLM and Why Enterprises Need It
byAivada
 
CompTIA Cybersecurity Analyst (CySA+) CS0-003: Unit 5
byVICTOR MAESTRE RAMIREZ
 
UiPath Autonomous Agents | Building and Orchestrating Agents End-to-End
byUiPathCommunity
 
Lab 1.5 - Sharing Secrets with GPG ~ 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
Access Control 2025: From Security Silo to Software-Defined Ecosystem
byMemoori
 
From Backup to Resilience: How MSPs Are Preparing for 2026
byMSP360
 
WHITE PAPER_ Ransomware Payment Policy and Resilience Strategy_ A Framework f...
byssuser0d171c
 
wob-report.pptxwob-report.pptxwob-report.pptx
byssuser0d171c
 
iRobot Post‑Mortem and Alternative Paths - Discussion Document for Boards and...
byDave Litwiller
 

Design, Build,and Maintain the Embedded Linux Platform

  • 1.
    Embedded Linux Platform 林上智(SZ LIN) szlin@debian.org Date: 2020/7/15
  • 2.
    2 林上智 (SZ LIN) •Software Engineer • Embedded Linux Design and Development • Kernel Contributor • Civil Infrastructure Platform TSC Member • Open Source Development and Governance • Debian Developer • OpenChain Project Governing Board Member • Cybersecurity • CISSP – ISSAP, CSSLP • ISA/ IEC 62443 Cybersecurity Expert About Me 178F 8338 B314 01E3 04FC 44BA A959 B38A 9561 F3F9
  • 3.
  • 4.
    Before Using EmbeddedLinux Something You Should Know 4
  • 5.
    5 Copyright Patent A patentgives its owner the right to exclude others from making, using, selling, and importing an invention for a limited period of time, usually twenty years. src: https://en.wikipedia.org/wiki/Patent Copyright is a legal right, that grants the creator of an original work exclusive rights to determine whether, and under what conditions, this original work may be used by others src: https://en.wikipedia.org/wiki/Copyright
  • 6.
    6 Copyright Patent Identify keyrecommended processes for effective open source management [1]. It is a shared defensive patent pool with the mission to protect Linux [2].
  • 7.
  • 8.
    8 Longevity + Stability+ Security Performance Real-time SafetyResource Limited
  • 9.
    9 Longevity + Stability+ Security Performance Real-time SafetyResource Limited Performance Real-time SafetyResource Limited SoftwareHardware
  • 10.
  • 11.
    11 Hardware Survey –Key Components Short LifecycleLong Lifecycle (> 10 years) SoC e.g., Intel, ARM, MIPS Non-volatile memory e.g., NAND flash, eMMC, SD, SSD DRAMRTC Watchdog PMIC Cellular module Wi-Fi module PHY Switch module UART module WPAN module External LAN controller
  • 12.
    12 Lifecycle of EmbeddedLinux Platform Maintenance PhaseDevelopment Phase System Preparation Regular UpdateBuild and Testing 4321 Long-term Test
  • 13.
    13 13 Maintenance PhaseDevelopment Phase System Preparation RegularUpdateBuild and Testing 4321 Long-term Test More info: Manage Vulnerabilities in Open Source Components in ICS Industrial Control Systems (ICS) Cyber Security Conference Singapore, 2020 [3]
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
    18 Category License SupportedPlatforms Supported UEFI Maintainer Das U-Boot [4] GPL-2+ 68k, ARM, Blackfin, MicroBlaze, MIPS, Nios, SuperH, PPC, RISC-V, x86 (on top of Coreboot) Y DENX Software Engineering Coreboot [5] GPL-2 IA-32, x86-64, ARMv7, ARMv8, MIPS, RISC-V, POWER8 Y coreboot.org GRUB GPL-3 IA-32, x86-64, IA-64, ARM, PowerPC, MIPS and SPARC Y GNU Project rEFInd [6][7] GNU GPLv3, Modified BSD License (original program), additional components released under various licenses x86, x86-64, or ARM64 Y Roderick W. Smith
  • 19.
  • 20.
    Choose Proper Kernel Basedon the application requirement 20
  • 21.
    21 Linux Kernel ComparisonTable Category Latest version Target Application Maintainer Linux kernel 5.7 • Performance • Resource Limited [8] [9] Kernel.org Preempt RT kernel 5.6 • Real-time • Functional safety • Resource Limited Real Time Linux collaborative project
  • 22.
    22 SoC Board SupportPackage Kernel • Kernel version depends on SoC vendors – Well made but not well maintained • Contain lots of in-house patches – Errata patches – Specific feature patches – … • Different SoC might use different versions of kernel • The lifetime is unsure
  • 23.
    23 LTS: Long TermStable Kernel [10] Extend software uptime for stable kernel • Only accept bug fixes and security fixes img: https://www.kernel.org/category/releases.html
  • 24.
    24 LTSI: Long TermSupport Initiative [11] • Linux Foundation collaborative project – Based on LTS – Add another chance to include further patches on top of LTS – Auto Test framework – Same lifetime with LTS (yearly release and 2 years life time)
  • 25.
    25 CIP (Civil InfrastructurePlatform) [12] • Linux Foundation collaborative project – Support kernel and core package – Auto Test framework – Maintenance period • 10 years and more (10-20 years) More info: Activities of Super Long Term Support Kernel Workgroup in Civil Infrastructure Platform Project Embedded Linux Conference Europe, 2019[51]
  • 26.
    26 Linux Kernel SourceComparison Table Version Maintenance Period (years) Features Latest Version Supported Real- time kernel Maintainer SoC BSP kernel ? Bug fixes ? N SoC vendor kernel team LTS kernel 2 ~ ? • Bug fixes • Security fixes 5.4 N Kernel.org LTSI kernel 2 ~ ? • Bug fixes • Security fixes • Specific features • New features 4.14 N LTSI (Linux Foundation Projects) CIP kernel 10 + • Bug fixes • Security fixes • Specific features • New features 4.19 Y CIP (Linux Foundation Projects)
  • 27.
    27 ELISA: Safety-Critical Systems[13] • Linux Foundation collaborative project – Build and certify Linux-based safety-critical applications – Define and maintain a common set of tools and processes • SIL2LinuxMP [14] project and the Linux Foundation’s Real-Time Linux project – IEC 61508
  • 28.
    28 Longevity + Stability+ Security Performance Real-time SafetyResource Limited Mutually Exclusive ?
  • 29.
    Multiple Kernel InSingle Platform To fulfill multiple user scenarios 29
  • 30.
    30 FIT (Flattened ImageTree) • Tree data structure • Handle multiple types of image – kernel : kernel image – fdt : dtb file – ramdisk : root file system • Image hashing – md5 – sha1 • Image signing • Each node in configurations has their image configuration in booting stage
  • 31.
  • 32.
  • 33.
    Choose Proper C Libraryand Toolchain 33
  • 34.
    34 C Library andToolchain Comparison Table Category License Features Target Application Maintainer User glibc [15] LGPL 2.1 • Stable ABI • Backward compatibility • Fully symbol versioning • Stack smashing protection/ heap corruption detection • Profiling • Performance • Security GNU uClibc-ng [16] LGPL 2.1 • No-MMU architecture support • Tiny size • Resource Limited uclibc-ng.org Musl [17] MIT • Stable ABI • Backward compatibility • Stack smashing protection/ heap corruption detection • Resource Limited • Security musl-libc.org * Be aware of year 2038 problem [19]Other option [72]
  • 35.
    35 Year 2038 Problem[20] • The time_t datatype is a data type in the ISO C library and kernel structure defined for storing system time values. • 32-bit system can represent dates from  Dec 13 1901  Jan 19th 2038 • It causes integer overflowing on 03:14:08 UTC 19 January 2038
  • 36.
  • 37.
    37 Init System ComparisonTable Category License C Library User Note busybox GPL 2.0 uClinux-ng Glibc musl ProteanOS PiBox Resource-limited application sysvinit GPL 2.0+ uClinux-ng glibc musl Devuan systemd LGPL 2.1+ glibc Arch, CentOS, CoreOS, Debian, Fedora, Mint, OpenSUSE, Redhat, Ubuntu Linux only openrc 2-clause BSD musl glibc Gentoo Alpine Linux upstart GPL 2.0 glibc Chromium OS Linux only
  • 38.
  • 39.
    39 Root Filesystem ComparisonTable Category Maintenance Period (years) Number of packages C Library Security Tracker CI Busybox ? 300 ~ 400 applets • uClibc • glibc ? ? Yocto Latest release the previous two releases It depends on meta- * • glibc • musl Y Y Buildroot 1 2000+ [42] • glibc • musl • uClibc-ng Y Y Debian 3 + 2 (i386, amd64, armel, armhf and arm64) 51000+ • glibc • musl Y Y
  • 40.
  • 41.
    41 System Development ToolsComparison Table Root filesystem System Development Tools Toolchain System Development Tools License Busybox Yocto OE-Core MIT Yocto Yocto OE-Core MIT Buildroot Buildroot Buildroot GPL 2.0+ Debian ISAR Debian toolchain Metadata: MIT Others: GPL 2.0 ELBE Debian toolchain GPL 3.0+ Yocto Deby (meta-debian) OE-Core MIT Live-build Debian toolchain GPL 3.0+
  • 42.
    42 Why to ChooseDebian [40] Good system security [41] Everything is open Usually, fixed packages are uploaded within a few days Stability unstable → testing → stable Scalability Server, Desktop, Laptop, Embedded devices Long term support 5 more years by Debian-LTS project (i386, amd64, armel, armhf and arm64) Multiple architectures alpha, amd64, armel, armhf, aarch64, hppa, i386, ia64, mips, mipsel, powerpc, s390, and sparc Open source compliance All packages in the Debian archive must to comply with the Debian Free Software Guidelines (DFSG)
  • 43.
  • 44.
    v1.0 Security fixes Newfeatures v1.1 v2.0 Project inside the organization v1.0 v1.1 v2.0 In-house security or bug patches Bug fixes Upstream 44
  • 45.
    •The project sharesits results with the upstream •The project fulfills longer time maintenance and security fixes •The project develops their code very quickly •The project faces difficulties to backport upstream patches due to conflicts as time goes by 45
  • 46.
    46 46 Maintenance PhaseDevelopment Phase System Preparation RegularUpdateBuild and Testing 4321 Long-term Test More info: Building, Deploying and Testing an Industrial Linux Platform Open Source Summit Japan, 2017 [42]
  • 47.
  • 48.
    48 CI/ CD AutomaticRelease Pipeline Building Testing Deploying 432 1 Release Continuous Delivery Continuous DeploymentCode Continuous Integration
  • 49.
    49 CI/ CD AutomaticRelease Pipeline Building Testing Deploying 432 1 Release Code Continuous Integration
  • 50.
  • 51.
    51 Static Testing CasesManagement - Jenkins Static analysis #1 Static analysis #2 … Static analysis #n
  • 52.
  • 53.
    53 Distributed Compiler • Software –Icecream/ IceCC was created by SUSE based on distcc [43][44] • Improve performance of compile jobs in parallel • Add dynamic scheduler of the compilation jobs • Support multiple platform • Support cross compiling • Hardware - for each node – SSD – Large capacity memory – Gigabit LAN
  • 54.
    54 CI/ CD AutomaticRelease Pipeline Building Testing Deploying 432 1 Release Continuous Delivery Code Continuous Integration
  • 55.
  • 56.
  • 57.
  • 58.
    58 Fuego [47][48] • Testframework for testing embedded Linux – Official automated test framework for the LTSI project. • BSD 3-Clause license in default • Over 100 pre-packaged tests – Ability for 3rd parties to initiate or schedule tests on our hardware, and the ability to share our test results with others.
  • 59.
  • 60.
  • 61.
    More info: Long-termMaintenance Model of Embedded Industrial Linux Distribution Open Source Summit China 2017 61 61 Maintenance PhaseDevelopment Phase System Preparation Regular UpdateBuild and Testing 4321 Long-term Test
  • 62.
    62 Notification Test Results * Testcases are managed by test framework
  • 63.
    63 For Stable KernelMaintenance • Automated Linux Kernel Testing [49][50] – Detect, bisect, report and fix regressions on upstream Kernel trees before release – Short tests on many configurations KernelCI
  • 64.
    64 Reproducible Builds [51] •Create an independently-verifiable path from source to binary – Ensure builds have identical results – Act as part of a chain of trust – Prove the source code has not been tampered/modified
  • 65.
    65 Open Source TestingTools Continuous Integration • Jenkins [55] • Jenkins X [56] Continuous Delivery/ Deployment • LAVA 2 [45] Distributed compiler service • icecc • GOMA [57][58] • distcc [51] Test Case Management • Jenkins • LAVA 2 • Fuego [47][48] Version Control • Git with gitlab [64] Static Program Analysis • Coding style • OWASP • Infer [53] • Sonarqube [54] Dynamic Program Analysis • Gcov [59] • Valgrind [60] • Profiling tools [61] Vulnerability Scanning • OpenVAS [63] • Vuls [65] Fuzzing Testing • Syzkaller [66] • Trinity [68] • OSS-fuzz [69] • The Fuzzing Project [70]
  • 66.
    66 CI/ CD/ LTare concepts of software engineering instead of tools or procedures
  • 67.
    67 67 Maintenance PhaseDevelopment Phase System Preparation RegularUpdateBuild and Testing 4321 Long-term Test More info: Software Update for Embedded Systems COSCUP Taiwan 2019
  • 68.
    68 Why We NeedSoftware Update? Bug fixes Security fixes New userspace program New kernel features
  • 69.
    69 The Components MightBe Updated Components Size Update frequency Risk Peripheral devices firmware < 10 MB Rarely Mid Bootloader (including SPL) < 1 MB Rarely High Device tree <100 kB Rarely High Linux kernel < 10 MB Regularly High Root file system Variant Regularly High System configuration < 1 MB Rarely Low Application Variant Often Low
  • 70.
    70 Software Update Requirements BasicFeatures Fail-safe Roll-back Size reduction Signatures Multiple storage type support (e.g., NOR/NAND flash, eMMC) Build system integration Remote access (e.g., OTA)
  • 71.
    71 Software Update Requirements AdditionalFeatures Online and offline updates Encryption Delta-updates Successful update detection Proactive updating
  • 72.
    72 Update Approaches Components SizeComplexity Time Cost Image/ block based Large Low Very High File based Variant Low Variant Package based (e.g., deb, rpm) Variant Low Variant Delta based Low Very High Low
  • 73.
  • 74.
  • 75.
    75 Comparison - Features CategoryFail-Safe Roll-Back Delta- Updates Signatures Multiple Storage Type Support Build System Integration SWUpdate Y Y librsync Y •NOR NAND flashes •UBI volumes •SD / eMMC Yocto/ Buildroot RAUC Y Y casync Y •NOR NAND flashes •UBI volumes •SD / eMMC Yocto/ Buildroot OSTree N Y archive- z2 Y ? Yocto
  • 76.
    76 Comparison - Others Method Asymmetric/ Symmetric ImageUpdates Type Language License SWUpdate Both Image-based File-based C99 GPLv2 With openssl exception RAUC Both Image-based File-based C LGPLv2.1 OSTree Asymmetric File-based C/C++ MPL 2.0 /LGPLv2+
  • 77.
    77 Conclusion Community Collaboration Differentapproach for multiple target applications Preparedness Planning Longevity, stability and security
  • 78.
  • 79.
    79 References [1] https://www.openchainproject.org [2] https://www.openinventionnetwork.com/ [3]https://icss20.sched.com/event/ZjMw/managing-vulnerabilities-in-open-source- components-in-ics [4] https://en.wikipedia.org/wiki/Das_U-Boot [5] https://en.wikipedia.org/wiki/Coreboot [6] http://www.rodsbooks.com/refind/ [7] https://en.wikipedia.org/wiki/REFInd [8] https://tiny.wiki.kernel.org/start [9] https://bootlin.com/pub/conferences/2017/jdll/opdenacker-embedded-linux-in-less- than-4mb-of-ram/opdenacker-embedded-linux-in-less-than-4mb-of-ram.pdf [10] https://www.kernel.org/category/releases.html [11] https://ltsi.linuxfoundation.org/ [12] https://www.cip-project.org/ [13] https://elisa.tech/
  • 80.
    80 References [14] http://www.osadl.org/SIL2LinuxMP.sil2-linux-project.0.html [15] https://www.gnu.org/software/libc/ [16]https://uclibc-ng.org/ [17] http://events.linuxfoundation.org/sites/events/files/slides/uclibc-still-makes-sense-brodkin- elce2017.pdf [18] https://www.musl-libc.org/ [19] https://en.wikipedia.org/wiki/Year_2038_problem [20] http://elinux.org/images/6/6e/End_of_Time_--_Embedded_Linux_Conference_2015.pdf [21] https://en.wikipedia.org/wiki/Linux_startup_process [22] http://upstart.ubuntu.com/faq.html [23] https://en.wikipedia.org/wiki/Systemd [24] https://sysdfree.wordpress.com/2019/03/09/135/ [25] https://wiki.gentoo.org/wiki/Comparison_of_init_systems [26] https://elinux.org/images/6/69/Demystifying_Systemd.pdf [27] http://proteanos.com/ [28] https://www.piboxproject.com/ [29] https://lists.debian.org/debian-devel/2016/02/msg00122.html [30] https://busybox.net/FAQ.html#libc [31] https://wiki.yoctoproject.org/wiki/Stable_branch_maintenance
  • 81.
    81 References [32] https://www.debian.org/social_contract#guidelines [33] https://bootlin.com/pub/conferences/2018/elc/petazzoni-buildroot-whats-new/petazzoni-buildroot- whats-new.pdf [34]https://events.static.linuxfound.org/sites/events/files/slides/libc-talk.pdf [35] http://events17.linuxfoundation.org/sites/events/files/slides/ELC%202016%20- %20Designing%20a%20distro%20from%20scratch%20using%20OpenEmbedded.pdf [36] https://github.com/meta-debian/meta-debian [37] https://events.linuxfoundation.org/wp-content/uploads/2017/12/ELCE2018_Debian-Yocto-State-of- the-Art_r6_Kazuhiro-Hayashi.pdf [38] https://events.linuxfoundation.org/wp-content/uploads/2017/12/Buildroot-vs-Yocto-Differences-for- Your-Daily-Job-Luca-Ceresoli-AIM-Sportline.pdf [39] https://events.static.linuxfound.org/sites/events/files/slides/belloni-petazzoni-buildroot-oe_0.pdf [40] https://www.debian.org/intro/why_debian.en.html [41] https://www.debian.org/security/index.en.html [42] http://events.linuxfoundation.org/sites/events/files/slides/Building%2C%20Deploying%20and%20Testing% 20an%20Industrial%20Linux%20Platform.pdf [43] https://github.com/icecc [44] https://www.slideshare.net/szlin/distributed-compiler-icecc
  • 82.
    82 References [45] https://validation.linaro.org/static/docs/v2/# [46] http://elinux.org/images/3/35/LAVA_Project_Update.pdf [47]https://elinux.org/Fuego [48] http://fuegotest.org/ [49] https://kernelci.org/ [50] https://reproducible-builds.org/ https://fosdem.org/2019/schedule/event/kernelci_a_new_dawn/attachments/slides/3300/export/events/attachments/ker nelci_a_new_dawn/slides/3300/gtucker_kernelci_fosdem_2019_v2_3_1024x768.pdf [51] https://github.com/distcc/distcc [52] https://wiki.jenkins-ci.org/display/JENKINS/Plugins [53] http://fbinfer.com/ [54] https://www.sonarqube.org/ [55] https://jenkins.io [56] https://jenkins.io/projects/jenkins-x/ [57] https://chromium.googlesource.com/infra/goma/server/ [58] https://chromium.googlesource.com/infra/goma/client [59] https://gcc.gnu.org/onlinedocs/gcc/Gcov.html [60] http://valgrind.org/ [61] https://perf.wiki.kernel.org/index.php/Main_Page [62] http://linux-test-project.github.io/ [63] http://www.openvas.org/
  • 83.
    83 References [64] https://about.gitlab.com/ [65] https://vuls.io/ [66]https://github.com/google/syzkaller [67] https://kselftest.wiki.kernel.org [68] http://codemonkey.org.uk/projects/trinity/ [69] https://github.com/google/oss-fuzz [70] https://fuzzing-project.org/ [71] https://mkrak.org/wp-content/uploads/2018/04/FOSS-NORTH_2018_Software_Updates.pdf [72] https://android.googlesource.com/platform/bionic [73] https://events.linuxfoundation.org/wp-content/uploads/2017/12/Strategies-for-Developing-and- Deploying-your-Embedded-Applications-and-Images-Mirza-Krak-Mender.io_.pdf [74] http://events17.linuxfoundation.org/sites/events/files/slides/ELC2017_SWUpdate.pdf [75] https://events.linuxfoundation.org/wp-content/uploads/2017/12/ELCE-2018-Update-Tools-BoF_Jan- Lubbe.pdf [76] https://events.linuxfoundation.org/wp-content/uploads/2017/12/ELCE-2018-Update-Tools-BoF_Jan- Lubbe.pdf [77] https://elinux.org/images/f/f5/Embedded_Systems_Software_Update_for_IoT.pdf [78] https://rauc.readthedocs.io/en/latest/ [79] http://elinux.org/images/6/6e/End_of_Time_--_Embedded_Linux_Conference_2015.pdf

Editor's Notes

  • #11 Why we need hardware survey
  • #29 Security, performance and safety <-> Resource limited
  • #46 Upstream first. Don’t use open source as a close source Leverage open source power
  • #54 如果能縮短 compile time, 就能增加 test time 時間是相對