Software Update for Embedded
Systems
By Using Open Source Software
SZ Lin (林上智)
Embedded Linux Development Center,
Software R&D Engineer
08/18, 2019
COSCUP 2019
About Me
2
SZ LIN (林上智)
 Software Engineer at Moxa
 Debian Developer
 CIP Technical Steering Committee
 Linux Foundation Project
 Blog - https://szlin.me
Before Using Open Source Software
Something You Should Know
3
4
Copyright Patent
A patent gives its owner the right to
exclude others from making, using,
selling, and importing an invention
for a limited period of time, usually
twenty years.
src: https://en.wikipedia.org/wiki/Patent
Copyright is a legal right, that grants
the creator of an original work
exclusive rights to determine whether,
and under what conditions, this
original work may be used by others
src: https://en.wikipedia.org/wiki/Copyright
5
Copyright Patent
Identify key recommended processes
for effective open source management.
It is a shared defensive patent pool
with the mission to protect Linux.
6
Open
Source
Compliance
Policies
Processes
Training
Tools
7
8
Open Source
Policy
Software
Staff
1.1.1
1.1.2
Program
Open Source
inquiry contact info.
2.1.1
Open Source
Contribution Policy
5.1.1
5.1.3
training, internal
wiki, other
practical
communication
Contribution
Process
5.1.2
Documentation
1. Awareness of participants
2. Program scope
1.3.1 1.4.1
Conformance
Documentation
1. Program meet
specification
2. Keep at least 18
months
6.1.1, 6.2.1
Review
Process
1.5.1
2.1.2
Release
Process
4.1.*
Development
Documentation
1. R&R
2. Role competencies
3. Competence evaluation
4. Name of role
5. Review and remediation
of non-compliant cases.
1.2.*
2.2.*
Open Source
Obligation
Compliance
Artifacts
Supplied
Software
Open Source
Version
Copyright
3.1.*
3.2.1
Processes, Tooling and Support
9
OpenChain
Trust between entities in the supply chain
The OpenChain Project builds
trust in open source by making
open source license
compliance simpler and more
consistent
SPDX
Trust for software packages
Software Package Data Exchange
(SPDX) is a file format used to
document information on the
software licenses under which a
given piece of computer
software is distributed.
FOSSology
Free scanning technology
FOSSology is a open source
license compliance software
system and toolkit
Bug fixes
Security
fixes
New userspace
program
Kernel feature
enhancement
Why We Need Firmware Update?
Over 10+ years
Maintenance release
10
Smart
Grid
Smart
Rail
Smart
Oil Field
Smart
Factory
Smart
Transportation
Smart
Marine
Application
Device
Network
Management
Industrial
Routers
Industrial
Ethernet
Industrial
Wireless
LAN
Network Infrastructure
Industrial Scenarios
Serial
Connectivity
I/O
Connectivity
Edge Connectivity
Embedded
Computers
Industrial
Computing
LTS: Long Term Stable Kernel
12
Extend software uptime for stable kernel
• Only accept bug fixes and security fixes
Stable Kernel Security Vulnerabilities
13
src :https://www.cvedetails.com/vulnerability-list/vendor_id-33/product_id-47/version_id-261041/Linux-
Linux-Kernel-4.19.html
14
Glibc Security Vulnerabilities
15
src: https://www.cvedetails.com/vulnerability-
list.php?vendor_id=72&product_id=767&version_id=&page=1&hasexp=0&opdos=0&opec=0&opov=0&opcsrf=0&opgpriv=0&opsqli=0&opxss=0&opdirt=0&opme
mc=0&ophttprs=0&opbyp=0&opfileinc=0&opginf=0&cvssscoremin=0&cvssscoremax=0&year=2019&month=0&cweid=0&order=3&trc=8&sha=acd5a2dd9b1f9198
1459836dbf3180626b3ee4fa
Year 2038 Problem
• The time_t datatype is a data type in the ISO C
library and kernel structure defined for storing
system time values.
• 32-bit system can represent dates from
 Dec 13 1901
 Jan 19th 2038
• It causes integer overflowing on 03:14:08 UTC 19
January 2038
16
17
src :https://www.phoronix.com/scan.php?page=news_item&px=Glibc-Y2038-Slow-Review
18
User Applications
GNU/ Linux
GNU C library
Init system
User
Space
Kernel
Space
Hardware and peripheral devices
Architecture-dependent firmware
Bootloader
Kernel
System call interface
Toolchain
Root filesystem
The Components Might Be Updated
19
Components Size Update frequency Risk
Peripheral devices
firmware
< 10 MB Rarely Mid
Bootloader
(including SPL)
< 1 MB Rarely High
Device tree <100 kB Rarely High
Linux kernel < 10 MB Regularly High
Root file system Variant Regularly High
Application Variant Often Low
System
configuration
< 1 MB Rarely Low
Characteristics of Industrial Embedded
Linux Devices
20
1
Multiple devices
Remote management
5
Middle of nowhere
Human-less warehouse or site
Harsh environment
Unreliable network and power supply
2
Bandwidth limited
Wireless focus
3
Multiple version supported
Rollback version
4
Longevity
Long-term support at least 10 years life cycle
6
The Media for Firmware Update
21
Wire cable
Wireless
network
Portable
storage
On-site
Suggested Requirements
22
Basic Features
Fail-safe
Roll-back
Exception report
Signatures
Multiple storage type support (e.g., NOR/NAND flash, eMMC)
Build system integration
Remote access (e.g., OTA)
Additional Features
Online and offline updates
Encryption
Delta-updates
Successful update detection
Proactive updating
Update Approaches
23
Components Size Complexity Time Cost
Image/ block based Large Low Very High
File based Variant Low Variant
Package based
(e.g., deb, rpm)
Variant Low Variant
Delta based Low Very High Variant
Partition Architecture
24
Application Framework
Middleware/ Libraries
Linux kernel,Device tree
Bootloader/ Firmware
Application
Root file system
User Data/ Configuration
OS
Asymmetric/ Symmetric Firmware Updates [1]
25
Bootloader/ Firmware
Recovery OS
User Data/ Configuration
Main OS
Bootloader/ Firmware
Main OS – A (Active)
User Data/ Configuration
Main OS – B (Inactive)
Asymmetric Firmware Updates
• Fail-safe
• Downtime
Symmetric Firmware Updates
• Seamless update
• Roll-back
• Fail-safe
• Double copy of OS
Industrial/ Harsh Environments
Including smart rail, smart grid, intelligent transportation,
factory automation, oil & gas, marine, and more
26
Comparison - Features
27
Category
Fail-
Safe
Roll-
Back
Delta-
Updates
Signatures
Multiple Storage
Type Support
Build
System
Integration
SWUpdate Y Y librsync Y
•NOR NAND flashes
•UBI volumes
•SD / eMMC
Yocto/
Buildroot
RAUC Y Y casync Y
•NOR NAND flashes
•UBI volumes
•SD / eMMC
Yocto/
Buildroot
OSTree N Y
archive-
z2
Y ? Yocto
Comparison - Others
28
Method
Asymmetric/
Symmetric
Image
Updates
Type Language License
SWUpdate Both
Image-based
File-based
C99
GPLv2
With openssl
exception
RAUC Both
Image-based
File-based
C LGPLv2.1
OSTree Asymmetric File-based C/C++
MPL 2.0
/LGPLv2+
Comparison (Date: 2019/8/17)
29
Method Supporter
Github
Star
User
Debian
Archive
Contributor
SWUpdate DENX 482
Siemens
Witekio
WIP 61
RAUC Pengutronix 199 ? WIP 42
ostree GNOME 344
EndlessOS
Flatpak
Y 95
© Moxa Inc. All rights reserved.
30
Thank You
References
[1] https://mkrak.org/wp-content/uploads/2018/04/FOSS-
NORTH_2018_Software_Updates.pdf
[2] https://events.linuxfoundation.org/wp-content/uploads/2017/12/Strategies-
for-Developing-and-Deploying-your-Embedded-Applications-and-Images-
Mirza-Krak-Mender.io_.pdf
[3] System upgrade with SWUpdate
http://events17.linuxfoundation.org/sites/events/files/slides/ELC2017_SWU
pdate.pdf
[4] https://events.linuxfoundation.org/wp-content/uploads/2017/12/ELCE-2018-
Update-Tools-BoF_Jan-Lubbe.pdf
[5] https://events.linuxfoundation.org/wp-content/uploads/2017/12/ELCE-2018-
Update-Tools-BoF_Jan-Lubbe.pdf
[6]
https://elinux.org/images/f/f5/Embedded_Systems_Software_Update_for_I
oT.pdf
[7] https://rauc.readthedocs.io/en/latest/
[8] https://sbabic.github.io/swupdate/swupdate.html
31
References
[9] https://wiki.yoctoproject.org/wiki/System_Update
32

Software update for embedded systems

Editor's Notes

  • #9 綠底是文件 藍底是process
  • #12 To maintain our position as the leading provider of connectivity, we are supported by three strong connectivity solutions divisions: Edge Connectivity*, Industrial Computing*, and Networking Infrastructure*. The value we bring with our connectivity infrastructure solutions is to build reliable networks that can connect all things in automation and improve the productivity and efficiency in many industrial applications such as rail, power, oil & gas, ITS, factory, marine, and more. (*Edge Connectivity: Serial connectivity: Commit stable long-term supply to the year 2025 and beyond. Driver/OS support from the outdated Windows CE to the latest Windows 10. I/O connectivity: offers coding-free control logic) (*Industrial Computing: System-view design: fanless with wide temperature operating range, high performance with 4G LTE connectivity in a compact size) (*Network Infrastructure: Industrial Ethernet: cover edge-to-core product portfolio with advanced redundancy technologies Industrial wireless LAN: Seamless wireless roaming communication < 150ms Industrial routers: 3-in-1 design with firewall/NAT/VPN Network management: GUI design for industrial automation usage)
  • #17 Glibc has architecture optimized str/mem routines Stack smashing protection Heap corruption detection glibc is the C library from the GNU project. It's the C library used by virtually all desktop and server GNU/Linux systems. It's feature-full, portable, complies to standards, but a bit bloated. Embedded GLIBC (EGLIBC) is a variant of the GNU C Library (GLIBC) optimized for embedded systems. Its goals include reduced footprint, support for cross-compiling and cross-testing, while maintaining source and binary compatibility with GLIBC. The project is discontinued. uClibc is an alternate C library, which features a much smaller footprint. This library can be an interesting alternative if flash space and/or memory footprint is an issue. However, the space advantages gained using uClibc are becoming less important as the price of memory and flash continues to drop. It is still useful C library for embedded systems without an MMU. uClibc-ng is a spin-off of uClibc C library. The main goal of the spin-off is to do regular releases and do a lot of automatic runtime testing. musl New standard C library. musl is lightweight, fast, simple, free, and strives to be correct in the sense of standards-conformance and safety.
  • #19 Toolchain Init system include C library?
  • #21 LPWAN
  • #27 To deliver success for mission-critical applications, our solutions also come with a comprehensive coverage of industrial approvals such as class 1 Div. 2 certified solutions for oil and gas applications and EN50155-compliant railway solutions that adhere to the durability and reliability requirements for operation in various harsh environments.
  • #28 Fail-safe Roll-back Signatures Multiple storage type support (e.g., NOR/NAND flash, eMMC) Build system integration Remote Access (e.g., OTA)
  • #29 What if we use its
  • #30 What if we use its