Download as PDF, PPTX
Uploaded byChris Simmonds
Linux field-update-2015
Updating Embedded Linux devices in the field requires robust, atomic, and fail-safe software update mechanisms to fix bugs remotely without rendering devices unusable. A commonly used open source updater is SWUpdate, a Linux application that can safely install updates downloaded over the network or from local media using techniques like separate recovery systems and ping-ponging between OS images. It aims to provide atomic system image updates with rollback capabilities and audit logs to ensure devices remain functional after updates.
More Related Content
![XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM Systems](https://cdn.slidesharecdn.com/ss_thumbnails/xendevsummit2019-babchuk-op-tee-190812095541-thumbnail.jpg?width=640&height=640&fit=bounds)
Linux field-update-2015
- 1. Updating Embedded Linuxdevices in the field Updating Embedded Linux devices in the field 1 Copyright © 2011-2015, 2net Ltd
- 2. License These slides areavailable under a Creative Commons Attribution-ShareAlike 3.0 license. You can read the full text of the license here http://creativecommons.org/licenses/by-sa/3.0/legalcode You are free to • copy, distribute, display, and perform the work • make derivative works • make commercial use of the work Under the following conditions • Attribution: you must give the original author credit • Share Alike: if you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one (i.e. include this page exactly as it is) • For any reuse or distribution, you must make clear to others the license terms of this work Updating Embedded Linux devices in the field 2 Copyright © 2011-2015, 2net Ltd
- 3. About Chris Simmonds •Consultant and trainer • Working with embedded Linux since 1999 • Android since 2009 • Speaker at many conferences and workshops "Looking after the Inner Penguin" blog at http://2net.co.uk/ https://uk.linkedin.com/in/chrisdsimmonds/ https://google.com/+chrissimmonds Updating Embedded Linux devices in the field 3 Copyright © 2011-2015, 2net Ltd
- 4. Overview • The needfor software update of embedded devices • Update by file or system image • Update agent: bootloader or application • An Linux update agent: SWupdate • Update strategies • Failsafe Updating Embedded Linux devices in the field 4 Copyright © 2011-2015, 2net Ltd
- 5. The problem • Embeddedsoftware is non-trivial (=> has bugs!) • Devices are often connected to the Internet • Allowing hackers to exploit the bugs • Result: problems with the IoT • We need an update mechanism to fix those bugs in the field • (Not to mention the desire to deploy new features, better performance, etc.) Updating Embedded Linux devices in the field 5 Copyright © 2011-2015, 2net Ltd
- 6. Components of embeddedLinux • Bootloader • Linux Kernel (+ Device Tree and kernel modules) • Root file system (basic operating system libraries and utilities, usually read-only) • System apps (often a separate partition, but may be combined with root file system) • User data (configuration settings, logs, user-supplied data) Updating Embedded Linux devices in the field 6 Copyright © 2011-2015, 2net Ltd
- 7. Frequency of update Frequency Easeof update Bootloader Kernel Root file system System applications Updating Embedded Linux devices in the field 7 Copyright © 2011-2015, 2net Ltd
- 8. Requirements of anupdater • Robust: must not render the device unusable • Atomic: must not apply half an update • Fail-safe: have a fall-back mode if all else fails • Remote, unattended update (as an option) • Audit trail: so you can tell what updates have been applied • Preserve user data Updating Embedded Linux devices in the field 8 Copyright © 2011-2015, 2net Ltd
- 9. A simple approach •Majority of updates are to root filesystem and system apps, which are "just files" • Many projects have an updater based on tarballs and some shell scripts • They all fail, because: • Lack atomicity • Can’t update kernel in raw flash memory • (Usually) no audit trail • Not fail-safe • Remember: if it can go wrong, it will go wrong (eventually) 1 1Murphy’s law Updating Embedded Linux devices in the field 9 Copyright © 2011-2015, 2net Ltd
- 10. Atomic file updates •Update is either complete or not done at all • "Easy" to achieve for individual files, e.g. using POSIX rename (2) and careful checkpointing with fsync (2) • More difficult to keep relationships between files consistent • Example: application and dependant libraries • (Can be done if you pay attention to library versioning) • Mainstream Linux package managers (RPM and dpkg) solve a lot of the problems but are still not bulletproof Updating Embedded Linux devices in the field 10 Copyright © 2011-2015, 2net Ltd
- 11. Atomic system imageupdates • Instead of a large number of individual files, consider a system as a smaller number of system images • Kernel, root filesystem, system apps, etc • Embedded devices are typically put together like this • Need a method to update safely • Examples: • Dual copies of OS and ping-pong between them • Have separate recovery OS just for updating Updating Embedded Linux devices in the field 11 Copyright © 2011-2015, 2net Ltd
- 12. Two copies ofthe OS Bootloader User data Boot flag OS Copy 1 OS Copy 2 • Update OS copy 2 from OS copy 1; set boot flag when done • "ping-pong" between OS copies • But, requires double the flash storage Updating Embedded Linux devices in the field 12 Copyright © 2011-2015, 2net Ltd
- 13. Recovery OS Bootloader Main OS RecoveryOS User data Boot flag • To perform update, set boot flag and reboot into recovery OS • Then update main OS; clear boot flag when done • Reboot to start updated main OS Updating Embedded Linux devices in the field 13 Copyright © 2011-2015, 2net Ltd
- 14. Bootloader as updateagent • Seems an attractive option: simple environment with no OS running • But, bootloaders have limited support for peripherals, file system formats and network protocols • Incorporating the necessary software would make the bootloader more buggy • Remember that updating the bootloader itself is difficult, often a single point of failure • Conclusion: bootloader is not a good update agent! Updating Embedded Linux devices in the field 14 Copyright © 2011-2015, 2net Ltd
- 15. Linux application asupdate agent • Much better environment • Access to all peripherals • Read-write access to file systems • All network protocols • Maybe a display for user interaction • Few open source examples • One is SWupdate Updating Embedded Linux devices in the field 15 Copyright © 2011-2015, 2net Ltd
- 16. SWupdate • Written byStefano Babic • Source: https://github.com/sbabic/swupdate • Doc: https://github.com/sbabic/swupdate/blob/ master/doc/source/swupdate.rst • Yocto meta layer available: https://github.com/sbabic/meta-swupdate • Can use pre- and post- install scripts to set/reset the boot flag in the bootloader (U-Boot, for example) Updating Embedded Linux devices in the field 16 Copyright © 2011-2015, 2net Ltd
- 17. Local update Main OS image RecoveryOSSD SWupdate • Update supplied on local removable media, initiated by user or maintenance staff Updating Embedded Linux devices in the field 17 Copyright © 2011-2015, 2net Ltd
- 18. Remote update 1/2 MainOS image Recovery OS SD SWupdate Downloader Updater • Download update from network to local storage: allows update to be verified and installed at a convenient time • May be initiated by user or automatically • Requires spare local storage Updating Embedded Linux devices in the field 18 Copyright © 2011-2015, 2net Ltd
- 19. Remote update 2/2 MainOS image Recovery OS SWupdate Downloader Updater • Update directly from network stream: no local storage needed • Device is unavailable while updating, may lead to significant down time if link slow or broken • (Streaming not implemented in SWupadte yet) Updating Embedded Linux devices in the field 19 Copyright © 2011-2015, 2net Ltd
- 20. Failsafe • Robust softwareupdate is not the whole answer • Can’t protect against installing (correctly) a buggy update • Last line of defence: • Watchdog -> reboots if device is not functioning • Bootloader counts boots triggered by watchdog • Boot into recovery mode when exceed threshold and attempt to re-install Updating Embedded Linux devices in the field 20 Copyright © 2011-2015, 2net Ltd
- 21. Further reading • "BuildingMurphy-compatible embedded Linux systems" by Gilad Ben-Yossef https://www.kernel. org/doc/ols/2005/ols2005v1-pages-21-36.pdf • "Updating an embedded system :swupdate documentation" http://sbabic.github.io/swupdate/ • The rename (2) function: http://pubs.opengroup.org/onlinepubs/ 009695399/functions/rename.html • The fsync (2) function: http://pubs.opengroup.org/ onlinepubs/009695399/functions/fsync.html Updating Embedded Linux devices in the field 21 Copyright © 2011-2015, 2net Ltd
- 22. • Questions? Slides onSlide Share: http://www.slideshare.net/ chrissimmonds/linux-field-update-2015 Updating Embedded Linux devices in the field 22 Copyright © 2011-2015, 2net Ltd