Software update for IoT Embedded World 2017

Software update for IoT
the current state of play
Chris Simmonds
Embedded World 2017
Software update for IoT 1 Copyright © 2011-2017, 2net Ltd

License
These slides are available under a Creative Commons Attribution-ShareAlike 3.0 license. You can read the full
text of the license here
http://creativecommons.org/licenses/by-sa/3.0/legalcode
You are free to
• copy, distribute, display, and perform the work
• make derivative works
• make commercial use of the work
Under the following conditions
• Attribution: you must give the original author credit
• Share Alike: if you alter, transform, or build upon this work, you may distribute the resulting work only
under a license identical to this one (i.e. include this page exactly as it is)
• For any reuse or distribution, you must make clear to others the license terms of this work

About Chris Simmonds
• Consultant and trainer
• Author of Mastering Embedded Linux Programming
• Working with embedded Linux since 1999
• Android since 2009
• Speaker at many conferences and workshops
"Looking after the Inner Penguin" blog at http://2net.co.uk/
https://uk.linkedin.com/in/chrisdsimmonds/
https://google.com/+chrissimmonds

Overview
• Software update 101
• Software update for embedded Linux
• Over The Air (OTA) update
• Stateless rootfs

What could possibly go wrong?
• Mirai: a recent > 600 Gbps DDoS attack
• Very simple: looks for open Telnet ports
and logs on using default, well-known,
name and password
• Prime target: Dahua IP CCTV cameras
Details on PenTestPartners:
https://www.pentestpartners.com/blog/optimising-mirai-a-better-iot-ddos-botnet

Requirements for SW update
• Secure: must not become an attack vector to hijack the device
• Robust: must not render the device unusable
• Atomic: an update must be installed completely or not at all
• Fail-safe: should fall-back to last known working system if there are
errors

What to update?
Frequency
Ease of update
Bootloader
Kernel
Root ﬁle system
System applications

Update granularity
• File:
• not an option: hard to achieve atomicity over a group of file updates
• Package (e.g. RPM, deb):
• apt-get update works fine for servers but not for devices
• Entire filesystem image:
• the most common option: fairy easy to implement and verify
• Atomic differential update
• Uses clever tricks to perform atomic update of groups of files
• Container
• neat idea, so long as you have containerised applications

Device update != server update
• Server
• Secure environment, no power outage, no network outage
• If update fails, human intervention is possible
• Device:
• Intermittent power and network mean update quite likely to be
interrupted
• Failed update may be difﬁcult (and expensive) to resolve
Incremental package updates via RPM/deb are not atomic

Image update
Symmetric A/B (Android after
Nougat)
Bootloader User
data
Boot
ﬂag
OS Copy 1
OS Copy 2
Asymmetric normal/recovery
(Android before Nougat)
Bootloader
Main OS
Recovery OS
User
data
Boot
ﬂag

Examples of image updaters 1/2
• swupdate: http://sbabic.github.io/swupdate/index.html
• Symmetric and asymmetric image update client
• License: GPLv2
• Mender: https://mender.io
• Symmetric image update client
• Integrates with open source OTA update server
• License: Apache 2

Examples of image updaters 2/2
• RAUC (Robust Auto-Update Controller):
https://rauc.readthedocs.org/
• License: LGPLv2.1
• Android Things:
https://developer.android.com/things/hardware/index.html
• Android Things is cut-down Android for IoT
• License: Apache 2

Atomic differential update (OSTree)
• OSTree stores data in a "git-like" object repository
• Actual filesystem created from hard links to objects in the repository
• Checkout is atomic
• Pluses
• Updates are incremental: uses less disk space than A/B image update
• Reduced transfer time: update contains only changed files
• Minuses
• Physically there is only one filesystem: cannot recover from filesystem
corruption

Projects using OSTree
• Automotive Grade Linux (agl)
• Yocto Project meta-updater layer

Atomic differential update (swupd)
• Similar to OSTree
• swupd processes updates in bundles
• Bundle contains ﬁle updates
• Updates to bundles are applied atomically

Projects using swupd
• Clear Linux
• Ostro OS
• Yocto Project meta-swupd layer

Containerised updaters
• Consists of
• Immutable base OS
• Applications in containers
• Update client running in base OS is able to update the applications
atomically
• But, note that:
• Updates applications only
• Need another mechanism to update kernel and rootfs (e.g image A/B)

Examples of containerised updaters
• resin.io: https://resin.io
• Applications distributed in Docker containers
• Managed on device by Resin Container Engine
• Integrates with OTA update server (not open source)
• Licenses: Client: Apache2; Server: proprietary
• Snappy: https://snapcraft.io
• Containers are called snaps
• A snap contains a squashfs ﬁle system, mounted on /snap/[app]
• License: GPLv3

OTA update
Device software
build system
Firmware
images
Sign with
authentication
key
Update
server
Device
Update
agent

Complexities of OTA update
• Authentication (is this update legit?)
• Security (am I receiving what you are sending?)
• Roll-back (if update fails to boot, switch to previous version)
• Scale (roll out to large populations)
• Monitoring (keeping track of status of the population of devices)

Roll-back
• Boot limit count
• Feature of bootloader (e.g U-Boot)
• Increment count in bootloader
• Reset after successful boot
• If reboot with count > 0, bootloader knows boot failed and loads alternate
rootfs
• Hardware watchdog
• If hang in early boot, watchdog times out and resets CPU
• Bootloader checks reset reason
• If watchdog, loads alternate rootfs

Stateless filesystems
• Most of the updaters mentioned require a stateless rootfs
• You need to store run-time changes to files outside filesystems being
updated
• Methods:
• symlink or bind mount files to writeable storage
• modify existing code to keep state separate
• write new code to be stateless
• See https://www.slideshare.net/chrissimmonds/
readonly-rootfs-theory-and-practice

Conclusion
• Image update is well-understood and road tested
• swupdate, RAUC, Mender, Android Things
• Atomic update promises ﬁner granularity and smaller updates
• OSTree, swupd
• Containerised update is trendy, but only solves part of the problem
• resin.io, snappy
• Some projects integrate with OTA update servers
• Mender, resin.io, (swupdate, RAUC via Hawkbit
https://eclipse.org/hawkbit)
• Update is more robust with a stateless rootfs, but this is still a new
concept to many developers

• Questions?
Slides on Slide Share
https://www.slideshare.net/chrissimmonds/
software-update-for-iot-embedded-world-2017
This and other topics associated with building robust embedded systems
are covered in my training courses
http://www.2net.co.uk/training.html

Software update for IoT Embedded World 2017

More Related Content

What's hot

Viewers also liked

Similar to Software update for IoT Embedded World 2017

Recently uploaded

Software update for IoT Embedded World 2017