This document discusses machine learning on embedded edge devices with constraints. It notes that edge devices have limited resources like small storage, no OS, and lack floating point support. This makes pre-trained models difficult to deploy as-is on microcontrollers. The document recommends keeping models simple and optimized for the specific task by using quantization during and after training to reduce model size. Resources for learning TinyML and deploying to microcontrollers are also provided.

1. TinyML: TFLite for
Microcontrollers
Robert John
GDE ML & GCP
@robert_thas
Machine Learning on Embedded Edge Devices

2. Why Machine Learning?
Learning from example: activity detection, pet
classification
● We can’t write rules for everything
● We produce enough data for our examples
● We have powerful machines for training

3. Why Embedded Edge Devices?
Some computers do only one thing!
● Placed in hard-to-reach places
● Run on batteries
● Replaced once a year or even longer
● No keyboard or mouse
● Rely on sensors or inputs from other microcontrollers
● Control other microcontrollers or other outputs
● Smart temperature example

4. Sensors on Edge Devices
More sensors than the average computer
● Motion sensors
● Biometric sensors
● Acoustic sensors
● Environmental sensors
● Force sensors

5. Constraints on Edge Devices
Can’t eat your cake and have it
Raspberry Pi Pico HP Chromebook
(Celeron)
Raspberry Pi 4
Classification Microcontroller Personal Computer Single-Board Computer
Microprocessor RP2040 Celeron BCM2711
Processor Designer Raspberry Intel Broadcom
ISA Family Arm Cortex-M0+ IA32 (32-bit x86) Arm Cortex-A72
Clock speed 133MHz 3.5 GHz 1.5 GHz
RAM 264 KB 4GB/8GB 2GB/4GB/8GB
Storage 2MB Flash HDD/SSD HDD/SSD
Inputs GPIO Serial Keyboard/Mouse Keyboard/Mouse
Outputs GPIO Serial Monitor/Speakers Monitor/Speakers

6. Edge ML Deployment Challenges
Inference is all you need!
● Pre-trained models are hard to fit into MCUs
● No OS to act as hardware buffer
● Mobile-optimized models might use instructions not present on MCU
● Need to understand the underlying ISA
● Floating-point arithmetic usually not present
● Model is task specific, no need classify 1000 images!
● Might need to start from scratch!!
● Need to know Python and C/C++

7. Keep It Simple!
You only need inference!
● The model should only be as large as it needs to be to guaranty the required
accuracy
● Leverage quantization-aware training to avoid model degradation
● Implement post-training quantization to shrink models from floating-point
(32-bit) to integer (8-bit)
● It’s okay to lose a little bit of context during inference

8. Learn More
Important Resources
● TensorFlow: https://www.tensorflow.org/
● TensorFlow Lite: https://www.tensorflow.org/lite
● TFLu: https://www.tensorflow.org/lite/microcontrollers
● TinyML Foundation: https://www.tinyml.org/
● TinyML Course on EdX:
https://www.edx.org/professional-certificate/harvardx-tiny-machine-learning
● Embedded Systems Essentials:
https://www.edx.org/professional-certificate/armeducationx-embedded-systems-essentials
●

9. Thank You!
Robert John
GDE ML & GCP
@robert_thas