On last March 21, 2020, we participated in worldwide Arduino Day 2020 and organized the online event for Bandung, Indonesia. This is the deck I delivered for my talk and demo.

1. Intro to TinyML
Technical, implementation with Arduino
Arduino Day 2020 Online Event
Bandung, Indonesia
Mar 21, 2020
Andri Yadi
CEO, DycodeX
Microsoft MVP, Azure

3. Andri Yadi
Co-founder & CEO of DycodeX
Vice Chairman, Indonesia IoT Association (ASIOTI)
Microsoft MVP, Azure
a (at) dycodex.com | andriyadi.com | github.com/andriyadi
Physicist, Developer, Maker, Community Guy, Entrepreneur
About Me
MicrosoJ Most Valuable Professional (MVP) for 12 years
Code for food & passion for 20 years
Break & make electronic stuOs for 22 years
Trying to change the world through entrepreneurship, 15 years now

4. I’VE BEEN PROMOTING
AIOT & TINYML
20+ TALKS LAST YEAR

5. Communication
Networks
IoT High Level Architecture
Gateways /
Base Station
Things
(lots of them)
Internet
Apps
Cloud User
Internet
Ingestion
Infrastructure
Logics
API

6. GNSS
176
Gyroscope
130
Accelerometer
21
Bluetooth
100
Radio
800
Display
400
in mW
Clearly Radio consumes
almost 50% of power.
65% if no display!
Typical IoT Device’s Energy Consumption

7. GNSS
176
Gyroscope
130
Accelerometer
21
Bluetooth
100
Radio
800
Display
400
in mW
Clearly Radio consumes
almost 50% of power.
65% if no display!
Typical IoT Device’s Energy Consumption
LPWA for IoT wireless
connectivity is the option
LPWA = Low Power Wide Area Network

8. You oKen see…
about LPWA connectivity

9. You oKen see…
10 years baOery life is
GIMMICK!
about LPWA connectivity

10. You oKen see…
Achievable by 2-3 of these combinations:
Using big capacity ba_ery (e.g. 19Ah)
Transmi_ing data few times a day
Payload size 10s-100s bytes ➙ shob tx
10 years baOery life is
GIMMICK!
Notes:
NB-IoT (or other LPWA connectivities) does consume
much lower power than WiFi or 4G/LTE, due to its low
data rate. Only tens to hundreds mA per-tx
about LPWA connectivity

11. So, if you have…
Sensors:
GPS / GNSS
Accelerometer, Gyroscope
Body Temperature
Ambient Temperature & Humidity
Barometric Pressure
Ambient Light
MEMS Microphone
Device Removal Detector
SMARTernak electronics board

12. So, if you have…
Sensors:
GPS / GNSS
Accelerometer, Gyroscope
Body Temperature
Ambient Temperature & Humidity
Barometric Pressure
Ambient Light
MEMS Microphone
Device Removal Detector
You may only transmit:
Current lat, long, speed, direction
Current values or average motion
Current, peak, or lowest body temperature
Current, peak, or avg. temperature & humidity
Current, peak, or avg. Barometric Pressure
Current, peak, or lowest ambient Light
Current, peak, or lowest sound level in db
Detached or not
SMARTernak electronics board
99% of sensor data has to be discarded due to
connectivity’s bandwidth or power constraints

13. Motor failure detection
Dashboard
For that, usually we collect raw data from vibration sensor and transmit to the Cloud.
In the cloud, raw data are processed to classify whether motor is working normal or fault.
On the other hand, consider a use case…
MicrocontrollerMotor

14. Motor failure detection
Vibration
raw data
5D 1B 16
56 01 41 …
(1K Bytes)
Dashboard
For that, usually we collect raw data from vibration sensor and transmit to the Cloud.
In the cloud, raw data are processed to classify whether motor is working normal or fault.
On the other hand, consider a use case…
MicrocontrollerMotor

15. Motor failure detection
Vibration
raw data
5D 1B 16
56 01 41 …
(1K Bytes)
Dashboard
For that, usually we collect raw data from vibration sensor and transmit to the Cloud.
In the cloud, raw data are processed to classify whether motor is working normal or fault.
On the other hand, consider a use case…
MicrocontrollerMotor
Vibration
raw data
5D 1B 16
56 01 41 …
(1K Bytes)

16. Motor failure detection
Vibration
raw data
5D 1B 16
56 01 41 …
(1K Bytes)
Fault
detected
Dashboard
For that, usually we collect raw data from vibration sensor and transmit to the Cloud.
In the cloud, raw data are processed to classify whether motor is working normal or fault.
On the other hand, consider a use case…
MicrocontrollerMotor
Vibration
raw data
5D 1B 16
56 01 41 …
(1K Bytes)

17. Motor failure detection
Vibration
raw data
5D 1B 16
56 01 41 …
(1K Bytes)
Fault
detected
Dashboard
If we try to send 1KB of raw data over LPWA (e.g. LoRa),
it may take “forever”, and consume more power
For that, usually we collect raw data from vibration sensor and transmit to the Cloud.
In the cloud, raw data are processed to classify whether motor is working normal or fault.
On the other hand, consider a use case…
MicrocontrollerMotor
Vibration
raw data
5D 1B 16
56 01 41 …
(1K Bytes)
…it will beat the purpose of low power connectivity
LPWA

18. Motor failure detection
Dashboard
Due to bandwidth and power constraints, we only send sampled and calculated data
So, what we may do, instead
MicrocontrollerMotor
Peak
Peak value
FFT
Vibration
raw data
5D 1B 16
56 01 41 …
(1K Bytes)
Sampled/Calculated
LPWA

19. Motor failure detection
Dashboard
Due to bandwidth and power constraints, we only send sampled and calculated data
So, what we may do, instead
MicrocontrollerMotor
But we may miss lot of interesting events!
Peak
Peak value
FFT
Fault
detected
Vibration
raw data
5D 1B 16
56 01 41 …
(1K Bytes)
Sampled/Calculated
LPWA

20. Motor failure detection
Dashboard
There’s on-device “intelligence” to do complete analysis, right on the device.
Then, only transmit the conclusion (just few bytes of data) to the cloud.
Somehow…
MicrocontrollerMotor
On-device
processing
Vibration
raw data
5D 1B 16
56 01 41 …
(1K Bytes)
LPWA

21. Motor failure detection
Dashboard
There’s on-device “intelligence” to do complete analysis, right on the device.
Then, only transmit the conclusion (just few bytes of data) to the cloud.
Somehow…
MicrocontrollerMotor
Processed data
0A 0B 01 (➙ Normal)
0A 0B 02 (➙ Fault)
On-device
processing
Fault
detected
Vibration
raw data
5D 1B 16
56 01 41 …
(1K Bytes)
LPWA

22. Motor failure detection
Dashboard
There’s on-device “intelligence” to do complete analysis, right on the device.
Then, only transmit the conclusion (just few bytes of data) to the cloud.
Somehow…
MicrocontrollerMotor
That may save power, bandwidth,
and deliver more complete analysis
Processed data
0A 0B 01 (➙ Normal)
0A 0B 02 (➙ Fault)
On-device
processing
Fault
detected
Vibration
raw data
5D 1B 16
56 01 41 …
(1K Bytes)
LPWA

23. Motor failure detection
Dashboard
There’s on-device “intelligence” to do complete analysis, right on the device.
Then, only transmit the conclusion (just few bytes of data) to the cloud.
Back to…
MicrocontrollerMotor
Processed data
0A 0B 01 (➙ Normal)
0A 0B 02 (➙ Fault)
On-device
processing
What is this? Machine Learning?
Fault
detected
That may save power, bandwidth,
and deliver more complete analysis
Vibration
raw data
5D 1B 16
56 01 41 …
(1K Bytes)
LPWA

24. Machine Learning is oKen associated with…

25. Machine Learning
ClassiYcation
What’s happening right now?
Anomaly Detection
Is it “normal”?
Forecasting
What will happen in the future?
Some ML problems
Credit: h_ps://www.slideshare.net/janjongboom/adding-intelligence-to-lorawan-devices-the-things-conference-2020

26. Machine Learning
ClassiYcation
What’s happening right now?
Anomaly Detection
Is it “normal”?
Forecasting
What will happen in the future?
Some ML problems
For those problems,
ML Inference is possible on
ultra-low power, low cost,
tiny Microcontroller

27. High pelormance:
Less latency as no raw data need to go to cloud/backend
Still works omine:
Machine Learning inference happens right on-device.
No internet connectivity needed to get inference result from cloud.
Encient power and bandwidth consumption:
Only transmit necessary inference result, which are small in data size.
Possible to use Low Power Wide Area (LPWA) connectivity, e.g: LoRa or
NB-IoT
Using optimised Edge hardware, even possible to be powered by ba_ery
AI at the Edge
Why the hassle?

29. So, how can we achieve
TinyML?

30. ML-accelerated
processor
a class of microprocessor
designed as hardware
acceleration for AI
applications, e.g. for
neural networks, machine
vision and machine
learning
Example:
GPU
FPGA
ASIC
DSP
Development
Board /Accessories
a standalone computing
system with ML-
accelerated processor
(main or co-) as a main or
additional development
unit
Example:
Coral DevBoard or USB
Accelerator
Intel Neural Compute Stick
Sipeed Maix
Arduino Nano 33 BLE Sense
ESPectro32
SoKware Tools
ML compilers
ML Model design tool
ML Model conveber
SDK & libraries
OS
Example:
TensorFlow & TensorFlow
Lite (TFLite)
TFLite for Microcontroller
OpenVino
Kendryte’s nncase
EdgeImpulse
Suppos
Documentation
Model Zoo
Examples
Datasheets
Hardware SoKware
What do we need?

31. Optimize &
Conve
(compress, remove,
replace, conve)
Train ML Model
(on your awesome
machine or Cloud)
.h5
.pb
.cabemodel
ONNX
ML Model
Intermediate
Representation
(IR) File
Transfer IR qle
to Edge device
Inference
(Edge-optimized
Inference library)
Inference
Result
AI at the Edge General Pipeline

32. Optimize
(TOCO)
Train ML Model
(on your awesome
machine or Cloud)
.h5
.pb
TinyML Pipeline
TFLite
Flat
Buber
Conve
(xxd)
Integrate
into MCU
Firmware
C Byte
Array

33. Optimize
(TOCO)
Train ML Model
(on your awesome
machine or Cloud)
.h5
.pb
TinyML Pipeline
TFLite
Flat
Buber
Conve
(xxd)
Integrate
into MCU
Firmware
C Byte
Array
Custom
Logics
ML Model
Microcontroller (MCU) Firmware
TensorFlow Lite
for Microcontroller

34. TinyML Pipeline
e.g. leveraging Azure Machine Learning
Azure Machine Learning Pipeline
TensorFlow model Yle
Compress
(toco)TFLite
Flat
Buber
Conve
(xxd)
C Byte
ArrayIntegrate
into MCU
Firmware
TinyML Pipeline

35. Typical TinyML Model Training
Labelled
Raw Data
Label1
Label2
Label3
Signal
Processing
(Low/High Pass Filter,
FFT)
Neural
Network
Output
Features
Credit: EdgeImpulse.com. Check it out!
Processing Blocks
To extract features
Learning Blocks
To classify new data.
NN size can be signiqcantly small
due to the extracted features from
previous blocks
AKer Filter
Frequency Domain

36. using Arduino framework, TFLite, PlahormIO,
ESPectro32 board
Gesture Recognition
DEMO

37. NB-IoT + GPS/GNSS Kit LoRa/LoRaWAN +
GNSS Kit
Sensors & Actuators Kit
(20+ Peripherals)
3G + GNSS + Sensors
ESPectro32 v2
Optimized for learning STEM,
coding & electronics, also for fast
prototyping.
An AIoT Development Board:
Low-power, dual-core 240 MHz
Microcontroller
Built-in WiFi & Bluetooth connectivity
7x7 LED matrix for user intelace
Sensors:
Accelerometer & Gyro using LSM6DS3,
Digital temperature sensor, phototransistor
microSD card slot
Extensible via:
40 RPi-compatible pins, Grove connectors,
Micro:bit connector
Extensible with Backpacks:
NB-IoT/Cellular + GPS/GNSS, LoRa, Sensor
Kits, Motor Driver, and more
ESPectro32
Backpacks
hOps://shop.makestro.com/product/espectro32-v2/

38. Demo

39. git clone hOps://github.com/andriyadi/MagicWand-TFLite-ESP32

40. Train Model
If necessary. Then compress and conveb the model…. Or use pre-trained model, instead
Reference: h_ps://github.com/tensollow/tensollow/tree/master/tensollow/lite/micro/examples/magic_wand/train

41. Enjoy!
Open Serial Monitor to see the inference result
Recognised gestured also displayed on LED matrix*
* On latest source code, display is already rotated :)

42. Other Demo
Similar demo, but using Arduino Nano 33 BLE Sense board
hOps://www.youtube.com/watch?v=Lfv3WJnYhX0 hOps://github.com/andriyadi/MagicWand-TFLite-Arduino

43. TinyML:
TensorFlow Lite for Microcontrollers: h_ps://www.tensollow.org/lite/
microcontrollers
Great book: TinyML, by Daniel Situnayake, Pete Warden
Azure:
Azure Machine Learning: h_ps://azure.microsoJ.com/en-in/services/
machine-learning-service/
DycodeX:
ESPectro32 dev board: h_ps://shop.makestro.com/product/espectro32-
v2/
SMARTernak: h_ps://smabernak.com
Other IoT + AI products & solutions: h_ps://dycodex.com
Contact me:
andri (at) dycodex.com
h_p://github.com/andriyadi | hOps://slideshare.net/andri_yadi/
Call to Action

44. Want to put “AI” in “BrAIns”?
… a.k.a. adopting AI + IoT -> AIoT?

45. AI + IoT enabler
Keep in touch
hi (at) dycodex.com | https://dycodex.com
Bandung, Indonesia