Play hard learn harder: The Serious Business of Play
Remoticon - TinyML Workshop.pptx
1. TinyML
Embedded Machine Learning
Pre-flight checklist:
● Solder headers to mic breakout board
● Install STM32CubeIDE
● Install serial terminal program (e.g.
PuTTY)
● Create Gmail and Edge Impulse
accounts
Worksheet: github.com/ShawnHymel/ei-
keyword-spotting
These slides are released under the CC BY 4.0 license.
Find them here: bit.ly/remoticon-tinyml-slides
2. ● Running machine learning algorithms on embedded systems
● Why do we need this?
○ Classification, prediction, decision making with little or no Internet connection
○ Voice activation, object detect, anomaly detection, etc.
What is embedded machine learning?
3. ● Worksheet: https://github.com/ShawnHymel/ei-keyword-spotting
● Work through together (feel free to work ahead)
● Describe concepts during pauses (downloading, compiling, running, etc.)
● Not a canned demo
○ OK, it’s kinda canned
○ Edge Impulse: graphical ML training tool (vs. running Colab script for training)
○ Use auto-generated library to create embedded ML project
Syllabus
4. Data Collection
Steps
● Worksheet: https://github.com/ShawnHymel/ei-keyword-spotting
● Run Colab script through downloading/unzipping Google Speech Commands
Dataset
Google Speech Commands Dataset
● About: https://ai.googleblog.com/2017/08/launching-speech-commands-
dataset.html
● Collection of 1-second snippets of spoken words
● Use as a starting point
● Select 1 or 2 words from this set to recognize
5. Data Curation
Steps
● Make account and new project on https://www.edgeimpulse.com/
● Copy in API key
● Run script through curation
google_speech_commands
backward (1664 files)
bed (2014 files)
bird (2064 files)
cat (2031 files)
...
_background_noise_ (6 files)
_noise (1500 files)
_unknown (1500 files)
down (1500 files)
up (1500 files)
keywords_curated
dataset_curation.py
target
keywords
7. What is “Artificial Intelligence?”
John McCarthy coined the term “artificial intelligence” in 1956
“[AI] is the science and engineering of making intelligent machines, especially
intelligent computer programs...Intelligence is the computational part of the ability
to achieve goals in the world.” --John McCarthy, 2007
8. What is “Machine Learning?”
Arthur Samuel coined the term “machine learning” in 1959
"A computer program is said to learn from experience E with respect to some
class of tasks T and performance measure P, if its performance at tasks in T, as
measured by P, improves with experience E." --Tom Mitchell, 1997
9. What is “Deep Learning?”
Rina Dechter coined the term “deep learning” in 1986
“Deep learning is a class of machine learning algorithms that uses multiple layers
to progressively extract higher-level features from the raw input.” --Wikipedia
12. Upload Data
Steps
● Run final cell in Colab script to automatically upload data to Edge Impulse
20%
80%
Set aside for testing after training
Used for training
Dataset is randomly shuffled
Why? Models can “overfit”
training data, so we test with
unseen data
16. LOLWUT?
- MFCCs mimic how our ears/brain interpret sound
- Give an idea of “overall shape” of sound signature
- Ignore finer details
- Popular in automatic speech recognition (ASR)
- I wanna learn more! bit.ly/practical-cryptography-
mfccs
17. Train Neural Network
Steps
● Keep default training settings
● Start training
Reshape to 49x13 array
1D Convolution
MaxPooling 1D
1D Convolution
MaxPooling 1D
Flatten to vector
Softmax
P_noise P_unknownPdown Pup
MFCCs
Convolutional Neural Network (CNN)
Finds “features”
in the image
Classifier
LOLWUT?
- CNNs are common in deep learning
for image classification
- I wanna know how to design this!
coursera.org/learn/machine-learning
18. Evaluate on Test Set
Steps
● Examine confusion matrix for training set
● Perform inference on (unseen) test set
Check-in! Any questions or issues?
19.
20. Download and Add Model Files
Steps
● Deployment > Download auto-
generated C++ library with model
● Follow guide in Nucleo-L476 demo
project: bit.ly/ei-nucleo-l476
● Import project and model files
1. Import project from GitHub repo
2. Replace model-parameters and tflite-model
folders with ones downloaded from Edge Impulse
21.
22.
23.
24. Run It!
Steps
● Set build configuration to “Release”
● Compile (Flash: text + data, RAM: data + bss)
● Create Run Configuration
● Upload to board (“Run”)
● Connect serial terminal
○ 115,200 baud rate
○ 8-N-1
● If “buffer overrun”
○ Up baud rate to 256,000
