This session, delivered at DevFest Nairobi 2022, focused on deploying ML models on edge devices. I used a Raspberry Pi as an example of the needs and demonstrated how TFLite could be utilized to deploy a model from TFHub onto a Raspberry Pi. I also showed how such a model could be fine-tuned on a new dataset, and how the code could be implemented to run continuously while reading a media stream from a sensor such as a camera or a microphone.

1. Nairobi
Robert Thas John
GDE, ML & GCP
@robert_thas
Considerations for
Deploying ML Models on
Edge Devices

2. Photo by Daniel Korpai on Unsplash
Limited:
● storage
● processing
● Memory
● power

3. This is an example
of section title
Photo by Jainath Ponnala on Unsplash

4. This is an example
of section title
Photo by Jainath Ponnala on Unsplash

5. ● Low Latency
● Offline Availability
● Privacy Preserving

6. Models have to be
● Customized
● Lightweight & efficient
● Target to hardware
● Quantized

7. MediaPipe

8. TFLite Task Library
●Android
●iOS
●Python
●C++

9. ●Vision APIs
●NL APIs
●Audio APIs

11. pip install tflite-support

12. # Imports
from tflite_support.task import vision
from tflite_support.task import core
# Create ImageClassifier from options.
classifier =
vision.ImageClassifier.create_from_file('model.tflite')
# Run inference on Coral Edge TPU.
image = vision.TensorImage.create_from_file('image.jpg')
classification_result = classifier.classify(image)

13. Results:
Rank #0:
index : 671
score : 0.91406
class name : /m/01bwb9
display name: Passer domesticus
Rank #1:
index : 670
score : 0.00391
class name : /m/01bwbt
display name: Passer montanus
Rank #2:
index : 495
score : 0.00391
class name : /m/0bwm6m
display name: Passer italiae

14. TensorFlow Lite
Model Maker

15. ● Image Classification
● Object Detection
● Text Classification
● BERT QnA
● Audio Classification
● Recommendation
● Text/Image Search from DB

16. sudo apt -y install libportaudio2
pip install -q tflite-model-maker

17. import os
import numpy as np
import tensorflow as tf
assert tf.__version__.startswith('2')
from tflite_model_maker import model_spec
from tflite_model_maker import image_classifier
from tflite_model_maker.config import ExportFormat
from tflite_model_maker.config import QuantizationConfig
from tflite_model_maker.image_classifier import DataLoader
import matplotlib.pyplot as plt

18. data = DataLoader.from_folder(image_path)
train_data, test_data = data.split(0.9)
model = image_classifier.create(train_data)
loss, accuracy = model.evaluate(test_data)
model.export(export_dir='.')

19. TensorFlow Lite
●RPi 2,3,4
●RPi Zero 2

20. pip install tflite-support
pip install opencv-python
pip install libportaudio2

22. # Imports
from tflite_support.task import audio
from tflite_support.task import core
from tflite_support.task import processor
classifier =
audio.AudioClassifier.create_from_file('yamnet.tflite')
# Run inference
audio_file = audio.TensorAudio.create_from_wav_file('audio.wav',
classifier.required_input_buffer_size)
audio_result = classifier.classify(audio_file)

23. Stream audio from microphone

24. # Imports
from tflite_support.task import audio
from tflite_support.task import core
from tflite_support.task import processor
#stream from mic
audio.AudioClassifier.create_from_file('yamnet.tflite')
recorder = classifier.create_audio_record()
tensor =classifier.create_tensor_audio()
recorder.start_recording()

25. while True:
time.sleep(o.5)
tensor.load_from_audio_recorder(recorder)
result = classifier.classify(tensor)