Edge computing and machine learning have several advantages: lower latency for localized interactions, reduced network usage, and improved privacy. However, edge ML also faces challenges like limited computing power, memory, and battery life. TensorFlow Lite helps address these challenges by enabling simple and efficient ML models on edge devices. It provides tools to convert Keras models to TFLite format, optimize models for size and performance, and leverage device-specific accelerators. The document discusses techniques like quantization, pruning, and selecting only required operations to build high-quality edge ML models with TensorFlow Lite.

1. Edge and AI
By : Vishwas Narayan

3. https://arxiv.org/pdf/1801.04381.pdf

4. Edge ml is a explosion
● Lower latency & close knit interactions
● Network connectivity
● Privacy preserving

5. ML on Edge
● Latency
● Bandwidth
● Accuracy
● Complexity

6. Challenges
● Uses little compute power
● Works on limited memory platforms
● consume less battery

7. Simple Architecture(Simple to see)

9. Code is simple
# Build and save Keras model.
model = build_your_model()
tf.keras.experimental.export_saved_model(model, saved_model_dir)
# Convert Keras model to TensorFlow Lite model.
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
tflite_model = converter.convert()

10. Purpose
● APIs for simplifying pre- and post-processing (launched)
● Autogenerates pre- and post-processing (in progress)

11. With tflite support

14. Langage Bind
● New language bindings (Swift, Obj-C, C# and C) for iOS, Android and Unity
● Community language bindings (Rust, Go, Flutter/Dart)

15. How to improve model performance?
● Use quantization
● Use pruning
● Leverage hardware accelerator
● Use mobile optimized model architecture
● Per-op profiling

16. Utilizing quantization for CPU, DSP & NPU
optimizations
Reduce precision of static parameters (e.g. weights) and dynamic values (e.g.
activations)

17. Accelerators

18. GPU for the purpose
● 2–7x faster than the floating point CPU implementation
● Uses OpenGL & OpenCL on Android and Metal on iOS
● Accepts float models (float16 or float32)

19. DSP Delegation
● Use Hexagon delegate on Android O & below
● Use NN API on Android P & beyond
● Accepts integer models (uint8)
● Launching soon!

20. Android Deployment
● Enables graph acceleration on DSP, GPU and NPU
● Supports 30+ ops in Android P, 90+ ops in Android Q
● Accepts float (float16, float32) and integer models (uint8)

21. Benchmarks and expand operator and reduce size
● Utilize TensorFlow ops if op is not natively supported
● Only include required ops to reduce the runtime’s size

22. CPU and tf operator
● Enables hundreds more ops from TensorFlow on CPU
● Caveat: Binary size increase (~6MB compressed)

24. Reduce overall runtime size
● Selectively include only the ops required by the model
● Pares down the size of the binary

25. tflite

26. tflite converter

27. tflite model maker

28. Coral Tensorflow

30. Kernels
https://www.kaggle.com/gaur128/creating-tflite-model
https://www.kaggle.com/bhrt97/lr-tflite
https://www.kaggle.com/general/62267
https://www.tensorflow.org/lite/tutorials/model_maker_image_classification

31. links
https://www.tensorflow.org/lite/tutorials/model_maker_image_classification
https://www.tensorflow.org/lite/convert/
https://pub.dev/packages/tflite

32. Video Hyperlinks
https://www.youtube.com/watch?v=BInZBpX7bEo
https://www.youtube.com/watch?v=27Zx-4GOQA8
https://www.youtube.com/watch?v=4xuxOKveu9U
https://www.youtube.com/watch?v=iyUkOAI69ts
https://www.youtube.com/watch?v=2tnWHCxP8Bk

33. Tiny ml

34. Communities
https://aihub.cloud.google.com/u/0/p/products%2F6ca92c9c-6932-45f8-8f08-
15a1a6dfafc7
https://tfhub.dev/s?deployment-format=lite
TFUG bangalore:https://t.me/tfblr
tinyML:https://www.meetup.com/tinyML-Enabling-low-Power-ML-at-the-edge-
Bangalore-India/

41. Parallela

43. If not edge have a browser then you can do this

44. TensorFlow Federated
https://www.youtube.com/watch?v=JBNas6Yd30A

46. Federated Learning Paper
https://arxiv.org/abs/1912.04977
https://arxiv.org/pdf/1902.01046.pdf

48. Thank You
Any question queries please please free to ask or connect.

49. If you want to further learn Please do contact me
from any social media platform that I have listed
here
Email : vishwasnarayan2345@gmail.com
Linkedin Vishwas N : shorturl.at/aquyC
Twitter : shorturl.at/gios7