Using TensorFlow.JS, TensorFlow Lite and TensorFlow Hub, train and run inference on ML and DL models for the edge devices.

1. Machine Learning and Deep Learning
on Edge Devices usingTensorFlow
~ Aditya Bhattacharya
Lead AI/ML Engineer,
West Pharmaceuticals

2. About Me
Currently, I am leading the Data Science team at West Pharmaceutical
Services and previously worked in Microsoft for Azure Cloud Platform
development use cases and well seasoned in domains such as Machine
Learning, Deep Learning, Internet of Things (IoT), Robotics and Cloud
Computing. In the Data Science domain, I am well experienced with
ComputerVision, Time-Series Forecasting, NLP, Speech analysis.
Apart from my day job, I am an AI Researcher at an NGO called MUST
Research, and I am one of the faculty members at MUST Research
Academy. https://academy.must.co.in/
https://aditya-bhattacharya.net/
https://www.linkedin.com/in/aditya-bhattacharya-b59155b6/

3. Key topics for discussion
1. Training and inferencing of ML and DL models on client browsers and edge devices using
TensorFlow.js and Node.js
2. UsingTensorFlow lite for training and deployment ML and DL models on light-weight edge devices
like Raspberry Pi.
3. TensorHub for registering and versioning trained models which can be deployed on edge devices.
4. Case-studies where real-time model training and inferencing is required for light-weight edge
devices.

4. Introduction
• By 2025, it is estimated that there will be 21 Billion IoT devices in
the entire world.
• With the advancement of AI and IoT, there will be a huge
demand of ML and DL applications at Edge or Node devices.
• So the question is, how exactly can we perform real time
machine learning on the edge?
• Or, is it necessary to perform ML/DL at edge?

5. ML/DL at Edge for IoT devices
• IoT devices are smart devices that needs to take a real-time action based on
certain conditions. Example – Based on anomaly detection in manufacturing
plants, certain equipments may need dynamic adjustment of configurations
to mitigate the problem. So, these equipments would require capabilities to
take instant actions.

6. Why “at the Edge” ?
• Latency: there's no round-trip to a server
• Privacy: no data needs to leave the device
• Connectivity: an Internet connection isn’t available all the time
• Power consumption: network connections are power hungry

7. Typical Enterprise ML and DL Flow
User Interface Layer
Middleware
API Layer
Analytics LayerData Layer
Monitoring
Layer
Feedback
Layer

8. TensorFlow.js

9. TensorFlow.js
• TensorFlow.js is a library for machine learning in JavaScript.
• Along with node.js, another JavaScript based cross-platform environment,
TensorFlow.js can actually implement ML workflow outside the browser at
edge devices.
• Node.js can run a local server, and implement a client-server model, in
which TensorFlow.js can be used train or re-train a model in the local edge
server, and again TensorFlow.js can be used to run inference on the client
node application
URL - https://www.tensorflow.org/js

10. TensorFlow.js Layers

11. Let’s see a fun demo withTensorFlow.js

12. Training and Prediction in Node.js
https://www.tensorflow.org/js/tutorials/training/nodejs_training

13. ConvertTensorFlow Models toTensorFlow.Js Models

14. Typical use cases forTensorFlow.js for IoT Systems
• Real-time sensor anomaly detection at the edge
• Make sensors think ahead of its time by forecasting sensor values and send
necessary alerts.
• Manufacturing ShopFloors Auto-Failure Detection
• Track edge device performance using ML – example: Clinic.js
https://clinicjs.org/doctor/
• Node-red for drag and drop IoT workflows - https://nodered.org/

15. TensorFlow Lite

16. TensorFlow Lite
• TensorFlow Lite is an open source deep learning framework for on-device
inference.
• TensorFlow Lite works with a huge range of devices, from tiny microcontrollers to
powerful mobile phones.
• It has two main components:
 Converter – Can convert usual TF models to TF Lite format which is more optimized and
lightweight.
 Interpreter – Can run specially optimized models on embedded devices like Raspberry Pi.
• With TF Lite, we can do offline training for ML/DL models and online deployment.
URL - https://www.tensorflow.org/lite

17. https://www.tensorflow.org/lite

18. Link - https://youtu.be/mduuuLIOVQI?t=2

19. Supported Embedded Devices
• Raspberry Pi
• GoogleCoral
• Arduino Nano 33 BLE Sense
• SparkFun Edge
• STM32F746 Discovery kit
• Adafruit EdgeBadge
• AdafruitTensorFlow Lite for Microcontrollers Kit
• Adafruit Circuit Playground Bluefruit
• Espressif ESP32-DevKitC
• Espressif ESP-EYE

20. Typical use cases forTensorFlow Lite for IoT Systems
• Real-time security systems
• Real-time traffic monitoring systems
• Indoor navigation system

21. TensorFlow Hub

22. TensorFlow Hub
• TensorFlow Hub a library for reusable machine learning modules.
• Another way to make ML – DL scalable by making ML – DL models
portable and can be published in TensorFlow Hub
• Can be used to perform easy transfer learning, and thereby making it
easier to train models on a smaller dataset.
• The TensorFlow Hub library lets you download and reuse the latest
trained models with a minimal amount of code.
URL -https://tfhub.dev/

23. WhyTensorFlow Hub?
• A way to easily discover models
• Access state of the art models
• Ease in using and integrating with
model APIs
• Completely portable models with no
code dependencies
• Supports wide array of platforms
• Easy inference - perform inference with
your loaded model, especially
important for embedded devices if
model training is done offline.
URL -https://tfhub.dev/

24. Summary
Online ML/DL Flow at Edge Devices
• TensorFlow.JS with Node.JS
OfflineTraining but Online Inference at Edge Devices
• TensorFlow Lite Models
• Pre-trained models fromTensorFlow Hub.

25. - ADITYA BHATTACHARYA
Questions?
-Want to connect over LinkedIn ?
- Or email me at: aditya.bhattacharya2016@gmail.com
- Follow me at: https://aditya-bhattacharya.net/
That’s all for today!