Internet of Things (IoT) is growing rapidly in decades, various applications came out from academia and industry. IoT is an amazing future to the Internet, but there remain some challenges to IoT for human have never dealt with so many devices and so much amount of data. Machine Learning (ML) is the technique that allows computers to learn from data without being explicitly programmed. Generally, the aim is to make predictions after learning and the process operates by building a model from the given (training) data and then makes predictions based on that model. Machine learning is closely related to artificial intelligence, pattern recognition and computational statistics and has strong relationship with mathematical optimization. In this talk, we focus on ML applications to IoT. Specially, we focus on the existing ML techniques that are suitable for IoT. We also consider the issues and challenges for solving the IoT problems using ML techniques.
1. Introduction
ML and IoT
Our Work
UbiMedia Lab
Machine Learning Applications to IoT
Dr. Hari Prabhat Gupta and Dr. Tanima Dutta
Assistant Professor(s),
Department of Computer Science and Engineering
Indian Institute of Technology (BHU) Varanasi, India
Email: {hariprabhat.cse,tanima.cse}@iitbhu.ac.in
1/55
2. Introduction
ML and IoT
Our Work
UbiMedia Lab
Outline
1 Introduction
Internet of Things (IoT)
Machine Learning (ML)
2 ML and IoT
ML and IoT: An Example Scenario
Major Components of IoT
3 Our Work
Smart Connectivity
Smart Education
Smart Life
Smart Environment
4 UbiMedia Lab
About UbiMedia Lab
Ongoing Projects
2/55
3. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
Types of Internet of Things (IoT)
Security and Privacy
Smart (Cities and Homes)
Devices (Wearable and Connecting)
BlockChains
Industrial automation
3/55
4. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
Security and Privacy
Highly security-centric “life-and-death” applications
Intermediate security uses that include: smart home
Lower security casual uses such as: games
All images source: https://www.google.com/
4/55
5. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
Smart
Home Area Network (Zigbee, BLE, WiFi): Smart Homes
Wide Area Network: Smart Cities
Field Area Networks: Smart End-to-End
5/55
7. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
Govt. of India
7/55
8. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
Devices:Wearable and Connecting
8/55
9. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
BlockChains
9/55
10. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
Machine Learning
Tom M. Mitchell:
A computer program is said to learn from Experience E with
respect to some Task T and some Performance measure P, if
its performance on T, as measured by P, improves with
experience E.
Learning means Improving with Experience at some Task
Example: Smart Homes
T: Estimate the desired temperature
E: Learning from temperature dataset
P: Accuracy of the desired temperature
10/55
11. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
Traditional vs Machine Learning algorithms
(Walk/Run)
Accelerometer Data
(Rules Learned)
Walk vs Run
Machine Learning
for many different people)
(Accelerometer data collected
Algorithm
(Walk/Run)
Accelerometer Data
(Programmed)
(Hard−coded Instruction)
Computer Program
Walk/Run
Traditional Programming:
Machine Learning:
11/55
12. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
Traditional vs Machine Learning algorithms
12/55
13. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
ML algorithms
13/55
14. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
ML Algorithms: Regression
How to estimate the price of a fruit if freshness (in days) is given?
14/55
15. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
ML Algorithms: Regression
Freshness of fruits (in days, 100−0)
Price(inINR0−100)
15/55
16. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
ML Algorithms: Regression
How to estimate the price of a fruit if freshness (in days) is
given?
Input
Training Data (labeled instances (x1, y1), (x2, y2), . . . , (xn, yn))
e.g., freshness of fruits and prices
Objective
Develop a relation (or rule f : x → y) to predict y for given x
e.g., a new fruit xnew with given fresh or not
Output
Discrete-valued y
e.g., Fresh or not ynew of the fruit xnew
16/55
17. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
ML Algorithms: Regression
Price(inINR0−100)
Freshness of fruits (in days, 100−0)
17/55
18. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
ML Algorithms: Classification
How to identify where a given fruit is fresh (if freshness is not
more than 50 days) or not?
18/55
19. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
ML Algorithms: Classification
19/55
20. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
ML Algorithms: Classification
How to identify where a given fruit is fresh (if freshness is
not more than 50 days) or not?
Input
Training Data (labeled instances (x1, y1), (x2, y2), . . . , (xn, yn))
e.g., freshness of fruits and prices
Objective
Develop a relation (or rule f : x → y) to predict y for given x
e.g., a new fruit xnew with given freshness (in days)
Output
Real-valued y
e.g., Price ynew of the fruit xnew
20/55
21. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
ML Algorithms: Classification
21/55
22. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
ML Algorithms: Clustering
How to put the fruits in two buckets?
22/55
23. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
Input
Training Data (Unlabeled instances x1, x2, . . . , xn)
Objective
Learn more about the data distribution
Output
Discover the inherent groupings in data.
23/55
24. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
Supervised & Unsupervised ML Algorithms
1 How to estimate the price of a fruit if freshness (in days)
is given?
2 How to identify where a given fruit is fresh (if freshness
is not more than 50 days) or not?
3 How to put the fruits in two buckets?
Supervised ML algorithms.
1 Regression: Estimate the real-value.
2 Classification: Estimate the Discrete-value.
Unsupervised ML algorithms
1 Clustering: Grouping the given items.
24/55
25. Introduction
ML and IoT
Our Work
UbiMedia Lab
Internet of Things (IoT)
Machine Learning (ML)
Example of ML Algorithms
1 Supervised: Decision trees, Nearest-neighbors methods,
regression and classification
2 Unsupervised: Clustering, Dimensionality reduction,
Association
In this workshop, we focus on:
Linear Regression
Logical Regression
K-NN
Decision trees
Support Vector Machine
K-mean Clustering
25/55
26. Introduction
ML and IoT
Our Work
UbiMedia Lab
ML and IoT: An Example Scenario
Major Components of IoT
Internet
26/55
27. Introduction
ML and IoT
Our Work
UbiMedia Lab
ML and IoT: An Example Scenario
Major Components of IoT
Major Components of IoT
Sensor technology
Communication technology
Machine Learning
Human-machine interface (UI/UX)
27/55
28. Introduction
ML and IoT
Our Work
UbiMedia Lab
ML and IoT: An Example Scenario
Major Components of IoT
Sensor technology: Smartphone, Arduino,
Communication technology: BLE, WIFI
Machine Learning: 10 ML algorithms
Human-machine interface (UI/UX): Android
28/55
29. Introduction
ML and IoT
Our Work
UbiMedia Lab
ML and IoT: An Example Scenario
Major Components of IoT
Traditional vs Machine Learning algorithms
29/55
30. Introduction
ML and IoT
Our Work
UbiMedia Lab
ML and IoT: An Example Scenario
Major Components of IoT
Abstract View of ML
30/55
31. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
ML → Internet and ML → Things
Machine Learning to Internet
Machine Learning to Things
31/55
32. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
Introduction
Smart Connectivity in IoT
Smart Education, Home, and Environment
Deployment
SDN and BlockChains
32/55
33. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
Connectivity in IoT for smart home
BLE Mesh Network
Samsung R&D Bangalore, “A Mesh Network for Mobile Devices
using Bluetooth Low Energy”, In IEEE Sensors 2015, South
Korea, pp. 1-4, 2015.
33/55
34. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
BLE Mesh Network at Samsung R&D Bangalore
34/55
35. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
BLE Mesh Network at Samsung R&D Bangalore
35/55
36. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
BLE Mesh Network at Samsung R&D Bangalore
36/55
37. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
Connectivity in IoT for smart home
Edge-Fog-Cloud Network
Surbhi Saraswat, Hari Prabhat Gupta, and Tanima Dutta, “Fog
based Energy Efficient Ubiquitous System”.
37/55
38. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
Edge-Fog-Cloud Network
Edge Management Framework
Fog Layer
Cloud Layer
Fog Management Framework
Edge Layer
Sensor
Actuator
Edge Relay
Fog Device
Gateway Device
Edge Device
Sensors
Fog Relay
38/55
39. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
Connectivity in IoT for smart home
SDN Network
Vishal Agarwal, Surbhi Saraswat, Hari Prabhat Gupta, and
Tanima Dutta, “A Priority based Traffic Engineering Framework in
Software Defined Networks”.
39/55
40. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
SDN Network
Switch
Switch
Switch
Switch
Switch
Network Topology
Network Constraints
Control
Admission
Active Flows
Load Monitor
Policy
Pricing
Path Assignment
h1
1
2
3
4
5
h2
Controller
App 2 App 3App 1
4
40/55
41. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
Smart Pencil
Smart Pencil
“S-Pencil: A Smart Pencil Grip Monitoring System for Kids using
Sensors”, in proc. of the IEEE Globecom 2017, Singapore,
December 04 - 08, 2017.
41/55
42. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
A Smart Pencil Grip Monitoring System using Sensors
(f)
(a) (b) (c)
(e)(d)
42/55
43. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
A Smart Pencil Grip Monitoring System using Sensors
(a) (b)
Bluetooth
PressureSensor
Pressure Sensor
Accelerometer
43/55
44. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
A Smart Pencil Grip Monitoring System using Sensors
Z−axis
X−axis Y−axis
Accelerometer
44/55
45. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
A Smart Pencil Grip Monitoring System using Sensors
Writing Activity Set
Ë Ð
Ø ÓÒ
Ë Ò×ÓÖ Ø
ÈÖ ÔÖÓ
×× Ò
Ø
ÜØÖ
Ø ÓÒ
ØÙÖ
Ø ×
Ø Ú ØÝ
Ê
Ó Ò Ø ÓÒ
ÕÙ × Ø ÓÒ
ÏÖ Ø Ò
Ø Ú ØÝ
45/55
46. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
Smart Life
Smart Life
“A Continuous Hand Gestures Recognition Technique for
Human-Machine Interaction using Accelerometer and Gyroscope
sensors”, IEEE Sensors Journal, Vol. 16, no. 16, pp. 6425-6432,
2016.
46/55
47. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
A Continuous Hand Gestures Recognition Technique
g1 g3g2 g4 g5 g6
47/55
48. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
A Continuous Hand Gestures Recognition Technique
Accelerometer sensor
Gyroscope sensor
Gesture EventSensor Data
Action
Command
mapping module
Gesture −> Action
AllShare −> Triggering
action on appliance
CHG Technique
Appliance
48/55
49. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
A Continuous Hand Gestures Recognition Technique
0
20
40
60
80
100
Variencesofthesensorydata
Coded sensory data
Gesture
Gesture
Gesture
Gesture
Coded sensory data of x-axis
Coded sensory data of y-axis
Coded sensory data of z-axis
Mean value of the coded sensory data
49/55
50. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
Smart Environment
Smart Environment
“Leveraging Smart Devices for Automatic Mood-transferring in
Real-time Oil Painting”, IEEE Transactions on Industrial
Electronics, Volume 64, Issue 2, pp. 1581-1588, 2017.
50/55
51. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
Automatic Mood-transferring in Real-time Oil Painting
Romantic
Refreshing
Displeased
Calm
Energetic
Nervous
Pleasant
Digital Oil Painting
Mood Selection
Color Selection
Light Selection
51/55
52. Introduction
ML and IoT
Our Work
UbiMedia Lab
Smart Connectivity
Smart Education
Smart Life
Smart Environment
Automatic Mood-transferring in Real-time Oil Painting
(c)(b)(a)
52/55
53. Introduction
ML and IoT
Our Work
UbiMedia Lab
About UbiMedia Lab
Ongoing Projects
Ubiquitous and Multimedia Lab
Lab Space: 1000m2
Sponsor agencies: SERB, University of Chicago, Tata
Centre for Development Chicago, Samsung R&D
Terminals: 50+
High End Machines: Servers (Unix, Windows, and Solaris)
and High Performance Computing (HPC)
Sensors and Smart Devices: Texas Instruments, Libelium
kits, Gopro Cameras, Hikvision PTZ cameras setups and
other different 200+ type of Sensors and Smart Devices
Current Students
Doctoral Student: 10+
Post Graduate Students: 2
Junior Research Fellow: 2
Undergraduate (in projects): 10+
Ongoing short term courses, workshops, seminars: 5+
53/55
54. Introduction
ML and IoT
Our Work
UbiMedia Lab
About UbiMedia Lab
Ongoing Projects
Running Projects
An Energy-efficient WSN for Precision Agriculture
Agency: SERB, Govt. of India, Duration: 2017-2020.
Smart Life
Samsung R&D Bangalore, Duration: May-July 2017.
Water Monitoring using Sensors
with Prof. Supratik Guha (University of Chicago),
Agency: TCD Chicago, Duration: 2018-2020.
A Robust Medical Image Forensics System for Smart
Healthcare
Agency: SERB, Govt. of India, Duration: 2018-2020.
54/55