This document discusses using machine learning for IoT device virtualization. It outlines challenges with IoT application development like hardware availability and simulating real scenarios. It then describes how KnowThings uses machine learning to automatically discover data protocols from network traffic and generate adaptive virtual devices without expert knowledge. Their solution captures traffic, generates a "living" model using genome sequencing algorithms, and allows the virtual device to playback responses. This enables more effective testing of IoT applications during development.

1. A Novel Approach:
IoT Device Virtualization using
Machine Learning
Menka Goyal
CTO & CoFounder, KnowThings.io
1

2. Agenda
IOT APPLICATIONS DEVELOPMENT
IOT APPLICATION DEVELOPMENT CHALLENGES
SERVICE VIRTUALIZATION IN IOT
MACHINE LEARNING IN IOT DEVICE VIRTUALIZATION
KNOW THINGS SOLUTION
1
2
3
4
5

3. The Internet of Developer Things

4. Dev Integration
Test
Stress
Test
Acceptance
Test
Modern Software Dev cycle
Continuous
Integration
Shift-Left
Behavior-
Driven Dev
Continuous
Delivery
Test-Driven
Dev
Agile Dev
Enterprise Development Processes/Tools

5. Challenges in IoT Solution Development
Hardware
Availability
Heterogeneous
Networks
Legacy
Systems
Simulating
Real
Scenarios

6. Service Virtualization ( Shift Left – Test Smarter – Increase
Productivity)
6

7. Smart
Switches
Smart
Switches
Smart
Switches
Medical
Monitor
Medical
Monitor
Medical
Monitor
Smart
Meters
Smart
Meters
Smart
Meters
SensorsSensorsSensors
SensorsSensorsIP Device
Device
GW
Device
GW
Device
GW
Device
GW
802.15.4
ZWave
MQTT
BT
REST
Monitor/
Control
Big Data / Analytics
Aggregation/
Data
Collection
REST
AMQP
Applications
and
Services
SOAP
REST
Web
MobileHTTP
HTTP

8. Smart
Switches
Smart
Switches
Smart
Switches
Medical
Monitor
Medical
Monitor
Medical
Monitor
Smart
Meters
Smart
Meters
Smart
Meters
SensorsSensorsSensors
SensorsSensorsIP Device
Device
GW
Dvice
GW
Device
GW
802.15.4
ZWave
MQTT
BT
REST
Monitor/
Control
Big Data / Analytics
Aggregation/
Data
Collection
REST
AMQP
Applications
and
Services
SOAP
REST
Web
MobileHTTP
HTTP
TIPS
TIPS
TIPS
TIPS - Things Intelligence Processing System
Device
GW
Device
GW
Device
GW
Device
GW
TIPS

9. Client
VS
Match
DPH
Decode
Data
Traditional Virtualization Approach
 Requires expert knowledge and data protocol handler
 Extract fields and values
 Requires set of defined rules

10. 10
 Machine learning to automatically discover protocol
 Requires no expert knowledge and data protocol handler
 Uses data mining methods to refine modelling and playback
 Applies Needleman Wunsch Genome Sequencing Algorithm
 Automatically discover byte-level patterns
Things Intelligence Processing System

11. KnowThings: Smart Virtualization for IoT Devices
Capture Model Playback
Capture network traffic
between Gateway and SUT
Generate a living, breathing “live”
model, Adaptive Virtual Device
Run AVD using Genome
Sequencing ML Algorithm

12. AVD - Models IoT Device Behavior
12
Sending online data requires:
 Knowing the order of messages
 Generating the values (model)
 Establishing dependencies (function)
Why is it challenging?
 Access to all the predictor elements
 Complexity of the model
 Dependencies extraction

13. Needleman Wunsch Algorithm
13

14. KnowThings Greenhouse App 14
KT Greenhouse
Device
Network
Traffic KT Gateway
KT Greenhouse
Dashboard

15. Greenhouse Message Traces
Request Response
POST /ktgh/requestor/sensors
[{"name": "Temperature", "ID": "TMP1"},
{"name": "Humidity", "ID": "HUM1"}]
HTTP/1.1 200 OK
POST /ktgh/requestor/actuators
[{"name": "Fan", "ID": "FAN1"}]
HTTP/1.1 200 OK
PUT /ktgh/requestor/sensors/TMP1
{"dataValue": 37.20527648925781}
HTTP/1.1 200 OK
PUT /ktgh/requestor/sensors/HUM1
{"dataValue": 32.28166961669922}
HTTP/1.1 200 OK
GET /ktgh/requestor/actuators/FAN1 HTTP/1.1 200 OK
{"ID":"FAN1","name":"Fan","value":89.20875276}

16. Character Frequencies in Alignment, using MSA
16

17. Entropy/Weights from Character Frequencies
17
Shannon Index Entropy Measure Hyperbolic Scaler Weighting Calculation

18. Cluster to Discover Operation Types
Request Response
POST /ktgh/requestor/sensors
[{"name": "Temperature", "ID": "TMP1"},
{"name": "Humidity", "ID": "HUM1"}]
HTTP/1.1 200 OK
POST /ktgh/requestor/actuators
[{"name": "Fan", "ID": "FAN1"}]
HTTP/1.1 200 OK
Request Response
GET /ktgh/requestor/actuators/FAN1 HTTP/1.1 200 OK
{"ID":"FAN1","name":"Fan",
"value":89.20875276}
Request Response
PUT /ktgh/requestor/sensors/TMP1
{"dataValue": 37.20527648925781}
HTTP/1.1 200 OK
PUT /ktgh/requestor/sensors/HUM1
{"dataValue": 32.28166961669922}
HTTP/1.1 200 OK
Cluster 1
(POST)
Cluster 2
(PUT)
Cluster 3
(GET)

19. KnowThings Greenhouse Playback 19
KT Greenhouse
Device
Adaptive Virtual Device replaces the
actual device
KT Gateway
Server
KT Gateway
KT Greenhouse
Dashboard
Traffic to AVD
Traffic from
AVD
Virtual Device 1
Virtual Device 2
Traffic to/from
AVD
Virtual Device 3

20. KnowThings Greenhouse Playback 20
KT Greenhouse
Device
Adaptive Virtual Device replaces the
Greenhouse Gateway
KT Gateway
Server
KT Gateway
Server
KT Greenhouse
Dashboard
Traffic from
AVD
Traffic from
AVD
Traffic from
actual device
Virtual Device 1
Virtual Device 2

21. How ML Device Simulation Differ from Other
Solutions
Realistic Data
Quickly Learns New ProtocolsFast Model Creation
Test at Scale

22. Publications and Patents
 Five US patents filed
 Six international journal and conference publications
 Versteeg, S., Du, M., Schneider, J. G., Han, J., Grundy, J., & Goyal, M. (2016). Opaque Service Virtualisation: A Practical Tool for
Emulating Endpoint Systems, 38th International Conference on Software Engineering (ICSE2016)
 Du, M., Schneider, J. G., Hine, C., Grundy, J., & Versteeg, S. (2013, June). Generating service models by trace subsequence substitution. In
Proceedings of the 9th international ACM Sigsoft conference on Quality of software architectures (pp. 123-132). ACM.
 Du, M., Versteeg, S., Schneider, J. G., Grundy, J., & Han, J. (2015). From Network Traces to System Responses: Opaquely Emulating Software
Services. arXiv preprint arXiv:1510.01421.
 Du, M., Versteeg, S., Schneider, J. G., Han, J., & Grundy, J. (2015). Interaction Traces Mining for Efficient System Responses Generation. ACM
SIGSOFT Software Engineering Notes, 40(1), 1-8.
 Du, M. Automatic Generation of Interaction Models for Enterprise Software Environment Emulation. 22nd Australasian Software Engineering
Conference (ASWEC2013).
 Versteeg, S., Du, M., Schneider, J. G., Han, J., Grundy, J., & Muller, S. (2015). Opaque Virtualization of Service Endpoints for Continuous Delivery
Enablement, IEEE Software

23. Research & Development Team
23
• Steve Versteeg
• Prof. Jean-Guy Schneider
• Zeinab Farahmandpour
• Ms. Miao Du
• Prof. Jun Han
• Anand Kameswaran
• Menka Goyal

24. Questions ?
24