It is an exciting time in computing with the sea-change happening both on the technology fronts and application fronts. Networked sensors and embedded platforms with significant computational capabilities with access to backend utility computing resources, offer a tremendous opportunity to realize large-scale cyber-physical systems (CPS) to address the many societal challenges including emergency response, disaster recovery, surveillance, and transportation. Referred to as Situation awareness applications, they are latency-sensitive, data intensive, involve heavy-duty processing, run 24x7, and result in actuation with possible retargeting of sensors. Examples include surveillance deploying large-scale distributed camera networks, and personalized traffic alerts in vehicular networks using road and traffic sensing. This talk covers ongoing research in Professor Ramachandran’s embedded pervasive lab to provide system support for Internet of Things.

1. System Support for Internet of Things
Kishore Ramachandran
(Kirak Hong, Dave Lillethun, Dushmanta Mohapatra)

2. System Support for Internet of Things
Kishore Ramachandran
(Kirak Hong, Dave Lillethun, Dushmanta Mohapatra)
BIG DATA Industry Forum sponsored by the Institute for Data & High
Performance Computing, Scheller College of Business, Center for Data
Analytics and Center for High Performance Computing at the Georgia
Institute of Technology, Atlanta, Georgia, U.S.A.
March 27, 2014

3. Abstract
System Support for Internet of Things
It is an exciting time in computing with the sea-change happening both on the technology fronts and
application fronts. Networked sensors and embedded platforms with significant computational
capabilities with access to backend utility computing resources, offer a tremendous opportunity to realize
large-scale cyber-physical systems (CPS) to address the many societal challenges including emergency
response, disaster recovery, surveillance, and transportation. Referred to as Situation awareness
applications, they are latency-sensitive, data intensive, involve heavy-duty processing, run 24x7, and
result in actuation with possible retargeting of sensors. Examples include surveillance deploying large-
scale distributed camera networks, and personalized traffic alerts in vehicular networks using road and
traffic sensing. This talk covers ongoing research in Professor Ramachandran’s embedded pervasive lab
to provide system support for Internet of Things.

4. Overview
• Motivation
• Mobile Fog: A Distributed Programming Model for the IoT
• The Analysis of Things
• Virtualization for the IoT
• Conclusion

5. Rise of Internet of Things

6. Enable New Knowledge
Agriculture
Energy Saving (I2E)‫‏‬
Predictive maintenance
Enhance Safety & Security
Smart Home
Healthcare
Defense
Intelligent Buildings
Smart Grid
Industrial Automation
Transportation and Connected Vehicles
A Broad Set of IoT Applications

7. Our Lab Focus
System Support for IoT

8. Overview
• Motivation
• Mobile Fog: A Distributed Programming Model for the IoT
• The Analysis of Things
• Virtualization for the IoT
• Conclusion

9. Future Internet Applications on IoT
• Common Characteristics
• Dealing with real-world data streams
• Real-time interaction among mobile
devices
• Wide-area analytics
• Requirements
• Dynamic scalability
• Low-latency communication
• Efficient in-network processing

10. Fog Computing
• New computing paradigm proposed by Cisco
• Extending the cloud utility computing to the edge
• Provide utility computing using resources that are
• Hierarchical
• Geo-distributed Mobile / Sensor
Edge
Core

11. Problem
• How can we easily develop future Internet applications on the fog
computing infrastructure?
• Need a right programming model that
• Provides a simple API
• Ensures dynamic scalability
• Supports efficient in-network processing for wide-area applications
• Allows real-time interaction among nearby edge devices

12. Limitations of Existing Platform as a Service
• Based on large data centers
High latency / poor bandwidth for data-intensive apps
• API designed for traditional web applications
Not suitable for the future Internet apps

13. Network Performance of Cloud VS. Fog
0
0.5
1
1.5
2
2.5
Bandwidth (Mbytes / sec)
Bandwidth (Iperf)
Nearby EC2
• Bandwidth and latency measured between a mobile node and a
computing node connected through GT-wifi
(measured on Apr. 16. 2014)
0
2
4
6
8
10
12
14
16
18
Latency (ms)
Latency (Ping)
Nearby EC2

14. Mobile Fog
• PaaS programming model on the Internet of Things
• Design Goals
• Simplicity: minimal interface with a single code base
• Scalability: allows dynamic scaling
• Context-awareness: network-, location-, resource-, capability-awareness
• Assumes Fog computing infrastructure
• Infrastructure nodes are placed in the fog
• IaaS interface for utility computing

15. Mobile Fog – Application Model
• Mobile Fog application
consists distributed
processes connected in a
hierarchy
• Each process covers a
specific geographical
region
Location
Network
Hierarchy

16. Mobile Fog – API (App Deployment)
App
Code
Runtime
Mobile Fog
Process
Mobile Fog
AppKey
Region
Level
Capacity
Connect_Fog(appkey)
Start_App()
On_create()
On_create()
On_new_parent()
On_new_child()
On_child_leave()

17. Mobile Fog – API (Communication)
on_message()
send_to()

18. Mobile Fog – API (Context-awareness)
query_level()
query_resource()
query_capacity()
query_location()
Level 0
Level 1
Level 2Level 2
Level 3 Camera, speed, etc.
CPU, RAM, storage

19. Mobile Fog – Scalability Level 0
Level 1
Level 2 Level 2
• Application scales at
runtime based on the
workload
• Developer specifies scaling
policy for each level
• Load balancing based on
geo-locations
App
Context
send_up() send_up() send_up() send_up()

20. F1
F2
F3
F5F4
B (F4)
A (F2)
C (F5)
F3 = B + C
F1 = A + B + C
Spatial Distribution
Temporal Distribution
AA
B
C
now – 3 mins
3 mins – 5 mins
5 mins – 8 mins
F2 F1 F1
F4 F3 F1
F5 F3 F1
time
old
new
space
CBA
Mobile Fog – Spatio-temporal Object Store
• App context object
is tagged by key,
location, and time
• get_object(key,
location, time)
put_object(key,
location,time)
• Context objects are
migrated when
scaling

21. Use Case – Vehicle Tracking using Cameras
Motion Filter
Image Capture
Target Tracking
License Plate Recognition
Target Location
Aggregation
send_up(motion_frame)
send_up(vehicle_ID)
send_down(PTZ_control)

22. Experiments
• Quad-tree topology
covering the urban
area
• Edge infrastructure
nodes directly
communicate with
simulated vehicles
• 1000 vehicles in a 7.7
KM x 3.5 KM grid
• SUMO traffic pattern
• OMNET++ net sim

23. Experiments (Vehicle to Vehicle Video
Streaming)
Fog-based approach
shows better latency
and less net traffic
relative to the cloud

24. Experiments (Traffic Monitoring)
• Location events from
mobile devices are stored
in Fog or Cloud
• Fog significant advantage
when query range is
within 0.5 KM
• Cloud-based approach is
better when query range
is large (> 0.8 KM)

25. Overview
• Motivation
• Mobile Fog: A Distributed Programming Model for the IoT
• The Analysis of Things
• Virtualization for the IoT
• Conclusion

26. 26
Challenges to Application Development
• Sensor / actuator access
• Data transport
• Widely-distributes sensors / actuators
• Requires computational resources
• Distributed computation
• for parallelism
• for distributed sensors / actuators
• Application dynamism
• need elastic resources
• Massive scale
• number of sensors / actuators
• amount of data produced
• geographic diversity
• And more…

27. 27
What’s missing from IoT?
• Systems Support for Live Streaming Analysis
• algorithms analyze data streams as they are produced
• brings intelligence to IoT
• proven by the success of Big Data Analytics in static data / batch processing world
• applies in many contexts:
• situation awareness
• cyberphysical systems
• financial analysis
• we refer to Live Stream Analysis in IoT as the Analysis of Things (AoT)

28. Internet of Things, in detail…
• Myriad sensors and actuators, not just user devices
• digital representations of the real world
• Mobility
• Wireless
• Both “simple” and “rich” sensing
• feature extraction and advanced analysis required for “rich”
sensing
• Heterogeneity
• Dynamic environment
• Machine-to-machine (M2M) and cyberphysical
applications
• sense-process-actuate loop -> low latency!
• Massive scale
• number of devices
• amount of data
• Geographical diversity (and geography matters!)
• location-sensitive & location-aware applications
• Hierarchical structure
• latency tolerance vs. time scale (data liveliness)

29. Hierarchical Applications
Each Store:
customer interaction (e.g., e-
cupons, product
recommendations)
Regional:
what to carry in stores,
product interactions
Corporate-wide:
marketing decisions, corporate
strategy

30. System Requirements to Support AoT
• Programming Model
• Execution Environment
Abstract away system-level issues
• Execution EnvironmentExecute applications on distributed resources
• StampedeRT
Efficient and scalable stream transport
• Stream RegistryStream registration and discovery
• Operator StoreComponent-based design
• Drivers
Connecting operator graphs to arbitrary inputs &
outputs
• Proposed WorkWidely distributed architecture

31. The Programming Model
Operator
State Declarations
Initialization
Routine
Handler
(Algorithm)
Interface
Operator
Store
Operator
Operator
Operator
Operator
Operator
Input / Output
Stream Types

32. Worker Node
Worker Node
Worker Node
Worker Node
Resource
Manager
Operator
Store
Stream
Registry
Operator
Code
Operator
Graph
System Architecture

33. Worker Node
Resource
Manager
Operator
Store
Stream
Registry
StampedeRT
workerd
Operator
Container
Running Application
Operator
Operator
Container
Operator

34. Stream Registration Example
Sensor Type
{ type : “camera”,
capabilities : [
“color”,
“PTZ” ],
location : “KACB 1212”,
gps : {
lat : 12.345,
long : 54.321 }
}
Stream Type
{ type : “video”,
mime : “video/mp4”,
encoding : “H.264”,
resolution : {
x : 800,
y : 600 }
framerate : 10
}
+ Handle (how to connect to stream & produce/consume)

35. Sharing Analysis Results
Operator
OperatorOperator
Operator
Shared Stream
Operator
Operator
Operator
Operator
Operator
Stream
Registry

36. Sensor Threshold Application
∑
S
S
S
S
S
S
S
∑
∑
∑
∑
Alert!
s
s
AoT Systemsource
drivers
sink
drivers
sum/mean
operators
threshold
operators{ id, GPS,
sensor value }
{ cent. GPS,
sum, count }
{ list( GPS,
sensor value ) }

37. Object Tracking Application
DEC
DEC
ENC
ENC
AoT System
source
drivers
{ M-JPEG video }
FG Track
FG Track
S
S
s
s
sink
drivers
JPEG
decoder
JPEG
encoder
foreground
detector
object
detector
/ tracker
{ M-JPEG video
w/ objects }
{ raw video }
{ foreground mask }
{ raw video
w/ objects }

38. Overview
• Motivation
• Mobile Fog: A Distributed Programming Model for the IoT
• The Analysis of Things
• Virtualization for the IoT
• Conclusion

39. How Virtualization Fits In IoT?
• Servers, Cloud Setups
• Server Consolidation, Load Balancing, Usage Accounting, Fairness, Security
• Fog Nodes
• Security, Data/ Computation migration to Clouds
• Dealing with the underlying heterogeneity of the fog nodes
• Client Devices
• Versatility, Security in smart phones (other hand-held devices)
• Security of the critical components in automobiles
• Data/ Computation migration to fogs and clouds

40. Need for new research
• Virtualization has been well researched in the context of servers and cloud
setups
• Introduction into fog nodes & client devices brings new challenges
• Rethink virtualization in the context of heterogeneity presented by fog nodes
• Set of applications running in the client devices may have different requirements
• Implications for the design and implementation of mechanisms
• Resource management in the context of needs of new applications/ usage-modes
• Migration becomes significantly harder
• Granularity of migration (whole VM/ process/ something else?)
• Dealing with the complexity of architectural difference among the nodes
• Dealing with difference in capabilities of I/O devices
• Dealing with geographical separation of the fog nodes and its implications for system and network
virtualization

41. Examples of Relevant Research
• Virtualization getting introduced into client devices
• Xen-ARM
• VMWARE Mobile Virtualization
• OKL4 Mobile Virtualization
• Research Efforts: Cells, others…
• Initial investigations about introducing into automobile software systems
• Cyber Foraging
• On-Demand Virtualization
• Resource Management

42. Memory Resource Management
• Core research problem: Dynamically partition the available physical memory
among the set of virtual machines
• Why is it necessary?
• Memory is finite (as opposed to CPU, I/O Cycles)
• Amount of allocated memory has direct impact on the performance of a VM
• Memory need of VMs varies over time
• Static partitioning may be enough in case of over-provisioned systems
• Over-provisioning may result in under-utilization of resource for a majority of time
• Client devices, Automobile systems are not over-provisioned
• May be rather resource-constrained

43. Known Mechanisms
• Ballooning
• Based on a pseudo-driver called balloon driver inside each VM
• Two operations: Inflate, Deflate
• Transfers memory pages between the VMs and the hypervisor
• Transcendent Memory (T-Mem)
• Collect memory from the hypervisor and the guests into a
centralized pool
• May use ballooning for collecting memory
• Provide indirect page-copy based interface to the pool
• get-page() / put-page()
• To be used by the VMs during memory pressure
• Other hybrid mechanisms

44. Why are these not good enough?
• Primary Reason: Not designed with
the client device applications in mind
• Application Characteristics:
• Occasional Memory Spikes
• Latency Sensitivity
• Mix of critical and non-critical VMs
• Other Reasons:
• Stand-alone ballooning is not on-demand
• Lack of coordination among the VMs
• T-Mem
• Does not necessarily provide
performance/usage benefit as compared
to ballooning
Youtube-Client

45. How to improve ballooning
• Directed Ballooning
• Coordinated effort among the
VMs to deal with memory spikes
• Implemented according to Xen
split-driver principle
• Issues
• Interference from other system
constructs
• Latency overhead

46. Improving latency
• Reduce scheduler interference
• Optimization inside VM kernel
• Capture the event when a ballooning related work item gets allocated to a kernel thread
• Increase the priority level of the kernel thread for the time it does ballooning work
• Optimization in the hypervisor
• A vcpu doing ballooning work is not de-scheduled
• Appropriate working set analysis in deciding the VMs to be used for memory
balancing
• Could also be used for deciding the pages to be freed
• Modify the design and implementation to not use Xen-Store based
coordination protocol
• More active involvement of the hypervisor
• GOAL: To have a latency within a few-factors of in-kernel mechanisms like
‘malloc’

47. Conclusion
• Variety of IoT applications require proper system support
• Complexity of developing large-scale applications
• Requirement for real-time interaction and dynamic workload handling
• Applications run on resource-poor heterogeneous devices
• System support for IoT
• Programming Model
• Runtime System
• Virtualization