This document discusses using semantic data abstraction and indoor localization to improve situation awareness in cyber-physical systems. It aims to reduce information overload on operators by semantically annotating sensor data. An indoor localization algorithm is proposed to identify event locations. The algorithm fuses radio signal strength and time difference of arrival data to estimate distances between sensors. It was shown to estimate positions with root mean square errors under 2 cm. Semantic abstraction and localization could help first responders better understand situations from sensor data in GPS-denied environments.

1. Situation Awareness in Cyber-Physical Systems
using Indoor Localization and Semantic Data
Abstraction
Presenter Pratikkumar Desai
Advisor Dr. Kuldip Rattan
PhD Seminar - 11/2/2012
Department of
Electrical Engineering

2. Outline
• Introduction
• Cyber-Physical System
• Situation Awareness
• Motivation
• Problem Statement
• Semantic Sensor Web
• Indoor Localization
Domain: Cyber-physical Systems
Application: Situation Awareness
Department of
Electrical Engineering

3. Cyber-Physical Systems
Cyber : Computation, communication, and control that are discrete, logical, and
switched
Physical : Natural and human-made systems governed by the laws of physics
and operating in continuous time
Cyber-Physical Systems (CPS) : Systems in which the cyber and physical
systems are tightly integrated at all scales and levels
What it is not?
• Not desktop computing
• Not traditional embedded/real-time systems
• Not today‟s sensor nets
Department of
Electrical Engineering

4. CPS Examples
eHealth
Military
Smart Home
Department of
Electrical Engineering

5. Situation Awareness
Projection
of the future
state or actions
Comprehension of the
current situation
Perception of the elements in the current
situation
Department of
Electrical Engineering

6. Motivation Scenario
First responders
(Hazardous condition)
• Responders are using sensor equipped mobile robot.
• Building is equipped with a system which can provide location.
• Need to Identify the situation from available sensor data.
• Need to identify location of the event.
Department of
Electrical Engineering

7. Perceptions & Comprehensions
• Sensors on Mobile Robot: • Fire:
• Temperature • Gas
• Pressure • Liquid
• Humidity • Wood
• CO2 • Mix
• CO • Dry heat
• Chemical sensors
• Dry Ice
• Sensors in home: • False Alarm
• Fire alarm
• Chemical leakage
• Thermostat
• Human observation
• Background knowledge
Department of
Electrical Engineering

8. Information Overload
Department of
Electrical Engineering

9. Problem Statements
(1) Reduce information overload on the operator
Solutions:
• Semantic abstractions of sensor data
• Semantic Sensor Web based automatic event comprehension
(2) Identify the location of the event
Solution:
• Indoor localization in GPS denied environments
Department of
Electrical Engineering

10. Semantic Sensor Web
Department of
Electrical Engineering

11. Semantic Web
“The Semantic Web is a major research initiative of the
World Wide Web Consortium (W3C) to create a metadata-
rich Web of resources that can describe themselves not
only by how they should be displayed (HTML) or
syntactically (XML), but also by the meaning of the
metadata.”
From W3C Semantic Web Activity Page
“The Semantic Web is an extension of the current web in
which information is given well-defined meaning, better
enabling computers and people to work in cooperation.”
Tim Berners-Lee, James Hendler, Ora Lassila,
The Semantic Web, Scientific American, May 2001
Department of
Electrical Engineering

12. Semantic Sensor Web
• An approach of annotating raw sensor data with semantic,
spatial and temporal metadata to increase interoperability.
• Provide abstraction to low level sensor data for more
intuitive representation.
• Provide Comprehension abstractions for event
identification.
Department of
Electrical Engineering

13. Cyber-Physical Systems for Situation
Awareness
Semantic Sensor
Web
Department of
Electrical Engineering

14. System Architecture of Semantic Web
Implementation
Department of
Electrical Engineering

15. Abstraction
Bipartite Graph
Sensor Perception and Event
comprehension abstractions
for the Example Scenario
Department of
Electrical Engineering

16. Detecting a Fire
• Temperature (local, on robot):
= 100 C
• Temperature (IR):
= 150 C
• IR light (looking straight):
= 800
• IR light (looking down):
= 800
• Carbon Dioxide:
= 2000 ppm
Department of
Electrical Engineering

17. Simulation Results
Fire
Robot
Department of
Electrical Engineering

18. Future Work
Unknown Event
• Temperature (local, on robot):
= 100 C
• Temperature (IR):
= 22 C
• IR light (looking straight):
=0
• IR light (looking down):
= 800
• Carbon Dioxide:
= 2000 ppm
Department of
Electrical Engineering

19. Future Work
Fuzzy Abstraction Set
• Fuzzy abstraction sets can be useful in
handling the problem of uncertain
perception conditions.
• An uncertain sensor perception can infer
more than one situation in some cases.
Department of
Electrical Engineering

20. Indoor Localization
Department of
Electrical Engineering

21. GPS denied Environments
Department of
Electrical Engineering

22. Traditional Indoor Localization Techniques
• Active Badge and Active Bat system.
• RADAR: An In-building RF-based user location
and tracking system. RF
• RFID radar
Camera
• Object tracking with multiple cameras
• Computer vision based localization
• Wireless Sensor Network
TDoA
Department of
Electrical Engineering

23. The Cricket Motes
• High performance MICA2 wireless location system.
• Ultrasound transmitter and receiver for time of flight
ranging.
• Decentralized and scalable operation.
• Applications:-
• Indoor location system
• Ubiquitous computing
• Asset/person tracking
Department of
Electrical Engineering

24. Beacon & Listener
Ceiling
Listeners
• Dynamic
• Receivers
Beacons
• Static
• Pseudo-Satellites
floor
Department of
Electrical Engineering

25. TDoA in The Cricket System
Department of
Electrical Engineering

26. Localization: Trilateration
Number of nodes = 3.
Outlier Rejection and Multilateration are used to improve location results.
Department of
Electrical Engineering

27. TDoA Based Positioning System
Pros Cons
• Distance estimation accuracy • Always need Line-of-Sight
is less than 2 cm for 10 (LOS) for distance estimation.
meters. • Do not work in presence
• Appropriate for localizing of ultrasonic noise.
small vehicles such as Mobile • Slow update rate for dynamic
robots or a person. application.
• Least expensive.
Department of
Electrical Engineering

28. The Proposed Algorithm
• Utilizes fusion of RSSI and TDoA data for accurate
distance estimation.
• The algorithm stages,
• RSSI data training
• Distance estimation
• Localization
• Uses TDoA as a primary distance estimation technique.
• RSSI data is trained and converted into appropriate
distance measurements.
• The proposed algorithm can be used in absence of one or
many TDoA links.
Department of
Electrical Engineering

29. Initial Conditions
• Distances between all beacons are known and fixed
Department of
Electrical Engineering

30. Beacon B1 Transmit Data
RSSI Link
TDoA Link
0 ? ? ? 0 ? ? ?
? 0 ? ? R12 0 ? ?
? ? 0 ? ? ? 0 ?
? ? ? 0 B1 ? ? ? 0
B2
0 ? ? ?
? 0 ? ?
? ? 0 ? B4
R14 ? ? 0
B3
L
0 ? ? ? R1L T1L 0 ? ? ?
? 0 ? ? ? ? ? 0 ? ?
? ? 0 ? ? ? R13 ? 0 ?
? ? ? 0 ? ? ? ? ? 0
Department of
Electrical Engineering

31. Beacon B4 Transmit Data
RSSI Link
TDoA Link
0 R21 R31 R41 0 ? ? ?
R12 0 ? ? R12 0 R32 R42
R13 R23 0 ? R13 R23 0 ?
R14 R24 R34 0 B1 R14 R24 R34 0
B2
0 ? ? ?
R12 0 ? ?
R13 R23 0 ? B4
R14 R24 R34 0
B3
L
0 ? ? ? R1L T1L 0 ? ? ?
R12 0 ? ? R2L T2L R12 0 ? ?
R13 R23 0 ? R3L T3L R13 R23 0 R43
R14 R24 R34 0 R4L T4L R14 R24 R34 0
Department of
Electrical Engineering

32. Distance Estimation (i)
• In traditional RSSI based estimation schemes, distance is
estimated with following equation,
2
Pr GtGr
(d ) 2 2
Pt (4 ) d L
2
ˆ 2 GtGr Pt 1
d 2
(4 ) L R SSI
ˆ A
d
R SSI
Where, A is an environment loss factor
Department of
Electrical Engineering

33. Distance Estimation (ii)
Where, „n‟ is number of data collected
ˆ
A
ˆ
d trained
RSSI
Department of
Electrical Engineering

34. IPS Results – Proposed Algorithm
RMS error in position estimation
(using proposed algorithm).
Department of
Electrical Engineering

35. Summary
• An approach of solving future situation awareness
problems using Cyber-Physical Systems.
• Introduced Semantic Web based sensor data abstraction
to reduced the operator information overload.
• A novel indoor localization algorithm to pinpoint the
location of event being monitored.
Department of
Electrical Engineering

36. Acknowledgement
• Dr. Kuldip Rattan (Advisor)
• Dr. Amit Sheth ( Co-advisor / Director, Kno.e.sis)
Department of
Electrical Engineering

37. Questions ?
Department of
Electrical Engineering

38. Demo
[If time permits]
SECURE : Semantic Empowered resCUe enviRonmEnt
Department of
Electrical Engineering