The document discusses the evolution of web technologies towards semantic integration and its application in physical-cyber-social systems, particularly for enhancing situational awareness and decision support. It highlights the role of semantic models and ontologies in processing diverse data types for knowledge discovery and insight generation, with examples in fields like healthcare and environmental monitoring. Additionally, it presents demonstrations of real-time sensor data applications, emphasizing the integration of various technologies for effective analysis and response to events like inclement weather.

1. Semantics empowered
Physical-Cyber-Social Systems for
EarthCube
Presentation at theEarthCubeFace Face-to-Face Workshop of Semantics & Ontologies
Workgroup: April 30-May 1, 2012, Ballston, VA.
Amit Sheth
Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing
Wright State University, Dayton, OH, USA
http://knoesis.org
Special thanks & contributions: Cory Henson, PramodAnantharam
1

2. Web (and associated computing) is evolving
Computing for Human Experience
Enhanced Experience,
Tech assimilated in life Web as an oracle / assistant / partner
- “ask the Web”: using semantics to leverage
Situations, 2007 text + data + services
Events - Powerset, Siri, Watson Web 3.0
Objects Web ofpeople, Sensor Web
- social networks, user-createdcasualcontent
- 40 billionsensors
Patterns Web 2.0
Web of resources
- data, service, data, mashups
Keywords - 4 billionmobilecomputing
Web of databases
1997 - dynamically generated pages
- web query interfaces
Web of pages
- text, manually created links Web 1.0
- extensive navigation

3. Sensors everywhere ..sensing, computing,
transmitting
• 2009: 1.1 billion PCs,
4 billion mobile devices,
40+ billion mobile sensors
(Nokia: Sensing the World with Mobile Devices)
• 6 billion intelligent sensors
– informed observers, rich local knowledge
Christmas Bird Count
3

4. Data & Knowledge Ecosystem
Situational Awareness
Decision Support
Insight Knowledge Discovery
Analysis (eg Patterns)
Understanding & Perception Data Mining
SSW/
W3C-SSN Search Browsing Integration
OGC SWE
Multimedia Data
Structured,
Textual Data: Scientific Literature, Web Pages, News, Blogs,
Semistructured
Reports, Wiki, Forums, Comments, Tweets
Unstructured
Observational Data Experimental Data Data
Transactional Data
4

5. Semantics as core enabler, enhancer @ Kno.e.sis
15 faculty
~50 PhD students
Excellent Industry collaborations
(MSFT, GOOG, IBM, Yahoo!, HP)
Well funded
Exceptional Graduates
Multidisciplinary:
Health/Clinical
Biomedical Sc
Social Sc
…
5

6. Semantic
Models Search
Integration
Analysis
Discovery
Relationship Web Question
Answering
Patterns / Inference / Reasoning
Situational
Meta data / Awareness
Semantic
Annotations
Metadata Extraction
RDB
Text
Structured and Semi- Multimedia Content Sensor Data
structured Data and Web Data

7. From simple ontologies
Knowledge Enabled Information and Services Science

8. Drug Ontology Hierarchy
(showing is-a relationships)
formulary_
non_drug_ interaction_ property formulary
reactant property
indication
indication_ property
owl:thing
monograph property
_ix_class prescription interaction_
_drug_ with_non_
brandname_ prescription
brand_name drug_reactant
prescription individual _drug interaction
_drug_
property brandname_
brandname_ composite prescription interaction_
undeclared _drug_ with_mono
interaction_
generic graph_ix_cl
with_prescri
cpnum_ generic_ ass
ption_drug
group composite
generic_
individual
Knowledge Enabled Information and Services Science

9. to complex ontologies
Knowledge Enabled Information and Services Science

10. N-Glycosylation metabolic pathway
GNT-I
attaches GlcNAc at position 2
N-glycan_beta_GlcNAc_9 N-acetyl-glucosaminyl_transferase_V
N-glycan_alpha_man_4
GNT-V
attaches GlcNAc at position 6
UDP-N-acetyl-D-glucosamine + alpha-D-Mannosyl-1,3-(R1)-beta-D-mannosyl-R2
<=>
UDP + N-Acetyl-$beta-D-glucosaminyl-1,2-alpha-D-mannosyl-1,3-(R1)-beta-D-mannosyl-$R2
UDP-N-acetyl-D-glucosamine + G00020 <=> UDP + G00021
Knowledge Enabled Information and Services Science

11. A little bit about semantic metadata
extractions and annotations
Knowledge Enabled Information and Services Science

12. Extractionfor Metadata Creation
Nexis Digital Videos
UPI
AP
... ...
Feeds/ Data Stores
Documents
WWW, Enterprise Digital Maps
Repositories
...
Digital Images Digital Audios
Create/extract as much (semantics)
metadata automatically as possible;
Use ontlogies to improve and enhance EXTRACTORS
extraction
METADATA
Knowledge Enabled Information and Services Science

13. Automatic Semantic Metadata
Extraction/Annotation of Textual Data
Knowledge Enabled Information and Services Science

14. Semantic Sensor Web Infrastructure

15. Semantically Annotated O&M
<om:Observation>
<om:samplingTime><gml:TimeInstant>...</gml:TimeInstant>
</om:samplingTime>
<om:procedurexlink:role="http//www.w3.org/2009/Incubator/ssn/ontologies/SensorOntolgy.owl#Sensor“
xlink:href="http//www.w3.org/2009/Incubator/ssn/ontologies/SensorOntolgy.owl#sensor_xyz"/>
<om:observedPropertyxlink:href="http//www.w3.org/2009/Incubator/ssn/ontologies/SensorOntolgy.owl#temperature"/>
<featureOfInterestxlink:href="http://sws.geonames.org/5758442/"/>
<om:resultuom="http//www.w3.org/2009/Incubator/ssn/ontologies/SensorOntolgy.owl#fahrenheit">42.0</om:result>
</om:Observation>
15

16. Semantic Sensor ML – Adding Ontological Metadata
Domain Person
Ontology
Company
Spatial
Ontology Coordinates
Coordinate System
Temporal
Ontology
Time Units
Timezone
Mike Botts, "SensorML and Sensor Web Enablement," 16
Earth System Science Center, UAB Huntsville

17. Workflow Architecture for Managing Streaming Sensor Data

18. Weather Application
Weather Application
Detection of events, such as blizzards, from weather
station observations on LinkedSensorData
18
Demos: Real-Time Feature Streams

19. SECURE: Semantics Empowered Rescue Application
Weather Environment
Rescue robots detect different types of fires, which may require different
methods/tools to extinguish, and relays this knowledge to first responders.
Demo: SECURE: Semantics Empowered Rescue Environment 19

20. A Challenging Example Query
What schools in Ohio should now be closed due to inclement
weather?
Need domain ontologies and rules to describe type of inclement
weather and severity.
Integrationof technologies needed to answer query
1. Spatial Aggregation
2. Semantic Sensor Web
3. Machine Perception
4. Linked Sensor Data
5. Analysis of Streaming Real-Time Data
More details in: Spatial Semantics for Better Interoperability and Analysis: Challenges and Experiences in
Building Semantically Rich Applications in Web 3.0
20

21. Technology 1
Spatial Aggregation
• What schools are in Ohio?
• What weather sensors are near each of the
school?
21

22. Technology 2
Semantic Sensor Web (SSW)
• What is inclement weather?
• What sensors in Ohio are capable of detecting inclement
weather?
• What sensors are near schools in Ohio?
• What observations are these sensors generating NOW?
22

23. Technology 3
Active Machine Perception
• Are these observations providing evidence for
inclement weather?
23

24. Technology 4
Linked Sensor Data
• What schools are in Ohio?
• What inclement weather necessitates school closings?
• What sensors in Ohio are capable of detecting inclement
weather?
• What sensors are near schools in Ohio?
• What observations are these sensors generating NOW?
24

25. Technology 5
Analysis of Streaming Real-Time
Data
• What observations are these sensors
generating
NOW?
25

27. Demos
• Real-Time Feature Streams
• SECURE(presentation:
• SECURE: Semantics Empowered resCUe EnviRonmEnt )Amit
• Trusted Perception Cycle
• Sensor Discovery on Linked Data
• Semantic Sensor Observation Service (SemSOS)
Related Talk
• Spatial Semantics for Better Interoperability and Analysis:
Challenges and Experiences in Building Semantically Rich
Applications in Web 3.0: Amit Sheth delivers talk at the 3rd Annual
Spatial Ontology Community of Practice Workshop:
Development, Implementation and Use of Geo-Spatial Ontologies
and Semantics, 3 October 2010, USGS, Reston, VA.