Knowledge Enabled Information and Services Science
Relationship Web:
Trailblazing , Analytics and Computing for Human Expe...
Knowledge Enabled Information and Services Science
Evolution of the Web
2005
1990s
Web of databases
- dynamically generate...
Knowledge Enabled Information and Services Science
A Compelling Vision: Trailblazing
“(Human mind works) by association. W...
Knowledge Enabled Information and Services Science
Keywords, Documents, Objects, Events,
Insight, Experience
“An object by...
Knowledge Enabled Information and Services Science
Semantics and Relationships on a
Web scale
Increasing depth and sophist...
Knowledge Enabled Information and Services Science
Relationship Web takes you away
from “which document” could have
data/i...
Knowledge Enabled Information and Services Science
Structured text
(biomedical
literature)
Informal Text
(Social Network
c...
Knowledge Enabled Information and Services Science
Issues - Relationships
Understanding and modeling relationships
Identif...
Knowledge Enabled Information and Services Science
Data, Information, and Insight
What: Which thing or which particular on...
Knowledge Enabled Information and Services Science
EventWeb [Jain]
RelationshipWeb [Sheth]
A Web in which each node is an ...
Knowledge Enabled Information and Services Science
Supporting Relationships
in the Real World
Objects and event represent ...
Knowledge Enabled Information and Services Science
Karthik
Gomadam
Amit
Sheth
Attended
Google IO
Moscone Center, SFO
May 2...
Knowledge Enabled Information and Services Science
Identifying & Representing Relationships
Implicit Relationships
–Statis...
Knowledge Enabled Information and Services Science
Identifying & Representing Relationships
Explicit Linguistic Relationsh...
Knowledge Enabled Information and Services Science
Identifying & Representing Relationships
Formal Relationships
–Subsumpt...
Knowledge Enabled Information and Services Science
Why is This a Hard Problem?
Are objects/entities equivalent/equal(same)...
Knowledge Enabled Information and Services Science
Why is This a Hard Problem?
Semantic ambiguity
When information (knowle...
Knowledge Enabled Information and Services Science
Faceted Search and Semantic Analytics –
Some Early Work where relations...
Knowledge Enabled Information and Services Science
InfoQuilt (1996-2000): Using metadata PatchQuilt and user models/ontolo...
Knowledge Enabled Information and Services Science
Physical Link to Relationship
<TITLE> A Scenic Sunset at Lake Tahoe </T...
Knowledge Enabled Information and Services Science
MREF (1996-1998)
Metadata Reference Link -- complementing HREF
Creating...
Knowledge Enabled Information and Services Science
Metadata Reference Link
(<A MREF …>)
<A HREF=“URL”>Document Description...
Knowledge Enabled Information and Services Science
Correlation Based on
Content-Based Metadata
Some interesting informatio...
Knowledge Enabled Information and Services Science
Abstraction Layers
METADATA
DATA
METADATA
DATA
ONTOLOGY
NAMESPACE
ONTOL...
Knowledge Enabled Information and Services Science
MREF (1998)
Model for Logical
Correlation using
Ontological Terms
and M...
Knowledge Enabled Information and Services Science
An Example RDF Model for MREF (1998)
<?namespace href="http://www.foo.c...
Knowledge Enabled Information and Services Science
Domain Specific Correlation
Potential locations for a future shopping m...
Knowledge Enabled Information and Services Science
Domain Specific Correlation
=> media-independent relationships between ...
Knowledge Enabled Information and Services Science
TIGER/Line DB
Population:
Area:
Boundaries:
Land cover:
Relief:
Census ...
Knowledge Enabled Information and Services Science
Knowledge Enabled Information and Services Science
Complex Relationships
Some relationships may not be manually asserted, ...
Knowledge Enabled Information and Services Science
Complex Relationships
Relationships (mappings) are not always simple
ma...
Knowledge Enabled Information and Services Science
A Simple Relationship ?
Smoking CancerCauses
Graph based / form fitting...
Knowledge Enabled Information and Services Science
Complex Relationships
Cause-Effects & Knowledge Discovery
VOLCANO
LOCAT...
Knowledge Enabled Information and Services Science
Knowledge Discovery - Example
Earthquake Sources Nuclear Test Sources
N...
Knowledge Enabled Information and Services Science
Complex Relationships – Several
Challenges
–Probabilistic relations
Num...
Knowledge Enabled Information and Services Science
Complex Relationships …
For each group of earthquakes with magnitudes i...
Knowledge Enabled Information and Services Science
Inter-Ontological Relationships
A nuclear test could have caused an ear...
Knowledge Enabled Information and Services Science
Entity, Relationship, Event Extraction
and Semantic Annotation
From con...
Knowledge Enabled Information and Services Science
© Semagix, Inc.
Semagix Freedom for building
ontology-driven informatio...
Knowledge Enabled Information and Services Science
WWW, Enterprise
Repositories
METADATA
EXTRACTORS
Digital Maps
Nexis
UPI...
Knowledge Enabled Information and Services Science
Automatic Semantic Metadata
Extraction/Annotation
Knowledge Enabled Information and Services Science
Limited tagging
(mostly syntactic)
COMTEX Tagging
Content
‘Enhancement’...
Knowledge Enabled Information and Services Science
Enabling powerful linking
of actionable information
and facilitating im...
Knowledge Enabled Information and Services Science
Video with
Editorialized
Text on the
WebAuto
Categorization
Semantic Me...
Knowledge Enabled Information and Services Science
Extraction
Agent
Enhanced Metadata AssetWeb Page
Ontology-Directed Meta...
Knowledge Enabled Information and Services Science
Some real-world or commercial
semantic applications
Knowledge Enabled Information and Services Science
VisualHarness Interface (1998):
Keyword, Attribute and Content Based Ac...
Knowledge Enabled Information and Services Science
Taalee’s Semantic (Facted) Search (1999-2001):
Highly customizable, pre...
Knowledge Enabled Information and Services Science
SemanticEnrichment
Whatelsecanacontextdo?
Knowledge Enabled Information and Services Science
CreatingaWebof
relatedinformation
Whatcanacontextdo?
(acommercialperspe...
Knowledge Enabled Information and Services Science
Whatelsecanacontextdo?
(acommercialperspective)
SemanticEnrichment
Sema...
Knowledge Enabled Information and Services Science
BLENDED BROWSING & QUERYING INTERFACE
ATTRIBUTE & KEYWORD
QUERYING
unif...
Knowledge Enabled Information and Services Science
Semantic/Interactive Targeting (1999-2001)
Buy Al Pacino Videos
Buy Rus...
Knowledge Enabled Information and Services Science
Keyword, Attribute
and Content Based Access
Blended Semantic Browsing a...
Knowledge Enabled Information and Services Science
Search for
company
‘Commerce One’
Links to news on companies
that compe...
Knowledge Enabled Information and Services Science
System recognizes ENTITY & CATEGORY
Relevant portion
of the Directory i...
Knowledge Enabled Information and Services Science
Users can explore
Semantically related
Information.
Semantic Directory
Knowledge Enabled Information and Services Science
Semantic Relationships
Knowledge Enabled Information and Services Science
Focused relevant
content
organized
by topic
(semantic
categorization)
A...
Knowledge Enabled Information and Services Science
Watch list Organizatio
n
Company
Hamas
WorldCom
FBI Watchlist
Ahmed
Yas...
Knowledge Enabled Information and Services Science
Global Investment Bank
Example of Fraud
prevention application
used in ...
Knowledge Enabled Information and Services Science
demostration
Knowledge Enabled Information and Services Science
How Background Knowledge helps
Informal Content Analysis
Knowledge Enabled Information and Services Science
A Community’s Pulse is often Informal
•Wealth of information available ...
Knowledge Enabled Information and Services Science
Background Knowledge Improves Content
Analysis
 Metadata creation
 Ex...
Knowledge Enabled Information and Services Science
Results - Pulse of a Music Community
•Mining artist popularity from cha...
Knowledge Enabled Information and Services Science
830.9570 194.9604 2
580.2985 0.3592
688.3214 0.2526
779.4759 38.4939
78...
Knowledge Enabled Information and Services Science 70
Semantic Sensor Web
71
Semantically Annotated O&M
<swe:component name="time">
<swe:Time definition="urn:ogc:def:phenomenon:time" uom="urn:ogc:...
Knowledge Enabled Information and Services Science 72
Person
Company
Coordinates
Coordinate System
Time Units
Timezone
Spa...
Knowledge Enabled Information and Services Science 73
Semantic Query
Semantic Temporal Query
Model-references from SML to ...
Knowledge Enabled Information and Services Science
demo of Semantic Sensor Web
http://wiki.knoesis.org/index.php/SSW
Knowledge Enabled Information and Services Science
Relationship/Fact Extraction from Text
Knowledge Engineering approach
–...
Knowledge Enabled Information and Services Science
Relationship/Fact Extraction from Text
Machine learning approaches
–Sup...
Knowledge Enabled Information and Services Science
Schema-Driven Extraction of
Relationships from Biomedical Text
Cartic R...
Knowledge Enabled Information and Services Science
Method – Parse Sentences
in PubMed
SS-Tagger (University of Tokyo)
SS-P...
Knowledge Enabled Information and Services Science
Method – Identify entities and
Relationships in Parse Tree
TOP
NP
VP
S
...
Knowledge Enabled Information and Services Science
Resulting Semantic Web Data in RDF
Modifiers
Modified entities
Composit...
Knowledge Enabled Information and Services Science
Blazing Semantic Trails in Biomedical
Literature
Cartic Ramakrishnan, E...
Knowledge Enabled Information and Services Science
Relationships -- Blazing the Trails
“The physician, puzzled by her pati...
Knowledge Enabled Information and Services Science
Semantic Browser
Knowledge Enabled Information and Services Science
demo of Semantic Browser
http://knoesis.wright.edu/library/demos
Knowledge Enabled Information and Services Science
Original Documents
PMID-10037099
PMID-15886201
Knowledge Enabled Information and Services Science
Semantic Trail
Knowledge Enabled Information and Services Science
“Everything's connected, all along the line.
Cause and effect.
That's t...
How are Harry Potter and
Dan Brown related?
Leonardo Da Vinci
The Da Vinci code
The Louvre
Victor Hugo
The Vitruvian man
S...
Knowledge Enabled Information and Services Science
Semantic Trails Over All Types of Data
Semantic Trails can be built ove...
Knowledge Enabled Information and Services Science
Discovering Complex Connection Patterns
Discovering informative subgrap...
Knowledge Enabled Information and Services Science
Discovering Complex Connection Patterns
We defined an interestingness m...
Knowledge Enabled Information and Services Science
Heuristics
Two factor influencing interestingness
Knowledge Enabled Information and Services Science
Discovery Algorithm
• Bidirectional lock-step growth from S and T
• Cho...
Knowledge Enabled Information and Services Science
Discovery Algorithm
Model the Candidate graph as an electrical circuit
...
Knowledge Enabled Information and Services Science
Discovery Algorithm
Results
–Arnold Schwarzenegger, Edward Kennedy
Othe...
Knowledge Enabled Information and Services Science
Results
Knowledge Enabled Information and Services Science
Hypothesis Driven Retrieval Of Scientific Text
Knowledge Enabled Information and Services Science
Spatial
Temporal
Thematic
Events: 3 Dimensions – Spatial, Temporal
and ...
Knowledge Enabled Information and Services Science
Events and STT dimensions
Powerful mechanism to integrate content
–Desc...
Knowledge Enabled Information and Services Science
Events and STT dimensions
Many relationship types
Spatial:
–What events...
Knowledge Enabled Information and Services Science
Events and STT dimensions
Going further
Can we use
–What? Where? When? ...
Knowledge Enabled Information and Services Science 102
High-level Sensor (S-H)
Low-level Sensor (S-L)
A-H E-H
A-L E-L
H L
...
Knowledge Enabled Information and Services Science 103
Sensor Data Pyramid
Raw Sensor (Phenomenological) Data
Feature Meta...
Knowledge Enabled Information and Services Science 104
Collection
Feature Extraction
Entity Detection
RDF
KB
Semantic Anal...
Knowledge Enabled Information and Services Science
Current Research Towards STT
Relationship Analysis
Modeling Spatial and...
Knowledge Enabled Information and Services Science
Current Research Towards STT
Relationship Analysis
Graph Pattern querie...
Knowledge Enabled Information and Services Science
Upper-Level Ontology Modeling Theme
and Space
Occurrent
Continuant
Name...
Knowledge Enabled Information and Services Science
Occurrent
Continuant
Named_Place
Spatial_Occurrent
Dynamic_Entity
Perso...
Knowledge Enabled Information and Services Science
Temporal RDF Graph: Platoon Membership
E1:Soldier
E3:Platoon E5:Soldier...
Knowledge Enabled Information and Services Science
ORDBMS Implementation: DB Structures
Unlike thematic relationships whic...
Knowledge Enabled Information and Services Science
Sample STT Query
Scenario (Biochemical Threat Detection): Analysts must...
Knowledge Enabled Information and Services Science
Going places …
Formal
Social,
Informal
Implicit
Powerful
Knowledge Enabled Information and Services Science
looking into my crystal ball
Knowledge Enabled Information and Services Science
TECHNOLOGY THAT FITS RIGHT IN
“The most profound technologies are those...
Knowledge Enabled Information and Services Science
imagine
Knowledge Enabled Information and Services Science
imagine when
Knowledge Enabled Information and Services Science
meets
Farm Helper
Knowledge Enabled Information and Services Science
with this
Latitude: 38° 57’36” N
Longitude: 95° 15’12” W
Date: 10-9-200...
Knowledge Enabled Information and Services Science
that is sent to
Geocoder
Farm Helper
Services Resource
Sensor Data Reso...
Knowledge Enabled Information and Services Science
and
Knowledge Enabled Information and Services Science
Six billion brains
Knowledge Enabled Information and Services Science
imagination today
Knowledge Enabled Information and Services Science
impacts our experience tomorrow
Knowledge Enabled Information and Services Science
Knowledge Enabled Information and Services Science
Computing For Human Experience
Prof. Amit P. Sheth,
LexisNexis Eminent ...
Upcoming SlideShare
Loading in …5
×

Relationship Web: Trailblazing, Analytics and Computing for Human Experience

2,901 views

Published on

Amit Sheth, "Relationship Web: Trailblazing, Analytics and Computing for Human Experience," Keynote talk at 27th International Conference on Conceptual Modeling (ER 2008) Barcelona, October 20-23 2008.

See associated discussion at:
http://knoesis.org/amit/publications/index.php?page=9
http://knoesis.org/library/resource.php?id=00190

Published in: Education
1 Comment
1 Like
Statistics
Notes
No Downloads
Views
Total views
2,901
On SlideShare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
95
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide
  • phrase “he beat Randy Johnson” in the first paragraph, after being passed through a relationship extraction [12] and pronominal resolution [14] engine, can be converted into the triple Dontrelle WillisbeatRandy Johnson. The notions of Model and Model Theoretic Semantics: Expressions in a formal language are interpreted in models. The structure common to all models in which a given language is interpreted (the model structure for the model-theoretic interpretation of the given language) reflects certain basic presuppositions about the “structure of the world” that are implicit in the language. The Principle of Compositionality: The meaning of an expression is a function of the meanings of its parts and the way they are syntactically combined. In other words, the semantics of an expression are computed using the semantics of its parts, obtained using an interpretation function.
  • phrase “he beat Randy Johnson” in the first paragraph, after being passed through a relationship extraction [12] and pronominal resolution [14] engine, can be converted into the triple Dontrelle WillisbeatRandy Johnson. The notions of Model and Model Theoretic Semantics: Expressions in a formal language are interpreted in models. The structure common to all models in which a given language is interpreted (the model structure for the model-theoretic interpretation of the given language) reflects certain basic presuppositions about the “structure of the world” that are implicit in the language. The Principle of Compositionality: The meaning of an expression is a function of the meanings of its parts and the way they are syntactically combined. In other words, the semantics of an expression are computed using the semantics of its parts, obtained using an interpretation function.
  • What is deep web ?Example for well formed text ?
  • Animation needs work
  • Animation needed
  • Going from syntactic to semantic metadata – Semantic Sensor MLInsert model references into tagsDomain Ontology e.g. people, companiesSpatial Ontology e.g. coordinate systems and coordinatesTemporal Ontology e.g. time units, time zones
  • Example Scenario – Sensor WorkDifferent sensors capture different aspects of the same entities and eventsChallenges for entity identification and resolution – being worked on by our colleaguesAfter these steps – more detailed STT analysis e.g. is a vehicle a threat?
  • Realizing this scenario – Different types of data/metadata needed
  • Overall big picture.SML + ontologies  large RDF KBNow can do STT analytics over this large graph semantically integrating the content
  • Weiser’s human-centered vision of ubiquitous computing… new mode ofhuman–computer interaction.
  • Relationship Web: Trailblazing, Analytics and Computing for Human Experience

    1. 1. Knowledge Enabled Information and Services Science Relationship Web: Trailblazing , Analytics and Computing for Human Experience 27th International Conference on Conceptual Modeling (ER 2008) Barcelona, Oct 20-23 2008 Amit Sheth Kno.e.sis Center, Wright State University, Dayton, OH This talk also represents work of several members of Kno.e.sis team, esp. the Semantic Discovery and Semantic Sensor Data. http://knoesis.wright.edu Thanks, K. Godam, C. Henson, M. Perry, C. Ramakrishnan, C. Thomas. Also thanks to sponsors: NSF (SemDis and STT), NIH, AFRL, IBM, HP, Microsoft.
    2. 2. Knowledge Enabled Information and Services Science Evolution of the Web 2005 1990s Web of databases - dynamically generated pages - web query interfaces Web of pages - text, manually created links - extensive navigation Web of services - data = service = data, mashups - ubiquitous computing Web of people - social networks, user-created content - GeneRIF, Connotea Web as an oracle / assistant / partner - “ask to the Web” - using semantics to leverage text + data + services + people 2010 Computing for Human Experience
    3. 3. Knowledge Enabled Information and Services Science A Compelling Vision: Trailblazing “(Human mind works) by association. With one item in its grasp, it snaps instantly to the next that is suggested by the association of thoughts, in accordance with some intricate web of trails carried by the cells of the brain.“ Dr. Vannevar Bush, As We May Think, 1945
    4. 4. Knowledge Enabled Information and Services Science Keywords, Documents, Objects, Events, Insight, Experience “An object by itself is intensely uninteresting”. – Grady Booch, Object Oriented Design with Applications, 1991 Keywords | Search (data) Entities | Integration (information) Relationships, Events | Analysis, Insight (knowledge)
    5. 5. Knowledge Enabled Information and Services Science Semantics and Relationships on a Web scale Increasing depth and sophistication in dealing with semantics –From searching to integration to analysis, insight, discovery and decision making –Relationships • Heart of semantics • Allows to continue progress from syntax and structure to semantics A. Sheth, et al. Relationships at the Heart of Semantic Web: Modeling, Discovering, and Exploiting Complex Semantic Relationships, in Enhancing the Power of the Internet (Studies in Fuzziness and Soft Computing, V. 139). SpringerVerlag., 2004. http://knoesis.wright.edu/library/resource.php?id=00190
    6. 6. Knowledge Enabled Information and Services Science Relationship Web takes you away from “which document” could have data/information I need, to interconnecting Web of information embedded in the resources to give knowledge, insights and answers I seek. Amit P. Sheth, Cartic Ramakrishnan: Relationship Web: Blazing Semantic Trails between Web Resources. IEEE Internet Computing July 2007.
    7. 7. Knowledge Enabled Information and Services Science Structured text (biomedical literature) Informal Text (Social Network chatter) Multimedia Content and Web data Web Services Metadata Extraction Patterns / Inference / Reasoning Domain Models Meta data / Semantic Annotations Relationship Web Search Integration Analysis Discovery Question Answering
    8. 8. Knowledge Enabled Information and Services Science Issues - Relationships Understanding and modeling relationships Identifying Relationship (extraction) Discovering and Exploring Relationships (reasoning) Hypothesizing and Validating Complex Relationships Using/exploiting Relationships for Semantic Applications (in search, querying, analysis, insight, discovery)
    9. 9. Knowledge Enabled Information and Services Science Data, Information, and Insight What: Which thing or which particular one Who: What or which person or persons Where: At or in what place When: At what time How: In what manner or way; by what means Why: For what purpose, reason, or cause; with what intention, justification, or motive © Ramesh Jain
    10. 10. Knowledge Enabled Information and Services Science EventWeb [Jain] RelationshipWeb [Sheth] A Web in which each node is an event or object, and connected to other nodes using –Linguistic Relationships –Referential Links: hypertext links to related information (HREF); links with metadata (MREF), links referring to model (model reference in SAWSDL, SA-REST) –Structural Links: showing spatial and temporal relationships –Causal Links: establishing causality –Relational Links: giving similarity or any other relationship Adapted/extended from Ramesh Jain
    11. 11. Knowledge Enabled Information and Services Science Supporting Relationships in the Real World Objects and event represent or model real world – More complex relationships • Living organisms: Saprotropism, Antagonism, Exploitation, Predation, Ammensalism or Antibiosis, Symbiosis • Relationships between humans: …
    12. 12. Knowledge Enabled Information and Services Science Karthik Gomadam Amit Sheth Attended Google IO Moscone Center, SFO May 28-29, 2008 Is advised by Ph.D Student Researcher is_advised_by Assistant Professor Professor Research Paper Journal Conference Location published_in published_in has_location Image Metadata Event Causal kno.e.sis DirectsRelational
    13. 13. Knowledge Enabled Information and Services Science Identifying & Representing Relationships Implicit Relationships –Statistical representation of interactions between entities, co-occurrence of terms in the same cluster, tag cloud... Linking of one document to another via a hyperlink… –Two documents’ belonging to categories that are siblings in a concept hierarchy.
    14. 14. Knowledge Enabled Information and Services Science Identifying & Representing Relationships Explicit Linguistic Relationships “He beat Randy Johnson” converted into the triple Dontrelle WillisbeatRandy Johnson
    15. 15. Knowledge Enabled Information and Services Science Identifying & Representing Relationships Formal Relationships –Subsumption, partonomy, …. Domain specific vs domain-independent relationships –Example of domain independent relationships: time, space/location Complex entity and relationships (example later)
    16. 16. Knowledge Enabled Information and Services Science Why is This a Hard Problem? Are objects/entities equivalent/equal(same)? How (well) are they related? –Implicit vs explicit: • Statistical representation of interactions between entities, co- occurrence of terms in the same cluster, tag cloud... • formal/assertional vs social consensus based • powerful (beyond FOL): partial, probilistic and fuzzy match –Degrees of relatedness and relevance: semantic similarity, semantic proximity, semantic distance, … • [differentiation, disjointedness] • related in a “context”
    17. 17. Knowledge Enabled Information and Services Science Why is This a Hard Problem? Semantic ambiguity When information (knowledge) is –Incomplete – inconsistent –Approximate –Even is-a link involves different notions: identify, unity, essense (Guarino and Wetley 2002) A.Sheth, et al, “Semantics for The Semantic Web: the Implicit, the Formal and the Powerful,” International Journal on Semantic Web & Information Systems, 1 (no. 1), 2005, pp. 1–18.
    18. 18. Knowledge Enabled Information and Services Science Faceted Search and Semantic Analytics – Some Early Work where relationships played key role InfoHarness/VisualHarness (1993-1998) http://www.knoesis.org/library/resource.php?id=00275 http://www.knoesis.org/library/resource.php?id=00245 http://knoesis.wright.edu/library/resource.php?id=00267 InfoQuilt (1996-2000) http://www.knoesis.org/library/resource.php?id=00178 MREF (1996-1998) OBSERVER (1996-2000) http://www.knoesis.org/library/resource.php?id=00273 http://www.knoesis.org/library/resource.php?id=00093 Taalee Semantic (Faceted) Search, Semantic Directory and Semagix Freedom enabled analytics (1999-2006) http://www.knoesis.org/library/resource.php?id=00193 http://knoesis.wright.edu/library/resource.php?id=00149
    19. 19. Knowledge Enabled Information and Services Science InfoQuilt (1996-2000): Using metadata PatchQuilt and user models/ontologies to support queries and analytics over globally distributed heterogeneous media repositories
    20. 20. Knowledge Enabled Information and Services Science Physical Link to Relationship <TITLE> A Scenic Sunset at Lake Tahoe </TITLE> <p> Lake Tahoe is a popular tourist spot and <A HREF = “http://www1.server.edu/lake_tahoe.txt”> some interesting facts</A> are available here. The scenic beauty of Lake Tahoe can be viewed in this photograph: <center> <IMG SRC=“http://www2.server.edu/lake_tahoe.img”> </center> Traditionally, correlation is achieved by using physical links Done manually by user publishing the HTML document
    21. 21. Knowledge Enabled Information and Services Science MREF (1996-1998) Metadata Reference Link -- complementing HREF Creating “logical web” through Media independent metadata based on correlation http://www.knoesis.org/library/resource.php?id=00274 http://knoesis.org/library/resource.php?id=00294
    22. 22. Knowledge Enabled Information and Services Science Metadata Reference Link (<A MREF …>) <A HREF=“URL”>Document Description</A> physical link between document (components) <A MREF KEYWORDS=<list-of-keywords>; THRESH=<real>>Document Description</A> <A MREF ATTRIBUTES(<list-of-attribute-value- pairs>)>Document Description</A>
    23. 23. Knowledge Enabled Information and Services Science Correlation Based on Content-Based Metadata Some interesting information on dams is available here “information on dams” defined by MREF to keywords and metadata (may be used for a query) height, width and size water.gif (Data) Metadata Storage water.gif ……mpeg ……ppm Major component(RGB) Blue Content based Metadata Content Dependent Metadata
    24. 24. Knowledge Enabled Information and Services Science Abstraction Layers METADATA DATA METADATA DATA ONTOLOGY NAMESPACE ONTOLOGY NAMESPACE MREF in RDF
    25. 25. Knowledge Enabled Information and Services Science MREF (1998) Model for Logical Correlation using Ontological Terms and Metadata Framework for Representing MREFs Serialization (one implementation choice) K. Shah and A. Sheth, "Logical Information Modeling of Web-accessible Heterogeneous Digital Assets", Proc. of the Forum on Research and Technology Advances in Digital Libraries," (ADL'98), Santa Barbara, CA, May 28-30, 1998, pp. 266-275. M R E F R D F X M L Figure 3: XML, RDF, and MREF
    26. 26. Knowledge Enabled Information and Services Science An Example RDF Model for MREF (1998) <?namespace href="http://www.foo.com/IQ" as="IQ"?> <?namespace href="http://www.w3.org/schemas/rdf-schema" as="RDF"?> <RDF:serialization> <RDF:bag id="MREF:12345> <IQ:keyword> <RDF:resource id="constraint_001"> <IQ:threshold>0.5</IQ:threshold> <RDF:PropValue>dam</RDF:PropValue> </RDF:resource> </IQ:keyword> <IQ:attribute> <RDF:resource id="constraint_002"> <IQ:name>majorRGB</IQ:color> <IQ:type>string</IQ:type> <RDF:PropValue>blue</RDF:PropValue> </RDF:resource> </IQ:attribute> </RDF:bag> </RDF:serialization>
    27. 27. Knowledge Enabled Information and Services Science Domain Specific Correlation Potential locations for a future shopping mall identified by all regions having a population greater than 500 and area greater than 50 sq meters having an urban land cover and moderate relief <A MREF ATTRIBUTES(population < 500; area < 50 & region-type = ‘block’ & land-cover = ‘urban’ & relief = ‘moderate’)>can be viewed here</A>
    28. 28. Knowledge Enabled Information and Services Science Domain Specific Correlation => media-independent relationships between domain specific metadata: population, area, land cover, relief => correlation between image and structured data at a higher domain specific level as opposed to physical “link-chasing” in the WWW
    29. 29. Knowledge Enabled Information and Services Science TIGER/Line DB Population: Area: Boundaries: Land cover: Relief: Census DB Map DB Regions (SQL) Boundaries Image Features (IP routines) Repositories and the Media Types
    30. 30. Knowledge Enabled Information and Services Science
    31. 31. Knowledge Enabled Information and Services Science Complex Relationships Some relationships may not be manually asserted, but according to statistical analyses of text, experimental data, etc. – allow association of provenance data with classes, instances, relationship types and direct relationships or statements
    32. 32. Knowledge Enabled Information and Services Science Complex Relationships Relationships (mappings) are not always simple mathematical / string transformations Examples of complex relationships –Associations / paths between classes –Graph based / form fitting functions –Probabilistic relational E 1 :Reviewer E 6:Person E 5 :Person E 2:Paper E4 :Paper E 7 :Submission E 3 :Person author _of author _of author _of author _of author _of knows knows
    33. 33. Knowledge Enabled Information and Services Science A Simple Relationship ? Smoking CancerCauses Graph based / form fitting functions
    34. 34. Knowledge Enabled Information and Services Science Complex Relationships Cause-Effects & Knowledge Discovery VOLCANO LOCATION ASH RAIN PYROCLASTIC FLOW ENVIRON. LOCATION PEOPLE WEATHER PLANT BUILDING DESTROYS COOLS TEMP DESTROYS KILLS
    35. 35. Knowledge Enabled Information and Services Science Knowledge Discovery - Example Earthquake Sources Nuclear Test Sources Nuclear Test May Cause Earthquakes Is it really true? Complex Relationship: How do you model this?
    36. 36. Knowledge Enabled Information and Services Science Complex Relationships – Several Challenges –Probabilistic relations Number of earthquakes with magnitude > 7 almost constant. So if at all, then nuclear tests only cause earthquakes with magnitude < 7
    37. 37. Knowledge Enabled Information and Services Science Complex Relationships … For each group of earthquakes with magnitudes in the ranges 5.8-6, 6-7, 7-8, 8-9, and >9 on the Richter scale per year starting from 1900, find number of earthquakes Number of earthquakes with magnitude > 7 almost constant. So nuclear tests probably only cause earthquakes with magnitude < 7
    38. 38. Knowledge Enabled Information and Services Science Inter-Ontological Relationships A nuclear test could have caused an earthquake if the earthquake occurred some time after the nuclear test was conducted and in a nearby region. NuclearTest Causes Earthquake <= dateDifference( NuclearTest.eventDate, Earthquake.eventDate ) < 30 AND distance( NuclearTest.latitude, NuclearTest.longitude, Earthquake,latitude, Earthquake.longitude ) < 10000
    39. 39. Knowledge Enabled Information and Services Science Entity, Relationship, Event Extraction and Semantic Annotation From content with structure –Web pages –Deep Web From well-formed text (edited for rules of grammar) From informal or casual text –social networking sites From digital media
    40. 40. Knowledge Enabled Information and Services Science © Semagix, Inc. Semagix Freedom for building ontology-driven information system Extracting Semantic Metadata from Semistructured and Structured Sources
    41. 41. Knowledge Enabled Information and Services Science WWW, Enterprise Repositories METADATA EXTRACTORS Digital Maps Nexis UPI AP Feeds/ Documents Digital Audios Data Stores Digital Videos Digital Images . . . . . . . . . Create/extract as much (semantics) metadata automatically as possible; Use ontlogies to improve and enhance extraction Information Extraction for Metadata Creation
    42. 42. Knowledge Enabled Information and Services Science Automatic Semantic Metadata Extraction/Annotation
    43. 43. Knowledge Enabled Information and Services Science Limited tagging (mostly syntactic) COMTEX Tagging Content ‘Enhancement’ Rich Semantic Metatagging Value-added Voquette Semantic Tagging Value-added relevant metatags added by Voquette to existing COMTEX tags: • Private companies • Type of company • Industry affiliation • Sector • Exchange • Company Execs • Competitors Semantic Annotation (Extraction + Enhancement)
    44. 44. Knowledge Enabled Information and Services Science Enabling powerful linking of actionable information and facilitating important semantic applications such as knowledge discovery and link analysis (user’s task of manually retrieving all the information he needs to know is greatly minimized; he can spend more time making effective decisions) Semantic Metadata Content Tags Company: Cisco Systems, Inc. Classification: Channel Partners, E-Business Solutions Channel Partner: Siemens Network Channel Partner: Voyager Network Channel Partner: Siemens Network Channel Partner: Wipro Group E-Business Solution: CIS-1270 Security E-Business Solution: CIS-320 Learning E-Business Solution: CIS-6250 Finance E-Business Solution: CIS-1005 e-Market Ticker: CSCO Industry: Telecommunication, . . . Sector: Computer Hardware Executive: John Chambers Competition: Nortel Networks Syntactic Metadata Producer: BusinessWire Source: Bloomberg Date: Sept. 10 2001 Location: San Jose, CA URL: http://bloomberg.com/1.htm Media: Text XML content item with enriched semantic tagging, ready to be queried E-Business SolutionOntology Cisco Systems Voyager Network Siemens Network Wipro Group Ulysys Group CIS-1270 Security CIS-320 Learning CIS-6250 Finance CIS-1005 e-Market Channel Partner belongs to - - - Ticker representedby - - - - - - - - - - - - Industry channelpartnerof - - - - - - - - - - - - Competition competes with provider of - - - - - - - - - - - - Executives w orks for - - - - - - - - - - - - Sectorbelongsto Semantic Enhancement Uniquely exploiting real-world semantic associations in the right context Semantic Metadata Extraction (also syntactic) Content Tags Semantic Metadata Classification: Channel Partners, E-Business Solutions Company: Cisco Systems, Inc. Syntactic Metadata Producer: BusinessWire Source: Bloomberg Date: Sept. 10 2001 Location: San Jose, CA URL: http://bloomberg.com/1.htm Media: Text Channel Partners E-Business SolutionsClassification Content Tags Semantic Metadata Classification: Channel Partners, E-Business Solutions Classification Committee Knowledge-base, Machine Learning & Statistical Techniques Semantic Metadata Enhancement
    45. 45. Knowledge Enabled Information and Services Science Video with Editorialized Text on the WebAuto Categorization Semantic Metadata Automatic Classification & Metadata Extraction (Web Page)
    46. 46. Knowledge Enabled Information and Services Science Extraction Agent Enhanced Metadata AssetWeb Page Ontology-Directed Metadata Extraction (Semi-Structured Data)
    47. 47. Knowledge Enabled Information and Services Science Some real-world or commercial semantic applications
    48. 48. Knowledge Enabled Information and Services Science VisualHarness Interface (1998): Keyword, Attribute and Content Based Access
    49. 49. Knowledge Enabled Information and Services Science Taalee’s Semantic (Facted) Search (1999-2001): Highly customizable, precise and fresh A/V search Delightful, relevant information, exceptional targeting opportunity
    50. 50. Knowledge Enabled Information and Services Science SemanticEnrichment Whatelsecanacontextdo?
    51. 51. Knowledge Enabled Information and Services Science CreatingaWebof relatedinformation Whatcanacontextdo? (acommercialperspective)
    52. 52. Knowledge Enabled Information and Services Science Whatelsecanacontextdo? (acommercialperspective) SemanticEnrichment Semantic Targeting
    53. 53. Knowledge Enabled Information and Services Science BLENDED BROWSING & QUERYING INTERFACE ATTRIBUTE & KEYWORD QUERYING uniform view of worldwide distributed assets of similar type SEMANTIC BROWSING Targeted e-shopping/e-commerce assets access VideoAnywhere and Taalee Semantic Querying and Browsing (1998-2001)
    54. 54. Knowledge Enabled Information and Services Science Semantic/Interactive Targeting (1999-2001) Buy Al Pacino Videos Buy Russell Crowe Videos Buy Christopher Plummer Videos Buy Diane Venora Videos Buy Philip Baker Hall Videos Buy The Insider Video Precisely targeted through the use of Structured Metadata and integration from multiple sources
    55. 55. Knowledge Enabled Information and Services Science Keyword, Attribute and Content Based Access Blended Semantic Browsing and Querying (Intelligence Analyst Workbench) 2001-2003
    56. 56. Knowledge Enabled Information and Services Science Search for company ‘Commerce One’ Links to news on companies that compete against Commerce One Links to news on companies Commerce One competes against (To view news on Ariba, click on the link for Ariba) Crucial news on Commerce One’s competitors (Ariba) can be accessed easily and automatically Semantic Browsing/Directory (2001-….)
    57. 57. Knowledge Enabled Information and Services Science System recognizes ENTITY & CATEGORY Relevant portion of the Directory is automatically presented. Semantic Directory
    58. 58. Knowledge Enabled Information and Services Science Users can explore Semantically related Information. Semantic Directory
    59. 59. Knowledge Enabled Information and Services Science Semantic Relationships
    60. 60. Knowledge Enabled Information and Services Science Focused relevant content organized by topic (semantic categorization) Automatic Content Aggregation from multiple content providers and feeds Related relevant content not explicitly asked for (semantic associations) Competitive research inferred automatically Automatic 3rd party content integration Semantic Application Example – Research Dashboard (Voquette/Semagix: 2001-2004)
    61. 61. Knowledge Enabled Information and Services Science Watch list Organizatio n Company Hamas WorldCom FBI Watchlist Ahmed Yaseer appears on Watchlist member of organization works for Company Ahmed Yaseer: • Appears on Watchlist ‘FBI’ • Works for Company ‘WorldCom’ • Member of organization ‘Hamas’ Early Semantic Association: An Application in Risk & Compliance (Semagix 2004-2006)
    62. 62. Knowledge Enabled Information and Services Science Global Investment Bank Example of Fraud prevention application used in financial services User will be able to navigate the ontology using a number of different interfaces World Wide Web content Public Records BLOGS, RSS Un-structure text, Semi-structured Data Watch Lists Law Enforcement Regulators Semi-structured Government Data Scores the entity based on the content and entity relationships Establishing New Account
    63. 63. Knowledge Enabled Information and Services Science demostration
    64. 64. Knowledge Enabled Information and Services Science How Background Knowledge helps Informal Content Analysis
    65. 65. Knowledge Enabled Information and Services Science A Community’s Pulse is often Informal •Wealth of information available in blogs, social networks, chats etc. •Free medium of self-expression makes mass opinions / interests available •Polling for popular culture opinions is easier •Social Production undeniably affects markets • geo-specific retail ads, demographic interests in music
    66. 66. Knowledge Enabled Information and Services Science Background Knowledge Improves Content Analysis  Metadata creation  Example – music comments  Spot Artist, Track names and associated sentiments  Example comment  “Keep your smile on Lil.”  Smile here is a track from Artist Lilly Allen’s album  Background knowledge from Music Brainz taxonomy provides evidence  Annotate ‘smile’ as Track  ‘Lil’ as Lilly Allen  Background knowledge from Urban Dictionary for understanding Slang  I say: “Your music is wicked”  What I really mean: “Your music is good”
    67. 67. Knowledge Enabled Information and Services Science Results - Pulse of a Music Community •Mining artist popularity from chatter on MySpace - Lists close to listeners preferences vs. Bill Boards BB User Comments: May 07 User Comments: Jun 07 Rihanna Biffy Clyro Twang Maroon 5 McCartney Winehouse Rascal Rihanna Winehouse Maroon 5 Mccartney Biffy Clyro Twang Rascal Rihanna Winehouse Maroon 5 Mccartney Biffy Clyro Rascal Twang
    68. 68. Knowledge Enabled Information and Services Science 830.9570 194.9604 2 580.2985 0.3592 688.3214 0.2526 779.4759 38.4939 784.3607 21.7736 1543.7476 1.3822 1544.7595 2.9977 1562.8113 37.4790 1660.7776 476.5043 parent ion m/z fragment ion m/z ms/ms peaklist data fragment ion abundance parent ion abundance parent ion charge Mass Spectrometry (MS) Data Semantic Extraction/Annotation of Experimental Data
    69. 69. Knowledge Enabled Information and Services Science 70 Semantic Sensor Web
    70. 70. 71 Semantically Annotated O&M <swe:component name="time"> <swe:Time definition="urn:ogc:def:phenomenon:time" uom="urn:ogc:def:unit:date-time"> <sa:swe rdfa:about="?time" rdfa:instanceof="time:Instant"> <sa:sml rdfa:property="xs:date-time"/> </sa:swe> </swe:Time> </swe:component> <swe:component name="measured_air_temperature"> <swe:Quantity definition="urn:ogc:def:phenomenon:temperature“ uom="urn:ogc:def:unit:fahrenheit"> <sa:swe rdfa:about="?measured_air_temperature“ rdfa:instanceof=“senso:TemperatureObservation"> <sa:swe rdfa:property="weather:fahrenheit"/> <sa:swe rdfa:rel="senso:occurred_when" resource="?time"/> <sa:swe rdfa:rel="senso:observed_by" resource="senso:buckeye_sensor"/> </sa:sml> </swe:Quantity> </swe:component> <swe:value name=“weather-data"> 2008-03-08T05:00:00,29.1 </swe:value> Semantic Sensor Web @ Kno.e.sis
    71. 71. Knowledge Enabled Information and Services Science 72 Person Company Coordinates Coordinate System Time Units Timezone Spatial Ontology Domain Ontology Temporal Ontology Mike Botts, "SensorML and Sensor Web Enablement," Earth System Science Center, UAB Huntsville Semantic Sensor ML – Adding Ontological Metadata
    72. 72. Knowledge Enabled Information and Services Science 73 Semantic Query Semantic Temporal Query Model-references from SML to OWL-Time ontology concepts provides the ability to perform semantic temporal queries Supported semantic query operators include: – contains: user-specified interval falls wholly within a sensor reading interval (also called inside) – within: sensor reading interval falls wholly within the user-specified interval (inverse of contains or inside) – overlaps: user-specified interval overlaps the sensor reading interval Example SPARQL query defining the temporal operator ‘within’
    73. 73. Knowledge Enabled Information and Services Science demo of Semantic Sensor Web http://wiki.knoesis.org/index.php/SSW
    74. 74. Knowledge Enabled Information and Services Science Relationship/Fact Extraction from Text Knowledge Engineering approach –Manually crafted rules • Over lexical items <person> works for <organization> • Over syntactic structures – parse trees –GATE
    75. 75. Knowledge Enabled Information and Services Science Relationship/Fact Extraction from Text Machine learning approaches –Supervised –Semi-supervised –Unsupervised
    76. 76. Knowledge Enabled Information and Services Science Schema-Driven Extraction of Relationships from Biomedical Text Cartic Ramakrishnan, Krys Kochut, Amit P. Sheth: A Framework for Schema- Driven Relationship Discovery from Unstructured Text. International Semantic Web Conference 2006: 583-596 [.pdf]
    77. 77. Knowledge Enabled Information and Services Science Method – Parse Sentences in PubMed SS-Tagger (University of Tokyo) SS-Parser (University of Tokyo) (TOP (S (NP (NP (DT An) (JJ excessive) (ADJP (JJ endogenous) (CC or) (JJ exogenous) ) (NN stimulation) ) (PP (IN by) (NP (NN estrogen) ) ) ) (VP (VBZ induces) (NP (NP (JJ adenomatous) (NN hyperplasia) ) (PP (IN of) (NP (DT the) (NN endometrium) ) ) ) ) ) ) • Entities (MeSH terms) in sentences occur in modified forms • “adenomatous” modifies “hyperplasia” • “An excessive endogenous or exogenous stimulation” modifies “estrogen” • Entities can also occur as composites of 2 or more other entities • “adenomatous hyperplasia” and “endometrium” occur as “adenomatous hyperplasia of the endometrium”
    78. 78. Knowledge Enabled Information and Services Science Method – Identify entities and Relationships in Parse Tree TOP NP VP S NP VBZ induces NP PP NP IN of DT the NN endometrium JJ adenomatous NN hyperplasia NP PP IN by NN estrogenDT the JJ excessive ADJP NN stimulation JJ endogenous JJ exogenous CC or Modifiers Modified entities Composite Entities
    79. 79. Knowledge Enabled Information and Services Science Resulting Semantic Web Data in RDF Modifiers Modified entities Composite Entities estrogen An excessive endogenous or exogenous stimulation modified_entity1 composite_entity1 modified_entity2 adenomatous hyperplasia endometrium hasModifier hasPart induces hasPart hasPart hasModifier hasPart
    80. 80. Knowledge Enabled Information and Services Science Blazing Semantic Trails in Biomedical Literature Cartic Ramakrishnan, Extracting, Representing and Mining Semantic Metadata from Text: Facilitating Knowledge Discovery in Biomedicine, PhD Thesis, Wright State University, August 2008. Amit Sheth and Cartic Ramakrishnan, “Relationship Web: Blazing Semantic Trails between Web Resources,” IEEE Internet Computing, July–August 2007, pp. 84–88.
    81. 81. Knowledge Enabled Information and Services Science Relationships -- Blazing the Trails “The physician, puzzled by her patient's reactions, strikes the trail established in studying an earlier similar case, and runs rapidly through analogous case histories, with side references to the classics for the pertinent anatomy and histology. The chemist, struggling with the synthesis of an organic compound, has all the chemical literature before him in his laboratory, with trails following the analogies of compounds, and side trails to their physical and chemical behavior.” [V. Bush, As We May Think. The Atlantic Monthly, 1945. 176(1): p. 101-108. ]
    82. 82. Knowledge Enabled Information and Services Science Semantic Browser
    83. 83. Knowledge Enabled Information and Services Science demo of Semantic Browser http://knoesis.wright.edu/library/demos
    84. 84. Knowledge Enabled Information and Services Science Original Documents PMID-10037099 PMID-15886201
    85. 85. Knowledge Enabled Information and Services Science Semantic Trail
    86. 86. Knowledge Enabled Information and Services Science “Everything's connected, all along the line. Cause and effect. That's the beauty of it. Our job is to trace the connections and reveal them.” Jack in Terry Gilliam’s 1985 film - “Brazil”
    87. 87. How are Harry Potter and Dan Brown related? Leonardo Da Vinci The Da Vinci code The Louvre Victor Hugo The Vitruvian man Santa Maria delle Grazie Et in Arcadia Ego Holy Blood, Holy Grail Harry Potter The Last Supper Nicolas Poussin Priory of Sion The Hunchback of Notre Dame The Mona Lisa Nicolas Flammel painted_by painted_by painted_by painted_by member_of member_of member_of written_by mentioned_in mentioned_in displayed_at displayed_at cryptic_motto_of displayed_at mentioned_in mentioned_in How are Harry Potter and Dan Brown related?
    88. 88. Knowledge Enabled Information and Services Science Semantic Trails Over All Types of Data Semantic Trails can be built over a Web of Semantic (Meta)Data extracted (manually, semi-automatically and automatically) and gleaned from –Structured data (e.g., NCBI databases) –Semi-structured data (e.g., XML based and semantic metadata standards for domain specific data representations and exchanges) –Unstructured data (e.g., Pubmed and other biomedical literature) and –Various modalities (experimental data, medical images, etc.)
    89. 89. Knowledge Enabled Information and Services Science Discovering Complex Connection Patterns Discovering informative subgraphs Given a pair of end-points (entities) produce a subgraph with relationships connecting them such that the subgraph is small enough to be visualized and contains relevant “interesting” connections Cartic Ramakrishnan, William H. Milnor, Matthew Perry, Amit P. Sheth. (2005). Discovering informative connection subgraphs in multi-relational graphs. SIGKDD Explorations, 7(2), pp. 56-63.
    90. 90. Knowledge Enabled Information and Services Science Discovering Complex Connection Patterns We defined an interestingness measure based on the ontology schema –In future biomedical domain the scientist will control this with the help of a browsable ontology –Our interestingness measure takes into account • Specificity of the relationships and entity classes involved • Rarity of relationships etc. Cartic Ramakrishnan, William H. Milnor, Matthew Perry, Amit P. Sheth. (2005). Discovering informative connection subgraphs in multi-relational graphs. SIGKDD Explorations, 7(2), pp. 56-63.
    91. 91. Knowledge Enabled Information and Services Science Heuristics Two factor influencing interestingness
    92. 92. Knowledge Enabled Information and Services Science Discovery Algorithm • Bidirectional lock-step growth from S and T • Choice of next node based on interestingness measure • Stop when there are enough connections between the frontiers • This is treated as the candidate graph
    93. 93. Knowledge Enabled Information and Services Science Discovery Algorithm Model the Candidate graph as an electrical circuit –S is the source and T the sink –Edge weight derived from the ontology schema are treated as conductance values –Using Ohm’s and Kirchoff’s laws we find maximum current flow paths through the candidate graph from S to T –At each step adding this path to the output graph to be displayed we repeat this process till a certain number of predefined nodes is reached
    94. 94. Knowledge Enabled Information and Services Science Discovery Algorithm Results –Arnold Schwarzenegger, Edward Kennedy Other related work –Semantic Associations
    95. 95. Knowledge Enabled Information and Services Science Results
    96. 96. Knowledge Enabled Information and Services Science Hypothesis Driven Retrieval Of Scientific Text
    97. 97. Knowledge Enabled Information and Services Science Spatial Temporal Thematic Events: 3 Dimensions – Spatial, Temporal and Thematic
    98. 98. Knowledge Enabled Information and Services Science Events and STT dimensions Powerful mechanism to integrate content –Describes the Real-World occurrences –Can have video, images, text, audio all of the same event –Search and Index based on events and STT relations
    99. 99. Knowledge Enabled Information and Services Science Events and STT dimensions Many relationship types Spatial: –What events happened near this event? –What entities/organizations are located nearby? Temporal: –What events happened before/after/during this event? Thematic: –What is happening? –Who is involved?
    100. 100. Knowledge Enabled Information and Services Science Events and STT dimensions Going further Can we use –What? Where? When? Who? To answer –Why? / How? Use integrated STT analysis to explore cause and effect
    101. 101. Knowledge Enabled Information and Services Science 102 High-level Sensor (S-H) Low-level Sensor (S-L) A-H E-H A-L E-L H L • How do we determine if A-H = A-L? (Same time? Same place?) • How do we determine if E-H = E-L? (Same entity?) • How do we determine if E-H or E-L constitutes a threat? Example Scenario: Sensor Data Fusion and Analysis
    102. 102. Knowledge Enabled Information and Services Science 103 Sensor Data Pyramid Raw Sensor (Phenomenological) Data Feature Metadata Entity Metadata Relationship Metadata Data Information Semantics/Understanding/I nsight Data Pyramid
    103. 103. Knowledge Enabled Information and Services Science 104 Collection Feature Extraction Entity Detection RDF KB Semantic Analysis Data • Raw Phenomenological Data TML Fusion SML-S SML-S SML-S Ontologies Information • Entity Metadata • Feature Metadata Knowledge • Object-Event Relations • Spatiotemporal Associations • Provenance Pathways Sensors (RF, EO, IR, HIS, acoustic) • Object-Event Ontology • Space-Time Ontology Analysis Processes Annotation Processes O&M Sensor Data Architecture
    104. 104. Knowledge Enabled Information and Services Science Current Research Towards STT Relationship Analysis Modeling Spatial and Temporal data using SW standards (RDF(S))1 –Upper-level ontology integrating thematic and spatial dimensions –Use Temporal RDF3 to encode temporal properties of relationships –Demonstrate expressiveness with various query operators built upon thematic contexts 1. Matthew Perry, Farshad Hakimpour, Amit Sheth. "Analyzing Theme, Space and Time: An Ontology-based Approach", Fourteenth International Symposium on Advances in Geographic Information Systems (ACM-GIS '06), Arlington, VA, November 10 - 11, 2006 2. Matthew Perry, Amit Sheth, Farshad Hakimpour, Prateek Jain. “Supporting Complex Thematic, Spatial and Temporal Queries over Semantic Web Data", Second International Conference on Geospatial Semantics (GeoS ‘07), Mexico City, MX, November 29 – 30, 2007 3. Claudio Gutiérrez, Carlos A. Hurtado, Alejandro A. Vaisman. “Temporal RDF”, ESWC 2005: 93-107
    105. 105. Knowledge Enabled Information and Services Science Current Research Towards STT Relationship Analysis Graph Pattern queries over spatial and temporal RDF data2 –Extended ORDBMS to store and query spatial and temporal RDF –User-defined functions for graph pattern queries involving spatial variables and spatial and temporal predicates –Implementation of temporal RDFS inferencing 1. Matthew Perry, Farshad Hakimpour, Amit Sheth. "Analyzing Theme, Space and Time: An Ontology-based Approach", Fourteenth International Symposium on Advances in Geographic Information Systems (ACM-GIS '06), Arlington, VA, November 10 - 11, 2006 2. Matthew Perry, Amit Sheth, Farshad Hakimpour, Prateek Jain. “Supporting Complex Thematic, Spatial and Temporal Queries over Semantic Web Data", Second International Conference on Geospatial Semantics (GeoS ‘07), Mexico City, MX, November 29 – 30, 2007 3. Claudio Gutiérrez, Carlos A. Hurtado, Alejandro A. Vaisman. “Temporal RDF”, ESWC 2005: 93-107
    106. 106. Knowledge Enabled Information and Services Science Upper-Level Ontology Modeling Theme and Space Occurrent Continuant Named_PlaceSpatial_Occurrent Dynamic_Entity Spatial_Region Occurrent: Events – happen and then don’t exist Continuant: Concrete and Abstract Entities – persist over time Named_Place: Those entities with static spatial behavior (e.g. building) Dynamic_Entity: Those entities with dynamic spatial behavior (e.g. person) Spatial_Occurrent: Events with concrete spatial locations (e.g. a speech) Spatial_Region: Records exact spatial location (geometry objects, coordinate system info) occurred_at located_at occurred_at: Links Spatial_Occurents to their geographic locations located_at: Links Named_Places to their geographic locations rdfs:subClassOf property Spatio-Temporal-Thematic Query Processing @ Kno.e.sis
    107. 107. Knowledge Enabled Information and Services Science Occurrent Continuant Named_Place Spatial_Occurrent Dynamic_Entity Person City Politician Soldier Military_Unit Battle Vehicle Bombing Speech Military_Event assigned_to on_crew_of used_in gives participates_in trains_at Spatial_Region located_at occurred_at Upper-level Ontology Domain Ontology rdfs:subClassOf used for integration rdfs:subClassOf relationship type
    108. 108. Knowledge Enabled Information and Services Science Temporal RDF Graph: Platoon Membership E1:Soldier E3:Platoon E5:Soldier E2:Platoon E4:Soldier assigned_to [1, 10] assigned_to [11, 20] assigned_to [5, 15] assigned_to [5, 15] E1 is assigned to E2 from time 1 to 10 and then assigned to E3 from time 11 to 20 Also need to handle inferencing: (x rdf:type Grad_Student):[2004, 2006] AND (x rdf:type Undergrad_Student):[2000, 2004]  (x rdf:type Student):[2000, 2006] Time interval represents valid time of the relationship
    109. 109. Knowledge Enabled Information and Services Science ORDBMS Implementation: DB Structures Unlike thematic relationships which are explicitly stated in the RDF graph, many spatial and temporal relationships (e.g., distance) are implicit and require additional computation
    110. 110. Knowledge Enabled Information and Services Science Sample STT Query Scenario (Biochemical Threat Detection): Analysts must examine soldiers’ symptoms to detect possible biochemical attack Query specifies a relationship between a soldier, a chemical agent and a battle location (graph pattern 1) a relationship between members of an enemy organization and their known locations (graph pattern 2) a spatial filtering condition based on the proximity of the soldier and the enemy group in this context (spatial Constraint)
    111. 111. Knowledge Enabled Information and Services Science Going places … Formal Social, Informal Implicit Powerful
    112. 112. Knowledge Enabled Information and Services Science looking into my crystal ball
    113. 113. Knowledge Enabled Information and Services Science TECHNOLOGY THAT FITS RIGHT IN “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it. Machines that fit the human environment instead of forcing humans to enter theirs will make using a computer as refreshing as a walk in the woods.” Mark Weiser, The Computer for the 21st Century (Ubicomp vision) 114 Get citation
    114. 114. Knowledge Enabled Information and Services Science imagine
    115. 115. Knowledge Enabled Information and Services Science imagine when
    116. 116. Knowledge Enabled Information and Services Science meets Farm Helper
    117. 117. Knowledge Enabled Information and Services Science with this Latitude: 38° 57’36” N Longitude: 95° 15’12” W Date: 10-9-2007 Time: 1345h
    118. 118. Knowledge Enabled Information and Services Science that is sent to Geocoder Farm Helper Services Resource Sensor Data Resource Structured Data Resource Agri DB Soil Survey Lat-Long Soil Information Pest information … Weather Resource Location Date /Time Weathe r Data
    119. 119. Knowledge Enabled Information and Services Science and
    120. 120. Knowledge Enabled Information and Services Science Six billion brains
    121. 121. Knowledge Enabled Information and Services Science imagination today
    122. 122. Knowledge Enabled Information and Services Science impacts our experience tomorrow
    123. 123. Knowledge Enabled Information and Services Science
    124. 124. Knowledge Enabled Information and Services Science Computing For Human Experience Prof. Amit P. Sheth, LexisNexis Eminent Scholar and Director, kno.e.sis center Wright State University http://knoesis.org

    ×