mumbai, india<br />
november 26, 2008<br />
another chapter in the war against civilization<br />
 and<br />
 the world saw it<br />Through the eyes of the people <br />
 the world read it<br />Through the words of the people <br />
PEOPLE told their stories to PEOPLE<br />
A powerful new era in <br />Information dissemination had taken firm ground<br />
Making it possible for us to<br />create a global network of citizens<br />Citizen Sensors – <br />Citizens observing, pro...
12<br />
Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A comprehensive path towards event monitoring and situ...
Geocoder<br />(Reverse Geo-coding)<br />Address to location database<br />18 Hormusji Street, Colaba<br />VasantVihar<br /...
Research Challenge #1<br />Spatio Temporal and Thematic analysis<br />What else happened “near” this event location?<br />...
Spatial Analysis….<br />Which tweets originated from an address near 18.916517°N 72.827682°E? <br />
Which tweets originated during Nov 27th 2008,from 11PM to 12 PM <br />
Giving us<br />Tweets originated from an address near 18.916517°N, 72.827682°E during time interval27th Nov 2008 between 1...
Research Challenge #2:Understanding and Analyzing Casual Text<br />Casual text<br />Microblogs are often written in SMS st...
Understanding Casual Text<br />Not the same as news articles or scientific literature<br />Grammatical errors<br />Implica...
Nature of Microblogs<br />Additional constraint of limited context<br />Max. of x chars in a microblog<br />Context often ...
NL understanding is hard to begin with..<br />Not so hard<br />“commando raid appears to be nigh at Oberoinow”<br />Oberoi...
Social Context surrounding content<br />Social context in which a message appears is also an added valuable resource<br />...
Research Opportunities<br />NER, disambiguation in casual, informal text is a budding area of research<br />Another import...
What Drives the Spatio-Temporal-Thematic Analysis and Casual Text Understanding<br />Semantics with the help of<br />Domai...
And who creates these models?<br />YOU, <br />ME,<br />We DO!<br />
Domain Knowledge: A key driver<br />Places that are nearby ‘Nariman house’<br />Spatial query<br />Messages originated aro...
Research Challenge #3But Where does the Domain Knowledge come from?<br />Community driven knowledge extraction <br />How t...
The Wisdom of the Crowds<br /><ul><li>The most comprehensive and up to date account of the present state of knowledge is g...
Blogs
Wikipedia</li></li></ul><li>Wikipedia<br />		=    Concise concept descriptions<br />		+    An article title denotes a conc...
Wikipedia<br />		=    Concise concept descriptions<br />		+    An article title denotes a concept<br />       +   Communit...
Goal: Harness the Wisdom of the Crowds to Automatically define a domain with up-to-date concepts<br />We can safely take a...
Collecting Instances<br />
Creating a Hierarchy<br />
Creating a Hierarchy<br />
Hierarchy Creation - summary<br />
Snapshot of final Topic Hierarchy<br />
Great to know Explosion and Fire are related!But, knowing Explosion “causes” fire is powerful<br />Relationships at the he...
Identifying relationships: Hard, harder than many hard things <br />But NOT that Hard, When WE do it<br />
Games with a purpose<br />Get humans to give their solitaire time <br />Solve real hard computational problems<br />Image ...
OntoLablr<br />Relationship Identification Game<br /><ul><li>leads to
causes</li></ul>Explosion<br />Traffic congestion<br />
And the infrastructure<br />Semantic Sensor Web<br />How can we annotate and correlate the knowledge from machine sensors ...
Research Challenge #4: Semantic Sensor Web<br />
Semantically Annotated O&M<br />&lt;swe:component name=&quot;time&quot;&gt;<br />	&lt;swe:Time definition=&quot;urn:ogc:de...
Semantic Sensor ML – Adding Ontological Metadata<br />Domain<br />Ontology<br />Person<br />Company<br />Spatial<br />Onto...
46<br />Semantic Query<br />Semantic Temporal Query<br />Model-references from SML to OWL-Time ontology concepts provides ...
Kno.e.sis’ Semantic Sensor Web<br />47<br />
Synthetic but realistic scenario<br />an image taken from a raw satellite feed<br />48<br />
an image taken by a camera phone with an associated label, “explosion.”  <br />Synthetic but realistic scenario<br />49<br />
Textual messages (such as tweets) using STT analysis<br />Synthetic but realistic scenario<br />50<br />
Upcoming SlideShare
Loading in...5
×

Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A comprehensive path towards event monitoring and situational awareness

1,314

Published on

Amit Sheth, "Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A comprehensive path towards event monitoring and situational awareness," Keynote talk at From E-Gov to Connected Governance: the Role of Cloud Computing, Web 2.0 and Web 3.0 Semantic Technologies, Fall Church, VA, February 17, 2009.

Additional details at: http://knoesis.org/library/resource.php?id=00702

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,314
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
55
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Microblogs are one of the most powerful ways of talking of CSD
  • Implicit social context created by people responding to other messages. In this example we are showing how the system can identify that its is Nariman and not Hareemane
  • In the scenario, what techniques and technlologies are being brought together? Semantic + Social Computing + Mobile Web
  • Users are shown two images along with labels. Labels gotten from GI or similar data source. Users add relationships. When 2 users agree, the labels are tagged with this relationship. Multiple relationships, using ML techniques, the system will learn .
  • Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A comprehensive path towards event monitoring and situational awareness

    1. 1. mumbai, india<br />
    2. 2. november 26, 2008<br />
    3. 3. another chapter in the war against civilization<br />
    4. 4. and<br />
    5. 5.
    6. 6.
    7. 7. the world saw it<br />Through the eyes of the people <br />
    8. 8. the world read it<br />Through the words of the people <br />
    9. 9. PEOPLE told their stories to PEOPLE<br />
    10. 10. A powerful new era in <br />Information dissemination had taken firm ground<br />
    11. 11. Making it possible for us to<br />create a global network of citizens<br />Citizen Sensors – <br />Citizens observing, processing, transmitting, reporting<br />
    12. 12. 12<br />
    13. 13. Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A comprehensive path towards event monitoring and situational awareness<br />Amit P. Sheth, <br />LexisNexis Eminent Scholar<br />Kno.e.sis Center<br />Wright State University<br />
    14. 14. Geocoder<br />(Reverse Geo-coding)<br />Address to location database<br />18 Hormusji Street, Colaba<br />VasantVihar<br />Image Metadata<br />latitude: 18° 54′ 59.46″ N, <br />longitude: 72° 49′ 39.65″ E<br />Structured Meta Extraction<br />Nariman House<br />Income Tax Office<br />Identify and extract information from tweets<br />Spatio-Temporal Analysis<br />
    15. 15. Research Challenge #1<br />Spatio Temporal and Thematic analysis<br />What else happened “near” this event location?<br />What events occurred “before” and “after” this event?<br />Any message about “causes” for this event?<br />
    16. 16. Spatial Analysis….<br />Which tweets originated from an address near 18.916517°N 72.827682°E? <br />
    17. 17. Which tweets originated during Nov 27th 2008,from 11PM to 12 PM <br />
    18. 18. Giving us<br />Tweets originated from an address near 18.916517°N, 72.827682°E during time interval27th Nov 2008 between 11PM to 12PM?<br />
    19. 19. Research Challenge #2:Understanding and Analyzing Casual Text<br />Casual text<br />Microblogs are often written in SMS style language<br />Slangs, abbreviations<br />
    20. 20. Understanding Casual Text<br />Not the same as news articles or scientific literature<br />Grammatical errors<br />Implications on NL parser results<br />Inconsistent writing style<br />Implications on learning algorithms that generalize from corpus<br />
    21. 21. Nature of Microblogs<br />Additional constraint of limited context<br />Max. of x chars in a microblog<br />Context often provided by the discourse<br />Entity identification and disambiguation<br />Pre-requisite to other sophisticated information analytics<br />
    22. 22. NL understanding is hard to begin with..<br />Not so hard<br />“commando raid appears to be nigh at Oberoinow”<br />Oberoi = Oberoi Hotel, Nigh = high<br />Challenging<br />new wing, live fire @ taj 2nd floor on iDesi TV stream<br />Fire on the second floor of the Taj hotel, not on iDesi TV<br />
    23. 23. Social Context surrounding content<br />Social context in which a message appears is also an added valuable resource<br />Post 1: <br />“Hareemane Househostages said by eyewitnesses to be Jews. 7 Gunshots heard by reporters at Taj”<br />Follow up post<br />that is Nariman House, not (Hareemane)<br />
    24. 24. Research Opportunities<br />NER, disambiguation in casual, informal text is a budding area of research<br />Another important area of focus: Combining information of varied quality from a <br />corpus (statistical NLP), <br />domain knowledge (tags, folksonomies, taxonomies, ontologies), <br />social context (explicit and implicit communities)<br />
    25. 25. What Drives the Spatio-Temporal-Thematic Analysis and Casual Text Understanding<br />Semantics with the help of<br />Domain Models<br />Domain Models<br />Domain Models(ontologies, folksonomies)<br />
    26. 26. And who creates these models?<br />YOU, <br />ME,<br />We DO!<br />
    27. 27. Domain Knowledge: A key driver<br />Places that are nearby ‘Nariman house’<br />Spatial query<br />Messages originated around this place<br />Temporal analysis<br />Messages about related events / places<br />Thematic analysis<br />
    28. 28. Research Challenge #3But Where does the Domain Knowledge come from?<br />Community driven knowledge extraction <br />How to create models that are “socially scalable”?<br />How to organically grow and maintain this model?<br />
    29. 29. The Wisdom of the Crowds<br /><ul><li>The most comprehensive and up to date account of the present state of knowledge is given by</li></ul>Everybody<br /><ul><li>The Web in general
    30. 30. Blogs
    31. 31. Wikipedia</li></li></ul><li>Wikipedia<br /> = Concise concept descriptions<br /> + An article title denotes a concept<br /> + Community takes care of <br /> disambiguation<br />Collecting Knowledge<br />
    32. 32. Wikipedia<br /> = Concise concept descriptions<br /> + An article title denotes a concept<br /> + Community takes care of <br /> disambiguation<br /> + Large, highly connected, sparsely <br /> annotated graph structure that connects named entities<br /> + Category hierarchy<br />Collecting Knowledge<br />
    33. 33. Goal: Harness the Wisdom of the Crowds to Automatically define a domain with up-to-date concepts<br />We can safely take advantage of existing (semi)structured knowledge sources<br />
    34. 34. Collecting Instances<br />
    35. 35. Creating a Hierarchy<br />
    36. 36. Creating a Hierarchy<br />
    37. 37. Hierarchy Creation - summary<br />
    38. 38. Snapshot of final Topic Hierarchy<br />
    39. 39. Great to know Explosion and Fire are related!But, knowing Explosion “causes” fire is powerful<br />Relationships at the heart of semantics!<br />
    40. 40. Identifying relationships: Hard, harder than many hard things <br />But NOT that Hard, When WE do it<br />
    41. 41. Games with a purpose<br />Get humans to give their solitaire time <br />Solve real hard computational problems<br />Image tagging, Identifying part of an image<br /> Tag a tune, Squigl, Verbosity, and Matchin<br />Pioneered by Luis Von Ahn<br />
    42. 42. OntoLablr<br />Relationship Identification Game<br /><ul><li>leads to
    43. 43. causes</li></ul>Explosion<br />Traffic congestion<br />
    44. 44. And the infrastructure<br />Semantic Sensor Web<br />How can we annotate and correlate the knowledge from machine sensors around the event location?<br />
    45. 45. Research Challenge #4: Semantic Sensor Web<br />
    46. 46. Semantically Annotated O&M<br />&lt;swe:component name=&quot;time&quot;&gt;<br /> &lt;swe:Time definition=&quot;urn:ogc:def:phenomenon:time&quot; uom=&quot;urn:ogc:def:unit:date-time&quot;&gt;<br /> &lt;sa:swe rdfa:about=&quot;?time&quot; rdfa:instanceof=&quot;time:Instant&quot;&gt;<br /> &lt;sa:sml rdfa:property=&quot;xs:date-time&quot;/&gt;<br /> &lt;/sa:swe&gt;<br /> &lt;/swe:Time&gt;<br />&lt;/swe:component&gt;<br />&lt;swe:component name=&quot;measured_air_temperature&quot;&gt;<br /> &lt;swe:Quantity definition=&quot;urn:ogc:def:phenomenon:temperature“ uom=&quot;urn:ogc:def:unit:fahrenheit&quot;&gt;<br /> &lt;sa:swe rdfa:about=&quot;?measured_air_temperature“ rdfa:instanceof=“senso:TemperatureObservation&quot;&gt;<br /> &lt;sa:swe rdfa:property=&quot;weather:fahrenheit&quot;/&gt;<br /> &lt;sa:swe rdfa:rel=&quot;senso:occurred_when&quot; resource=&quot;?time&quot;/&gt;<br /> &lt;sa:swe rdfa:rel=&quot;senso:observed_by&quot; resource=&quot;senso:buckeye_sensor&quot;/&gt;<br /> &lt;/sa:sml&gt; <br /> &lt;/swe:Quantity&gt;<br />&lt;/swe:component&gt;<br />&lt;swe:value name=“weather-data&quot;&gt;<br /> 2008-03-08T05:00:00,29.1<br />&lt;/swe:value&gt;<br />
    47. 47. Semantic Sensor ML – Adding Ontological Metadata<br />Domain<br />Ontology<br />Person<br />Company<br />Spatial<br />Ontology<br />Coordinates<br />Coordinate System<br />Temporal<br />Ontology<br />Time Units<br />Timezone<br />45<br />Mike Botts, &quot;SensorML and Sensor Web Enablement,&quot; <br />Earth System Science Center, UAB Huntsville<br />
    48. 48. 46<br />Semantic Query<br />Semantic Temporal Query<br />Model-references from SML to OWL-Time ontology concepts provides the ability to perform semantic temporal queries<br />Supported semantic query operators include:<br />contains: user-specified interval falls wholly within a sensor reading interval (also called inside)<br />within: sensor reading interval falls wholly within the user-specified interval (inverse of contains or inside)<br />overlaps: user-specified interval overlaps the sensor reading interval<br />Example SPARQL query defining the temporal operator ‘within’<br />
    49. 49. Kno.e.sis’ Semantic Sensor Web<br />47<br />
    50. 50. Synthetic but realistic scenario<br />an image taken from a raw satellite feed<br />48<br />
    51. 51. an image taken by a camera phone with an associated label, “explosion.” <br />Synthetic but realistic scenario<br />49<br />
    52. 52. Textual messages (such as tweets) using STT analysis<br />Synthetic but realistic scenario<br />50<br />
    53. 53. Correlating to get<br />Synthetic but realistic scenario<br />
    54. 54. Create better views (smart mashups)<br />
    55. 55. A few more things<br />Use of background knowledge<br />Event extraction from text<br />time and location extraction <br />Such information may not be present<br />Someone from Washington DC can tweet about Mumbai<br />Scalable semantic analytics<br />Subgraph and pattern discovery<br />Meaningful subgraphs like relevant and interesting paths<br />Ranking paths <br />
    56. 56. The Sum of the Parts<br />Spatio-Temporal analysis<br />Find out where and when<br />+ Thematic <br />What and how<br />+ Semantic Extraction from text, multimedia and sensor data<br /> - tags, time, location, concepts, events<br />+ Semantic models & background knowledge<br />Making better sense of STT<br />Integration<br /> + Semantic Sensor Web<br />The platform<br /> = Situational Awareness<br />
    57. 57. Search<br />Integration<br />Analysis<br />Discovery<br />Question <br /> Answering<br />Situational <br /> Awareness<br />Domain Models<br />Patterns / Inference / Reasoning<br />RDB<br />Relationship Web<br />Meta data / Semantic Annotations<br />Metadata Extraction<br />Multimedia Content and Web data<br />Text<br />Sensor Data<br />Structured and Semi-structured data<br />
    58. 58. Interested in more background?<br />Semantics-Empowered Social Computing<br />Semantic Sensor Web <br />Traveling the Semantic Web through Space, Theme and Time <br />Relationship Web: Blazing Semantic Trails between Web Resources <br />Contact/more details: amit @ knoesis.org<br />Special thanks: Karthik Gomadam, MeenaNagarajan, Christopher Thomas<br />Partial Funding: NSF (Semantic Discovery: IIS: 071441, Spatio Temporal Thematic: IIS-0842129), AFRL and DAGSI (Semantic Sensor Web), Microsoft Research and IBM Research (Analysis of Social Media Content),and HP Research (Knowledge Extraction from Community-Generated Content).<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×