x<br />WEB DATA evolved over time<br />Real-Time Sensor, Social, Multi-media data<br />2010’s<br />Dynamic User Generated ...
x<br />Properties of Streaming Data<br />Huge Volume<br />Rapid<br />Continuous<br />Information Overload!!<br />Heterogen...
x<br />Some Statistics<br />“A cross-country flight from New York to Los Angeles on a Boeing 737 plane generates a massive...
48th ACM Southeast Conference. ACMSE 2010. <br />Oxford, Mississippi.  April 15-17, 2010.<br />From Sensor Streams to Feat...
x<br />Outline<br /> Introduction <br />Architecture<br /> Linked Sensor Data<br /> Feature Streams<br /> Demonstration<br...
x<br />Domain<br />Weather Domain<br />Features<br />Blizzard<br />Flurry<br />RainStorm<br />RainShower<br />7<br />
x<br />Explaining the title<br />Background  Knowledge<br />Blizzard<br />Rain Storm<br />ABSTRACTION<br />Huge amount of ...
x<br />Types of Abstractions<br />Summarization across Thematic Dimension<br />Summarization over the Temporal Dimension<b...
x<br />Types of Abstractions<br />Summarization across Thematic Dimension<br />Select<br />Join<br />Background Knowledge<...
x<br />An example problem?<br />11<br />“Find the sequence of weather events observed near Dayton James Cox Airport betwee...
x<br />Outline<br />Introduction<br />Architecture<br />Linked Sensor Data<br />Feature Streams<br /> Demonstration<br />1...
x<br />System Architecture<br />13<br />
x<br />Outline<br />Introduction <br />Architecture<br />Linked Sensor Data<br />Feature Streams<br /> Demonstration<br />...
48th ACM Southeast Conference. ACMSE 2010. <br />Oxford, Mississippi.  April 15-17, 2010.<br />Technology1: Linked Sensor ...
Sensor Discovery Application <br />Weather Station ID<br />Current Observations from MesoWest<br />Weather Station Coordin...
What is Linked Sensor Data<br />Weather Sensors<br />Sensor Dataset<br />GPS Sensors<br />Satellite Sensors<br />Camera Se...
What is Linked Sensor Data<br />Recommended best practice for exposing, sharing, and connecting pieces of data, informatio...
Linked Sensor Data on LOD<br /> - First Sensor Dataset on LOD<br /> - Among the largest dataset on LOD<br />19<br />
zn<br />Sensor Datasets<br />LinkedSensorDataset<br /><ul><li>RDF Descriptions of ~20,000 weather stations in US
 Average 5 sensors/weather station
 Spatial attributes of the weather station
 Links to locations in Geonames</li></ul>LinkedObservationDataset<br /><ul><li> RDF descriptions of Hurricanes and Blizzar...
 Observations generated by sensors described in LinkedSensorDataset</li></ul>20<br />
Data Generation Workflow<br />O&M2RDFCONVERTER<br />21<br />
Workflow – Phase 1<br />22<br />
Workflow – Phase 2<br />OGC (Open Geospatial Consortium) standard for encoding sensor observations<br />23<br />
Workflow – Phase 3<br />W3C SSN ontology<br />Ontology – formal representation of knowledge by a set of concepts and relat...
Workflow – Phase 3<br />Figure 1: System Components and Architecture<br />
Workflow – Phase 4<br />Open Source RDF store  by OpenLink Software for storing RDF data<br />PUBBY Linked Data Front End<...
Summarizing Linked Sensor Data<br />Find the sensor around <br />Dayton James Cox Airport?<br />Extract Data for the senso...
 MesoWest
Static + Dynamic
 20,000+ systems
 MesoWest
 ~Static
 230,000+ locations
Geonames
 ~Static</li></li></ul><li>x<br />Outline<br /> Introduction <br />Architecture<br />Linked Sensor Data<br />Feature Strea...
48th ACM Southeast Conference. ACMSE 2010. <br />Oxford, Mississippi.  April 15-17, 2010.<br />Technology 2: Feature Strea...
x<br />System Architecture<br />Streams Integration based on feature composition<br />Integrated Stream Analysis to check ...
x<br />Feature Composition<br />31<br />
x<br />System Capability<br />32<br />
x<br />System Feature Integration<br />SELECT<br />JOIN<br />33<br />
x<br />System Architecture<br />Integrated Stream Analysis to check if the feature is being detected<br />34<br />
x<br />Feature Definition<br />RainStorm = 	HighWindSpeed(above 35mph) AND <br />			Rain Precipitation AND <br />			Temper...
Upcoming SlideShare
Loading in...5
×

Real Time Semantic Analysis of Streaming Sensor Data

2,407

Published on

Harshal Patni, "Real Time Semantic Analysis of Streaming Sensor Data," MS Thesis Defense, Kno.e.sis Center, Wright State University, Dayton OH, March 21, 2001.

More at: http://wiki.knoesis.org/index.php/SSW
Dissertation Advisor: Prof. Amit Sheth

Published in: Education, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,407
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
75
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Good Morning Everyone. My name is Harshal Patni and I am here to present my thesis on Streaming Sensor Data but Before we begin lets have a look at how web data evolved over time
  • Social media is the dominant source of streaming data now, however in future sensors would …Data needs to be reduced
  • To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
  • Move this slide above
  • Remove the precipitation (in) and also show the general streamAdd the image taken on the phoneRemove the stuff on left when you show select, join and analyze`
  • Remove the precipitation (in) and also show the general streamAdd the image taken on the phoneRemove the stuff on left when you show select, join and analyze`
  • To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
  • To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
  • To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsAdd linked Sensor Data when highlightThe output of these phases is called LSD and its added on LOD
  • To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
  • Get all sensors using well known location names – Problem to be solveAssociate sensor descriptions to well know locations.
  • Get all sensors using well known location names – Problem to be solve
  • Say the numbers in the table
  • RDF because of LOD
  • Highlight the important points in MesoWest DataThe sensor data file just 3 linesMapping file - shorten
  • Emphasize semantically annotated O&amp;MAnd its an XMLTry to replace the cory/weather.owl
  • Use the ssn ontologyAdd the image of ontology for the (Sensor Ontology)http://www.w3.org/2005/Incubator/ssn/wiki/Report_Work_on_the_SSN_ontology
  • Add in block letters saying this is semantically annotated XML and RDF
  • Add Pubby to show derefenced dataPubby should be large to show what it is
  • To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
  • To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
  • Replace Air Temperature with Non Freezing Temperature
  • Replace Rain Precipitation with PrecipitationSame with airtempearure - temperature
  • To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
  • Highlight the query with 3 boxes to show the temp,windspeed and precipitation streamHighlight the feature results too
  • Talk about the observations and features storage
  • Remove the precipitation (in) and also show the general streamAdd the image taken on the phoneRemove the stuff on left when you show select, join and analyze`
  • Linked Data explodes
  • To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important partsMarket the datasets we added on LOD
  • Linked Data explodes
  • % of FeaturesThrow the text on the top for the statisticsMiddle of storm and hence we have 70 % data reductionElse it would be more
  • Real Time Semantic Analysis of Streaming Sensor Data

    1. 1.
    2. 2. x<br />WEB DATA evolved over time<br />Real-Time Sensor, Social, Multi-media data<br />2010’s<br />Dynamic User Generated Content<br />2000’s<br />Static Document and files <br />1990’s<br />2<br />
    3. 3. x<br />Properties of Streaming Data<br />Huge Volume<br />Rapid<br />Continuous<br />Information Overload!!<br />Heterogeneous<br />3<br />
    4. 4. x<br />Some Statistics<br />“A cross-country flight from New York to Los Angeles on a Boeing 737 plane generates a massive 240 terabytes of data”<br />- GigaOmni Media<br />“Sensors Networks will produce 10-20 times the amount of generated by social media in the next few years” <br /> - GigaOmni Media<br />“More data has been created in the last three years than in all the past 40,000 years”<br />- Teradata<br />Solution - “Meaningfully summarize this data”<br />4<br />
    5. 5. 48th ACM Southeast Conference. ACMSE 2010. <br />Oxford, Mississippi. April 15-17, 2010.<br />From Sensor Streams to Feature Streams <br />in Real Time<br />HarshalPatni<br />Ohio Center of Excellence in Knowledge enabled Computing (Kno.e.sis) <br />Wright State University, Dayton, OH<br />Part of Semantic Sensor Web @ Kno.e.sis<br />
    6. 6. x<br />Outline<br /> Introduction <br />Architecture<br /> Linked Sensor Data<br /> Feature Streams<br /> Demonstration<br />6<br />
    7. 7. x<br />Domain<br />Weather Domain<br />Features<br />Blizzard<br />Flurry<br />RainStorm<br />RainShower<br />7<br />
    8. 8. x<br />Explaining the title<br />Background Knowledge<br />Blizzard<br />Rain Storm<br />ABSTRACTION<br />Huge amount of <br />Raw Sensor Data<br />Features representing Real-World events<br />8<br />
    9. 9. x<br />Types of Abstractions<br />Summarization across Thematic Dimension<br />Summarization over the Temporal Dimension<br />9<br />
    10. 10. x<br />Types of Abstractions<br />Summarization across Thematic Dimension<br />Select<br />Join<br />Background Knowledge<br />Analyze<br />Features representing Real-World Events<br />10<br />
    11. 11. x<br />An example problem?<br />11<br />“Find the sequence of weather events observed near Dayton James Cox Airport between <br /> Jan 13th and Jan 18th?”<br />Spatial<br />Thematic<br />Temporal<br />Technologies required - <br />Linked Sensor Data<br />Feature Streams<br />
    12. 12. x<br />Outline<br />Introduction<br />Architecture<br />Linked Sensor Data<br />Feature Streams<br /> Demonstration<br />12<br />
    13. 13. x<br />System Architecture<br />13<br />
    14. 14. x<br />Outline<br />Introduction <br />Architecture<br />Linked Sensor Data<br />Feature Streams<br /> Demonstration<br />14<br />
    15. 15. 48th ACM Southeast Conference. ACMSE 2010. <br />Oxford, Mississippi. April 15-17, 2010.<br />Technology1: Linked Sensor Data<br />Find the sensor around Dayton James Cox Airport?<br />Extract Data for the sensor near Dayton James Cox Airport?<br />Harshal Patni, Cory Henson, Amit Sheth, 'Linked Sensor Data,' In: Proceedings of 2010 International Symposium on Collaborative Technologies and Systems (CTS 2010), Chicago, IL, May 17-21, 2010.<br />
    16. 16. Sensor Discovery Application <br />Weather Station ID<br />Current Observations from MesoWest<br />Weather Station Coordinates<br />Weather Station Phenomena<br />MesoWest – Project under Department of Meteorology, University of UTAH<br />GeoNames – Geographic dataset<br />16<br />
    17. 17. What is Linked Sensor Data<br />Weather Sensors<br />Sensor Dataset<br />GPS Sensors<br />Satellite Sensors<br />Camera Sensors<br />17<br />
    18. 18. What is Linked Sensor Data<br />Recommended best practice for exposing, sharing, and connecting pieces of data, information, and knowledge on the Web using URIs and RDF<br />GeoNames Dataset<br />RDF – language for representing data on the Web<br />locatedNear<br />Sensor Dataset<br />Publicly Accessible<br />18<br />
    19. 19. Linked Sensor Data on LOD<br /> - First Sensor Dataset on LOD<br /> - Among the largest dataset on LOD<br />19<br />
    20. 20. zn<br />Sensor Datasets<br />LinkedSensorDataset<br /><ul><li>RDF Descriptions of ~20,000 weather stations in US
    21. 21. Average 5 sensors/weather station
    22. 22. Spatial attributes of the weather station
    23. 23. Links to locations in Geonames</li></ul>LinkedObservationDataset<br /><ul><li> RDF descriptions of Hurricanes and Blizzard observations in US
    24. 24. Observations generated by sensors described in LinkedSensorDataset</li></ul>20<br />
    25. 25. Data Generation Workflow<br />O&M2RDFCONVERTER<br />21<br />
    26. 26. Workflow – Phase 1<br />22<br />
    27. 27. Workflow – Phase 2<br />OGC (Open Geospatial Consortium) standard for encoding sensor observations<br />23<br />
    28. 28. Workflow – Phase 3<br />W3C SSN ontology<br />Ontology – formal representation of knowledge by a set of concepts and relationship between those concepts<br />
    29. 29. Workflow – Phase 3<br />Figure 1: System Components and Architecture<br />
    30. 30. Workflow – Phase 4<br />Open Source RDF store by OpenLink Software for storing RDF data<br />PUBBY Linked Data Front End<br />
    31. 31. Summarizing Linked Sensor Data<br />Find the sensor around <br />Dayton James Cox Airport?<br />Extract Data for the sensor?<br />Observation<br />KB<br />Sensor KB<br />Location KB<br />(Geonames)<br />location<br />procedure<br />location<br />location<br />procedure<br />720F<br />Thermometer<br />Dayton Airport<br /><ul><li> ~2 billion triples
    32. 32. MesoWest
    33. 33. Static + Dynamic
    34. 34. 20,000+ systems
    35. 35. MesoWest
    36. 36. ~Static
    37. 37. 230,000+ locations
    38. 38. Geonames
    39. 39. ~Static</li></li></ul><li>x<br />Outline<br /> Introduction <br />Architecture<br />Linked Sensor Data<br />Feature Streams<br /> Demonstration<br />28<br />
    40. 40. 48th ACM Southeast Conference. ACMSE 2010. <br />Oxford, Mississippi. April 15-17, 2010.<br />Technology 2: Feature Streams<br />What feature is currently being detected by sensor near Dayton Airport?<br />Harshal Patni, Cory Henson, Amit Sheth, Pramod Ananthram, ‘From Real Time Sensor Streams to Real Time Feature Streams,' Kno.e.sis Technical Report, January 2011.<br />
    41. 41. x<br />System Architecture<br />Streams Integration based on feature composition<br />Integrated Stream Analysis to check if the feature is being detected<br />30<br />
    42. 42. x<br />Feature Composition<br />31<br />
    43. 43. x<br />System Capability<br />32<br />
    44. 44. x<br />System Feature Integration<br />SELECT<br />JOIN<br />33<br />
    45. 45. x<br />System Architecture<br />Integrated Stream Analysis to check if the feature is being detected<br />34<br />
    46. 46. x<br />Feature Definition<br />RainStorm = HighWindSpeed(above 35mph) AND <br /> Rain Precipitation AND <br /> Temperature(greater than 32F)<br />SPARQL query for RainStorm<br />Temperature<br />Rain Precipitation<br />WindSpeed<br />35<br />Rain Storm NOAA definition<br />
    47. 47. x<br />Feature Analysis<br />RDF Feature Stream<br />36<br />
    48. 48. x<br />Revisiting Abstractions<br />Summarization across Thematic Dimension<br />Select<br />Join<br />Background Knowledge<br />Analyze<br />Features representing Real-World Events<br />37<br />
    49. 49. Summarizing Feature Streams<br />Feature Streams<br />KB<br />Find sequence of events near Dayton Airport?<br />Observation<br />KB<br />Sensor KB<br />Location KB<br />(Geonames)<br />procedure<br />location<br />location<br />procedure<br />720F<br />Thermometer<br />Dayton Airport<br /><ul><li> ~2 billion triples
    50. 50. MesoWest
    51. 51. Static + Dynamic
    52. 52. 20,000+ systems
    53. 53. MesoWest
    54. 54. ~Static
    55. 55. 230,000+ locations
    56. 56. Geonames
    57. 57. ~Static</li></li></ul><li>x<br />Answering the query<br />39<br />“Find the sequence of weather events observed near Dayton James Cox Airport between <br /> Jan 13th and Jan 18th?”<br />Feature Streams<br />Linked Sensor Data<br />
    58. 58. x<br />Outline<br /> Introduction <br />Architecture<br />Linked Sensor Data<br />Feature Streams<br /> Demonstration<br />40<br />
    59. 59. x<br />Demo<br />41<br />Feature Streams Demo<br />http://knoesis1.wright.edu/EventStreams<br />
    60. 60. x<br />Evaluation<br /><ul><li>Data Used: Nevada Blizzard (April 1st – April 6th)</li></ul>70% Data clear<br />30% Feature Observed<br />42<br />
    61. 61. WORKSHOP PAPERS<br />Harshal Patni, Satya S. Sahoo, Cory Henson, Amit Sheth, Provenance Aware Linked Sensor Data, 2nd Workshop on Trust and Privacy on Social and Semantic Web,Co-Located with ESWC, Heraklion Greece, May 30th - June 3rd 2010<br />Harshal Patni, Cory Henson, Amit Sheth, Linked Sensor Data, In: Proceedings of 2010 International Symposium on Collaborative Technologies and Systems (CTS 2010), Chicago, IL, May 17-21, 2010<br />TECHNICAL REPORT<br />Harshal Patni, Cory Henson, Amit Sheth, and Pramod Ananthram. From Real Time Sensor Streams to Real Time Feature Streams, Kno.e.sis Center Technical Report, December 2009<br />Joshua Pschorr, Cory Henson, Harshal Patni, and Amit Sheth. Sensor Discovery on Linked Data, Kno.e.sis Center Technical Report, December 2009<br />JOURNAL PAPER (In Progress)<br />Semantic Sensor Web: Design and Application towards weaving a meaningful sensor web<br />Publications<br />43<br />
    62. 62. Thank You Committee<br />44<br />
    63. 63. Thank You<br />Semantic Sensor Web<br />45<br />
    64. 64. Demos, Papers and more at: http://wiki.knoesis.org/index.php/SSW<br />Semantic Sensor Web @ Kno.e.sis<br />QUESTIONS<br />46<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×