Downloaded 19 times
Uploaded byArtificial Intelligence Institute at UofSC
Real-Time Analysis of Streaming Sensor Data
The document discusses real-time analysis of streaming sensor data. It describes how sensor data has evolved over time from static documents to dynamic user-generated content. It also discusses the properties of streaming data, including its huge volume, continuous and rapid nature, and heterogeneous and information overload characteristics. The document then proposes using semantic technologies to analyze and summarize streaming sensor data by composing features representing real-world events and integrating background knowledge. It provides an example of querying for weather events near an airport between dates and demonstrates the system's capabilities.
More Related Content
![Context Aware Harassment Detection in Social Media [Overview]](https://cdn.slidesharecdn.com/ss_thumbnails/harassment-short-v3-nov15-151119171214-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Real-Time Analysis of Streaming Sensor Data
Real-Time Analysis of Streaming Sensor Data
- 2. x WEB DATA evolvedover time Real-Time Sensor, 2010’s Social, Multi-media data Dynamic User 2000’s Generated Content 1990’s Static Document and files 2
- 3. x Properties of StreamingData Huge Volume Rapid Continuous Information Overload!! Heterogeneous 3
- 4. x Some Statistics “A cross-country flight from New York to Los “Sensors Networksbeenproduce in the last the amount “More on a Boeing 737 plane 10-20 times massive will created Angeles data has generates a of generated -bythan in mediasummarize this data” Solution “Meaningfully in the next few years” three years social all the past 40,000 years” 240 terabytes of data” - GigaOmni Media - Teradata - GigaOmni Media 4
- 5. 48th ACM SoutheastConference. ACMSE 2010. Oxford, Mississippi. April 15-17, 2010. Real-Time Analysis of Streaming Sensor Data Harshal Patni, Cory Henson, Michael Cooney, AmitSheth, ThirunarayanKrishnaprasad Ohio Center of Excellence in Knowledge enabled Computing (Kno.e.sis) Wright State University, Dayton, OH Semantic Sensor Web @ Kno.e.sis
- 6. x RT feature stream Background Knowledge Rain Storm Blizzard ABSTRACTION Huge amount of Features representing Raw Sensor Data Real-World events 7
- 7. x Types of Abstractions Summarization across Thematic Dimension Summarization over the Temporal Dimension 8
- 8. x Types of Abstractions Summarization across Thematic Dimension Select Join Background Knowledge Analyze Features representing Real-World Events 9
- 9. System Architecture x 10
- 10. An example problem? x “Find the sequence of weather events observed near Dayton James Cox Airport between Jan 13th and Jan 18th?” Thematic Spatial Temporal Technologies required - 1. Linked Sensor Data 2. Feature Streams 11
- 11. Sensor Discovery Application Weather Station ID Current Observations from MesoWest Weather Station Coordinates Weather Station Phenomena MesoWest – Project under Department of Meteorology, University of UTAH GeoNames – Geographic dataset Sensors near Dayton James Cox Airport 12
- 12. Linked Sensor Data OGC MesoWest Observation and Virtuoso RDF RDF Instance Service Data Measurement store (O&M) O&M2RDFCONVERTER 13
- 13. Summarizing Linked SensorData Extract Data for the sensor? Find the sensor around Dayton James Cox Airport? procedure location Observation location Location KB Sensor KB KB (Geonames) procedure location 720F Thermometer Dayton Airport • ~2 billion triples • 20,000+ systems • 230,000+ locations • MesoWest • MesoWest •Geonames •Static + Dynamic • ~Static • ~Static
- 14.
- 15.
- 16. x System Feature Integration SELECT JOIN 17
- 17. Feature x Definition • Rain Storm NOAA definition RainStorm = HighWindSpeed(above 35mph) AND Rain Precipitation AND SPARQL query for Temperature(greater than 32F) RainStorm Temperature Rain Precipitation WindSpeed 18
- 18. FeaturexAnalysis RDF Feature Stream 19
- 19. Summarizing Feature Streams Feature Streams KB Find sequence of events near Dayton Airport? procedure Observation location Location KB Sensor KB KB (Geonames) procedure location 720F Thermometer Dayton Airport • ~2 billion triples • 20,000+ systems • 230,000+ locations • MesoWest • MesoWest •Geonames •Static + Dynamic • ~Static • ~Static
- 20. x Answering thequery “Find the sequence of weather events observed near Dayton James Cox Airport between Jan 13th and Jan 18th?” Linked Sensor Data Feature Streams 21
- 21. x Demo on-line video: http://www.youtube.com/watch?v=_ews4w_eCpg 22
- 22. Related Publications WORKSHOP PAPERS • Harshal Patni, Satya S. Sahoo, Cory Henson, Amit Sheth, Provenance Aware Linked Sensor Data, 2nd Workshop on Trust and Privacy on Social and Semantic Web,Co-Located with ESWC, Heraklion Greece, May 30th - June 3rd 2010 • Harshal Patni, Cory Henson, Amit Sheth, Linked Sensor Data, In: Proceedings of 2010 International Symposium on Collaborative Technologies and Systems (CTS 2010), Chicago, IL, May 17-21, 2010 TECHNICAL REPORT • Harshal Patni, Cory Henson, Amit Sheth, and Pramod Ananthram. From Real Time Sensor Streams to Real Time Feature Streams, Kno.e.sis Center Technical Report, December 2009 • Joshua Pschorr, Cory Henson, Harshal Patni, and Amit Sheth. Sensor Discovery on Linked Data, Kno.e.sis Center Technical Report, December 2009 JOURNAL PAPER (In Progress) • Semantic Sensor Web: Design and Application towards weaving a meaningful sensor web 23
- 23.
- 24. QUESTIONS Demos, Papersand more at: http://semantic-sensor-web.com Semantic Sensor Web @ Kno.e.sis 25
Editor's Notes
- #3 Web data evolved over time from Static documents -> dynamic user generated content (Web 2.0) -> streaming (real-time) dataSo what is real-time data and what are its properties
- #4 Properties of streaming sensor data
- #5 Social media is the dominant source of streaming data now, however in future sensors would …Data needs to be reduced
- #6 Hence analyzing data in real-time becomes important
- #7 Define a feature and introduce what do we mean by a feature
- #8 So when we say Real-Time semantic analysis of streaming sensor data, we mean analyzing huge amount of lower level heterogeneous sensor data using background knowledge to create meaningful features (features that humans understand)
- #9 Abstraction across the temporal dimension over a single stream vs abstraction across the thematic dimension over multiple, multimodal, heterogenous streams using background knowledge
- #11 High level architecture
- #12 To walk through the implementation, lets take a sample questionThe question might look trivial, but it contains 3 important parts
- #13 Get all sensors using well known location names – Problem to be solveAssociate sensor descriptions to well know locations.
- #14 Convert the data raw sensor data collected and convert to RDF using W3C’s sensor ontology
- #19 Highlight the query with 3 boxes to show the temp, wind speed and precipitation streamHighlight the feature results too
- #20 Talk about the observations and features storage
- #22 Linked Data explodes
- #23 Online demo – http://knoesis1.wright.edu/?q=rtfs