Your SlideShare is downloading. ×
  • Like
GSN Global Sensor Networks for Environmental Data Management
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

GSN Global Sensor Networks for Environmental Data Management

  • 317 views
Published

GSN overview for the Mountain Observatories conference

GSN overview for the Mountain Observatories conference

Published in Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
317
On SlideShare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
19
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Global Sensor Networks An evolving middleware for sensor data stream processing Jean-Paul Calbimonte, Julien Eberle, Sofiane Sarni, Karl Aberer LSIR EPFL Mountain Observatories 2014, Reno Nevada 16.07.2014 http://gsn.epfl.ch
  • 2. Sensor deployments everywhere Mountains Glaciers Snow regions Sea Coastal Agriculture … DIY Mobile Participatory
  • 3. We want the data • Open data repositories • Accessible research datasets • Discoverability • Reuse datasets • Metadata
  • 4. GSN: Global Sensor Networks
  • 5. GSN: Global Sensor Networks Help managing sensor datasets Help publishing the data Help making the data discoverable and reusable
  • 6. GSN in a nutshell • Middleware: Sensor network deployment • Virtual Sensor (VS): Process streaming data • Hosts & manages multiple VSs 6
  • 7. Where is GSN? 7 • Sensor Network: • Sensing • In network • Data/query Processing • Filtering • Aggregation • Data Management: • Data: • Management. • Publishing. • Expensive processing. • Archives. Data Management Sensor Network GSN goes here
  • 8. Collecting data from different sources 8 • GSN works in a distributed fashion • Data can be kept locally • Break data silos • Put sensor data on the web GSN nodes
  • 9. GSN Distributed Deployment 9 Integrity Service Access Control GSN/Web/Web-Services Notification Manager Query Processor Query Repository Storage Manager Virtual Sensor Manager Input Stream Manager Stream Quality Manager Life Cycle Manager Pool Of Sensing Devices
  • 10. GSN Virtual Sensors 10 • A virtual sensor, any kind of data producer • a real sensor, a wireless camera, a desktop computer, GPS sensor, network traffic, etc. • combination of other virtual sensors. • Logical view of the sensor network. • Described in an XML file: • Functional/non-function properties. Source 1 Source 2 … Source n Application logic and processing Output Stream VirtualSensor
  • 11. Virtual Sensor configuration 11 <virtual-sensor name="room-monitor" > <addressing> <predicate key="geographical">BC143</predicate> <predicate key="usage"> room monitoring</predicate> </addressing> <life-cycle pool-size="10" /> <output-structure> <field name="image" type="binary:image/jpeg" /> <field name="temp" type="int" /> </output-structure> <storage permanent="true" history-size="10h" /> <input-streams> <input-stream name="cam"> <stream-source alias="cam" storage-size="1“ sampling-rate=“1”> <address wrapper=“tinyos2.x"> <predicate key=“host">tinybox.epfl.ch </predicate> <predicate key=“port">9001</predicate> </address> select * from WRAPPER </stream-source> <stream-source alias="temperature1“ storage-size="1m“ sampling-rate=“1”> <address wrapper="remote"> <predicate key="type">temperature</predicate> <predicate key="geographical">BC143-N </predicate> </address> select AVG(temp1) as T1 from WRAPPER </stream-source> <stream-source alias="temperature2“ storage-size="1m“> <address wrapper="remote"> <predicate key="type“>temperature</predicate> <predicate key="geographical“>BC143-S </predicate> </address> select AVG(temp2) as T2 from WRAPPER </stream-source> <query> select cam.picture as image, temperature.T1 as temp from cam, temperature1 where temperature1.T1 > 30 AND temperature1.T1 = temperature2.T2 </query> </input-stream> </input-streams> </virtual-sensor>
  • 12. Data in GSN through Wrappers 12 Common abstractions, independent of applications, hardware Simple integration & data correlation. 5140 GSN Various Applications Plug & Play deployment On-the-fly reconfiguration GSN GSN
  • 13. Some available mappings 13 • HTTP generic wrapper • devices accessible via HTTP GET or POST requests, e.g., the AXIS206W wireless camera • Serial forwarder wrapper • enables interaction with TinyOS compatible motes (standard access in TinyOS) • USB camera wrapper • local USB connection. • supports cameras with OV518 and OV511 chips. • RFID wrapper • access to Texas Instruments Series 6000 S6700 multi-protocol RFID readers • Alien Technologies long range RFID reader 8950 EU. • WiseNode wrapper • access to WiseNode sensors (CSEM, Switzerland, http://www.csem.ch/) • Generic UDP wrapper • any device using the UDP protocol • Generic serial/bluetooth wrapper • supports sensing devices which send data through the serial port, e.g., EPuck robots, etc.
  • 14. Wrappers: Lines of Code 14 50RFID reader (TI) 50Generic HTTP 300Wired camera 180Generic serial 45Generic UDP 75WiseNode 160TinyOS Lines of codeWrapper type
  • 15. Open source project: Available in Github • Open Source License • Mainly in Java • Community Support • Used in several projects
  • 16. Releases available in Github
  • 17. So what can I do with it? • Get data from my sensors (API, web interface) • Store and archive the data • Put it online, available for download • Put it online, available for discovery and querying • Apply post-processing to the data • Combine different data sources • Use the data from an R script • More stuff…
  • 18. Data Validation through Measurements and Modelling over Multiple Scales Lagrangian Dispersion Model High resolution urban atmospheric pollution maps Model Input Terrain, meteorology, source strength, background Sensor Data Crowd-sensors, mobile sensors, monitoring stations
  • 19. GSN Storage 19 • Centralized RDBMS  Trends: Data & Users  Evaluate a NoSQL solution Scalability Fault-tolerance Performance
  • 20. GSN Storage Extension 20 LSIR-Cloud HBASE Java API Client Experimental Platform CPU: 24 cores x 2.3GHz Mem: 64 GB Disk: 2.8 TB Nodes: 8 CPU: 8 x 12 cores x 2.3GHz Mem: 32 GB DFS Disk: 43 TB Network: 1 Gbps HDFS Cluster Put / Get HBASE Exporter HBASE Wrapper HBASE Query Handler User Requests VS data Store VS data Read VS data Execute Query
  • 21. SSN Ontology with other ontologies 21 W3C SSN Ontology tool for modeling our sensor data combine with domain ontologies
  • 22. GSN Access Control (AC) • VS has an owner: decides user access 22 VS: Virtual Sensor AC ISSUES REASON Private VS Features not visible VS Availability should be provided No Notifications Faster responses, if notified No Access Time Limitations Enable owner to control access Manual VS management Automation of the VS activation No AC in REST services Enable alternative data access
  • 23. More things we’re doing • Integration: integrate with Geo-enabled repositories (e.g. GeoNetworks) • Standards: NetCDF, OGC standards, OpenDAP • Metadata: add semantics to the data • Web standards: RDF and Linked Data • tinyGSN: for mobile devices
  • 24. Some example of Use • OpenIoT: Smart agriculture, manufacturing, etc. • SwissExperiment: environmental sensing • PlanetData: traffic data observation • OpenSense: air quality measurements • Permasense: mountain and snow observatory • Etc..
  • 25. OSPER - Swiss Experiment Open support platform for environmental research Multidisciplinary research team • Real world data + problems Facilitating research in: • Precipitation patterns in mountains • Evaporation in Africa • Return periods of Natural Hazards • Stream flows in Alpine catchments • Permafrost in the Alps managing environmental sensor data &metadata Platform http://swiss-experiment.ch Data heterogeneous sensing devices summarization, filtering, compression, interpolation continuous processing, streaming, geospatial, aggregation pattern discovery, correlation, regression metadata management, semantics data services, visualization, standards acquisition processing querying analysis discovery provision
  • 26. OpenSense2 global concern highly location-dependent time-dependent Crowdsourcing High-Resolution Air Quality Sensing Air Pollution Accurate location-dependent and real-time information on air pollution is needed Integrated air quality measurement platform  Heterogeneous devices and data  Human activity assessment, lifestyle and health data • Link high-quality and low-quality data • Integration of pure statistical models and physical dispersion models • Better coverage through crowdsensing • Incentives for crowd data provision • Finer temporal and spatial resolutions • Utilitarian approach for trade-off between model complexity, privacy and accuracy • Higher accuracy of pollution maps models http://opensense.epfl.ch Institutional stations OpenSense infrastructure Personal mobile sensors CrowdSense
  • 27. OpenIoT FP7 Open Source Cloud solution for the Internet of Things http://openiot.eu Established Open-source platform for IoT • Integrate sensors & things with cloud computing • Configure, deploy and use IoT services • Auditing/assessing privacy of IoT apps in the cloud • Semantic annotations of internet-connected objects • Energy-efficient data harvesting • Publish/subscribe for continuous processing and sensor data filtering • Mobility of sensors and QoS aspects in IoT https://github.com/OpenIotOrg/openiot Use cases and validation scenarios Smart Manufacturing Campus Guide Air Monitoring Agriculture Sensing
  • 28. Thanks a lot! Global Sensor Networks Jean-Paul Calbimonte LSIR EPFL http://gsn.epfl.ch