-
© Copyright 2013
OpenIoT Consortium
EU OpenIoT Project
FP7 ICT-2011 1.3: Internet-connected Objects
Open Source blueprint for large
scale self-organizing cloud
environments for IoT applications
OpenIoT X-GSN
OpenIoT Summer School, Galway, August 5, 2014
Jean-Paul Calbimonte
EPFL, Switzerland
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 2
Outline
• Why GSN
• Global Sensor Networks in a nutshell
– Virtual Sensors
– Wrappers
– Processing classes
– Distributed processing
– Storage
– Services
• X-GSN in OpenIoT
– Semantics
– Sensor & Observation annotations
– Linked Sensor Middleware
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 3
Sensor deployments everywhere
Mountains
Glaciers
Snow regions
Sea
Coastal
Agriculture
…
DIY
Mobile
Participatory
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 4
We want the data
• Open data repositories
• Accessible research datasets
• Discoverability
• Reuse datasets
• Metadata access
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 5
Sensor Networks and the Web
5
Sensor Networks
users
applications
data streams
Volume
Velocity
Variety WEB
Universal Web-based access to Sensor
data
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 6
Motivation
6
from Sensor Networks
to the Sensor Web
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 7
GSN: Global Sensor Networks
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 8
GSN: Global Sensor Networks
Help managing sensor datasets
Help publishing the data
Help making the data discoverable and reusable
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 9
GSN: things to know
GSN
GSN exposes Virtual Sensors
Sensing Device
temp_vs
timed: datetime
temperature: float
Virtual Sensor
GSN
Sensing Device
temp_vs
timed: datetime
temperature: float
Virtual Sensor
Aggregate, process, fusion
A virtual sensor, any kind of data producer
a real sensor, a wireless camera, a desktop computer, GPS sensor, network traffic, etc.
combination of other virtual sensors.
Logical view of the sensor network.
Described in an XML file
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 10
Virtual Sensor Config
<virtual-sensor name="testsensor_1" priority="10" >
<processing-class>
<class-name>org.openiot.gsn.vsensor.LSMExporter</class-name>
<output-structure>
<field name="noise" type="double" />
</output-structure>
</processing-class>
<description>sensorscope station</description>
<life-cycle pool-size="10"/>
<streams>
<stream name="input1">
…
<query>select timed, noise as noise from source1</query>
</stream>
</streams>
</virtual-sensor>
Virtual sensor name
Streams & wrappers
data fields
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 11
GSN: things to know
GSN
Virtual Sensors get the data through wrappers
wrapper
Examples:
•HTTP generic wrapper
•Serial forwarder wrapper
•USB camera wrapper
•RFID wrapper
•WiseNode wrapper
•Generic UDP wrapper
•Generic serial/bluetooth wrapper
•CSV wrapper
Or implement your own wrapper:
50RFID reader (TI)
50Generic HTTP
300Wired camera
180Generic serial
45Generic UDP
75WiseNode
160TinyOS
Lines of codeWrapper type
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 12
Wrapper config
<address wrapper="csv">
<predicate key="file">data/station_1061.csv</predicate>
<predicate key="fields">timed, noise</predicate>
<predicate key="formats">timestamp(d/M/y H:m), numeric</predicate>
<predicate key="bad-values">NaN,6999,-6999,null</predicate>
<predicate key="timezone">Etc/GMT-2</predicate>
<predicate key="sampling">5000</predicate>
<predicate key="check-point-directory">csv-check-points</predicate>
</address>
Wrapper type
File location
Fields
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 13
GSN: things to know
GSN
Virtual Sensors produce streams of data
Streams are processed on the fly by a Processing Class
Data Processing
Filtering
Aggregation
Outlier detection
Storing
Publishing
Expensive processing.
Archiving
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 14
GSN: things to know
• GSN works in a distributed fashion
– Data can be kept locally
– Break data silos
• Put sensor data on the web
14
GSN nodes
Remote wrappers can get data from another GSN instance
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 15
GSN Distributed Deployment
15
Integrity Service
Access Control
GSN/Web/Web-Services
Notification Manager
Query Processor
Query Repository
Storage Manager
Virtual Sensor Manager
Input Stream Manager
Stream Quality Manager
Life Cycle Manager
Pool Of Sensing Devices
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 16
Motivation
16
from Sensor Networks
to the Sensor Web
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 17
Heterogeneity problems
17
Discoverability
Heterogeneity
Metadata quality
Integration
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 18
Querying the semantic sensor Web
18
e.g. publish sensor data as RDF/Linked Data?
URIs as names of things
HTTP URIs
useful information when URI
is dereferenced
Link to other URIs
users
applicatio
ns
WEB
Use ontology models to continuously query real-
time data streams originated from sensors?
static vs. streams
one-off vs. continuous
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 19
The OpenIoT Architecture
Sensor data
management
Semantic data
management
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 20
We need ontologies!
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 21
We need ontologies!
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 22
X-GSN Sensor annotation
sensorName=opensense_1
sourceType=lausanne
sensorType=lausanne
information=Air Quality Sensors from Lausanne station 1
author=opensense
feature="http://lsm.deri.ie/OpenIoT/opensensefeature"
fields="humidity,temperature"
field.temperature.propertyName="http://lsm.deri.ie/OpenIoT/Temperature"
field.temperature.unit=C
field.humidity.propertyName="http://lsm.deri.ie/OpenIoT/Humidity"
field.humidity.unit=Percent
latitude=46.529838
longitude=6.596818
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 23
X-GSN Observation annotations
Open Source blueprint for large scale self-organizing
cloud environments for IoT applications
© Copyright 2013
OpenIoT Consortium 24
Motivation
24
from Sensor Networks
to the Sensor Web
and the Semantic Sensor Web
-
© Copyright 2013
OpenIoT Consortium
EU OpenIoT Project
FP7 ICT-2011 1.3: Internet-connected Objects
Open Source blueprint for large
scale self-organizing cloud
environments for IoT applications
OpenIoT X-GSN
OpenIoT Summer School, Galway, August 5, 2014
Jean-Paul Calbimonte
EPFL, Switzerland

X-GSN in OpenIoT SummerSchool

  • 1.
    - © Copyright 2013 OpenIoTConsortium EU OpenIoT Project FP7 ICT-2011 1.3: Internet-connected Objects Open Source blueprint for large scale self-organizing cloud environments for IoT applications OpenIoT X-GSN OpenIoT Summer School, Galway, August 5, 2014 Jean-Paul Calbimonte EPFL, Switzerland
  • 2.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 2 Outline • Why GSN • Global Sensor Networks in a nutshell – Virtual Sensors – Wrappers – Processing classes – Distributed processing – Storage – Services • X-GSN in OpenIoT – Semantics – Sensor & Observation annotations – Linked Sensor Middleware
  • 3.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 3 Sensor deployments everywhere Mountains Glaciers Snow regions Sea Coastal Agriculture … DIY Mobile Participatory
  • 4.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 4 We want the data • Open data repositories • Accessible research datasets • Discoverability • Reuse datasets • Metadata access
  • 5.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 5 Sensor Networks and the Web 5 Sensor Networks users applications data streams Volume Velocity Variety WEB Universal Web-based access to Sensor data
  • 6.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 6 Motivation 6 from Sensor Networks to the Sensor Web
  • 7.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 7 GSN: Global Sensor Networks
  • 8.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 8 GSN: Global Sensor Networks Help managing sensor datasets Help publishing the data Help making the data discoverable and reusable
  • 9.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 9 GSN: things to know GSN GSN exposes Virtual Sensors Sensing Device temp_vs timed: datetime temperature: float Virtual Sensor GSN Sensing Device temp_vs timed: datetime temperature: float Virtual Sensor Aggregate, process, fusion A virtual sensor, any kind of data producer a real sensor, a wireless camera, a desktop computer, GPS sensor, network traffic, etc. combination of other virtual sensors. Logical view of the sensor network. Described in an XML file
  • 10.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 10 Virtual Sensor Config <virtual-sensor name="testsensor_1" priority="10" > <processing-class> <class-name>org.openiot.gsn.vsensor.LSMExporter</class-name> <output-structure> <field name="noise" type="double" /> </output-structure> </processing-class> <description>sensorscope station</description> <life-cycle pool-size="10"/> <streams> <stream name="input1"> … <query>select timed, noise as noise from source1</query> </stream> </streams> </virtual-sensor> Virtual sensor name Streams & wrappers data fields
  • 11.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 11 GSN: things to know GSN Virtual Sensors get the data through wrappers wrapper Examples: •HTTP generic wrapper •Serial forwarder wrapper •USB camera wrapper •RFID wrapper •WiseNode wrapper •Generic UDP wrapper •Generic serial/bluetooth wrapper •CSV wrapper Or implement your own wrapper: 50RFID reader (TI) 50Generic HTTP 300Wired camera 180Generic serial 45Generic UDP 75WiseNode 160TinyOS Lines of codeWrapper type
  • 12.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 12 Wrapper config <address wrapper="csv"> <predicate key="file">data/station_1061.csv</predicate> <predicate key="fields">timed, noise</predicate> <predicate key="formats">timestamp(d/M/y H:m), numeric</predicate> <predicate key="bad-values">NaN,6999,-6999,null</predicate> <predicate key="timezone">Etc/GMT-2</predicate> <predicate key="sampling">5000</predicate> <predicate key="check-point-directory">csv-check-points</predicate> </address> Wrapper type File location Fields
  • 13.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 13 GSN: things to know GSN Virtual Sensors produce streams of data Streams are processed on the fly by a Processing Class Data Processing Filtering Aggregation Outlier detection Storing Publishing Expensive processing. Archiving
  • 14.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 14 GSN: things to know • GSN works in a distributed fashion – Data can be kept locally – Break data silos • Put sensor data on the web 14 GSN nodes Remote wrappers can get data from another GSN instance
  • 15.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 15 GSN Distributed Deployment 15 Integrity Service Access Control GSN/Web/Web-Services Notification Manager Query Processor Query Repository Storage Manager Virtual Sensor Manager Input Stream Manager Stream Quality Manager Life Cycle Manager Pool Of Sensing Devices
  • 16.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 16 Motivation 16 from Sensor Networks to the Sensor Web
  • 17.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 17 Heterogeneity problems 17 Discoverability Heterogeneity Metadata quality Integration
  • 18.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 18 Querying the semantic sensor Web 18 e.g. publish sensor data as RDF/Linked Data? URIs as names of things HTTP URIs useful information when URI is dereferenced Link to other URIs users applicatio ns WEB Use ontology models to continuously query real- time data streams originated from sensors? static vs. streams one-off vs. continuous
  • 19.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 19 The OpenIoT Architecture Sensor data management Semantic data management
  • 20.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 20 We need ontologies!
  • 21.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 21 We need ontologies!
  • 22.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 22 X-GSN Sensor annotation sensorName=opensense_1 sourceType=lausanne sensorType=lausanne information=Air Quality Sensors from Lausanne station 1 author=opensense feature="http://lsm.deri.ie/OpenIoT/opensensefeature" fields="humidity,temperature" field.temperature.propertyName="http://lsm.deri.ie/OpenIoT/Temperature" field.temperature.unit=C field.humidity.propertyName="http://lsm.deri.ie/OpenIoT/Humidity" field.humidity.unit=Percent latitude=46.529838 longitude=6.596818
  • 23.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 23 X-GSN Observation annotations
  • 24.
    Open Source blueprintfor large scale self-organizing cloud environments for IoT applications © Copyright 2013 OpenIoT Consortium 24 Motivation 24 from Sensor Networks to the Sensor Web and the Semantic Sensor Web
  • 25.
    - © Copyright 2013 OpenIoTConsortium EU OpenIoT Project FP7 ICT-2011 1.3: Internet-connected Objects Open Source blueprint for large scale self-organizing cloud environments for IoT applications OpenIoT X-GSN OpenIoT Summer School, Galway, August 5, 2014 Jean-Paul Calbimonte EPFL, Switzerland

Editor's Notes

  • #2 Cover Slide – Type 1
  • #3 Content Slide
  • #26 Cover Slide – Type 1