Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Ur2016

This presentation illustrates the open source software istSOS which allows to easily manage and share sensor data following a open standard. A real application case and software special features are presented.

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

Ur2016

  1. 1. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system 2016 UNDERSTANDING RISK FORUM –MAY 16 - 20, 2016, VENICE Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HydroMet sensor data management system Massimiliano Cannata, Milan Antonovic
  2. 2. Lake Verbano shores are exposed to flooding events that cause extensive damages Picture by Julien Daulte
  3. 3. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Large basins (6’386 km2) compared to limited lake area (213 km2) Intense reainfall events compared to maximum discharge (2,000 m3/s) Locarno: almost flat area highly urbanized
  4. 4. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Demo • But HOW did we achieve this How to Picture from David Oesch Building an Early Warning and Descision Support System the risk manage
  5. 5. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system EW-DSS components and workflow Meteorological Forecasts Hydrological modelling Data Validation Data Sharing Early Warning & DSS HydroMet(s) monitoring network Sensor Data GIS Data
  6. 6. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system International Basin 5 HydroMet networks
  7. 7. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Open Source Technolgies Open standard to foster interoperability Free and Open Source Sowftare to guarantee durability and flexibility Past experiences with proprietary solutions showed limitations that killed us 
  8. 8. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system 13/05/2016 8 HydroMet monitoring network of Canton Ticino, southern Switzerland Management of the
  9. 9. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Real time data collection In 2010, switch from analogue modem (collection every 4 hours) to GPRS sensor data transmission (real time) Needs for smarter system for collecting and dispatch measurements 12 maggio 2016 999 02 1 0
  10. 10. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Looking for a solution... 13/05/2016 10 Simple (possibly Python… shhh!) OpenStandard
  11. 11. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system The Sensor Observation Service standard 11 From http://www.opengeospatial.org/standards/sos “The Sensor Observation Service standard is applicable to use cases in which sensor data needs to be managed in an interoperable way. This standard defines a Web service interface which allows querying observations, sensor metadata, as well as representations of observed features. Further, this standard defines means to register new sensors and to remove existing ones. Also, it defines operations to insert new sensor observations. This standard defines this functionality in a binding independent way; two bindings are specified in this document: a KVP binding and a SOAP binding (in version 2.0).”
  12. 12. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system SOS User types 12
  13. 13. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system The software Open Source Software (GPL v2) 13
  14. 14. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system The istSOS software 14 mod_wsgi isodate psycopg2 pytz Database
  15. 15. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Some of the special features... 15
  16. 16. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system istSOS service for Canton Ticino HydroMet 16 151 registered sensors 15 observed properties (air-temperature, air-rainfall, water-height, water-height, air-humidity, water-discharge, water-height, air-pressure, air-radiation, water-conductivity, water-temperature, water-temperature, battery-tension, water-tension, air-relative_humidity) 40 years of data (1976-2016) 88.5 Mio registered observations 32 GB of database 8 GB of data served in April (4.5 GB with SOS requests & 3.21 GB with istSOS RESTful API) 5.7 Mio served requests in April 1 Internal server error response (500) in April
  17. 17. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system istSOS service for Canton Ticino HydroMet
  18. 18. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system istSOS service for Canton Ticino HydroMet 18 RAW data istSOS instance 10 min data istSOS instance 1 day data istSOS instance OBS: 49,6 Mio STAT: 151 SIZE: 13 GB SOS: 3.1 GB WA: 0.76 GB Req: 3.3 Mio Years: 26 OBS: 38 Mio STAT: 151 SIZE: 19 GB SOS: 1.4 GB WA: 2.45 GB Req: 2.4 Mio Years: 26 OBS: 0.9 Mio STAT: 151 SIZE: 184MB SOS: 10 MB WA: 3 MB Req: 190 Years: 40
  19. 19. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system EWS components with open standard and open software
  20. 20. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system The application for the civil protection
  21. 21. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Mode «Dams»: access to dam-break hazard zones and wave arrival times Mode «Alarm»: access to forecasted lake levels and exposed elements data (detect exposed elements with a given lake level, see plots of forecasted lake level, access instructions to secure exposed elements, etc.) Mode «Evacuation»: access to datails of population to manage evacuation (create evacauation event, search and locate person, count people in an area, assign people an evacauatio status, etc.) Mode «Regular»: access to data and funtions for resources management (search for parcel, address or person, query and eventually edit elements details, etc.)
  22. 22. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Access protected for data sensibility
  23. 23. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system The event of November 2014
  24. 24. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Lake Verbano events 12/05/2016 24 Basin = 6,386 km2 Lake = 213 km2 Discharge max = ca. 2,000 m3/s ave pre-alarm alarm 5 Lake Levels 193.66 194.85 195.75
  25. 25. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Flood of the 12-21.11.2014 max lake level 196.42 (alarm level 5) 12/05/2016 25
  26. 26. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Unique Visitors: 50 Requests / Second: 2.5 - 30 REQUESTS Requests Bandwidth SITGAP Web Services 25’372 0.83 GB WMS 190’112 2.57 GB Total 215’484 3.4 GB REQUESTS % ERROR CODE 205132 95.20 OK - The request sent by the client was successful 200 9568 4.44 Not Modified - Resource has not been modified 304 401 0.19 Unauthorized - Request needs user authentication 401 365 0.17 Bad Request - The syntax of the request is invalid 400 5 0.002 Document Not Found - Requested resource could not be found 404 10 0.004 Internal Server Error 500 SITGAP2: Average Daily Statistics during the last flood
  27. 27. istSOS
  28. 28. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Looking for a solution... 12/05/2016 28 Simple Python OpenStandard
  29. 29. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system The Sensor Observation Service standard 29 From http://www.opengeospatial.org/standards/sos “The Sensor Observation Service standard is applicable to use cases in which sensor data needs to be managed in an interoperable way. This standard defines a Web service interface which allows querying observations, sensor metadata, as well as representations of observed features. Further, this standard defines means to register new sensors and to remove existing ones. Also, it defines operations to insert new sensor observations. This standard defines this functionality in a binding independent way; two bindings are specified in this document: a KVP binding and a SOAP binding (in version 2.0).”
  30. 30. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system SOS User types 30
  31. 31. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system istSOS one enabler of the IOT 31 The Internet of Things (IoT) is the collection of billions of end devices intelligently connected and interoperating with servers and services. Combine “Things” like Lego blocks to create applications, services, values…. How to chose the blocks? what are they? how do they fits together? We need well known language: standards for machine – machine interaction! SOS is an OGC standard to: • register new sensor (THING) to the Web, • storing new observations • retrieving desired data
  32. 32. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system The software 32 Open Source Software (GPL v2)
  33. 33. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Suported system types (up-to-now) 12/05/2016 33 in-situ - fixed - point in-situ - mobile - points
  34. 34. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Supported versions 12/05/2016 34 SOS v1.0.0 & v2.0 (core)
  35. 35. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system The istSOS software 35 mod_wsgi isodate psycopg2 pytz Database
  36. 36. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Some of the special features... 36
  37. 37. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Easy installation and initialization and management using GUI (WebAdmin Interface) 37
  38. 38. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Data viewer 12 maggio 2016 38
  39. 39. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Output formats from GetObservation request XML – CSV - JSON 39
  40. 40. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Aggregation on-the-fly with GetObservation request AVG, SUM, MAX, MIN Extra parameters: aggregateInterval = PT1H & aggregateFunction = AVG 40
  41. 41. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Real Time Quality Check Configuration of quality index on the observed property level or sensor specific. Automatically data quality check assigning quality index on each observation based on specific constraints 41
  42. 42. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Raw data Assignment of quality indices during insertion: Quality index example 42 Raw data Meaningful data (Leve 1) Statistically sound (Level 1 + 2)
  43. 43. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Time-Zone support eventTime 2013-01-01T01:00:00+01 / 2013-01-07T01:00:00+01 eventTime 2013-01-01T02:00:00+02 / 2013-01-07T02:00:00+02 20th September 2013 43
  44. 44. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Virtual procedures (EVAPOTRANSPIRATION) 44 GRABOW Real Sensor Air Temperature Relative humidity Wind velocity Solar radiation V_GRABOW Virtual procedure Reference evapotranspirationFAO56 Calculation
  45. 45. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Only welcome page open (NEW) Security configurations 45 Fully Open Fully closed Only SOS requests open Only non-transactional SOS requests open
  46. 46. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system (NEW) User profiles 13 maggio 2016 46 datamanager: users with admin role can modify only measures and procedures metadata admin: users with admin role have access to all the istSOS features visitor: users with admin role can only view measures
  47. 47. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system (NEW) Acquisition Job Scheduler With Advanced Python Scheduler (APScheduler, embedded in istSOS) you can schedule sensor data to be inserted at times of your choosing 16 July 2014 47 Sensor File Transfer of Raw Data Server Scheduler Data acquisition istSOS
  48. 48. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system (NEW) MQTT support Temperature sensor 21.3 C MQTT broker21.3C subscribe MQTT broker Humidity sensor Time series analysis IoT - Event based behaviour publish insertObs
  49. 49. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system (New) istSOS Alert & Notification Service 16 July 2014 49
  50. 50. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Status Page (New) Sensor status at a glance taking in account acquisition delay and inserting errors 16 July 2014 50 AKNOWLEDGED BY ADMINISTATOR NO ERRORS OR DELAY PENDING ERRORS OR DELAY
  51. 51. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system (New) GSoC 2016 • istSOS widget HTM embaddable elements • istSOS Android API library to build-up Android application with enabled istSOS features • VistSOS client API library to enable complex visualization of istSOS time-series data 51
  52. 52. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system istSOS workshop @ FOSS4G2016 August 23rd, Tuesday – WS23 52
  53. 53. Lake flooding risk mitigation in Locarno (Switzerland ) using istSOS as HyMet sensor data management system Thanks www.supsi.ch/ist geomatica@supsi.ch 12/05/2016 53 http://geomatica-supsi.blogspot.ch/ http://geoservice.ist.supsi.ch/+geomatics 53

×