Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Daata Management service #2
CIP-ICT-PSP-2012-6 – 32516...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Daata Management service #2
CIP-ICT-PSP-2012-6 – 32516...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Daata Management service #2
CIP-ICT-PSP-2012-6 – 32516...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Daata Management service #2
CIP-ICT-PSP-2012-6 – 32516...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Daata Management service #2
CIP-ICT-PSP-2012-6 – 32516...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Daata Management service #2
CIP-ICT-PSP-2012-6 – 32516...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Daata Management service #2
CIP-ICT-PSP-2012-6 – 32516...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Daata Management service #2
CIP-ICT-PSP-2012-6 – 32516...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
Upcoming SlideShare
Loading in …5
×

S.2.h Meter Data Management Service

663 views

Published on

SUNSHINE Project: meter data management service

Published in: Technology
  • Be the first to comment

  • Be the first to like this

S.2.h Meter Data Management Service

  1. 1. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Daata Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). D4.5 Meter Data Management Service #2 WP 4 – Integration of SUNSHINE pilot smart urban services Task 4.2 – Meter data management service Revision: v1.0 Authors: Stefano Pezzi (SGIS) Luca Giovannini (SGIS)
  2. 2. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Daata Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Dissemination level PU (public) Contributor(s) Stefano Pezzi, Martina Forconi, Oscar Benedetti, Enrico De Guidi, Luca Giovannini (SGIS), Luis Torres (Meteogrid) Reviewer(s) Federico Prandi (Graphitech) Editor(s) Raffaele De amicis (Graphitech) Partner in charge(s) SGIS Due date 31/01/2015 Submission Date 31/03/2015 REVISION HISTORY AND STATEMENT OF ORIGINALITY Revision Date Author Organisation Description 0.1 08/02/2015 Luca Giovannini SGIS Document creation 0.5 19/03/2015 Luis Torres Meteogrid Document review 1.0 20/03/2015 Luca Giovannini SGIS Document finalization Statement of originality: This deliverable contains original unpublished work except where clearly indicated otherwise. Acknowledgement of previously published material and of the work of others has been made through appropriate citation, quotation or both. Moreover, this deliverable reflects only the author’s views. The European Community is not liable for any use that might be made of the information contained herein.
  3. 3. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Daata Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Table of content Table of content........................................................................................................................... 3 Acronyms...................................................................................................................................... 3 1 Introduction ....................................................................................................................... 5 2 Meter data management service....................................................................................... 5 2.1 Overall architecture....................................................................................................... 6 3 Hardware components ...................................................................................................... 9 3.1 Main platform servers ................................................................................................. 10 3.1.1 Backend server...................................................................................................... 10 3.1.2 FTP server ............................................................................................................. 10 3.1.3 Webserver server.................................................................................................. 10 3.1.4 Firewall server....................................................................................................... 11 4 Software components...................................................................................................... 12 4.1 Sensor DB (52° North SOS 4.0)..................................................................................... 12 4.1.1 Database structure ............................................................................................... 13 4.1.2 Database management......................................................................................... 16 4.2 FTP ingestion................................................................................................................ 18 4.3 Meter data processing................................................................................................. 19 4.4 CSV loader.................................................................................................................... 20 4.5 Weather ingestion ....................................................................................................... 21 4.6 GreenButton ingestion ................................................................................................ 23 4.7 Sensor data access....................................................................................................... 25 5 Sunshine Weather Platform............................................................................................. 27 5.1 Platform architecture .................................................................................................. 27 5.1.1 Weather Data Collecting Service .......................................................................... 28 5.1.2 Weather Data Exhibition Service - WMS – WFS ................................................... 28 5.2 Software components.................................................................................................. 29 5.2.1 Core modules........................................................................................................ 29 5.2.2 Zope Object Database........................................................................................... 30 5.2.3 Data ingestion in Sigym3 ...................................................................................... 31 5.2.4 Data Access........................................................................................................... 31 Annex I - Sensor DB schema (52° North SOS 4.0)....................................................................... 32 Annex II – Mapping weather WFS to Sensor DB........................................................................ 33 Acronyms
  4. 4. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Daata Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). CPU Central Processor Unit CSV Comma Separated Values DB Data Base DMZ DeMilitarized Zone FTP File Transfer Protocol IT Information Technology LAN Local Area Network SOS Sensor Observation Service
  5. 5. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Daata Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 1 Introduction This document is an annex to the final deliverable of the Meter Data Management Service of the Sunshine project. The following chapters describe the implementation of the service with respect to hardware components, interaction with pilot infrastructures and functional software modules. The reader can also refer to deliverable D1.5 “SUNSHINE System architecture – final” for an overview of this service and of its relation with the other elements of the Sunshine platform. 2 Meter data management service During the development of SUNSHINE’s platform the scope of Meter Data Management Service has been extended from the management of meter data to the management of data coming more generally from sensors. The reason being that SUNSHINE’s pilots are providing not just consumption data from meters, but also several other types of data from other sensors that also need to be collected and managed to support SUNSHINE’s services and piloting activities. More specifically, the Meter Data Management service manages:  Consumption data from energy meters in pilot buildings;  Indoor temperature readings from indoor sensors in pilot buildings;  Weather observation and forecasts data from a dedicated weather service from Meteogrid;  Status data (dimming level, hours of use, etc) from pilot lamps and light-lines. All these data types are stored in the same Sensor DB and managed with common procedures, which will be described in this deliverable. In order to avoid redundancies, the detailed description of the management of status data for lamps and light-lines, which is also part of the Remote System Management service, has not been repeated here and can be found in deliverable D4.6 “Remote System Management Service n2”. In brief, the purpose of the Meter Data Management Service is to:  Gather all the data coming from meters/sensors;  Elaborate them and generate derived data, if necessary (Section 4.6);  Store them in a normalized format into a repository (Section 4.1) from which they can be retrieved through a standard interface (Section 4.7). For consumption data from meters two data flows are supported by the platform, in order to enable both the pilots with low IT capacities and those who are able to develop software components:
  6. 6. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Daata Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp).  Simple FTP transfer of CSV file;  Feed from Green Button web service. Two service endpoints correspond to these types of flows:  FTP Ingestion service (Section 4.2);  Green Button Ingestion service (Section 4.6). To maximize the reuse of the developed software, the Green Button Ingestion service doesn’t load directly the data in the DB, but produces a CSV file that is treated by a dedicated component, the CSV loader (Section 4.4), that is the same used by the FTP Ingestion service. The flow of indoor temperature readings is channelled through the Green Button web service (Section 4.6) as all the pilots actually providing such data flow have implemented the needed infrastructure. The flow of weather data (Section 4.5) is established via a scheduling procedure that is planned to pull data daily from the dedicated Web Feature Services (WFS) set up by partner Meteogrid (whose dedicated platform is described in detail in Chapter 5). The flow of lamp status data is managed by the Remote System Management service (see deliverable D4.6 “Remote System Management Service n2”) that ensures the uniform ingestion of such data via the ingestion API of the Sensor DB. 2.1 Overall architecture The overall architecture of Sunshine is shown in Figure 1, we’ll now roughly describe the whole system and then we’ll focus on the components involved in the meter data management service. The diagram is quite complex and the different colours differentiate the various functional sections of the entire system:  Red modules compose the security layer;  Green modules compose the meter data management service, overseeing the ingestion of different data flows into Sunshine’s sensor database;  Blue modules compose the remote system management service, managing light line control and monitoring;  Grey modules compose the building pre-certification service, managing building geodata and estimating building heath needs. Services are implemented by the boxes representing functional units of software and by the blue circles that define the interfaces. In the left side of the diagram there are some of the nodes that deliver information to the Sunshine’s platform plus the Sunshine’s client application. An exception is represented by the
  7. 7. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Daata Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). brown node on the right side, at the bottom, that is a virtual machine with City of Ferrara’s software components that is hosted at Sinergis’ premises1 . The server components are on the right portion of the diagram: here the red ones represent the security layer that will be described more in detail in the D4.6 Annex on Security Layer. The layer doesn’t add any additional functionality, but intercepts the requests that are sent to services that need to be protected. Even the reverse proxies defined inside the web server are in red, because this way to expose services on the internet adds also a little degree of protection besides other advantages. 1 Physically the VM is located in the same LAN of the Sunshine’s platform, but the interactions with its components are made via web protocols so it can be deployed to the pilot’s premises without any problem.
  8. 8. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Daata Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Figure 1 - Overall deployment diagram
  9. 9. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 3 Hardware components Figure 2 - Physical and virtual server The SUNSHINE specific servers are deployed on a virtualized IT infrastructure based on VMware vSphere software (the “Back end VM Server” in Figure 2). The underlying hardware has the following features: CPU Architecture x86_64 CPU Number 2 (6 core each) CPU Clock 2300 MHz Hard Drive Space 12TB As SGIS plays in SUNSHINE also the role of technical partner of the Ferrara pilot, the same infrastructure hosts as well the pilot’s server (sunshine-fe.nco.inet) used to collect the meter data from the various buildings and to transmit them to the SUNSHINE platform. The Ferrara’s server shares only the virtualization infrastructure and the internet domain (sinergis.it) with the central platform, but all the communications between the SUNSHINE platform and the pilot’s server passes through internet protocols. Other servers that are functional to the SUNSHINE platform, but play generic roles (firewall, web server, ftp server) are based on separate hardware in order to achieve a correct and secure network topology and to fit in the existent SGIS internet infrastructure. In particular we
  10. 10. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). have the FTP server (ftp-vm) that is a VM hosted in another virtualization infrastructure located in the DMZ. Then we have the web server (coreweb2) that is a physical server located in the DMZ and finally the firewall. More details are given in the following paragraphs. In the diagram of Figure 2 also a sunshine-revdb.nco.inet virtual server is shown, but its role is descripted in D4.3. 3.1 Main platform servers The main components are deployed on the backend server that is positioned in the internal LAN in SGIS’ premises in Casalecchio di Reno, Bologna; the others server involved are:  the FTP server, that is located in a separate VM server positioned in the DMZ for security reasons;  the web server is also a physically separated server; this server is used by SGIS company not only for the SUNSHINE project, but for all the other company’s project and customers.  the firewall; also this server is a company’s asset that is functional to all projects and that is used to guarantee security for the server that are exposed on the internet. 3.1.1 Backend server Operating System Ubuntu 12.04.3 LTS 64bit Host Name IP address 3.1.2 FTP server Operating System Ubuntu 12.04.3 LTS 64bit Host Name IP address DNS name 3.1.3 Webserver server Operating System CentOS 6.4 64bit Host Name IP address
  11. 11. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 3.1.4 Firewall server Operating System Slackware 12.0 64bit Host Name IP address
  12. 12. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 4 Software components The components are described in details in the sections of this chapter and their mutual relation is described in the figure below (in green). Figure 3 - The meter Data Management service components 4.1 Sensor DB (52° North SOS 4.0) It is the final repository of all sensor data. A database implementation of OGC Sensor Observation Service2 (SOS) standard protocol developed by 52° North3 has been installed and configured on Sunshine’s platform for this purpose. It is implemented as a PostgreSQL DB and also makes use of the spatial extension PostGIS. 2 http://www.opengeospatial.org/standards/sos 3 http://52north.org/communities/sensorweb/sos/index.html
  13. 13. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 4.1.1 Database structure Annex I of this document reports the UML schema of the sensor DB. The main constructs of the DB schema are the following:  Codespace: univocally identifies the pilot;  Feature of Interest: represents the geo-object who is measured by sensors. The FOI is the means of locating (geocoding) the measuring points via their coordinates;  Observable Property: identifies the property observed by the sensor (e.g. gas consumption, temperature or lamp dimming level);  Procedure: identifies a physical /virtual sensor which produces the observations associated with the offerings;  Offering: a logical grouping of observations that are related to each other, which are jointly offered by a service. The following sub-sections describe the uniform and consistent naming convention that has been defined for these constructs. Codespace CodeSpaceUri (examples) CodeSpace Pilot http://www.sunshineproject.eu/swe/project-pilot/ROV ROV Rovereto http://www.sunshineproject.eu/swe/project-pilot/FER FER Ferrara http://www.sunshineproject.eu/swe/project-pilot/BAS BAS Bassano http://www.sunshineproject.eu/swe/project-pilot/LAM LAM Lamia http://www.sunshineproject.eu/swe/project-pilot/PAO PAO Paola http://www.sunshineproject.eu/swe/project-pilot/TRN TRN Trentino http://www.sunshineproject.eu/swe/project-pilot/VDN VDN Val di Non http://www.sunshineproject.eu/swe/project-pilot/HRV HRV Croatia CodeSpaceUri: <radix>/CodeSpace Feature of Interest FeatureofinterestUri (examples) http://www.sunshineproject.eu/swe/featureOfInterest/FER/building/ScuolaACosta http://www.sunshineproject.eu/swe/featureOfInterest/FER/building/PalazzinaMarfisaDEste http://www.sunshineproject.eu/swe/featureOfInterest/HRV/weatherstation/06 http://www.sunshineproject.eu/swe/featureOfInterest/ROV/lamp/101 http://www.sunshineproject.eu/swe/featureOfInterest/ROV/lamp/102 http://www.sunshineproject.eu/swe/featureOfInterest/BAS/lightline/PiazzaGuadagnin http://www.sunshineproject.eu/swe/featureOfInterest/LAM/building/SchoolTechApplications2 http://www.sunshineproject.eu/swe/featureOfInterest/ROV/lightline/RotondaMarco
  14. 14. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). FeatureofinterestUri: <radix>/CodeSpace/<natura_della_Feature>/<Identificativo> FoI nature Building Lightline Lamp Weatherstation Observable property ObservablepropertiesUri (examples) http://www.sunshineproject.eu/swe/observableProperty/ELEC http://www.sunshineproject.eu/swe/observableProperty/TEMP http://www.sunshineproject.eu/swe/observableProperty/IPOW http://www.sunshineproject.eu/swe/observableProperty/DIMM http://www.sunshineproject.eu/swe/observableProperty/LIFE http://www.sunshineproject.eu/swe/observableProperty/IRRA http://www.sunshineproject.eu/swe/observableProperty/GASR ProcedureUri: <radix>/<CodeProperty> CodeProperty Description ELER Electrical Energy Consumption, relative GASR Gaseous Fuel Consumption, relative THER Thermal Energy Consumption, relative LIQR Liquid Fuel Consumption, relative SLDR Solid Fuel Consumption, relative STMR Steam Fuel consumption, relative ELEC Electrical Energy Cost GASC Gaseous Fuel Cost THEC Thermal Energy Cost LIQC Liquid Fuel Cost SLDC Solid Fuel Cost STMC Steam Fuel Cost ELEA Electrical Energy Consumption, absolute GASA Gaseous Fuel Consumption, absolute THEA Thermal Energy Consumption, absolute IPOW Lamp instantaneous power DIMM Lamp dimming LIFE Lightline usage hours
  15. 15. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). ONOF Lamp CB Status NALL Lightline CB Number N-OK Lightline CB Accreditate SUNR Lightline sunrise time SUNS Lightline sunset time ICUR Lamp current COSF Lamp Cos(phi) IVOL Lamp voltage SCDL Lamp scheduler ACNT Lamp accounting TEMP Temperature WIND Wind Speed Module IRRA Short Wave Irradiation Procedure ProcedureUri (examples) http://www.sunshineproject.eu/swe/procedure/ROV-100 http://www.sunshineproject.eu/swe/procedure/VDN-F00 http://www.sunshineproject.eu/swe/procedure/BAS-F00 http://www.sunshineproject.eu/swe/procedure/TRN-M01 http://www.sunshineproject.eu/swe/procedure/TRN-T01 ProcedureUri: <radix>/<codespace>-<alphaNumeric> alphaNumeric:  Building/shelter consumption data: numeric between 000 and 999  Lamp/line consumption and state data: o Hundreds identify the line o Tens and units identify lamps belonging to the same line  Weather Forecast data: F<n> with n ∈ [0..99]  Weather Meteorological conditions: M<n> with n ∈ [0..99]  Indoor Temperature data: T<n> with n ∈ [0..99] Offering Offering (examples) http://www.sunshineproject.eu/swe/offering/ROV-F01_TEMP_CEL_1h http://www.sunshineproject.eu/swe/offering/LAM-M01_TEMP_CEL_1h http://www.sunshineproject.eu/swe/offering/HRV-001_ELER_KWH_1h
  16. 16. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). http://www.sunshineproject.eu/swe/offering/FER-001_GASR_MCU_1d OfferingUri: <radix>/CodeSpace- <alphaNumeric>_<codeProperty>_<uom>_<frequenza_di_campionamento> Sampling frequency Description 10 Every 10 minutes 15 Every 15 minutes 1h Hourly 1d Daily 1m Monthly ir Irregular frequency Unit of measure Description KWH "kWh" CEL "°C" MCU "m3" EUR "Euro" WAT "W" PRC "%" HRS "hours" BOO "0/1" NUM "elements" HMS "date" AMP "Ampere CDL "codelist" KWA "kW" KMH "m/s" WM2 "W/m2" 4.1.2 Database management The management of the database structure have been performed via:  The use of the SOS APIs for the creation/deletion of new procedures (insertSensor, deleteSensor);  The use of custom SQL scripts for the creation/deletion of new offerings. Scripts take care to modify all the entries related to offerings in the various DB tables. The creation of a new offering implies the following actions on the DB:
  17. 17. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). DB TABLE Action offering Create entry with identifier, e.g. “LAM- M01_TEMP_CEL_1h”. observableproperty Verify entry existence for “Observable Property” of the offering, e.g. “TEMP”. unit Verify entry existence for "Unit" of the offering, e.g. “CEL”. featureofinterest Verify entry existence for “Feature of Interest” of the offering, specifying:  featureofinteresttypeid among options in table featureofinteresttype  codespaceid series Insert entry with  featureofinterestid  observablepropertyid  procedureid observationconstellation Insert entry with  observablepropertyid  procedureid  observationtypeid  offeringid offeringallowedobservationtype Insert entry with  offeringid  observationtypeid offeringallowedfeaturetype Insert entry with  offeringid  featureofinteresttypeid relatedfeature Verify entry existence with  Featureofinterestid offeringhasrelatedfeature Insert entry with  relatedfeatureid  offeringid
  18. 18. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). DB management view The DB management view is a fundamental tool for the set-up and maintenance of the DB structure and for the monitoring of the correct flowing of data into the DB. At runtime the Sensor DB has hundreds of offerings and a manual check on the correctness of all offerings would be unfeasible. As Figure 4 shows, the view has one entry per offering and for each entry it specifies:  Feature of Interest identifier  Procedure identifier  Offering identifier  Observable Property identifier  Unit of measure  Reading Frequency  Observations count for the offering  Timestamp of first observation  Timestamp of last observation Figure 4 The Db management view 4.2 FTP ingestion It’s a scheduled program that checks if any CSV file to be loaded into the database is arrived on the dedicated FTP folders. The files are removed from the folders and passed to the CSV loader component that does the real job of inserting data in the DB. Detailed guidelines have been prepared describing how to format name and content of CSV files in order to allow the automatic ingestion procedure to take place smoothly. Guidelines are available in the deliverable D2.4 Annex “Specifications_for_data_ingestion_via sunshine_FTP”.
  19. 19. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). In order to configure the service, pilots compiled a mapping file that has the purpose of describing the basic properties of the meter/sensor and allows to identify which pilot building or light line does it belong to. An example of the content of a meter mapping file for the pilot in Lamia is shown in Figure 5. Figure 5 – Meter mapping file 4.3 Meter data processing Some elaborations are needed before the real data ingestion. For instance, some aggregations are pre-calculated in order to avoid this task to the client. Another kind of processing is the interpolation for those readings that have been collected at irregular intervals of time (some kinds of data flows have this characteristic). This module is specific to the consumption data from sensor reading and its role is that of guaranteeing a uniform format for its storage and therefore a cleaner server/client interaction. Stored consumption data has the following format:  Consumption is relative to the last reading interval;  Readings have a regular frequency;  Available frequencies are: every 15 minutes (10 minutes for TNET), hourly and daily. These constraints do not apply to consumption data coming from billing documentation that are stored in the BD as is. In order to guarantee the above-mentioned format, three main processing tasks that have been developed:  Data resampler, from irregular frequency to regular frequency;  Differential consumption, from absolute readings to relative readings (keeping track of the last known absolute reading);  Data aggregator, from high frequency regular relative readings to low frequency regular relative readings (e.g. from 15 minutes frequency to hourly and daily).
  20. 20. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 4.4 CSV loader This component accesses the DB for inserting the readings in the proper tables. Before doing that, if it has been configured to do so, the files are pre-processed by the “Meter data processing” component. Readings are inserted in the DB using SOS insertion API, insertObservation. What follows is an example of an insertObservation call: <?xml version="1.0" encoding="UTF-8"?> <env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.w3.org/2003/05/soap-envelope http://www.w3.org/2003/05/soap-envelope/soap-envelope.xsd"> <env:Body> <sos:InsertObservation service="SOS" version="2.0.0" xmlns:sos="http://www.opengis.net/sos/2.0" xmlns:swes="http://www.opengis.net/swes/2.0" xmlns:swe="http://www.opengis.net/swe/2.0" xmlns:sml="http://www.opengis.net/sensorML/1.0.1" xmlns:gml="http://www.opengis.net/gml/3.2" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:om="http://www.opengis.net/om/2.0" xmlns:sams="http://www.opengis.net/samplingSpatial/2.0" xmlns:sf="http://www.opengis.net/sampling/2.0" xmlns:xs="http://www.w3.org/2001/XMLSchema" xsi:schemaLocation="http://www.opengis.net/sos/2.0 http://schemas.opengis.net/sos/2.0/sos.xsd http://www.opengis.net/samplingSpatial/2.0 http://schemas.opengis.net/samplingSpatial/2.0/spatialSamplingFeature.xsd"> <sos:offering>http://www.sunshineproject.eu/swe/offering/ROV- 213_ICUR_AMP_ir</sos:offering> <sos:observation> <om:OM_Observation gml:id="o1"> <gml:identifier codeSpace="http://www.sunshineproject.eu/swe/project- pilot/ROV">http://www.sunshineproject.eu/swe/observation/ROV-213_ICUR_AMP_ir/2015-03-11 11:53:24</gml:identifier> <om:type xlink:href="http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_Measurement"/> <om:phenomenonTime> <gml:TimePeriod gml:id="Ki.ObsTime.1"> <gml:beginPosition>2015-03- 11T04:50:47.000+01:00</gml:beginPosition> <gml:endPosition>2015-03-11T05:50:47.000+01:00</gml:endPosition> </gml:TimePeriod> </om:phenomenonTime> <om:resultTime> <gml:TimeInstant gml:id="Ki.resTime.1"> <gml:timePosition>2015-03- 11T05:50:47.000+01:00</gml:timePosition> </gml:TimeInstant> </om:resultTime> <om:procedure xlink:href="http://www.sunshineproject.eu/swe/procedure/ROV-213"/> <om:observedProperty xlink:href="http://www.sunshineproject.eu/swe/observableProperty/ICUR"/> <om:featureOfInterest xlink:href="http://www.sunshineproject.eu/swe/featureOfInterest/ROV/lamp/213"/>
  21. 21. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). <om:result xsi:type="gml:MeasureType" uom="A">29</om:result> </om:OM_Observation> </sos:observation> </sos:InsertObservation> </env:Body> </env:Envelope> 4.5 Weather Data ingestion Weather data are served by two WFS services set up by partner Meteogrid from a dedicated platform described in detail in Chapter 5. The ingestion process queries the WFS services, collects the data for each meteorological quantity (temperature, irradiation, wind speed), and loads it into the Sensor DB. More in details here are the WFS services:  WFS forecast A three-day forecast with hourly resolution is available daily for each pilot’s location.  WFS observation Measured or interpolated weather conditions are available with hourly resolution for each pilot’s location. From both WFSs the feature types that are read and ingested to the SOS are: 1. temperature_s: temperature in °C (Celsius Degrees) 2. wind_speed_s: wind module in m/s 3. short_wave_irradiation_s: short wave irradiation in W/m2 Each feature type has the following properties, as can be seen in Figure 6:  msGeometry (gml:Point)  name: name of observatory (example: ZAGREB165)  timestamp (data are available with hourly frequency)  value  type  observatory_id <gml:featureMember> <sigym:temperature_s fid="temperatures_2015-03-13 01:00:00+01:00,temperatura_obs_int,165"> <sigym:msGeometry> <gml:Point srsName="EPSG:4326"> <gml:coordinates>15.965225,45.807468999999998</gml:coordinates> </gml:Point> </sigym:msGeometry> <sigym:name>ZAGREB165</sigym:name> <sigym:timestamp>2015-03-13T01:00:00+01:00</sigym:timestamp> <sigym:value>6.0</sigym:value> <sigym:mode/> <sigym:type>temperatura_obs_int</sigym:type>
  22. 22. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). <sigym:observatory_id>165</sigym:observatory_id> </sigym:temperature_s> </gml:featureMember> Figure 6 - Feature example of temperature_s from observation WFS Each feature type of both WFS (forecast and observation) has 83 feature members representing the observatories. The observatory name and id are the same in both WFSs. Annex II describes the mapping between feature members of the WFS with procedures of the SOS. The mapping between WFS and SOS has the following main characteristics:  a feature member of the WFS (observatory) correspond in the SOS to: o one FeatureOfInterest; o two Procedures (one for the forecast and one for the observation); o six Offerings; three offering (temperature, wind, irradiation) for forecast and for observation.  a value of the WFS correspond to an Observation in the SOS The Procedure name in the SOS is: <pilot>-F/M<nn> where F=forecast and M=measure (observation) The service that reads the WFS and ingest the data to SOS:  is scheduled once a day at 10:10:00 AM  the service makes 3 GetFeature calls to the Forecast WFS and 3 GetFeature calls to the Observation WFS, one call for each feature type (temperature_s, wind_speed_s, short_wave_irradiation_s);  the GetFeature call response for the Forecast WFS returns the information for the next 3 days (72 hours) for the 83 observatories. The service deletes the old forecast and ingests the values into the SOS.  for the Observation WFS the service calls a GetFeature using the TIME parameter to specify a range covering one day (from the day before of current date of scheduling to the current scheduling). The response to the GetFeature call returns the information for 24 hours for each of the 83 observatories; the values are ingested into the SOS (and no deletion of old values is done). The ingestion module refers to a configuration file (importMeteoDataConfiguration.xml) that allows configuring:  Ingestion scheduling (hour, minute, second)  URL of WFSs  WFS Featuretype names  Mapping between “WFS observatory id” and “SOS procedure code”
  23. 23. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp).  CodeSpace Identifier of the Pilot  FeatureOfInterest Identifier 4.6 GreenButton ingestion A Smart Energy Grid has many different nodes (generation facility, distribution network, utility company data storage facility, smart meters, local data concentrator, etc) and actors (energy producer, energy deliverer, energy seller, energy customer, service provider company, etc) and consequently many different types of communication channels, each with its own set of possible communication protocols. In the context of the Sunshine project, the pilot partner has both the role of energy user and the role of data storage facility, while the Sunshine platform is the service provider company that needs access to the consumption data stored in the storage facility in order to provide value-adding services. This is exactly the use case typology that is covered by the GreenButton protocol4 , the international data exchange standard for consumption data that have been chosen to be used in SUNSHINE’s platform. The Sunshine platform does not communicate directly with the smart meters of the pilots and is therefore not implementing any protocol related to direct communication with smart meters. Green Button is an initiative of the U.S. Government that aims to allow consumers to access their own meter data about energy consumption. In the original context, the name Green Button denotes the whole initiative, but in our project we will call GB only the suite of web services that makes the data access possible. GreenButton protocol and API were identified as the best option to implement the use case described above for several reasons:  It is already in operational use;  It is supplied with examples and demos;  It is a standard aiming at becoming international5 So GreenButton provides Sunshine with a protocol already implemented and operational in a real use case scenario. This module consumes a REST service that returns an XML that is compliant with the AtomPub standard that is the underlying standard of the Green Button. Concerning the handling of authentication and authorization in the data communication process between the pilots and the Sunshine platform, GreenButton protocol relies on OAuth protocol which has a widespread use for the use cases we are concerned with (three-partner authorisation between data owner, data custodian, data user). For the sake of truth, we must highlight that there is not a real “service endpoint” for this kind of flow. As shown in Figure 7, the meter data producers actually acts like server, publishing their readings and the SUNSHINE platform 4 http://energy.gov/data/green-button 5 http://www.sgip.org/PAP-20-Green-Button-ESPI-Evolution
  24. 24. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). acts like a consumer of the feeds. The roles are therefore switched and the GreenButton ingestion service must be configured with the address of the pilot’s feed to read. Detailed guidelines have been prepared describing the GreenButton standard, its security and authentication module based on OAuth protocol6 and providing instruction on how to implement a web service supporting the protocol. Guidelines are available in deliverable annex “D4.5 Annex - Specifications_for_data_ingestion_via_GreenButton_v1.2”. Figure 7 - Consuming a GreenButton service Support was given via Skype telcos and mail exchange to pilots that decided to implement a web service based on GreenButton protocol to expose consumption data. The most relevant issues were related to the securization of the infrastructure:  exchange of OAuth security token between pilot and central infrastructure;  activation of HTTPS secure protocol; 6 http://oauth.net/
  25. 25. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Moreover, a mapping was established for each pilot between the data structure of the GreenButton DB residing in the pilot infrastructure and the data structure of the central Sensor DB. This mapping was requested in the guidelines disseminated to pilots, where also a template for the mapping itself was provided. Figure 8 shows a compiled example of such mapping table. Figure 8 – Mapping between GreenButton DB structures (usage point, meter reading, reading type) and Sensor DB (meter identifier, last column). 4.7 Sensor data access The last module, in logical order, is the sensor data access. Access to sensor data is guaranteed by two services:  Web feature service (see Figure 9): o To publish the identifiers of the different sensors and data flows via the linking with their geographical position.  Sensor observation service APIs: o GetCapabilities: returns an XML service description with information about the interface (offered operations and endpoints) as well as the available sensor data, such as the period for which sensor data is available, sensors that produce the measured values, or phenomena that are observed;
  26. 26. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). o DescribeSensor: provides sensor metadata in SensorML. The sensor description can contain information about the sensor in general, the identifier and classification, position and observed phenomena, but also details such as calibration data; o GetObservation: allows pull-based querying of observed values, including their metadata. The measured values and their metadata is returned in the Observations and Measurements format (O & M); o GetResult: provides the ability to query for sensor readings without the metadata given consistent metadata (e.g. sensor, observed object). Figure 9 – Sensor locations (blue dots) and identifiers (attributes in the table) exposed via a Web Feature Service.
  27. 27. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 5 Sunshine Weather Platform This chapter describes the platform implemented at Meteogrid premises that is responsible for the provision of weather data, both observation and localized forecasts, to the central SUSNHINE’s platform for all pilot locations. 5.1 Platform architecture In order to support SUNSHINE’s platform services Meteogrid’s team has designed and set up an IT platform that can generate specific weather forecasts for the SUNSHINE pilot areas, collect information from NOAA, ECMWF and local weather centres and compute a specific forecast downscaling for each building. This platform consists of two dedicated servers  MeteoGrid Weather Forecast server;  Sigym3 MeteoServer. MeteoGrid Weather Forecast server houses a WRF implementation. WRF - Weather Research and Forecasting model is a numerical weather prediction system customized in much of the public and private meteorological services worldwide. Below major Sunshine customization implementation details are offered:  Latitudes: 34.77º - 51.59º with resolution of 11.7 km.  Longitudes: 3.12º - 30.08º with resolution of 11.7 km  Temporal resolution output: 1 hour  Temporal scope: 72 hours  Output variables: T2, Q2, PSFC, ACSWDNB, ACLWDNB, RAINNC, rainc, U10, V10, CLDFRA, CLT, CLL  settings: - Micro-physics: WRF Single-moment three-class and five-class Schemes; Hong, Song- You, Jimy Dudhia, and Shu-Hua Chen, 2004: A revised approach to ice microphysical Processes for the bulk parameterization of clouds and precipitation. Mon. Wea. Rev., 132, 103-120. - Cumulus parameterization: Kain-Fritsch Scheme; Kain, John S., 2004: The Kain-Fritsch convective parameterization: An update. J. Appl. Meteor., 43, 170-181. - Shortwave and Longwave: Shortwave and Longwave CAM Schemes; Collins, William D., et al., 2004: Description of the NCAR Community Atmosphere Model (CAM 3.0). NCAR Tech. Note NCAR / TN-464 + STR. 214 pp.  Urban Surface: Urban Canopy Model
  28. 28. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Sigym3 MeteoServer integrates a reinterpretation service of weather forecasting based on precision procedures and statistical downscaling algorithms; a set of algorithms and models for the generation of derived variables of interest; and a GIS platform that integrates meteorological data, derived variables and other spatial data, considering and managing the time axis. In addition to the necessary adaptations to meet the needs of the project, specific services for collection and presentation of the information managed by the system have been developed:  the Weather Data Collecting Service  the Weather Data Exhibition Service 5.1.1 Weather Data Collecting Service In brief, the purpose of the Weather Data Collecting Service is to:  Gather local observations collected from local weather services or building meteorological stations;  Gather forecast data from NOAA, ECMWF and MeteoGrid Weather Forecast dedicated server;  Elaborate them and generate derived data, geo-interpolation processes plus unit conversion (different weather services use different units for meteorological variables);  Store them in a normalized format into a repository from which they can be retrieved through a standard interface (WFS). The flow of ECMWF and MeteoGrid forecast data are channelled through an FTP Ingestion service. Data are received twice a day to update forecast information. The flow of NOAA forecast data and METAR observations data are channelled through the FTP Ingestion service. Data are gathered via a scheduling procedure that pulls data twice a day to update forecast information and meteorological observations inside the Weather Platform. The flow of local weather observation data is managed by the Web Scraper Ingestion and the FTP Ingestion services that ensure the ingestion of such data every hour. 5.1.2 Weather Data Exhibition Service - WMS – WFS Collected and processed data is exposed through WMS and WFS services. The data flows towards Sunshine’s platform via the WFS service. SGIS has set up a scheduled procedure to pull data daily from the dedicated Web Feature Services (WFS).
  29. 29. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Sunshine-Weather specific servers are deployed on a virtualized IT infrastructure based on KVM allowing a flexible architecture. The underlying hardware has the following features: Sigym3 MeteoServer CPU Architecture x86_64 CPU Number 2 (8 cores) CPU Clock 2300 MHz Hard Drive Space 2x2TB (RAID 1) MeteoGrid Weather Forecast server CPU Architecture AMD ABU DHABI CPU Number 2 (16 cores each) CPU Clock 2300 MHz Hard Drive Space 8TB 5.2 Software components All the software used in this platform is free Open-source software (OSS). All the used free software is widely tested and used in all the software community, throughout the whole lifecycle of the software projects, it is worldwide tested and it is used in all types of projects, from small to large scale and, also, from small to large companies. 5.2.1 Core modules Operating System: Ubuntu (http://www.ubuntu.com/). Ubuntu is a Debian-based Linux operating system. Ubuntu releases updated versions predictably – every six months – and each release receives free support for nine months (eighteen months prior to 13.04) with security fixes, high-impact bug fixes and conservative, substantially beneficial low-risk bug fixes. Data Base: PostGIS (http://postgis.net/). PostGIS is a spatial database extender for PostgreSQL object-relational database. It adds support for geographic objects allowing location queries to be run in SQL. Programming Language: Python (https://www.python.org/). Python is a widely used general-purpose, high-level programming language. Its design philosophy emphasizes code readability, and its syntax allows programmers to express concepts in a few lines of code. The language provides constructs intended to enable clear programs on both a small and large scale. Apache HTTP Server (https://httpd.apache.org/).
  30. 30. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Apache HTTP Server provides a secure, efficient and extensible server that provides HTTP services in sync with the current HTTP standards. NGINX (http://nginx.org/). NGINX is an open source reverse proxy server for HTTP, HTTPS, SMTP, POP3, and IMAP protocols, as well as a load balancer, HTTP cache, and a web server (origin server). 5.2.2 Zope Object Database Zope Object Database (ZODB) (http://www.zodb.org/en/latest/). ZOPE is an object-oriented database for transparently and persistently storing Python objects. Features of the ZODB include: transactions, history/undo, transparently pluggable storage, built-in caching, multiversion concurrency control (MVCC) and scalability across a network. Pyramid (http://www.pylonsproject.org/projects/pyramid/about). Pyramid is a very general open source Python web framework. Pyramid is a minimalistic, platform-independent object publishing web framework. It also has integration with the Zope Object Database (ZODB). Main objects on the ZODB used for meteorological forecasts and observation gathering are shown below. Figure 10 – Main objects on the Zope Object database. GEO Projections Bounds Resolution Types Periodicity Actual Data - Raster - Features Day 1 Day 2 Day n STORES SOURCESCONTEXTS
  31. 31. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 5.2.3 Data ingestion in Sigym3 Below is a detailed diagram of the FTP ingestion services and the meteorological server modules. Figure 11 – Data ingestion in Sigym3. FTPRelayer, developed by Meteogrid (https://github.com/meteogrid/FTPRelayer), is a component based on Linux’s inotify that redistributes files which arrive at a machine to several machines, optionally processing them before sending them. It allows passive waiting for incoming files, avoiding polling. It is freely available in the provided URL. 5.2.4 Data Access Web Map Service (WMS) is a standard protocol for serving georeferenced map images over the Internet that are generated by a map server using data from a GIS database. Web Feature Service (WFS) provides an interface allowing requests for geographical features across the web using platform-independent calls. Proftpd FtPRelayer Mailprocessor Collector Genserver Webserver WMS WFS PostGIS ZODB
  32. 32. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Annex I - Sensor DB schema (52° North SOS 4.0)
  33. 33. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Annex II – Mapping weather WFS to Sensor DB ID WFS Name WFS Procedure SOS notes 86 CLES (VAL DI NON) VDN-F00 87 BASSANO DEL GRAPPA BAS-F00 88 PAOLA PAO-F00 89 SCHWECHAT no more in the project 90 FERRARA FER-F00 91 ROVERETO91 ROV-F00 92 ROVERETO92 ROV-F01 93 TRENTINO93 TRN-F00 94 TRENTINO94 TRN-F01 95 TRENTINO95 TRN-F02 96 TRENTINO96 TRN-F03 97 TRENTINO97 TRN-F04 98 TRENTINO98 TRN-F05 99 TRENTINO99 TRN-F06 100 TRENTINO100 TRN-F07 101 TRENTINO101 TRN-F08 102 TRENTINO102 TRN-F09 103 TRENTINO103 TRN-F10 104 TRENTINO104 TRN-F11 105 TRENTINO105 TRN-F12 106 TRENTINO106 TRN-F13 107 TRENTINO107 TRN-F14 108 TRENTINO108 TRN-F15 109 TRENTINO109 TRN-F16 110 TRENTINO110 TRN-F17 111 TRENTINO111 TRN-F18 112 TRENTINO112 TRN-F19 113 TRENTINO113 TRN-F20 114 TRENTINO114 TRN-F21 115 TRENTINO115 TRN-F22 116 TRENTINO116 TRN-F23 117 TRENTINO117 TRN-F24 118 TRENTINO118 TRN-F25 119 TRENTINO119 TRN-F26 120 TRENTINO120 TRN-F27 121 TRENTINO121 TRN-F28 122 TRENTINO122 TRN-F29 123 TRENTINO123 TRN-F30 124 TRENTINO124 TRN-F31 125 TRENTINO125 TRN-F32 126 TRENTINO126 TRN-F33 127 TRENTINO127 TRN-F34
  34. 34. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 128 TRENTINO128 TRN-F35 129 TRENTINO129 TRN-F36 130 TRENTINO130 TRN-F37 131 TRENTINO131 TRN-F38 132 TRENTINO132 TRN-F39 133 TRENTINO133 TRN-F40 134 TRENTINO134 TRN-F41 135 TRENTINO135 TRN-F42 136 TRENTINO136 TRN-F43 137 TRENTINO137 TRN-F44 138 TRENTINO138 TRN-F45 139 TRENTINO139 TRN-F46 140 TRENTINO140 TRN-F47 141 TRENTINO141 TRN-F48 142 TRENTINO142 TRN-F49 143 TRENTINO143 TRN-F50 144 TRENTINO144 TRN-F51 145 TRENTINO145 TRN-F52 146 TRENTINO146 TRN-F53 147 TRENTINO147 TRN-F54 148 TRENTINO148 TRN-F55 149 TRENTINO149 TRN-F56 150 TRENTINO150 TRN-F57 151 TRENTINO151 TRN-F58 152 TRENTINO152 TRN-F59 153 TRENTINO153 TRN-F60 154 LAMIA154 LAM-F00 155 LAMIA155 LAM-F01 156 LAMIA156 LAM-F02 157 LAMIA157 LAM-F03 158 LAMIA158 LAM-F04 159 ZAGREB159 HRV-F00 160 ZAGREB160 HRV-F01 161 ZAGREB161 HRV-F02 162 ZAGREB162 HRV-F03 163 RIJEKA HRV-F04 no buildings 164 ZAGREB164 HRV-F05 165 ZAGREB165 HRV-F06 166 SPLIT HRV-F07 no buildings 167 VARAZDIN HRV-F08 168 KRIZ HRV-F09

×