Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
WP4 – Integration of SUNSHINE pilot smart urban services
D4.5 Meter Data Management service #2
CIP-ICT-PSP-2012-6 – 325161...
Upcoming SlideShare
Loading in …5
×

S.2.f Specifications for Data Ingestion via Green Button

799 views

Published on

SUNSHINE Project: specifications for data ingestion via Green Button

Published in: Technology
  • Be the first to comment

  • Be the first to like this

S.2.f Specifications for Data Ingestion via Green Button

  1. 1. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 1 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Annex to D4.5 - Meter Data Management Service #2 Specifications for Data Ingestion via Green Button WP 4 – Integration of SUNSHINE pilot smart urban services Task 4.9 – Integration of new smart services with existing service infrastructures Task 4.2 – Meter data management service Revision: v1.1
  2. 2. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 2 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). REVISION HISTORY AND STATEMENT OF ORIGINALITY Revision Date Author Description v. 0.1 3rd September 2014 Luca Giovannini Document created. v. 0.3 4th September 2014 Rossana Bambili First draft of guidelines v. 0.9 18th September 2014 Luca Giovannini Final draft v. 1.0 14th November 2014 Stefano Pezzi Release v. 1.1 7th January 2015 Stefano Pezzi Revision after pilot installation v.1.2 16th January 2015 Stefano Pezzi Added some information on the pre-population of the datacustodian tables Statement of originality: This deliverable contains original unpublished work except where clearly indicated otherwise. Acknowledgement of previously published material and of the work of others has been made through appropriate citation, quotation or both. Moreover, this deliverable reflects only the author’s views. The European Community is not liable for any use that might be made of the information contained herein.
  3. 3. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 3 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Table of contents 1 Purpose...............................................................................................................................5 1.1 Green Button in a nutshell.......................................................................................................5 1.1.1 Green Button & Sunshine.....................................................................................................6 1.2 OAuth.....................................................................................................................................9 1.2.1 OAuth v.1.0 vs v.2.0 .......................................................................................................... 14 1.3 OAuth & Green Button ......................................................................................................... 16 1.4 Green Button in Sunshine scenario ........................................................................................ 17 2 Software Installation .........................................................................................................20 2.1 Software Requirements (DataCustodian) ............................................................................... 20 2.2 Installation instructions......................................................................................................... 20 2.2.1 Java JRE............................................................................................................................. 21 2.2.2 Apache Tomcat.................................................................................................................. 21 2.2.3 MySQL............................................................................................................................... 22 2.2.4 DataCustodian application................................................................................................. 27 3 Configuration and Use.......................................................................................................29 3.1 datacustodian DB structure ................................................................................................... 29 3.2 Populating the datacustodian database ................................................................................. 31 3.2.1 ‘retail_customers’ table ..................................................................................................... 31 3.2.2 ‘application_information’ table.......................................................................................... 32 3.2.3 ‘application_information_scopes’ table.............................................................................. 32 3.2.4 ‘time_configurations’ table................................................................................................ 32 3.2.5 ‘usage_points’ table........................................................................................................... 33 3.2.6 ‘reading_types’ table ......................................................................................................... 34 3.2.7 ‘meter_readings’ table....................................................................................................... 34 3.3 Import data service ............................................................................................................... 35 3.3.1 Service url ......................................................................................................................... 36
  4. 4. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 4 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 3.3.2 Consumption data XML...................................................................................................... 36 3.4 Export data ........................................................................................................................... 38 3.5 Security underneath the application layer.............................................................................. 39 3.6 Generation of the access token.............................................................................................. 39 Annex 1 – DST rules..................................................................................................................42 Annex 2 – Codelists ..................................................................................................................44 ServiceCategoryKind................................................................................................................................ 44 AccumulationBehaviourType .................................................................................................................. 44 CommodityType ...................................................................................................................................... 44 TimeAttributeType .................................................................................................................................. 45 UomType ................................................................................................................................................. 45 CurrencyCode .......................................................................................................................................... 45
  5. 5. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 5 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 1 Purpose The purpose of this guideline is to support the pilots in implementing at their own premises a Green Button web service, the DB and all the backend supporting services behind it. Through this web service, they will be able to expose their consumption data to the central Sunshine platform. 1.1 Green Button in a nutshell Green Button is an initiative of the U.S. Government that aims to allow consumers to access their own meter data (called Energy Usage Information, EUI) about energy consumption. In the original context, the name Green Button (in the rest of the document we’ll use GB) denotes the whole initiative, but in our project we will call GB only the suite of web services that makes the data access possible. There are three actors involved in the scenarios of making meter data available: Retail Customer (RC) = is the person (a real one or an entity) that consumes energy and is the actual owner of the data. Data Custodian (DC) = is typically the utility that delivers the energy and measures it with meters in order to get it paid by the retail customer. To do this, the data custodian has to record consumptions data and store them for its operational purposes. Third Party (TP) = is a service provider that offers value-added services to the retail customer, services that elaborate his/her meters data These three actors interact among them usually in these two alternative ways: 1. Retail Customer directly accesses his/her own data by using data custodian services. 2. Retail Customer delegates a Third Party to access his/her own data. The Third Party uses the Retail Customer’s data to provide some sort of special services to the Retail Customer, probably using some other information available from further data providers (meteorological data, global energy consumption data …). Some examples of services could be: advanced reporting, hints about energy saving, etc. In GB jargon, the first is called “GB Download My Data” and the second “GB Connect My Data”.
  6. 6. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 6 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 1.1.1 Green Button & Sunshine The Sunshine scenario can be described by the latter of the two GB interactions (GB Connect My Data) with some peculiarity. In Sunshine the actors are the following: Pilot = is the owner or conductor of a building that consumes energy and is the actual owner of the consumption data. Pilot technical partner = is a subject that acts on behalf of the utility company in the sense that it is in charge of gathering the consumption data in various manner (in real time or off line). It is an intermediary of the utility company because this one could not be active part of the project: by means of smart meters, retrofitted meters, or simply reading the old bills, consumption data are retrieved and made available. The Pilot Technical Partner is strictly related to the Pilot and sometimes it can overlap completely with it. Sunshine platform = is the platform of services, data and software components that Sunshine has put in place. Let see know the correspondence with GB actors: GB actor Sunshine actor Retail Customer Pilot Data Custodian Pilot Technical Partner Third Party Sunshine platform Table 1 - Correspondence between GB and Sunshine actors Like sketched in Figure 1, the Retail Customer is actually the owner of the building where the energy is consumed and measured. In Sunshine’s consortium, the building owners are either the Pilot Technical Partners (see HEPESCO) or the Pilot (i.e. Municipality of Ferrara buildings).
  7. 7. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 7 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Utility company Pilot Technical Partner Sunshine Pilot EUI EUI (GB) Energy Data Custodian & Retail Customer Third Party Figure 1 - Energy Usage Information flow in Sunshine The real Data Custodian should be the utility company that distributes the energy, but, due various technical and “political” problems, it’s quite impossible to get the data directly from the utility company. So we can consider the pilot’s technical partner as a surrogate of a Data Custodian, since it manages to get the meter data either by asking them to the utility in a non-automatic manner or by adding its own meter reading infrastructure to the utility’s one. The Third Party is the Sunshine platform that needs to access the consumption data to process them and provide value added services. Figure 2 - The GB scenario
  8. 8. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 8 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). The first two actors are almost the same and this is the peculiarity of the Sunshine scenario: Retail Customer could easily access directly its data, because it is very close to the Data Custodian or sometimes it’s the Data Custodian itself, but requires the services that Sunshine platform can offer. The original context of the GB scenario is the one depicted in Figure 2. Here we can see the flows between the 3 actors. Notice the two button icons that indicate the respective flows of data: in the diagram red arrows show the “GB Download My Data” flow, while the blue ones are involved in the “GB Connect My Data” use case. The Sunshine’s context is a profile of the “GB Connect My Data” context of GB scenario, and we can simplify the diagram in that of Figure 3, taking away the “GB Download My Data”. Figure 3 – The Sunshine scenario The dotted line arrows represent the authorization flows, in particular those of the delegated access implemented by OAuth. The term “One-Time Authorization” is used to highlight the fact that once has happened it may last for a long time in order to avoid a new authorization delegation exchange every time a data access has to be performed. The solid blue line arrow is the real data exchange between Third Party and Data Custodian, the one that interests most. The green line arrow is an interaction between Third Party and Data Custodian that aims to register the Third Party in the Data Custodian system as a subject that is being given a generic pre-authorization to use the Data Custodian APIs or services. This registration operation allows creating a trust between DC and TP,
  9. 9. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 9 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). and letting the DC know all the details of the actions that the TP could execute on the RU resources (i.e. read and add documents from a web drive) once obtained the authorization from the owner. The registration operation can take place offline although GB services allow to do it dynamically; in order to simplify the exchange of data in Sunshine scenario, since the DC and the TP are well known from the start, we will consider only the offline registration. The OAuth protocol, on which GB bases its security features, relies also on HTTP redirection because the main use case is the one in which the user access the Third Party services by means of his/her browser (in the diagrams it’s called “agent”) and the first redirection is from TP’s web portal to DC’s web portal, where the RU can authenticate and delegate access to the TP services. In Sunshine the access delegation is like the TP registration: it can be done once and offline for the same reasons; furthermore the DC web portal and the TP web portal, in the sense of the web portal functionality involved in the authorization delegation operations, does not play any roles. So we can achieve a further simplification of the diagram as we show in the Figure 4. Figure 4 - Further simplification of Sunshine context 1.2 OAuth As we said before, the three actors’ scenario of GB recalls the more generic one of delegated access: this is an increasingly widespread situation in which a web user is involved almost every day. The user owns some resources (emails, images, documents, address book, path recordings, bank …) that are maintained by some provider (Google, Flickr, bank …). Other service providers offer more advanced services on these data
  10. 10. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 10 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). (professional printing, special visualization, instant messaging, spatial analysis …), but, of course, they need access to the original data do some work on them. This kind of interaction strongly involves security aspects like granting access to other subjects to own protected resources without having to share own authentication credential. OAuth1 protocol is about this problem and points out a particular authorization flow among the involved subjects in order to fulfil the descripted objective. In the OAuth terminology, the actors involved in the delegated authorization scenario are: Resource Owner = the owner of the resources that we want to secure. Client = an application making protected resource requests on behalf of the Resource Owner and with its authorization. The term “client” does not imply any particular implementation characteristics (e.g., whether the application executes on a server, a desktop, or other devices). Resource Server = The server hosting the protected resources, capable of accepting and responding to protected resource requests using access tokens, that is the artefact that substitutes the Resource Owner credential. Authorization Server = the server issuing access tokens to the client after successfully authenticating the Resource Owner and obtaining authorization. User-agent = typically a web browser or other software under the control of the Resource Owner. We are talking of a three actors’ scenario, but here we’ve just listed five subjects: some of these actors actually overlap in a higher level architecture and their roles are split only from an implementation and technological point of view. In particular: Resource Server ≡ Authorization Server Resource Owner ≡ User-agent Other terms used in the OAuth protocols are the following2 : protected resources = an access-restricted resource that can be obtained from the server using an OAuth-authenticated request. 1 OAuth v.1.0 is based on two proprietary protocols used by Google and Flickr: “AuthSub” and “API Auth”, respectively. 2 http://tools.ietf.org/html/rfc5849
  11. 11. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 11 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). credentials = Credentials are a pair of a unique identifier and a matching shared secret. OAuth defines three classes of credentials: client, temporary, and token, used to identify and authenticate the client making the request, the authorization request, and the access grant, respectively. token = A unique identifier issued by the server and used by the client to associate authenticated requests with the resource owner whose authorization is requested or has been obtained by the client. Tokens have a matching shared-secret that is used by the client to establish its ownership of the token, and its authority to represent the resource owner. In the sample flow below, let’s imagine an user Bob that wants to use some professional printing services offered by a web company PrintEveryThing that advertises its capacity of printing Facebook, Flickr, Google Drive, Picasa … web stored photos. The roles are the followings:  Bob = Resource Owner  Flickr = Resource Server  PrintEveryThing = Client The flow is described by the following steps; step # 0 represents the preconditions of the flow: 0. Bob owns some photos at Flickr. Flickr implements OAuth mechanism and has an Authentication Server and an Authorization Server as well. The PrintEveryThing is a registered Client at Flickr. This means that PrintEveryThing has been given an ID by Flickr and shares a secret for encrypting its messages to assure they come from it. 1. Bob accesses the PrintEveryThing web portal in order to use some of its services. 2. At a certain point of the interaction with the PrintEveryThing’s portal, Bob confirms that he wants to print his photo album at Flickr. This action makes PrintEveryThing communicates with Flickr asking for an unauthorized request token; the release of this token means that Flickr has recognized PrintEveryThing as a registered Third Party, but without an explicit authorization from the user, the token can’t be used to access the photos. 3. Bob’s browser gets redirected by PrintEveryThing’s portal to the Flickr portal. The redirection contains a call-back URL and the unauthorized request token, plus some other information. 4. Flickr’s portal asks for Bob’s credentials and authenticates him by means of its Authentication Server.
  12. 12. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 12 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 5. Flickr’s portal recognizes the unauthorized request token and, retrieving the information stored in the TP registration step (not shown in this flow), asks Bob what kind of permissions to give to PrintEveryThing and for which protected resources (photos). 6. Bob chooses the privileges and resources and then confirms his intention to let PrintEveryThing access them. This action makes Flickr produce a request token that is returned to PrintEveryThing by means of the next step. We can say that the user granting operation transforms the unauthorized request token into a request token. 7. Flickr redirects back BOB’s browser to the call-back URL at PrintEveryThing’s site bearing some other information: the request token. This token contains the information that the user has granted the Third Party the privileges to his resources, but can’t be used to directly access the resource yet. 8. Finally PrintEveryThing exchanges with Flickr its request token with an access token3 9. PrintEveryThing uses the access token to retrieve the protected photos from Flickr using its APIs and then does its job on them. In the diagram of Figure 5 the OAuth flow is shown with only the Client and the Resource Server highlighted; user (the Resource Owner) or user’s browser is involved in the passages represented with a solid line. Actually, the OAuth flow that we exemplified here is one of the possible because several variants can be sketched: for instance, the user could start interaction at the Resource Server portal instead of the Client’s and by means of some redirections that go the other way around, the same steps can be performed and the same result obtained. Instead of starting from the client’s portal, choosing the Resource Server 3 The access token is the final result of the OAuth flow: it is the string representing an access authorization issued to the client, rather than using the resource owner's credentials directly.
  13. 13. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 13 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Client Resource Server Requests unauthorized request token Grants unauthorized request token Redirects user to resource server Authenticates user Obtains user authorization Redirects user to Third Party Requests Access token Unauth. req token Request token Grants access token Request token Unauth. req token Accesses Protected resources Access token Protected resources Access token User browses portal Figure 5 - OAuth flow The three fundamental interchanges can be further simplified in the schema of Figure 6.
  14. 14. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 14 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Client Resource Server Authorization request Authorization grant Authorization grant Access Token Access Token Protected Resource Figure 6 – High level OAuth interchanges 1.2.1 OAuth v.1.0 vs. v.2.0 The OAuth authorization flow described up to here refers to version 1.0 that was first published at the end of 2007 and that became a standard (RFC 5849) in mid-2010 (after an update release 1.0a fixed a security leak). A subsequent version of OAuth, the v.2.0, has been released afterwards, the first draft in the very same period of the 2010; this version is not backwards compatible, but it maintains the overall architecture and the approach descripted above. Now the stable version4 is in a proposal state under the code RFC 6749. This new release has been also a source of controversy: some of the original creators of OAuth abandoned the project because some vulnerability has been introduced in exchange for more simplicity at client side, but here we don’t want to discuss this aspect. The fact is that the major web companies have adopted OAuth v.2.0 even if many features are still changing and being added. Green Button relies on v.2.0, so it’s better that we take a quick look at the most important differences; some of these are both simplifying the protocol and making it less secure, as critics says: 4 https://tools.ietf.org/html/rfc6749
  15. 15. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 15 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 1. Cryptography. Client applications do not have to use HMAC (keyed-hash message authentication code) to encrypt tokens and request string anymore, but they can send plaintext token over an https secure channel. 2. Signature. A simpler signature is used using only one secret5 instead of two. 3. Bearer token. It is a special type of token that can be used instead of an access token. Every request that has a bearer token with it, can access the protected resources for which it has been issued, no matter who is the requestor. No need of the “proof-of-possession” that is proving that who is sending the request is in possess of cryptographic key material. An access token instead is used along with a signature that identifies the requestor. 4. Access token refresh. Access tokens can be long-lived and even have unlimited lifetime. This could be a security issue in case an access token (or better, a bearer) is stolen. In OAuth v.2.0 server can issue short-lived access tokens in order to minimize the time windows in which is possible to use the access token fraudulently. The short-lived access token is released together with long-lived refresh tokens that can be used to reissue the access token. The difference is that to use a refresh token you need to use the secret and the client ID. 5. Separation of roles. OAuth v.2.0 separates the roles of Authorization server from the one of Resource Server. Authorization server is responsible for obtaining user authorization and issuing the relative tokens. Resource Server checks handles the API calls to the protected resources and enforces the authorization policies. The original flow of Figure 6 can be re-drawn like in Figure 7. In this diagram the client request to the resource owner is usually made indirectly via the authorization server as an intermediary. 5 In cryptography a secret is a data known only by the two parts involved in a secure communication. This data, usually a string or a big number, can be shared before the communication starts or at the start of it. The secret is used to encrypt the messages exchanged between the parts.
  16. 16. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 16 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Client Authorization request Authorization grant Authorization grant Access Token Access Token Protected Resource Resource Owner Authorization server Resource server Figure 7 - OAuth v.2.0 interchanges In the Sunshine scenario we don’t have a user interacting with either the Third Party’s portal or the Resource Server’s one. So, a part from the overlap of the Resource Owner with the Resource Server, we have this further characteristic of an absence of a human interaction. This means that all the outcomes of the interaction must be produced in another way, in advance or off-line. An example is given by the transformation of the “unauthorized request token” in a “request token” that is the result of the user granting authorization to the third party. 1.3 OAuth & Green Button The Green Button and the OAuth scenarios resembles very much. But these are not only analogies: as a matter of fact GB services are strongly based on the OAuth protocol, in particular OAuth v.2.0. In OAuth, the actors’ names are slightly different from the GB ones, so it’s better to remind the correspondence: GB actor OAuth actor Retail Customer Resource Owner (RO) or user
  17. 17. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 17 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Data Custodian Resource Server6 (RS) or server or service provider Third Party Client or consumer Table 2 - Correspondence between GB and OAuth actors So the use cases faced by GB can be classified in two main groups: the first that deals with the authorization interchanges and that resembles the one of OAuth and the second that handles the real data transmission. GB provide for dynamic authorization phase, in which actors “meet” them for the very first time and registration, trust release and specific authorization for the resources are all managed using the OAuth protocol and the OAuth interchanges. In Sunshine the parties involved are known from the start and some passages can be done offline in order to simplify the operations. In particular in the scenario “GB connect my data” the consumer is involved only in approving access to the data, but is not (necessarily) involved in the movement of the data from the Data Custodian to a third party service provider, and it results in a machine-to-machine communication. In particular we’ll see in the next chapter how to install and configure an open source implementation of Green Button services that offers a wide range of web APIs that can manage all the possible communication between the parties. We won’t use all these APIs precisely because we want to keep the system simple. 1.4 Green Button in Sunshine scenario In the Sunshine scenario, pilots that want to transmit data by means of the GB mechanism will have to: 1. Act like a retail customer and give Sunshine platform the authorization to access its data; this action will be executed by a human user that interacts with the software capable of generating an OAuth access token. The user will deal with both the Data Custodian and the Third Party software that will interchange messages and redirect the user’s browser like in the interactions described above. The access token will be finally stored in the Sunshine platform and used to access the resources. 2. Expose the EUI by means of a GB service; therefore the data shall be ingested from the original format into the database structures known by the service. 6 The Resource Server plays several roles that in OAuth can be highlighted as separated because they actually can be implemented by different components of the architecture: Authentication server and Authorization server. OAuth v.2.0 explicitly introduces this separation.
  18. 18. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 18 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). To achieve these results, Sunshine is going to reuse the software developed as a reference implementation by the initiative EnergyOS7 (Open Source for Energy Infrastructure) and the private company Pivotal Labs8 , called OpenESPI. In particular, the open source software packages implemented are the 1. DataCustodian module, that is the component that publishes the energy consumption data and the 2. ThirdParty module that communicates with it for the authorization steps. A third component, developed in the Sunshine project, is the GreenButton Client, that is the module that, once obtained the authorization token, will handle the mere EUI transfer operation. Another function that the DataCustodian offers is the ingestion of the EUI data from original sources. A service is exposed to facilitate the ingestion of the readings in the EUI database: the pilot can either activate the “B” flow using this service (encoding the readings in a XML request) or the “A” flow loading the data directly in the database with some custom scripts (see the blue arrows in Figure 8). Like written above, in Green Button jargon, Pilot plays the role of Data Custodian and Sunshine platform the Third Party one. Therefore the software module DataCustodian must be installed at the Pilot’s side that whilst the ThirdParty module will be installed on the Sunshine platform. DataCustodian ThirdParty EUI internal flow EUI Pilot Sunshine datacustodian DB (EUI) Token DB Original consumption data Oauth flow SOS GreenButton Client A B B Figure 8 – The interactions between Pilot and Sunshine platform 7 http://www.energyos.org/ 8 http://pivotallabs.com/
  19. 19. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 19 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). In the next chapter only the instruction to install the DataCustodian module will be given. Then, we’ll describe the procedure to assign to the Sunshine platform a token to allow the EUI access and finally how to feed the GB service with the consumption data.
  20. 20. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 20 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 2 Software Installation The DataCustodian is available as a web application packed in “war” file that is downloadable at the address: http://sunshine.sinergis.it/download/greenbutton/1.1/DataCustodian.war **The package is based on the version 1.1-SNAPSHOT released on 4th September 2014 and has been adapted specifically for Sunshine needs. The current version available at the link above and on which are shaped the current guidelines differs from the one that was available for download at the time of release of version v0.9 of these guidelines, so pilots are invited to download and use the current WAR version. 2.1 Software Requirements (DataCustodian) The installation of the following software is required:  Java JRE (Java Run-Time Engine) 1.7.0 or higher (downloadable from http://www.java.sun.com)  Servlet container Apache Tomcat v. 7.0.54 or higher (downloadable from http://jakarta.apache.org/tomcat) Moreover, the availability of a mySQL relational database is required9 to host the token store. The EUI database can be implemented either by a mySQL DB or by a PostgreSQL. 2.2 Installation instructions The deployment diagram of the DataCustodian application is shown in Figure 9. The DB Server and the Application Server may be one and the same server. The two databases maintain the consumption data that are published by the DataCustodian services, but also support the OAuth mechanism implemented by GB. They could be two instances installed on separate DB servers, but to keep it simple, consider them two SCHEMAs (same as two DATABASEs) in the same server and in the same mySQL instance. 9 Next release should support also PostgreSQL and HSQL
  21. 21. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 21 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Figure 9 - Deployment diagram of DataCustodian application 2.2.1 Java JRE The following instructions refer to a MS Windows environment (64bit), but are valid also for a Unix/Linux environment, provided that the relevant path adjustments are applied: 1. Create a subfolder named greenbutton in the root folder of your system. 2. Download and execute jdk-7u65-windows-x64.exe. 3. Select as installation folder: C: greenbuttonjre1.7.0_45. 4. Set the environment variable JAVA_HOME to JAVA_HOME = C: greenbutton jre1.7.0_45. Even if a JRE is already present, check that the installation path (step 3) has no blank spaces and that the environment variable JAVA_HOME (step 4) is properly set. 2.2.2 Apache Tomcat The following instructions refer to a MS Windows environment, but are valid also to a Unix/Linux environment, provided that the relevant path adjustments are applied: 1. Create a subfolder named greenbutton in the root folder of your system. 2. Download apache-tomcat-7.0.54.zip. 3. Unpack the package apache-tomcat-7.0.54.zip in the greenbutton folder. 4. Set the environment variable CATALINA_HOME to CATALINA_HOME = C: greenbuttonapache- tomcat-7.0.54.
  22. 22. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 22 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 5. Open file <CATALINA_HOME>/bin/startup.bat and set memory allocation parameters for Java Virtual Machine (JVM) as follows: set CATALINA_OPTS=-server –Xms2048m –Xmx2048m -XX:PermSize=256m -XX:MaxPermSize=256m. Even if Apache Tomcat is already installed in the server, check that the installation path has no blank spaces and that the environment variable CATALINA_HOME (step 4) is properly set. 2.2.3 MySQL The following instructions refer to a MS Windows environment, but are valid also to a Unix/Linux environment, provided that the relevant path adjustments are applied: 1. Download MySQL Installer from http://dev.mysql.com/downloads/installer/ and execute it. 2. Choose the appropriate Setup Type for your system (developer or custom). 3. Follow the wizard’s instructions. 2.2.3.1 DataCustodian DB (EUI) The datacustodian DB is meant to host the readings (EUI). It is automatically created by the application DataCustodian when it is run for the very first time with some particular configuration settings. See the paragraph 2.2.4.1 for the details; here the instruction for mySQL DB are illustrated, but the postgreSQL case is different just for the connection parameters and the configuration file name. Here and in the rest of the document the EUI database has been named “datacustodian”, but the pilot can choose whatever name and modify consequently the configuration files. 2.2.3.2 Token DB The Token DB is just for the authorization phase. It must be created manually running the SQL creation scripts that you’ll find in http://sunshine.sinergis.it/download/greenbutton/1.1/createSunshineTS_mySQL.sql and listed hereinafter. CREATE DATABASE IF NOT EXISTS `tokenstore` /*!40100 DEFAULT CHARACTER SET utf8 */; USE `tokenstore`; -- MySQL dump 10.13 Distrib 5.5.35, for debian-linux-gnu (x86_64) -- -- Host: 127.0.0.1 Database: tokenstore -- ------------------------------------------------------ -- Server version 5.5.32 /*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */; /*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */; /*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */; /*!40101 SET NAMES utf8 */; /*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */; /*!40103 SET TIME_ZONE='+00:00' */; /*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
  23. 23. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 23 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). /*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */; /*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */; /*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */; -- -- Table structure for table `clientdetails` -- DROP TABLE IF EXISTS `clientdetails`; /*!40101 SET @saved_cs_client = @@character_set_client */; /*!40101 SET character_set_client = utf8 */; CREATE TABLE `clientdetails` ( `appId` varchar(255) NOT NULL, `resourceIds` varchar(256) DEFAULT NULL, `appSecret` varchar(256) DEFAULT NULL, `scope` varchar(256) DEFAULT NULL, `grantTypes` varchar(256) DEFAULT NULL, `redirectUrl` varchar(256) DEFAULT NULL, `authorities` varchar(256) DEFAULT NULL, `access_token_validity` int(11) DEFAULT NULL, `refresh_token_validity` int(11) DEFAULT NULL, `additionalInformation` varchar(4096) DEFAULT NULL, `autoApproveScopes` varchar(256) DEFAULT NULL, PRIMARY KEY (`appId`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; /*!40101 SET character_set_client = @saved_cs_client */; -- -- Table structure for table `oauth_access_token` -- DROP TABLE IF EXISTS `oauth_access_token`; /*!40101 SET @saved_cs_client = @@character_set_client */; /*!40101 SET character_set_client = utf8 */; CREATE TABLE `oauth_access_token` ( `token_id` varchar(256) DEFAULT NULL, `token` mediumblob, `authentication_id` varchar(256) DEFAULT NULL, `user_name` varchar(256) DEFAULT NULL, `client_id` varchar(256) DEFAULT NULL, `authentication` mediumblob, `refresh_token` varchar(256) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8; /*!40101 SET character_set_client = @saved_cs_client */; -- -- Table structure for table `oauth_approvals` -- DROP TABLE IF EXISTS `oauth_approvals`; /*!40101 SET @saved_cs_client = @@character_set_client */; /*!40101 SET character_set_client = utf8 */; CREATE TABLE `oauth_approvals` ( `userId` varchar(256) DEFAULT NULL, `clientId` varchar(256) DEFAULT NULL, `scope` varchar(256) DEFAULT NULL, `status` varchar(10) DEFAULT NULL, `expiresAt` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  24. 24. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 24 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). `lastModifiedAt` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00' ) ENGINE=InnoDB DEFAULT CHARSET=utf8; /*!40101 SET character_set_client = @saved_cs_client */; -- -- Table structure for table `oauth_client_details` -- DROP TABLE IF EXISTS `oauth_client_details`; /*!40101 SET @saved_cs_client = @@character_set_client */; /*!40101 SET character_set_client = utf8 */; CREATE TABLE `oauth_client_details` ( `client_id` varchar(255) NOT NULL, `resource_ids` varchar(256) DEFAULT NULL, `client_secret` varchar(256) DEFAULT NULL, `scope` varchar(256) DEFAULT NULL, `authorized_grant_types` varchar(256) DEFAULT NULL, `web_server_redirect_uri` varchar(256) DEFAULT NULL, `authorities` varchar(256) DEFAULT NULL, `access_token_validity` int(11) DEFAULT NULL, `refresh_token_validity` int(11) DEFAULT NULL, `additional_information` varchar(4096) DEFAULT NULL, `autoapprove` varchar(256) DEFAULT NULL, PRIMARY KEY (`client_id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; /*!40101 SET character_set_client = @saved_cs_client */; -- -- Table structure for table `oauth_client_token` -- DROP TABLE IF EXISTS `oauth_client_token`; /*!40101 SET @saved_cs_client = @@character_set_client */; /*!40101 SET character_set_client = utf8 */; CREATE TABLE `oauth_client_token` ( `token_id` varchar(256) DEFAULT NULL, `token` mediumblob, `authentication_id` varchar(256) DEFAULT NULL, `user_name` varchar(256) DEFAULT NULL, `client_id` varchar(256) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8; /*!40101 SET character_set_client = @saved_cs_client */; -- -- Table structure for table `oauth_code` -- DROP TABLE IF EXISTS `oauth_code`; /*!40101 SET @saved_cs_client = @@character_set_client */; /*!40101 SET character_set_client = utf8 */; CREATE TABLE `oauth_code` ( `code` varchar(256) DEFAULT NULL, `authentication` mediumblob ) ENGINE=InnoDB DEFAULT CHARSET=utf8; /*!40101 SET character_set_client = @saved_cs_client */; -- -- Table structure for table `oauth_refresh_token` --
  25. 25. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 25 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). DROP TABLE IF EXISTS `oauth_refresh_token`; /*!40101 SET @saved_cs_client = @@character_set_client */; /*!40101 SET character_set_client = utf8 */; CREATE TABLE `oauth_refresh_token` ( `token_id` varchar(256) DEFAULT NULL, `token` mediumblob, `authentication` mediumblob ) ENGINE=InnoDB DEFAULT CHARSET=utf8; /*!40101 SET character_set_client = @saved_cs_client */; /*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */; /*!40101 SET SQL_MODE=@OLD_SQL_MODE */; /*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */; /*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */; /*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */; /*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */; /*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */; /*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */; Download the SQL script http://sunshine.sinergis.it/download/greenbutton/1.1/createSunshineTS_mySQL.sql and run inside the “mySQL workbench” or some other SQL console connected to the mySQL instance that will host the tokenstore DB . 2.2.3.3 Populating the token DB The tokenstore DB should be prepopulated with some records in order to enable the application to create the access token for the Sunshine platform. A SQL script containing the insert statements can be downloaded at the address http://sunshine.sinergis.it/download/greenbutton/1.1/prepopulateSunshineTS_mySQL.sql. For the data exchange the needed record is the first one that describes the third party user (Sunshine Platform). The other records are respectively:  a user that the pilot can use to write and read his own data;  a user that the pilot can use to read all the authorizations released to the third party applications. In all these records a string “secret” is used as a default value for the field “client_secret”, but this should be changed: they are the string that are used to encrypt the messages exchanged between the parties involved in the OAuth flow. Obviously the three strings can be different: so change them in the SQL file
  26. 26. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 26 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). created from the below snippet. This string must be pre-shared in some way with the third party (Sunshine Platform)10 . The value of “scope” is a coded string that specifies the parameters of the reading that Sunshine Platform (the third party) is interested in. The “access_token_validity” is expressed in seconds and is set to one year, while the “refresh_token_validity” is set to five years. Very important is the value of the call back URL that is set to http://sunshine.sinergis.it/ThirdParty/espi/1_1/OAuthCallBack. It’s the address of the endpoint of a ThirdParty service exposed at the Sunshine Platform and necessary for the authorization token release. INSERT INTO `oauth_client_details` VALUES ('sunshine_platform', NULL, 'secret', 'read', 'authorization_code,refresh_token', 'http://sunshine.sinergis.it/ThirdParty/espi/1_1/OAuthCallBack', 'ROLE_USER', '31536000', '157680000', NULL, 'FALSE'); INSERT INTO `oauth_client_details` VALUES ('data_custodian_admin', NULL, 'secret', 'DataCustodian_Admin_Access', 'client_credentials', NULL, 'ROLE_DC_ADMIN', '31536000', NULL, NULL, 'FALSE'); INSERT INTO `oauth_client_details` VALUES ('third_party_admin', NULL, 'secret', 'ThirdParty_Admin_Access', 'client_credentials', NULL, 'ROLE_TP_ADMIN', '31536000', NULL, NULL, 'FALSE'); Download the SQL script located at http://sunshine.sinergis.it/download/greenbutton/1.1/prepopulateSunshineTS_mySQL.sql.  Change the “secret” strings contained in the script.  Run the script inside the “mySQL workbench” or some other SQL console connected to the mySQL instance that hosts the tokenstore DB . 10 Instead of using unsecure communication like mail or document transfer, one could use the web site https://onetimesecret.com/ that allows exchanging privately small amount of data in a very simple manner.
  27. 27. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 27 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp).  Communicate the “secret” string and the endpoint of your DataCustodian installation to the Sunshine Platform administrators in order to let them configure the ThirdParty. 2.2.4 DataCustodian application After downloading the war file, it is necessary to modify some configuration files inside it. It is important to do it inside the zipped war so use an editor that can do it or extract the file, modify it and then re-insert it in the war file. These are configurations that Tomcat must read the first time it is run, that is when unpacks and deploys the war file content in its folder <CATALINA_HOME>/webapps/ Then Tomcat is started for the first time and in this run it will do several one-shot actions; after this first run it must be stopped, some other configurations must be done (this time in the deployed unzipped files) and then Tomcat finally restarted to have the application working. 2.2.4.1 Pre-install configurations The files to be configured tell the application which is the database to connect to, the behaviour to adopt with the DB, and which tables create; the files that need to be modified are the following: 1. The first file is a “.properties”: WEB-INFclassesspringmysql-data-access.properties and must be modified setting the DB connection parameters: jdbc.url=jdbc:mysql://<dbserver>:<dbport>/datacustodian jdbc.username=<username> jdbc.password=<password> 2. Then there are two XML files; the first is : WEB-INFclassesspringbusiness-config.xml and must be modified inserting the “create-drop” value in the following element: <prop key="hibernate.hbm2ddl.auto">create-drop</prop> This tells the application to create the tables needed eventually dropping the existing ones. In the same file, insert the file name mysql-data-access.properties in the location attribute of the element context:property-placeholder as follows: <context:property-placeholder location="classpath:spring/mysql-data- access.properties" system-properties-mode="OVERRIDE"/> The second XML file is:
  28. 28. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 28 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). WEB-INFclassesspringdatasource-config.xml and in this file we must do the last same modification: <context:property-placeholder location="classpath:spring/mysql-data- access.properties" system-properties-mode="OVERRIDE"/> 3. The third configuration regards another XML file: WEB-INFclassesspringoauth-AS-config.xml Here find and edit the element “property” with attribute “baseURL” with the appropriate URL for the DataCustodian application: <property name="baseURL" value="http://<pilot DataCustodian server>/DataCustodian"/> Inside this file there are also a lot of references to the “token DB” and the jdbc-style connection URL with the user and relative password; these credentials must be changed to match the ones of the installed instance of the mySQL DB and the connection URL must be set to the real one if it is not the same server of the DataCustodian application (it’s default setting is jdbc:mysql://localhost:3306/tokenstore). 2.2.4.2 Installation and first run Copy the modified war file into folder: <CATALINA_HOME>/webapps/ when Apache Tomcat is not running. Then start Tomcat and check if it connects to the database and creates some tables in the “datacustodian” database inside the mySQL instance. 2.2.4.3 Post-installation configurations Stop Apache Tomcat again; the datacustodian tables structure has been created during the first run and now we must instruct the application not to drop and recreate the tables each time it is started. To do this, we must modify the file business-config.xml anew, but this time the unzipped one in the Tomcat’s folder: <CATALINA_HOME>webappsDataCustodianWEB-INFclassesspringbusiness-config.xml Open it and insert the value “update” in the following element, overwriting the previous value: <prop key="hibernate.hbm2ddl.auto">update</prop> You can also drop the war file from the webapp folder and finally start Tomcat again and the DataCustodian application should be up and running.
  29. 29. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 29 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 3 Configuration and Use In this chapter we will see how to set up the EUI database, feed it with the data coming from the meters and let them be harvested by the Sunshine platform. In order to do so, the pilot should set up a mechanism that feed the datacustodian DB ingesting the data from the original source (the meters loggers, the AMI, the files coming from the utilities company …) to the standard tables of the datacustodian database and expose the service that enables the transmission of data to Sunshine. DataCustodian application provides two services (import & export) that allow these operations: the former can be used by the pilot back-end procedures and the latter will be used by the Sunshine platform. Both are protected by OAuth mechanism so they can be exposed on the internet. 3.1 datacustodian DB structure The structure of the datacustodian DB that is relevant to the implementation of the Web Service and deals with the consumption data is described in Figure 10. Figure 10 - Database structure Table retail_customers stores the authentication information of the service’s users. The users to set up are two: the service administrator, who is represented by the pilot technical partner, and the “retail user”, that is the owner of the data.
  30. 30. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 30 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Table usage_points stores the meter information. Every meter is a usage point and one or more usage points can be associated to a single retail customer. Table time_configuration stores the information about time zone and daylight saving time rules for the area where the usage point is located. Consumption timestamps in the DB are stored in UTC format and this type of information is used in order to convert data into local time frame, if needed. For the purposes of Sunshine project conversion into local time is not needed, so the time configuration can be set to UTC (see the following sub-section). Table meter_reading lists the available types of reading coming from the usage points (the meters). Examples of meter readings form the same usage point (i.e. a gas meter) can be “the set of readings for a certain period” or “the readings at an interval of 4 hours” to be distinguished from “the readings at an interval of 15 min”. Typically however every usage point is associated to just one meter reading. Table reading_types describes the properties of the readings. Every meter reading is associated to a reading type describing i.e. the type of energy measured, the frequency, the nature of the measurement (absolute, integrated, etc). Table interval_blocks stores the information about the regular data feeds into the DB. Each ingestion of consumption data (be it a single reading or a chunk of readings) correspond to an interval block and is associated to a specific meter reading. Table interval_readings stores the actual data about consumption and cost. Each entry corresponds to a single reading (energy consumption and its related cost, if available) and is linked to the interval block it was part when it was imported in the DB. As Figure 10 shows, part of the tables require only a one-time set-up, while tables interval_blocks and interval_readings require regular updates as they store the actual dynamic consumption data. Section 3.2 describes how one-time tables can be set-up via direct SQL INSERT calls, while section 3.3 describes how to insert dynamical consumption data into the DB and section 3.4 illustrates a procedure to extract the data from the DB to verify its content.
  31. 31. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 31 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 3.2 Populating the datacustodian database This section is a collection of template INSERT statements to populate the one-time tables of datacustodian DB. Important note: each entry of every table has to have a unique UUID11 which has to be generated by the pilots. The insert statements are gathered in a SQL file available for download at the address http://sunshine.sinergis.it/download/greenbutton/1.1/prepopulateSunshineDC_postgreSQL.sql. Note that some information is duplicated and are present in the tokenstore DB as well as in the datacustodian DB. As a general consideration, all these tables contain fields with names “self_link_href” or “up_link_href” coupled with fields “self_link_rel” and “up_link_rel”. The values for these fields are respectively the URLs of the resource itself (self_link_href) or of the resource of the upper hierarchical level (up_self_link) and the qualifiers of these links. This information is used to form the response of the REST services following the principle of the HATEOAS constraint of the REST architecture. As a common characteristic, records have a UUID as external identifier, so this UUID must be regenerated and substituted before running the script. 3.2.1 ‘retail_customers’ table This table contains the users of the datacustodian application: the first user represents the retail customer and will have a role that allows him or her to give a third party the authorization to access the data; another role of this user is the feeding of the EUI data, using a specific import service as we’ll see in § 3.3. Another user is the data custodian itself that is enabled for some administration activities that are not important in this context. The ids of a resource (in this case of a retail customer) are very important as in REST syntax they are used to form the URIs that allow manipulating the resource itself: here we are using a progressive. Retail user (building owner) INSERT INTO retail_customers( 11 http://en.wikipedia.org/wiki/Universally_unique_identifier. An online UUID generator derived from the current time can be found here (just for testing purposes): http://www.famkruithof.net/uuid/uuidgen
  32. 32. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 32 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). id, description, published, self_link_href, self_link_rel, up_link_href, up_link_rel, updated, uuid, enabled, first_name, last_name, password, role, username) VALUES ( 1,'Ferrara municipality','2014-12-31','/RetailCustomer/1','self', '/RetailCustomer','up','2014-12-31','106E8B03-0299-467E-972A- A883ECDCC598','1','Servizio energia','Ferrara','F3rr@r@','ROLE_USER', 'ferrara'); Data custodian (Pilot admin) INSERT INTO retail_customers( id, description, published, self_link_href, self_link_rel, up_link_href, up_link_rel, updated, uuid, enabled, first_name, last_name, password, role, username) VALUES ( 2, 'Pilot Ferrara Data Custodian', '2014-12-31', '/RetailCustomer/2', 'self','/RetailCustomer','up','2014-12-31','106E8B03-0299-467E-972A- A883ECDCC599','1','Sinergis', 'Sinergis', '21n3r612','ROLE_CUSTODIAN', 'admin'); 3.2.2 ‘application_information’ table This table contains information about the applications that are registered in order to use the datacustodian services. Of particular importance is the application that has the attribute “kind” with value “THIRD_PARTY”, that is the Sunshine platform. 3.2.3 ‘application_information_scopes’ table This table contains the scopes related to the applications listed in the previous table. 3.2.4 ‘time_configurations’ table Here we insert the two records that represent the UTC and CET Summer time zones codifications. Time configuration: UTC
  33. 33. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 33 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). INSERT INTO time_configurations( id, description, uuid, dstoffset, tzoffset) VALUES ( 1, 'UTC', '/LocalTimeParameters/1', 'self', '/LocalTimeParameters', 'up', '106E8B03-0299-467E-972A-A883ECDCC593', 0, 0); Time configuration: Central European (Summer) Time Annex 1 – DST rules describes how to compute the values of dststartrule and dstendrule in order to define a Daylight Saving Time rule. INSERT INTO time_configurations( id, description, uuid, dstendrule, dstoffset, dststartrule, tzoffset) VALUES ( 2, 'Central European (Summer) Time', '/LocalTimeParameters/2', 'self', '/LocalTimeParameters', 'up', '106E8B03-0299-467E-972A-A883ECDCC595', decode('AE0E2000', 'hex'), 3600, decode('3E0E2000', 'hex'), 3600); If you use a time zone different from UTC or CET summer time, please add it. 3.2.5 ‘usage_points’ table A “usage point” in GreenButton is roughly equivalent to a single meter. It has a relation 1:n with the “retail_customers” table that is a retail customer can be linked to more than one usage point. The code list for the servicecategory_kind attribute is described in Annex 2 – Codelists. INSERT INTO usage_points( id, description, self_link_href, self_link_rel, up_link_href, up_link_rel, uuid, servicecategory_kind, localtimeparameters_id, retail_customer_id) VALUES ( 1, 'FER-001', '/RetailCustomer/1/UsagePoint/1', 'self', '/RetailCustomer/1/UsagePoint', 'up', '106E8B03-0299-467E-972A- A883ECDCC594', 1, 1, 1);
  34. 34. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 34 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). 3.2.6 ‘reading_types’ table A reading type is a group of readings that share the same characteristics in terms of:  type of commodity (gas, electricity …)  unit of measure  the type of measure (incremental or cumulative)  the currency for eventual costs and other more detailed and not mandatory features. A reading type is related to 1 or more meter reading. The codelists for the attributes accumulationbehaviour, commodity, currency, timeattribute and uom are described in Annex 2 – Codelists. Just as examples the accumulationbehaviour code indicates if a measure of energy regards an interval of time of the cumulative measure since the start of measuring; the commoditytype is the kind of energy mean; currency is always euro; timeattribute is the reading frequency and finally the uom is the unit of measure of the reading. INSERT INTO reading_types( id, description, self_link_href, self_link_rel, up_link_href, up_link_rel, uuid, accumulationbehaviour, commodity, currency, timeattribute, uom) VALUES ( 1, 'GASR_MCU_1h', '/ReadingType/1', 'self', '/ReadingType', 'up', '106E8B03-0299-467E-972A-A883ECDCC593', 4, 7, 0, 7, 42); Here is shown a gas meter reading, with a frequency of an hour. 3.2.7 ‘meter_readings’ table A meter reading is not the real reading coming from the meter, but a way to group them in order to simplify the relation with the usage point and the characteristics of the reading stored in the reading_types table. In fact the real readings are stored in the interval_block table where only the relation with the meter_reading table is necessary. INSERT INTO meter_readings(
  35. 35. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 35 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). id, description, self_link_href, self_link_rel, up_link_href, up_link_rel, uuid, reading_type_id, usage_point_id) VALUES ( 1, 'FER-001_GASR_MCU_1h', '/RetailCustomer/1/UsagePoint/1/MeterReading/1', 'self', '/RetailCustomer/1/UsagePoint/1/MeterReading', 'up', '106E8B03-0299-467E- 972A-A883ECDCC592', 1, 1); Here is shown the gas meter reading type is associated with the usage point #1. Download the SQL script located at http://sunshine.sinergis.it/download/greenbutton/1.1/prepopulateSunshineDC_postgreSQL.sql. (If you installed the DataCustodian DB in a mySQL instance, just add single quotes to table and field names and to values to get the mySQL syntax. Change the UUID, the date values, the client secret and the DataCustodian endpoint. These changes are indicated in the script file by comments. Even the retail user’s credential should be changed.  Run the SQL script inside a SQL console connected to the datacustodian DB.  Communicate to the Sunshine Platform admin the name of the registered retail users12 3.3 Import data service In order to populate the datacustodian DB with the dynamical consumption data coming from the energy meters, the DataCustodian application is provided with an import service. Consumption readings can be imported one by one or in a contiguous group (as meters may not always be capable of sending reading in real time, but instead may store them temporarily and send them in bundles) and each data import corresponds to a new interval block item in the DB. Consumption data has to be formatted in XML and posted to the import service whose URL specifies the RetailCustomer, UsagePoint and MeterReading the data refers to. The service is a RESTlike service whose 12 At the moment the ThirdParty application has not a self registration functionality.
  36. 36. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 36 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). URL indicates the resource to be modified (the IntervalBlock); a POST to this URL should be invoked with a payload coded as an XML. Service URL and XML template are described below. 3.3.1 Service url The URL of the import service is like this: http://<pilot-greenbutton-server>/DataCustodian/espi/1_1/resource/<interval-block-up-link> where <interval-block-up-link> has the form: RetailCustomer/X/UsagePoint/Y/MeterReading/Z/IntervalBlock Labels X, Y and Z are respectively the IDs of the RetailCustomer whose data are being uploaded and the relative UsagePoint and the MeterReading they refer to. Important note: generally speaking, different interval blocks of consumption data will correspond to different UsagePoints/MeterReadings (as each pilot have more than one meter), so be careful to set X, Y and Z values accordingly each time you call the import service. i.e. http://sunshine-fe.sinergis.it/DataCustodian/espi/1_1/resource/RetailCustomer/1/UsagePoint/3/ MeterReading/2/IntervalBlock Where "sunshine-fe.sinergis.it" is the base URL for the server exposing consumption data for the pilot in Ferrara, and RetailCustomer id = 1 ("Ferrara municipality"), UsagePoint id = 3 (gas meter in kindergarten Rampari) and MeterReading id = 4 (hourly reading of total cubic meters of consumed gas). 3.3.2 Consumption data XML The XML-formatted data provided below is a template for the format consumption data that has to be provided in to the data import service: <ns3:entry xmlns:espi="http://naesb.org/espi" xmlns:ns3="http://www.w3.org/2005/Atom"> <ns3:id>urn:uuid:3061204d-0148-1000-0000-000000000318</ns3:id> <ns3:link href="http://sunshine-fe.sinergis.it/DataCustodian/espi/ 1_1/resource/RetailCustomer/1/UsagePoint/3/MeterReading/4/IntervalBlock" rel="up"/> <ns3:content> <espi:IntervalBlock> <espi:interval> <espi:duration>7200</espi:duration> <espi:start>975628800</espi:start>
  37. 37. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 37 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). </espi:interval> <espi:IntervalReading> <espi:timePeriod> <espi:duration>3600</espi:duration> <espi:start>975628800</espi:start> </espi:timePeriod> <espi:value>1023</espi:value> <espi:cost>1987</espi:cost> </espi:IntervalReading> <espi:IntervalReading> <espi:timePeriod> <espi:duration>3600</espi:duration> <espi:start>975632400</espi:start> </espi:timePeriod> <espi:value>1107</espi:value> <espi:cost>2139</espi:cost> </espi:IntervalReading> </espi:IntervalBlock> </ns3:content> <ns3:published>2000-12-01T02:00:00+01:00</ns3:published> <ns3:updated>2000-12-01T02:00:00+01:00</ns3:updated> </ns3:entry></ns3:entry> <ns3:id> this will be the UUID of the interval block we are importing. As for all the items in the DB, it has to be unique and has to be generated by the user13 . <ns3:link > this is exactly equal to the import service URL specified in the previous sub-section and is likewise dependent on the specific IDs of the RetailUser, UsagePoint and MeterReading the imported consumption data refer to. <espi:interval> This block describes the overall start and duration of the uploaded group of consumption readings. Duration is expressed in seconds while the start date is expressed in Unix time14 , that is defined as the number of seconds that have elapsed since the midnight UTC of January 1st 1970, not counting leap seconds. Important note: This means that start date is expressed in UTC time reference frame. 13 See footnote 11 14 http://en.wikipedia.org/wiki/Unix_time. An online conversion tool (just for the sake of testing) can be found here: http://www.onlineconversion.com/unix_time.htm
  38. 38. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 38 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). <espi:IntervalReading> This block describes start date and duration of the single reading, following the same format of the espi:interval tag, as well as the measured consumption value and the related cost. Consumption value has to be expressed as thousandth (10-3 ) of the measuring unit (i.e. 1 kWh has to be written as 1.000). Cost has to be expressed as hundred-thousandth (10-5 ) of the currency unit (i.e. 1 Euro has to be written as 100.000). Cost can be omitted if not available. <ns3:published> Both tags ns3:published and ns3:updated have to be populated and they have to report the same value, corresponding to the timestamp of the end of the interval block, that is to say the start time of the espi:interval plus its duration. The format of the timestamp is the ISO Time15 format. Time can be specified in any time zone reference, but UTC reference is preferred (i.e. 2000-12-01T01:00:00Z [UTC] and 2000-12-01T02:00:00+01:00 [CET] are equivalent and are both accepted). 3.4 Export data Consumption data properly uploaded into the DB will be then periodically harvested by the central sunshine platform in order to feed the central sensor DB and be from there exposed to the final sunshine user. In order to allow sunshine central platform to properly ingest data coming from GreenButton export service, each pilot has to fill a mapping file along the lines of the following template. Pilots should also specify the ID of the RetailUser associated with the sunshine platform (recommended ID is ID = 1). Meter identification (from meter_mapping file) Meter Reading ID Reading Type ID Usage Point ID FER-003_GASA_MCU_1m 4 1 3 […] 15 http://en.wikipedia.org/wiki/ISO_8601
  39. 39. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 39 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Pilots can test the export service by calling the URL http://<pilot-greenbutton-server>/ DataCustodian/espi/1_1/resource/Batch/RetailCustomer/X/UsagePoint/Y where labels X and Y are respectively the IDs of the RetailCustomer (has to be the “sunshine platform” user defined in the previous section, typically id = 1) and UsagePoint of interest. The export service must be exposed on the internet and should be reachable with HTTP protocol. 3.5 Security underneath the application layer OAuth works at the same OSI layer of HTTP (application layer or layer 7). In order to achieve a complete security of the data flow in GB “Connect My Data” scenario, OAuth is not enough and Transport Layer Security (TLS) is required for all exchanges. Despite its name, TLS works at presentation layer (OSI 6 layer) and during the negotiation is on OSI layer 5, session layer. The developers of the protocol recommend that all implementations move to TLS version 1.2. However, most client web browsers are still at TLS 1.0 or at best TLS 1.1. So, the following levels of TLS are required for the different actors:  Data Custodian – Must implement TLS 1.2  Third Party – Must Implement TLS 1.1 or higher  Retail Customer – Can Implement TLS 1.0, 1.1, or 1.2 Each party must choose the highest level of TLS mutually supported during the TLS negotiation. Thus in the rest of the document whereas in an URL an http: schema is written, in production you should use the https: one. 3.6 Generation of the access token The generation of the access token is done interactively by the retail customer accessing the GUI of the DataCustodian web application installed at the pilot’s server and through a redirection to the GUI of the ThirdParty web application installed on the Sunshine Platform, reproducing the interaction of the OAuth protocol. The generation of the access token involves the active participation of two actors over the web: the pilot (DataCustodian) and the Sunshine Platform (ThirdParty). Thus they must know of each other and must see each other over the internet. In other word their configuration must provide the URL reference of the necessary service, the must share the same “secret”, the human user that does the operation must be registered on both platforms.
  40. 40. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 40 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). The steps are the following: 1. Access the DataCustodian GUI at the address: http://<pilot-greenbutton-server>/DataCustodian/ (i.e. http://sunshine-fe.sinergis.it/DataCustodian/ ) 2. Login the application Here the retail customer should authenticate him/herself with the credentials that have been assigned to him/her by the administrator of the DataCustodian application. 3. The application will show the user a welcome page where a menu bar allows to activate the following functions: a.Usage Points (list the configured usage points whose data are maintained in the DB) b.Third Parties (list the configured third parties known to the Data Custodian and that can be authorized to access the retail customer’s EUI data). c. Logout 4. Click the “Third Parties” menu item and a radio button list (probably formed by just one element) of organizations that can play the third party role will be shown. Aside the third party, there will be the URL to which the retail user will be redirected for the scope selection phase. This is an address of the ThirdParty application installed in the Sunshine platform. 5. Choose the “Sunshine platform” item and click next. 6. You will be redirected to the Third Party login page since you’re not authenticated in the Sunshine platform, but only at the pilot’s side. 7. After the login, a radio button list with the type of authorizations that can be granted (scope) will be shown. For our purposes, only a scope “read” will be in the list: choose it and click next. 8. The ThirdParty application will therefore redirect the user’s browser to a DataCustodian page (no authentication needed this time) to confirm the intention of giving “read” authorization access to “Sunshine platform”. Choose “Approve” and click “Submit”. 9. Now, another redirection back to ThirdParty application, to a page where the Access Token (and its relative Refresh Token) generated by the sequence of operations just concluded are shown. The generated tokens are also stored in the ThirdParty DB to be used by the Sunshine platform for accessing the EUI data.
  41. 41. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 41 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Generate the access token for the Sunshine Platform by following the steps above. Before beginning, assure that the retail user has been registered on the pilot’s DataCustodian application and on the Sunshine platform ThirdParty application. There’s no self-registration, so these operations must be done by the administrators. Take contact with the Sunshine Platform administrator to share the user name and his/her password.
  42. 42. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 42 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Annex 1 – DST rules Daylight Saving Time (DST) is defined by two rules, one identifying the start date of DST within a year and the second identifying the end date. Each country or group of countries define its own rules, for instance North America and EU use two slightly different set of rules. Each rule can be encoded in an 8-digit hex string following the following schema. The bit encoding: Bits 0 - 11: seconds 0 - 3599 Bits 12 - 16: hours 0 - 23 Bits 17 - 19: day of the week 0 = not applicable, 1 - 7 (Monday = 1) Bits:20 - 24: day of the month 0 = not applicable, 1 - 31 Bits: 25 - 27: operator (detailed below) Bits: 28 - 31: month 1 - 12 Rule value of 0xFFFFFFFF means rule processing/DST correction is disabled. The operators: 0: DST starts/ends on the Day of the Month 1: DST starts/ends on the Day of the Week that is on or after the Day of the Month 2: DST starts/ends on the first occurrence of the Day of the Week in a month 3: DST starts/ends on the second occurrence of the Day of the Week in a month 4: DST starts/ends on the third occurrence of the Day of the Week in a month 5: DST starts/ends on the forth occurrence of the Day of the Week in a month 6: DST starts/ends on the fifth occurrence of the Day of the Week in a month 7: DST starts/ends on the last occurrence of the Day of the Week in a month An example: EU DST rule for CET (UTC+01:00) Start: Last Sunday in March End: Last Sunday in October Time: 1.00 am (01:00) Greenwich Mean Time (GMT) http://wwp.greenwichmeantime.com/time-zone/rules/eu/
  43. 43. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 43 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). The hex strings: <dstEndRule>AE0E2000</dstEndRule> <dstStartRule>3E0E2000</dstStartRule> The conversion:
  44. 44. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 44 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). Annex 2 – Codelists The codelists reported in this annex are limited to the values that are used for the consumption meters in the project sunshine. The following subsections will provide the mapping between the meter properties already described by each pilot in the meter_mapping file into the proper value of the relevant codelist. Each meter listed in the meter_mapping file is associated with a meter identification string: <meterID>_<type><abs/rel>_<unit>_<freq> (i.e. FER-003_GASA_MCU_1m) Where the following relations with GreenButton DB schema apply: <meterID> UsagePoint description FER-003 <type> ServiceCathegory/Commodity GAS <abs/rel> AccumulationBehaviour A <unit> Uom MCU <freq> TimeAttribute 1m The following sections describe the mapping between the codelist values defined in the meter_mapping file and the corresponding relevant values of the GreenButton codelists. ServiceCategoryKind Meter_mapping ServiceCategoryKind (GreenButton) Value ELE 0 electricity GAS 1 gas THE 5 heat AccumulationBehaviourType Meter_mapping AccumulationBehaviour (GreenButton) Value A 3 Cumulative R 4 DeltaData CommodityType Meter_mapping CommodityType (GreenButton) Value ELE 1 Electricity secondary GAS 7 NaturalGas THE 12 HeatingFluid
  45. 45. WP4 – Integration of SUNSHINE pilot smart urban services D4.5 Meter Data Management service #2 CIP-ICT-PSP-2012-6 – 325161 Page 45 of 45 "This project is partially funded under the ICT Policy Support Programme (ICT PSP) as part of the Competitiveness and Innovation Framework Programme by the European Community" (http://ec.europa.eu/ict_psp). TimeAttributeType Meter_mapping TimeAttributeType (GreenButton) Value 15 2 15-minute 1h 7 60-minute 1d 11 Daily 1w 24 Weekly 1m 13 Monthly 1y 0 Not Applicable ir 0 Not Applicable UomType Meter_mapping UomType (GreenButton) Value KWH 72 Real energy (Watt-hours) MCU 42 m3 (Cubic Meter) LTR 134 Liter CurrencyCode Follows codes defined in ISO 421716 . CurrencyCode (GreenButton) Value 0 Not Applicable 840 US Dollar 978 Euro 16 http://en.wikipedia.org/wiki/ISO_4217

×