R is a statistical programming language and environment used for data analysis and visualization. It allows complex calculations and predictive modeling on big datasets. o Spark: fast and general engine for large-scale data processing. It is used to perform the complex calculations on the stored data in Hadoop. o Mahout: library for scalable machine learning. It provides algorithms to perform recommendation, clustering and classification on big datasets. o GraphLab: library for machine learning and graph analytics. It is used to perform predictive modeling and recommendation based on customer profiles and consumption patterns. So in summary, the Insight Engine is a scalable and modular architecture based on open source technologies that allows: - Fast and secure communication through the

1. Project co-funded by the European Commission within the IEE Programme
Dissemination Level
PU Public x
CO Consortium and EASME
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
EMPOWERING customers to save
energy by informative billing
Deliverable 4.1
First report on initial software implementation and
test
Project No. IEE/12/697/S12.645843

2. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
2
Authors
Name Organisation
Giovanni Pede SINERGIE
Stoyan Danov CIMNE
Fabio Renda CIMNE
Marina Pirazzi REGGIO EMILIA
Duilio Cangiari REGGIO EMILIA
Sara Cattabriga SINERGIE
Revision history
Revision Date Author Organisation Description
0.1 18/02/2014 Giovanni Pede SINERGIE Initial template
0.2 25/02/2014 Fabio Renda CIMNE
Empowering architecture and
technical testing
0.3 28/02/2014 Stoyan Danov CIMNE Revision and comments
0.4 10/04/2014 Giovanni Pede SINERGIE Final version
0.5 15/09/2014 Giovanni Pede SINERGIE
Including end-user feedback
from utility surveys
0.6 15/09/2014 Gabriela Ganeva CIMNE Overall revision

3. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
3
Table of contents
1 Introduction and Objectives..........................................................................................................4
2 Implementation of the EMPOWERING system...............................................................................5
2.1 Global architecture of the Insight Engine........................................................................................ 5
2.2 Application Programming Interface (API)........................................................................................ 7
2.2.1 Data collection..........................................................................................................................................................7
2.2.2 Analytic services .......................................................................................................................................................7
2.2.3 API Documentation page..........................................................................................................................................8
2.2.4 Security...................................................................................................................................................................10
3 Implementation at utilities ......................................................................................................... 11
3.1 Implementation of the Billing and Online tools ............................................................................ 11
3.1.1 Austria ....................................................................................................................................................................11
3.1.2 France.....................................................................................................................................................................13
3.1.3 Italy.........................................................................................................................................................................19
3.1.4 Spain.......................................................................................................................................................................23
4 Testing .......................................................................................................................................25
4.1 Technical test................................................................................................................................. 25
4.1.1 API communication tests........................................................................................................................................25
4.1.2 Testing of analytic services.....................................................................................................................................27
4.2 Test with users............................................................................................................................... 28
4.2.1 Usability and Visualisation assessed by experts.....................................................................................................28
4.2.2 Feedbacks from the utilities’ internal staff.............................................................................................................35
4.2.3 End-user feedback..................................................................................................................................................44
5 Conclusions and next steps ......................................................................................................... 56

4. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
4
1 Introduction and Objectives
The current document reports the initial implementation of the software packages and services in
EMPOWERING. It presents the status of development of the central analytical engine (Insight Engine), API
for data exchange and the initial development of the Billing and Online tools at the utilities. Results from
technical testing and usability issues are also presented.
The deliverable follows the initial EMPOWERING specifications and provides the basis for the improved
second version of implementation.
D2.1a Internal report on
service and data
D2.2a Internal report on
customer expectations
D2.3a Internal report on
use cases
D3.1 Initial specifications
of EMPOWERING
D4.1 First report on initial
implementation and test
D2.1b Report on service
and data requirements
D2.2b Publishable report
on customer expectations
D2.3b Publishable report
on use cases
D3.2 Specifications of
EMPOWERING
D4.2 Second report on
implementation and test
D2.4 Report on legal,
regulatory and data
Version1
Version2

5. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
5
2 Implementation of the EMPOWERING system
2.1 Global architecture of the Insight Engine
The figure below shows the final technical solution implemented in the Insight Engine. The solution is
entirely based (100%) on open source software. In order to describe the functionality of the whole system
it is divided into 4 conceptual domains:
API: In charge of the communication with external systems. The API is fully developed following the REST
standard, nowadays the most widely way to implement and enable the interaction with cloud systems.
TASK MANAGEMENT SYSTEM: a framework that permits the different technologies to work together, share
and schedule tasks and assuring high system performance.
ANALITYCS: a comprehensive set of technologies that enable the simultaneous parallel processing of big
quantities of data in order to generate the required results in reasonable time.
STORAGE: a combination of database technologies permitting the acquiring, allocation, extraction and
availability for analytics of big quantities of data in a scalable manner, maintaining low hardware costs.
2.1.1.1 API REST interface
Developed using python frameworks and highly extensible:
o Flask and Python-Eve: implements all the server functionalities in order to deploy a web API
server
o OpenAM: provides open source Authentication, Authorization, Entitlement and Federation
software
o Mongo DB: is the first data storage directly connected with the API in order to be scalable
and fast.
2.1.1.2 Task manager / Scheduler

6. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
6
In charge of manage and synchronize the operations in the system
o RabbitMq: fast internal messages queue system used to move the information through
different technologies
o Celery: job scheduling system, used to launch processes and analysis
2.1.1.3 Storage
This domain is the long term storage, in charge to save the big amount of data in a scalable way and offer
some basic tools of analysis.
This domain consists of short and long term storage.
o MongoDB: software package for buffer storage in fast environment for data reception and
sending.
o Hadoop: long term big data storage over which the analytical modules work
2.1.1.4 Analytics
Set of tools in charge to execute the processing of the stored data. It is a combination of technologies and
software libraries permitting complex calculations in big data environment
o R + RHIPE: Statistical programming language coupled with the RHIPE library that offers the
access directly to the Hadoop stored data and allowing to process big data quantities.
o Python libraries: collection of scientific software libraries (Pandas, SciPy, NumPy ) that
enables other advanced tools for data analysis as an alternative of R and permits to
optimise the calculations and reduce the data processing time.
2.1.1.5 Summary
The table summarises the status of implementation and testing of the technologies described in the
architecture of the entire system.
DOMAIN ARCHITECTURE MODULE STATUS
API Flask IMPLEMENTED/TESTED
API Python-Eve IMPLEMENTED/TESTED
API OpenAM IMPLEMENTED/TESTED
SCHEDULER RabbitMQ IMPLEMENTED/TESTED
SCHEDULER Celery IMPLEMENTED/TESTED
ANALITYCS R+RHIPE IMPLEMENTED/TESTED
ANALITYCS Python Libraries IMPLEMENTED/TESTED
STORAGE Hadoop family IMPLEMENTED/TESTED
STORAGE MongoDB IMPLEMENTED/TESTED

7. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
7
2.2 Application Programming Interface (API)
The API includes the definition of the resources for data exchange, the implementation of the Extract-
Transform-Load (ETL) operations needed for allocating the received data in Hadoop, and the processes for
returning the results from the analytics. The API permits bi-directional exchange of data between the utility
systems and the Insight engine.
2.2.1 Data collection
The data collection is done by systematising it into conceptual groups (resources). In the first version
implementation the following resources were created:
• COMPANIES: an internal resource not reflected in the API documentation for the utilities which
includes identifiers to separate the data by company
• CONTRACTS: this resource contains the customer data related to the contract with the company
and the metering device
• AMON_MEASURES: this resource groups the energy consumption data related to the metering
devices. The resource is based on the AMON open standard for energy data exchange (
https://github.com/AMEE/AMON ). It is internally separated into two resources:
o MEASUREMENTS: contains the values of the energy consumption parameters and the time
stamps
o READINGS: contains the type, period and physical unit of the energy consumption values.
API RESOURCES STATUS
COMPANIES IMPLEMENTED/TESTED
AMON_MEASURES IMPLEMENTED/TESTED
CONTRACTS IMPLEMENTED/TESTED
2.2.2 Analytic services
The initial implementation was done over a reduced set of services. For each utility a set of 3 services was
implemented. The services were tested successfully in the stages of data sending, processing and result
retrieving.
SERVICE ID STATUS
OT101 IMPLEMENTED/TESTED
OT103 IMPLEMENTED/TESTED
OT201 IMPLEMENTED/TESTED
OT106 IMPLEMENTED/TESTED
OT204 IMPLEMENTED/TESTED
BT111 IMPLEMENTED/TESTED

8. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
8
2.2.3 API Documentation page
The detailed API documentation and examples for implementation of the REST client for sending and
retrieving of data by the utilities are offered in the following web domain. The information is progressively
completed and offered as a common reference for the utilities.
https://api.empowering.cimne.com/v1/docs/
The next sections show screenshots of the documentation page. In order to access the full documentation
on the web an authorization by the EMPOWERING administrator is needed.
In continuation the methods allowed by the API are described.
2.2.3.1 POST
Description of how to make a POST in the API. This command pushes data to the Insight Engine.
2.2.3.2 GET
Description of how to make a GET. This command retrieves information from the Insight Engine.

9. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
9
2.2.3.3 PATCH
Description of how to make PATCH in the API. This command updates the data in the Insight Engine
2.2.3.4 DELETE
Description of how to make DELETE in the API. This command deletes the information in the Insight Engine
2.2.3.5 Service result example
The different EMPOWERING services are implemented as modules in the Insight Engine and are designated
with unique alphanumeric identifiers. The API offers complete description of the results for each module
and an example of how to retrieve it from the system.

10. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
10
2.2.4 Security
The security of the system has been assured through the implementation of the following technologies.
• SSL protocol for user and password sending
• X.509 certificate in server and client
• OpenAM: identification and access management platform
TECHNOLOGY STATUS
SSL IMPLEMENTED
X.509 Certificate IMPLEMENTED
OpenAM IMPLEMENTED

11. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
11
3 Implementation at utilities
3.1 Implementation of the Billing and Online tools
On the basis of the results achieved at the present date, each pilots developed a first version of the Billing
tool or Online tool to be provided to their users. In this section mock-ups and/or screenshots of
implemented services have been inserted.
3.1.1 Austria
Online tool
LINZ STROM will implement only the OT within the project EMPOWERING. Following some screenshots of
the services developed so far:
OT 201: screenshot of the service implemented in the utility website.
OT 103: screenshot of the service implemented in the utility website.

12. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
12
OT102: screenshot of the service implemented in the utility website

13. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
13
3.1.2 France
Online tool
The screenshot below shows the services implemented in the utility website, in a specific session.
Global Introduction “Welcome” page:
Followed by “My consumption page”

14. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
14
OT 106 screenshots (service that should migrate onto a BT106 for GEG)

15. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
15
Below are inserted other screenshots of services implemented within the website of GEG:
OT101
OT105
OT201

16. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
16
OT601

17. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
17
Billing tool
The Mock-up of the billing tool has been developed using the same screenshots of the OT. It represents
BT208, BT111

18. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
18

19. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
19
3.1.3 Italy
Online tool
The online tool of IREN has been implemented in the private network of IREN. The application below
already exists but it is not accessible by the public. The screenshots below shows OT101 and OT103.

20. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
20
Billing tool (electric energy billing)
IREN developed a first concept of the billing tools, one addressed to electricity users and one addressed to
district heating consumers. The concepts/layouts of the billing tools are inserted below.
In the electricity tool the following services are recognisable: BT101, BT102, BT108, BT110, BT201, BT204,
BT401

21. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
21

22. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
22
Billing tool (district heating billing)
In the thermal tool the following services are recognisable: BT101, BT102, BT108, BT110, BT201, BT207,
BT401

23. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
23
3.1.4 Spain
Online tool
The following services have been implemented in the utility website. In the screenshot are inserted: OT
101, OT201, OT103, OT303

24. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
24
Billing tool
Below is inserted the mock-up of the billing tool developed on the basis of the OT. The services inserted
are: BT101, BT201, BT106, BT303, BT401.

25. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
25
4 Testing
The testing at that initial stage had the objective to:
• Verify the technical capabilities of the Insight Engine and the API after the initial software
implementation
• Receive data, process data and return results at the agreed format permitting the visualisation at
the utility applications (Billing and Online tools).
• Check the usability of the visualisations developed by the utilities to provide the EMPOWERING
services to their customers.
The results of the tests and the users’ feedback should be taken into account for introducing improvements
in the next stage of development.
4.1 Technical test
The test checks the treating capacity of the system to work with the quantities of data expected to be
processed in the Empowering project.
The functionality of the API is tested taking in account a sufficient number of utilities and customers to
assure the right working condition of the Insight Engine.
4.1.1 API communication tests
In order to test the API all of the following methods were executed successfully:
- POST
- GET
- PATCH
- DELETE
The test was done during the month of February 2014 through exchanging data between the Insight Engine
and the utilities. The data quantities (number of contracts and metering data) were as shown in the table.
UTILITY NUMBER OF CONTRACTS NUMBER OF MEASURES
ELGAS 8670 212.000.000
IREN 298 3.417
GEG 2302 20.114
LINZ 1100 20.614

26. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
26
Additionally, in order to test the capabilities of the system to operate in a scale similar to the expected
widespread application, by means of on purpose developed Python script a randomly generated sample of
contracts and measures was obtained, and the data was pushed to the Insight Engine. The operation
simulated data from 1 hypothetic utility with smart meters delivering 15 minute readings:
- 100.000 contracts
- 1 year of measures (12 times of 288.000.000 measures per month)
The data was exchanged through the API in packages of 10.000 measures. The test was successfully
completed using the methods for the API resources shown in the table below.
METHOD RESOURCE STATUS
POST CONTRACT OK
POST AMON_MEASURES OK
GET CONTRACT OK
GET AMON_MEASURES OK
PATCH CONTRACT OK
DELETE CONTRACT OK
GET OT101Results OK
GET OT103Results OK
GET OT201Results OK
GET OT106Results OK
GET OT204Results OK
GET BT111Results OK

27. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
27
4.1.2 Testing of analytic services
The first implementation of the EMPOWERING analytic services (February 2014) consisted of the
implementation of 6 services from which 3 services were selected by each utility. The services were tested
using the meter data pushed by the utilities (refer to previous section). The Insight Engine correctly
returned the results necessary to generate the visualisation in the utility Billing and Online tools. The
variables containing the returned results for each customer are shown in the table.
SERVICE ID PILOT IMPLEMENTING THE
SERVICE
RESULTS RETURNED
OT101
AT, IT, ES
[ "consumption", "averageEffConsumption",
"averageConsumption", "diffAverageEffConsumption",
"diffAverageConsumption","numberCustomersEff",
"numberCustomers"]
OT103
AT, IT, ES
[ "monthConsumption", "averageEffConsumption",
"averageConsumption", "numberCustomersEff",
"numberCustomers"]
OT201
AT, IT, ES
["actualConsumption", "previousConsumption",
"diffConsumption"]
OT106 FR [ "monthConsumption", "averageEffConsumption",
"averageNoEffConsumption", "numberCustomersEff",
"numberCustomersNoEff"]
OT204 FR [“monthConsumption”]
BT111 FR ["consumption", "averageConsumption",
"aggregateConsumption", "averageEffConsumption",
"numberCustomers", "numberCustomersEff",
"typeConsumption"]

28. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
28
4.2 Test with users
The testing with users regarding usability and understanding of the services has been approached from 3
perspectives: by external experts in usability Focus Group, by survey among the internal staff of the utilities
involved in providing the services and by a survey among a sample of the end-users of the services (utility
customers).
4.2.1 Usability and Visualisation assessed by experts
The Usability Focus Group (FG) was organised on the 31st
of March in Reggio Emilia by Sinergie and
Extrafondente. Experts of different fields related to usability assessment were invited. The following
experts take part on the FG:
Giuliana Fornaciari, Architect, expert in energy infographics
Giulia Saija, graphic professional
Daria Dolfini, psychologist and expert in ergonomy
Alessandro Borghetti, web designer and developer
Omar Livoni, Energy manager
Faiza Mahri, cultural and linguistic mediator
The FG was conducted by Giovanni Pede and Marina Pirazzi. The aims of the FG were to collect experts’
opinions and feedback on visualisation and usability issues on the different mock-ups produced for each
pilot. The expert provides feedback on services developed in Italy, Austria, Spain and France. The FG
consisted on two separate sessions:
First session. The participants received four identical 3-item questionnaires (one for each project pilot) and
were given 45’ to fill them in by looking at the mock-ups hanged on the wall.
Second session. There was a debate when each participant had the opportunity to briefly express their idea
on the mock-ups presented.

29. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
29
Before discussing specifically on each pilot mock-ups, general results were collected:
a) All participants agreed that all mock-ups are too complex to be addressed to different target audiences.
They strongly suggest to work-out 3 “complexity levels”: 1) one very simple page as the only BT for all
clients receiving paper invoices and as the first page of the OT, with icons rather than graphs, and a few
lines commenting their energetic performance (in Italy the sentence would be in at least 2 languages in
addition to Italian). 2) and 3), pages including more articulated and complex information available on the
web for those wishing to access more sophisticated (and difficult to understand) information on their
performance.
b) Specify who are the neighbours: same district? Closest building or house? Etc.
c) Always add a few lines explaining the graph, following the example of the French mock-up.
d) Introduce icons to label the graph axes in order to ensure a better understanding and to make them more
friendly and intuitive. Following an example drawn by a FG participant:
e) In graphs, annual cumulative consumptions should be always highlighted the relation with the climate
impact
f) Utilities have to specify why they are providing such services as customers could be suspicious on their real
interests on communicating how to save energy
g) Tips are meaningful only if they are linked to actual performance data, type of heating system, seasons.
General tips are useless.
h) The information about expected performances for the current year have been very appreciated (it is a
service included in the French mock-up). The experts think that this kind of service should be added by all
partners.
i) Daily consumptions could be useful but only for advanced consumers; it should also be linked to the
advices in order to take notice on the domestic appliance in use that particular day/moment.
j) Information about savings in € would be the best motivator. Where it is not possible to provide it, such as
in Italy, express savings in KWh.
k) Add a reward to increase awareness and involvement of the users. The example provided during the FG
discussion is to use a “tree” to indicate the contribution the user is giving to the community, in addition to

30. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
30
his/her energy savings. The number or the dimension of the tree icon(s) can be used to show the
environmental impact of user behaviour, as the tree is symbolically decoded as positive in all
communities/nations/languages.
l) When implementing paper-based or online services, the utilities have to take into account the F-pattern. A
participant of the FG explained that it has been proved, using eye tracking tools, that a user reads
documents and web-pages in a F-shaped pattern and that this is also the way with which Google shows the
results to the users of the search engine. The image below show the pattern followed by users when
reading text and screens:
Red areas shows the points of the screen on which a reader focuses its attention. Blue areas are the one
which are not read by the user. F-pattern has been used to provide suggestions to the Italian mock-up for
the Billing Tool.
After the general discussion, pilot-specific feedbacks have been collected.
Austria
a) Graphs are very clear and intuitive
b) The mock-ups lacks on conclusions. Comments and conclusions on the user performance should be
inserted
c) Insert icons to make more friendly the graphs
d) The following service could be misleading. It remind overly on the energy classes/certification while in
the service the provided information is CO2

31. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
31
e) Always put scales, labels and units in the graphs. The following is not clear
France
a) The textual part is most detailed of all pilots. It will help less advanced users
b) In the BT, the textual explanation should be before the graph

32. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
32
c) Uniform colouring is not very understandable. It is better to adopt a gradation of colors in order to have
a better understanding of where begin the “good” are, where is the “bad” following the example of the
same service in the Spanish mock-up (on the right)
The colours should be changed with more evocative ones (e.g. the colours of the traffic light)
The labels on the x axis should be moved as this is the position only for the units and for the name of the
axis
Italy
Referring the Italian pilots, the BT mock-ups (District heating and electricity) and the OT mock-up have been
shown to the Focus Group participants. Major suggestions for the Italian mock-ups were:
a) The appearance of BT and OT is different, they seems as provided by different utilities
b) An icon to explain what is POD should be inserted
c) Users cannot understand what are degree days. Add explanations, preferably in a graphic way.
d) The following data should be better represented using two pie charts. Also percentages of improvement
on the total should be added

33. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
33
e) In order to show cumulated consumptions, the proposed graph in the mock-up of Iren is not very
understandable. Better to substitute it with a line chart
f) Taking into account usability tips and the F-Pattern:
- Users are attracted by icons
- In the line (3) inside the right square people expect to find conclusions
- First row (1) is completely read by users, the second (2) is partially read while in the third row (3) only the
left square is read, the rest is ignored
Minor changes to improve the usability of the Italian BT have been suggested.

34. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
34
The proposed BT mock up is on the left. On the right the suggested changes on the same mock up. Tips
should be moved to the first row in order to ensure they will be read by the users. Also, the smiling icon
attract the attention of users and should be placed in the position people expect to find the conclusions.
g) Referring the OT, the Focus Group participants find that more comments on graphs are needed. The
smile indicator is placed in a little visible position
h) In the OT the scroll menu should be removed. It is better to show directly the information the user
expect to see at first when he/she enter in the related page on the OT, namely the information on last
billing period. Allow users to shift to another billing period with an arrow button (left for previous periods
and right for the next) or for the mobile versions allow sliding left-right to see the other billing periods.

35. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
35
Spain
a) It would be effectiveness to include captions directly in the graphs, using icons to allow a more rapid
read
b) Modify colour gradation with more evocative gradation (green, yellow, red)
c) OT lacks on tips provision
d) Different services should be divided in separate rows. The following services should be divided
4.2.2 Feedbacks from the utilities’ internal staff
The purpose of the analysis was to investigate the level of understanding of the internal staff of the utilities,
not involved in the Empowering project so far, but that will be involved in the phase of the provision of the
Empowering services. The involved technicians were staff of the customer care offices, staff of the
information desks, programmers, technical assistants, managers.
Austria
In Austria 15 technicians responded to the questionnaire. The table below shows the responses for each
tested service and the related comments.

36. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
36
Service Questions and Answers
This service is useful
Strongly agree: 10/15
Agree: 5/15
This service is easy to understand
Strongly agree: 10/15
Agree: 5/15
I feel comfortable in providing explanation of this service to my
customer
Strongly agree: 10/15
Agree: 5/15
Comments
Instead of or in addition to metering point specify how the
customer can start with the service. Those information are often
not found in the invoice.
This service is useful
Strongly agree: 6/15
Agree: 9/15
This service is easy to understand
Strongly agree: 6/15
Agree: 9/15
I feel comfortable in providing explanation of this service to my
customer
Strongly agree: 6/15
Agree: 8/15
Disagree: 1/15
Comments
Neighbour comparison depends on too much factors (apartment
size, number of person etc). It is difficult to have a reliable data.
What if I have only 1 neighbour? There is a privacy problem.
It could be useful to specify different heating sources and not put
them all together in the comparison.
There are data protection problems. The utility must ensure the
anonymity of users

37. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
37
This service is useful
Strongly agree: 4/15
Agree: 10/15
Neither agree nor disagree: 1/15
This service is easy to understand
Strongly agree: 4/15
Agree: 9/15
Neither agree nor disagree: 1/15
Disagree: 1/15
I feel comfortable in providing explanation of this service to my
customer
Strongly agree: 4/15
Agree: 10/15
Neither agree nor disagree: 1/15
Comments
A benchmark only based on the apartment size is too much
imprecise
The indicator could be misunderstood as that's not the total CO2
consumption
France
In France 13 technicians responded to the questionnaire. The table below shows the responses for each
tested service.
Service Questions and Answers
As an expert in your field, do you find this this service clear?
Not very clear: 3/13
Quite clear: 7/13
Very clear: 3/13
Do you think that this mockup will be clear for your team?
Mostly disagree: 2/13
Quite agree: 9/13
Strongly agree: 2/13
Do you think that this mockup will be clear for customers
without additional explanations?
Strongly disagree: 1/13
Mostly disagree: 5/13

38. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
38
Quite agree: 6/13
Strongly agree: 1/13
Comments
//
As an expert in your field, do you find this this service clear?
Not very clear: 1/13
Quite clear: 7/13
Very clear: 5/13
Do you think that this mockup will be clear for your team?
Quite agree: 7/13
Strongly agree: 6/13
Do you think that this mockup will be clear for customers
without additional explanations?
Mostly disagree: 2/13
Quite agree: 7/13
Strongly agree: 4/13
Comments
//
As an expert in your field, do you find this this service clear?
Not very clear: 4/13
Quite clear: 8/13
Very clear: 1/13
Do you think that this mockup will be clear for your team?
Mostly disagree: 3/13
Quite agree: 9/13
Strongly agree: 1/13
Do you think that this mockup will be clear for customers
without additional explanations?
Strongly disagree: 1/13
Mostly disagree: 4/13
Quite agree: 7/13
Strongly agree: 1/13
Comments
//

39. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
39
As an expert in your field, do you find this this service clear?
Not clear at all: 1/13
Quite clear: 7/13
Very clear: 5/13
Do you think that this mockup will be clear for your team?
Mostly disagree: 1/13
Quite agree: 8/13
Strongly agree: 4/13
Do you think that this mockup will be clear for customers
without additional explanations?
Mostly disagree: 3/13
Quite agree: 8/13
Strongly agree: 2/13
Comments
//
As an expert in your field, do you find this this service clear?
Not very clear: 1/13
Quite clear: 5/13
Very clear: 7/13
Do you think that this mockup will be clear for your team?
Mostly disagree: 1/13
Quite agree: 7/13
Strongly agree: 5/13
Do you think that this mockup will be clear for customers
without additional explanations?
Mostly disagree: 1/13
Quite agree: 5/13
Strongly agree: 7/13
Comments
//
Italy
In Italy 11 technicians (4 for electricity and 7 for the district heating) responded to the questionnaire. The
table below shows the responses for each tested service and the related comments.

40. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
40
Service (Electricity) Questions and Answers
This service is easy to understand
Strongly agree: 2/4
Agree: 2/4
I feel comfortable in providing explanation of this service to my
customer
Strongly agree: 3/4
Agree: 1/4
Comments
It is necessary to insert a text for explanations
The graphic indicator is useful to summarize the contents of the
informative billing
Strongly agree: 4/4
This service is easy to understand
Strongly agree: 2/4
Agree: 2/4
I feel comfortable in providing explanation of this service to my
customer
Strongly agree: 2/4
Agree: 2/4
Comments
//
This service is easy to understand
Strongly agree: 1/4
Agree: 2/4
Disagree: 1/4
I feel comfortable in providing explanation of this service to my
customer
Strongly agree: 1/4
Agree: 2/4
Disagree: 1/4
Comments
It is necessary to insert a text for explanations

41. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
41
This service is easy to understand
Strongly agree: 1/4
Agree: 3/4
I feel comfortable in providing explanation of this service to my
customer
Strongly agree: 1/4
Agree: 3/4
Comments
It is necessary to insert a text for explanations
This service (tips) is easy to understand
Strongly agree: 1/4
Agree: 3/4
Comments
It is necessary to insert a text for explanations
Service (District Heating) Questions and Answers
This service is easy to understand
Strongly agree: 1/7
Agree: 6/7
The graphic indicator (smile) is useful to summarize the
contents of the informative billing
Agree: 6/7
Disagree: 1/7
I feel comfortable in providing explanation of this service to my
customer
Strongly agree: 1/7
Agree: 4/7
No opinion: 2/7
Comments
It is necessary to insert a text for explanations

42. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
42
This service is easy to understand
Strongly agree: 1/7
Agree: 5/7
Disagree: 1/7
I feel comfortable in providing explanation of this service to my
customer
Strongly agree: 1/7
Agree: 4/7
No opinion: 2/7
Comments
It is necessary to insert a text for explanations
This service is easy to understand
Strongly agree: 1/7
Agree: 6/7
I feel comfortable in providing explanation of this service to my
customer
Strongly agree: 1/7
Agree: 6/7
Comments
It is necessary to insert a text for explanations
It would be useful to insert a message with the foreseen future
consumption according to the trend of previous years
This service is easy to understand
Strongly agree: 1/7
Agree: 5/7
No opinion: 1/7
Comments
It is necessary to insert a text for explanations
It is necessary to insert tips related to the actual situation of the
customer
Provide more professional images
Spain
In Spain 8 technicians responded to the questionnaire. The table below shows the responses for each
tested service and the related comments.

43. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
43
Service Questions and Answers
The service description is clear
Strongly agree: 3/8
Agree: 5/8
This service is easy to understand
Strongly agree: 3/8
Agree: 5/8
I feel comfortable in providing explanation of this service to
my customer
Strongly agree: 3/8
Agree: 5/8
Comments
//
The service description is clear
Strongly agree: 2/8
Agree: 6/8
This service is easy to understand
Strongly agree: 2/8
Agree: 6/8
I feel comfortable in providing explanation of this service to
my customer
Strongly agree: 1/8
Agree: 7/8
Comments
//
The service description is clear
Strongly agree: 2/8
Agree: 6/8
This service is easy to understand
Strongly agree: 2/8
Agree: 6/8
I feel comfortable in providing explanation of this service to
my customer

44. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
44
Strongly agree: 1/8
Agree: 7/8
Comments
Granularity less than month – daily graph
Insert how the temperature of the month is computed
The service description is clear
Strongly agree: 2/8
Agree: 4/8
No Opinion: 2/8
This service is easy to understand
Strongly agree: 2/8
Agree: 5/8
No Opinion: 1/8
I feel comfortable in providing explanation of this service to
my customer
Strongly agree: 1/8
Agree: 5/8
No Opinion: 2/8
Comments
They have to be improved with customers’ feedbacks
4.2.3 End-user feedback
The collection of feedback from end-users has been implemented using the methodology and the tools
validated during a technical Focus Group on usability held in Reggio Emilia the 31 March 2014 (point 4.2.1
above). During this Focus Group experts have been asked to assess all mock ups developed in each pilot
using a 3-open-questions questionnaire which provide the base for a deep discussion on usability, graphic,
comprehensibility issues.
We asked the same questions to the end users after showing them the services implemented so far. We
decided to not collect too many questionnaires according to the Nielsen1
model, which demonstrates that
when investigating usability issues a number of 5 interviewees can recognize more than the 75% of
usability problems.
1
Nielsen, Jakob, and Landauer, Thomas K.: "A mathematical model of the finding of usability problems,"
Proceedings of ACM INTERCHI'93 Conference (Amsterdam, The Netherlands, 24-29 April 1993), pp.
206-213.

45. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
45
The table below shows a summary of collected questionnaires and the methodology used to address the
customers.
Pilot Austria France Italy Spain
n. of questionnaire 18
5 + collection of
feedbacks with
first access to the
OT
8 in Torino and 7
in Reggio Emilia
5
mean of collection
One to one
interviews
One to one
interviews
e-mail + One to
one interviews
e-mail
4.2.3.1 Austria
In Austria there were 18 users who responded to the questionnaire about the mock-up of the project. The
questions submitted concerned the understanding of the content, the quality of information and the
quality of the graphics.
To the first question - What information do you believe the user can draw by looking at the mock-up? -
Almost the whole part of users have indicated that they can identify their own energy consumption. Many

46. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
46
users have added about the comparison that is possible between some other families and their own.
Another response concerned about the possible energy consumption saving of their energy bill.
To the second question - Do you think the information service provided be complete or more information
Should be added in order to be fully understandable and effective? - The majority of users felt the
information in the mock-up sufficient and clear. Some users have suggested to add some indication about
the square meters of each user, to create a possible newsletter and an online website section with F.A.Q.
To the third question - Express your opinion on the graphic solutions ADOPTED by this mock-up - almost all
users have considered the mock-up graphics very clear and understandable. Only one user felt the graphics
unclear and another user has suggested to highlight the consumption peaks with some colour.
4.2.3.2 France
First visit on the Empowering platform
The Access to the website and the creation of a customer account has been found as complicated because
of some technical problems (e.g. provisional reference to be entered during subscription). Furthermore the
website sometimes is detected as untrusted content (this is related to the authentication certificate).
Although these difficulties, customers consider that the empowering platform offers a simple and user-
friendly navigation.
They appreciate:
- the navigation banner on the left which presents all the tools.
- The presence of several tools
- The graphics that allow a quick and comfortable reading.
They criticize:
- the information tabs « + » which are not visible
- the links to GEG advices which are not visible.
Interpretation and understanding of five services
OT208: Comparison of your consumption last 3 months compared to the consumption of the
same 3 months from the previous year

47. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
47
End-user feedback:
Difficulties on understanding the climate-adjusted consumption
The customers try to understand this graphic by themselves, without help text. The third bar of the graph
was interpreted as a projection for future consumption.
Unlike help text, the title is read by customers. So it would be appropriate to change it or add a subtitle, in
order to reflect the notion of climate-adjusted consumption. For example: To analyse trends in energy use,
taking into account the severity of the winter.
Some customers suggested improvement:
- add a pictogram on third bar evoking climate: snowflakes, thermometer…
- add « 2013 », « 2014 » et « 2014* » on graphic bars
For people who are curious, it would be necessary to propose more detailed explanations about
climate-adjustments.
OT106: Yearly presentation of monthly comparison between your consumption and the
averages of the 10% most efficient (min) and less efficient (max) consumers with equivalent
contracts (tariff & contracted power)

48. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
48
This graphic is a descriptive tool: it presents the evolution of consumption over one year.
This is also a normative tool (or standard-setting tool): it allows customers to compare themselves to other
customers in the same category (tariff, contract power).
Comparison groups in a normative way often trigger reactions: is the group coherent? Is the comparative
reference relevant?
“I would need to know about the area and the number of people. It lacks for positioning.
"(Paolo); "I wish I had the opportunity not only to compare my consumption to those
who have the same contract but also according to other criteria, for example those that
are in the same number as us in their homes, dwellings of the same area , etc.
"(Jonathan)
A couple (Aurélie et Sébastien) contracted low power (3kA) and try to avoid simultaneous operation of
electrical equipment in order to limit his consumption; this tools will be relevant for them to evaluate their
strategy.
Some customers suggested improvement about border colour, in order to help interpretation: high border
red, low border green.
OT111: Last 3 months consumption comparison of your monthly consumption with the average
consumption of your neighbours and the X% less consuming

49. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
49
This graphic is both a descriptive and a normative tool (like OT106).
Two respondents considered this intrusive:
“It is like to enter in other people’s houses. It is very voyeuristic (…) to know what others
do in their home” (Laura) ; “it is more than to point the finger. You want to show the
most environmentally responsible. I find it a bit intrusive and blaming. It puts me on
competition” (Océane).
The graphic title « My consumption, and similar households » finds greater acceptance than « my
consumption, my neighbourhood ». But comparison with neighbourhood is of particular concern to
customers, much more than general term like « similar household ».
Even if comparative reference are not considered relevant (different surface, number of occupants…), this
graphic encourages broader and deeper reflection about energy use.
"I can see that it is possible to consume less in any case, I have a margin. After I know I
have a smaller studio, I live alone so obviously I use less than my neighbors who are
often several. But it does not bother me to be compared to households not be
comparable, as I know (...) I find it very interesting even if the surface and equipment can
be different. Which could provide more, it would be a comparison with the neighborhood
average consumption etc. "(Anaïs)
Customer reaction depends on the results:
- if they are below the group average, a « rebound » effect may occur (and lead to increased
consumption)

50. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
50
- if they are above the group average, it generally leads to lower consumption.
To avoid « rebound » effect, sociologists recommend inserting emoticons like smiley.
Theory of behaviour change also calls for clear and achievable objectives, which enhance efficacy of
feedback tools.
Last, some customer would appreciate to compare here their heating consumption with the
neighbourhood, including gas consumption.
OT603: Prediction of billing for the next 3 periods.
Some misunderstanding about this graphics triggered reactions related to estimated consumption. Some
expert customers do not accept that auto-transfer data would not be integrated by empowering tools.
These customers call for smart meters deployment.
One customer (Thibaud) suggested a new presentation for this graphic:

51. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
51
OT105 : Comparison with energy ranges - 5 categories without letters
This tool is available only for dwelling heated by electricity. This concerns a few people; that is why we do
not have any user perception about.
Questioned about this graphic, customers expressed their expectations about gas services.

52. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
52
Benefits and limits of empowering services
These services provide customer satisfaction: comfortable reading, efficiency analysis…
“The value of these tools for me is to compare my consumption with that of others who
have similar units. This information does not appear on the bill, and so far receives when
there is no reference to whether our consumption is high or low” (Jonathan)
However they would appreciate:
 financial forecast : € and not only kWh
 proposition of eco-friendly behaviour, more operational than usual.
 opportunity to attach their comments on the platform : personal objectives for example
 easier access to platform (single account on GEG website)
 auto-transfer integration
4.2.3.3 Italy
The testing in Italy have been mainly focussed in assessing the billing tool as it will be the first starting
service and as it will be the less customisable. The assessment has been carried out sending the mock-ups
and the questionnaire via email to the users of Torino and via face-to-face interviews in Reggio Emilia.
BT 101
The involved users find that the graph provides good information on the comparison between the user’s
consumption and the average of similar users, with the division of the time bands which helps the
comprehension of the service.

53. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
53
What are the compared families should be clarified, specifying if they are families with similar areas,
compatible households and composition. If it is not possible to compare with similar household, in the BT
the characteristics of the compared family groups should be explained.
The used graph is not immediately comprehensible. It could be substituted by a simpler model like the
“blood test” on which users are more used. In the “blood test” model a table with min and max indicators
help to understand if you are doing well or bad.
Example:
Night consumption
F3 band
100 165 400
OK (or green smiling
face)
Night consumption
F1 band
100 500 400 NO(or red smiling face)
Advise: reduce not
necessary loads
For basic users a table like the one above is more easy and rapid to read. For advanced users a graph like
the one on the BT can be used or put as a summary of data reported in the table.
BT 102
The third column confuses the reading. It is not clear the meaning of “efficiency”. Efficiency in what?
Respect what? And how it is calculated?

54. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
54
BT 110 and 108
This graph does not seem full of meaning, despite the title “further details” suggests. The day of the week
or at least the weekends should be highlighted in the graph.
It could be hard for those not used in reading graphs to be understood.
It is not clear the purpose of this graph: do we want to force users to reduce their consumption in order to
fit the average trend?
BT 401
In this box “ad hoc” advises should be added. For example, in the case of a user which consume more than
in the previous period, in order to make him understand why he is consuming more, related information
should be added (e.g. this summer is hot, so the air conditioning is turned on).
At the end a summary on the average behaviour of the user according to monthly consumption and
cumulated consumption can be added.
An overall assessment (largest consumer, conscious consumer, efficient family etc.) of the user
performance can be added. Insert the kWh to be saved in order to become an “efficient family” in order to
spur the user in adopting a virtuous behaviour.

55. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
55
The user should rapidly understand if its consumption is too high. In this case the actions to be adopted to
save energy should be listed in the BT.
In this part also the average power per time band should be added. For example, the average Power (W) =
Consumed energy in F1 period/duration of F1 period. The average Power gives to residential users useful
information. Also, in the night period if there is abnormal consumption, the service should give information
on what is turned on advising on how to reduce excessive loads.
The median user understands better W or kW than kWh.
4.2.3.4 Spain
In Spain 5 users assess the online tool accessing it for the first time. In general users like the graphic
solution used for the OT and find it usable, with clear and simple information. They provide more specific
feedback for the single services they tried.
OT 101
Users ask to explain the criteria to find “all similar homes” and “efficient homes”. Some information on the
price would be appreciated.
The comparison help to understand if the consumptions are high or not because provide a reference to
know that.
OT 103

56. EMPOWERING: D4.1 First report on initial software implementation and test
The sole responsibility for the content of this deliverable lies with the authors. It does not necessarily reflect the opinion of the European Union.
Neither the EASME nor the European Commission are responsible for any use that may be made of the information contained therein.
56
The users cannot understand the meaning of the green line and how it is calculated. It should be useful to
add a legend.
5 Conclusions and next steps
The initial software implementation and test had the objective to develop a working prototype of the
EMPOWERING system and to verify its performance capabilities in conditions similar to those expected in
real operation. The test should also include usability issues of the visualisations offered by the utilities to
their customers.
In order to test the API and the calculation functionality of the Insight Engine (v1), tests with data provided
by the four partner utilities have been done. In anticipation of the future data volume expected to be
managed by the system during the pilot operation, an additional test has been also performed. The test
consisted in generating and pushing to the system of 100.000 fake contracts of a “hypothetic utility” with 1
year of 15 min measures generated randomly, thus simulating smart meter measurements.
The tests proved the capabilities of the EMPOWERING system (v1) to handle the expected data quantities
during the pilot operation and to produce correct results in the specified format. The next steps towards v2
include the implementation of the rest of the services according specifications and the optimisation of the
data handling processes to increase the system performance.
The analysis carried out referring the usability testing allowed the Consortium to delineate the subsequent
steps and the specific activities and results to achieve in the next tasks. More specifically:
- The results of the Focus Groups on usability allow utilities in understanding the changes that have to be
done in their mock-ups/services in order to make them more friendly and usable. In addition the Focus
Group permitted to validate the methodology and the tools developed for assessing the usability. These
would be replicated in order to assess the usability of the version 2 of services among the end users.
- There is a general understanding of the services among the internal staff of the utilities. There are changes
necessary in the developed services in order to make them more understandable to the staffs that will be
involved in the provision phase of the services to the customers. The tests with the end-users followed the
methodology on envisaging usability used in the Focus Groups. The surveys in general demonstrated that
the visualisations are useful and easy to understand, but in some cases improvements are necessary to
avoid confusion.
Users found that some additional help texts and legends for the graphics should be added to improve
understanding. This is especially necessary for the definition of which are the “similar households” and the
“efficient households”. In general, the formulation “similar household as yours” is better accepted by the
end-users as “your neighbours”. The criteria for similitude should be clearly stated when providing the
services. Some of the end-users claimed more targeted comparisons taking into account the size of
dwellings and number of occupants.
In some cases deficiencies of the visualisation were found, such as not visible tabs or links. Some colour
improvements were also suggested.
The understanding of the climatic adjustment of the consumption presented some difficulties for the end-
users. More intuitive and clear explanations should be searched to address this issue.
The more detailed graphics, like daily consumption, have been appreciated only by more knowledgeable
users. An improvement for distinguishing clearly the weekends from the weekdays by highlighting them in
the graphics has been suggested.
For the final version, the above user comments should be taken into account in order to improve the
effectiveness and the satisfaction from the services.