SlideShare a Scribd company logo
1 of 6
Download to read offline
Smart Data along the Value Chain of Electrification
User Needs and Requirements Analysis for a European Big Data Roadmap in the Energy Sector
Sebnem Rusitschka
Siemens AG Corporate Technology
Business Analytics and Monitoring
Sector Forum Lead for Energy and Transportation in EU BIG
Munich, Germany
Abstract—Massive amounts of data await the energy sector
stakeholders once the digital transformation has reached a
tipping point. In this paper, we are providing an overview of the
big data market and competition arenas, which emerge with the
increasing feasibility of data-driven scenarios in the energy
sector. From the use cases within these scenario categories of
Operational Efficiency, Customer Loyalty, and New Business
Models we derived user needs and requirements particular to
the European energy sector. We conclude with a discussion of
the findings and an outlook on the further work of defining a big
data value roadmap for infrastructure- and resource-centric
sectors in Europe.
Index Terms—Data systems, Big Data, Smart grids, Energy
management.
I. OVERVIEW
Whilst some are still hang up on whether big data is hype
or not – some have acknowledged that the masses and variety
of data in the industrial businesses is a consequence of
digitization and automation along their value chains. In the
past year we have interviewed stakeholders and researched
trends regarding big data in the energy and transportation
sector within the European project BIG – Big Data Public
Private Forum [1]. The other partners in the consortium
analyzed yet other sectors, as well as technological advances
from the original big data domains. Together we are looking
into cross-sectorial similarities, the different maturity levels,
even cross-sectorial scenarios.
Most importantly we have analyzed user needs and
requirements, and researched how current big data
technologies – and regulatory frameworks – need to be
adapted and further developed to give industrial businesses the
cutting edge advantage in the digital transformation of their
businesses. With the analysis of market and technological
trends across infrastructure- and resource-centric industries
such as energy and transportation with their similar enablers,
drivers and constraints to big data, we set the stage for
defining what big data means in these traditional sectors. It is
evident that big data value potential will not be confined in
clear cut industrial boundaries, but rather big data opportunity
and competition can also come from other industries: Business
models compete with business models in so-called arenas [2]
irrespective of industry. Fig. 1 depicts the emergence of these
new arenas around energy infrastructure, energy supply, and
energy as well as mobility services at the end user side. These
new arenas are filling with players reaching from car
manufacturers to data-driven startups; electricity retailers and
power generation companies are redefining their positioning.
We look at the big picture, with energy sourcing from Oil
& Gas to Renewables to Decentralized Generation. We
observed trends within sectors that are the biggest energy
consumers in Europe, i.e., transportation, manufacturing, and
households. The short cut that electrification represents for
energy efficiency is significant. The enabling technology
backbone is the digitization and automation along the value
chain of electrification: the smart infrastructure, i.e., smart
grid.
The “well-to-wheel” energy inefficiency in transportation
is indicative: losses amount to 70 percent either during fossil
power generation, transmission, and distribution for the
electrification of vehicles or when driving and idling at traffic
lights with petrol vehicles [3]. In light of these inefficiencies,
the integration of renewable energy sources and the full-
electric vehicle is a perfect technology match. Given that the
technological challenges are overcome, this match not only
enables zero-emission transportation, but electric cars that are
This work has received funding from the European Union’s Seventh
Framework Programme for research, technological development, and
demonstration under grant agreement no 318062.
Fig. 1 Competition in data-driven scenarios in the energy sector is no longer
confined within industries but takes place in new "arenas"
parked most of the time also represent a viable storage option
for the peak supply from Renewables, which occur at noon via
solar and at night via wind. This new arena, “mobility
services,” in which electricity and transportation meet, for
example, becomes populated by car manufacturers who
provide infrastructure or even energy supply [3] alongside the
incumbent energy sector players. The entrance of new players
also transforms these traditional segments into new arenas.
New players enable the efficient balance of energy supply
and demand through data-driven services providing demand
response [4], energy savings [5], and power [6]. Positive
regulatory drivers are significant: Data-driven virtual power
plant operation becomes lucrative through the direct
marketing potential of the small but numerous distributed
generation sources. Technological trends promise to disburden
some of the negative regulatory side effects, such as the higher
prices due to the current subventions for Renewables.
Especially, the manufacturing sector is pressured by high
electricity prices [7]. At the same time, industrial automation
leads to digital factories, which are enabled to automate
according to energy efficiency criteria [8]. The new arena,
“energy services,” becomes increasingly more active with
energy data-driven startups that offer savings and demand
response programs primarily offering services to industrial
consumers. Incumbent energy suppliers acknowledge the shift
in consumer demand as well as entirely data-driven business
actors do [9]. These energy services will eventually reach the
mass market of households, utilizing resources such as smart
thermostats or entire home automation systems [9].
All of these arenas as depicted in Fig. 1 have in common
the typical characteristics that generally define big data:
Volume of available data from digitization and automation
along the infrastructure as well as at the supply and demand
side: Phasor measurement units (PMUs) deliver up to 100 Hz
resolution GPS-synchronized data on grid parameters – a
current SCADA system with traditional RTUs, which poll
data every 2-4 seconds, has to process less than one per cent of
the data volume of the entire PMU data accumulating in the
same area [10]. Smart metering results in 3,000-fold increase
in data when 15 minutes intervals are measured [11]; the
software deployment for the smart infrastructure additionally
generates data in forms of error logs, etc.
Velocity of data, i.e. the rate at which data is produced and
consumed is also increasing [12], since the balancing of
energy supply and demand as well as the provision of capacity
becomes increasingly more real-time due to the added
uncertainties through Renewables, decentralization, and active
or mobile consumption.
Variety of data arises due to the diversification of energy
consumption and generation as well as consumerization and
liberalization. Market communication alone results in the
exchange of a variety of data also coming from new source
and roles. With the always connected end user, online sources
such as social media as well as smart phones add to this
inherent variety in directly and indirectly-related data sources
[13].
In our approach to analyzing the energy sector towards its
need for handling big data and the requirements towards the
usage of big data technology, we collected and discussed
various use cases with the above characteristics as briefly
described in Section II. The analysis of user needs and
requirements of the energy sector on the basis of these use
cases and scenarios are detailed in Section III. Section IV
concludes the paper with a discussion of the findings and an
outlook on the planned roadmapping activities.
II. DATA-DRIVEN SCENARIOS IN THE ENERGY SECTOR
The big data use cases we collected in interviews,
workshops, and online research are holding immense potential
for improvement and growth for business. In order to be able
to analyze industrial user needs and requirements, it is also
important to understand what big data really means in the
energy sector and how the value generation is affected by the
very nature of cyber-physical systems that are at the core of
the digitized infrastructure businesses. Thus, in a second top
down approach we clustered the use cases with similar
prerequisites into application scenarios that are congruent with
inherent business needs:
a) Operational efficiency subsumes all use cases that
involve improvements in maintenance and operations in real-
time or a predictive manner based on the data which comes
from infrastructure, stations, assets, and end users of energy.
Technology vendors, who develop the sensorization, i.e.
digitization and automation, of the infrastructure are the main
enablers. The market demand for such enhanced technologies
is increasing, because it also helps the businesses in energy to
better manage risk: The complexity pan-European
interconnected electricity markets with the integration of
Renewables and liberalization of electricity trading requires
more visibility of the underlying system and of the energy
flows in (near) real-time.
As a rule of thumb – anything with the adjective “smart”
falls into this category: smart grid, smart metering; smart
cities, smart (oil, gas) fields. Some examples of big data use
cases in operational efficiency are: (i) Predictive and real-time
analysis of disturbances in power systems and cost-effective
countermeasures; (ii) Operational capacity planning,
monitoring and control systems for energy supply and
networks, potentially with dynamic pricing or (iii) Optimizing
multimodal networks in energy as well as transportation
especially in urban settings, such as city logistics or eCar-
sharing for which the energy consumption and feed-in in the
transportation hubs could be cross-optimized with the
logistics.
All of the scenarios in this category have the main big
challenge of breaking up silos: be it across departments within
vertically integrated companies or across organizations along
the value chain of electrification, which now are liberalized
and oftentimes have conflicting objectives. The big data use
cases in the operational efficiency scenario require seamless
integration of data, communication and analytics across a
variety of data sources, which are owned by different
stakeholders.
b) Customer Loyalty: Understanding big data
opportunities regarding customer needs and wants is
especially interesting for companies in liberalized,
consumerized markets such as electricity, where entry barriers
for new players as well as the margins are decreasing.
Customer loyalty and continuous service improvement is what
enables energy players to grow in these markets. There are,
however, also B2B use cases mainly from technology vendors,
but also from energy data or mobility data start-ups.
Some examples of using big data to improve customer
experience are: (i) Continuous service improvement and
product innovation, e.g. individualized tariff offerings based
on detailed customer segmentation using smart meter or
device level consumption or feed-in data; (ii) Predictive
lifecycle management of assets, i.e. data from machines &
devices in energy combined with enterprise resource planning
and engineering data to offer services such as intelligent on-
demand spare-parts logistics, for example; or (iii) Industrial
demand-side management, which allows for energy efficient
production and hence increases competitiveness of
manufacturing businesses.
Big challenge is handling confidentiality and privacy of
private and business customers whilst getting to know and
anticipate their needs. Data originator, data owner, and data
users are different stakeholders, who need to collaborate and
share data to realize these application scenarios.
c) New Business Models revolve around monetizing
available data sources and existing data services in entirely
different ways. There are quite a few cases in which data
sources or analysis results from one sector represent insights
for stakeholders of another sector. The analysis of energy and
mobility data start-ups shows that there is a whole new way
of generating business value if the resources are owned by the
end user. Then the business is entirely customer- and service-
oriented; whereas the infrastructures of energy and
transportation with their existing stakeholders are utilized as
part of the service. We call these the intermediary business
models.
Energy consumer segment profiles, such as prosumer
profiles for PVs, CHPs; or actively managed demand side
profile, etc., from metering service providers could also be
offered for smaller energy retailers, network operators or
utilities who can benefit from improvements on the standard
profiles of energy usage but do not otherwise have access to
high resolution energy data of their own customers yet.
The big challenge is the unclear regulation around the
secondary use of energy data. Additionally, one of the other
scenarios needs to be realized ensuring the digitization of most
parts of the infrastructure and the sourcing of end usage data.
The connected end user is the minimal prerequisite for the
consumer focused new business models.
III. USER NEEDS & REQUIREMENTS IN THE EUROPEAN
ENERGY SECTOR
A. User Needs
Business user needs can be derived from the above
categories of business needs:
Ease of use: Big data technologies employ new paradigms
and mostly offer programmatic access only, e.g. R the
statistical programming language or MapReduce, a framework
for programming massively parallel task execution. Without
software development skills and a deep understanding of
distributed computing paradigm as well as application of data
analytics algorithms within such distributed environments,
scalable analytics of mass, streaming data is currently not
possible.
Semantics of correlations and anomalies that can be
discovered and visualized via big data analytics need to be
made accessible. Currently only domain and data experts
together can interpret the data outliers, business users are often
left with guesswork when looking at data analytics results.
Veracity of data needs to be guaranteed before it is used in
smart grid applications, such as distribution automation or
billing of variable tariffs. Because the increase in data that will
be used for these applications will be magnitudes bigger,
simple rules or manual plausibility checks no longer are
applicable.
Smart Data often is used by industrial stakeholders
emphasizing that a business user needs refined data – not raw
data (big data). In cyber-physical system as opposed to online
businesses, there is information and communication
technology (ICT) embedded in the entire system instead of
only in the enterprise IT backend. Especially infrastructure
operators have the opportunity to pre-process data in the field
and aggregate levels or distribute the intelligence for data
analytics along the entire installed ICT infrastructure to make
best use of computing and communication resources to deal
with volume and velocity of mass sensor data. The need for
smart data is driven by the need for cost-effective usage of all
installed ICT resources along the infrastructure.
Decision support or automation becomes a core need, as
the pace and structure of business is changing. European grid
operators today need to intervene almost daily to prevent
potentially large-scale blackouts, e.g. due to integration of
Renewables or liberalized markets. Business users need more
than the information that something is wrong. Visualizations
can be extremely useful, but the question of what needs to be
done remains to be answered either in real-time or in advance
of an event, i.e. in a predictive manner.
Strong need for scalable, advanced analytics, hence, will
push the envelope of state-of-the-art, such as smart metering
data analytics [11]: e.g. Segmentation based on load curves,
forecasting on local areas, scoring for non-technical losses,
pattern recognition within load curves, predictive modeling,
etc.
End users needs are not directed towards big data or
energy data as is. However, big data scenarios in energy offer
many improvements for the end users: Operational Efficiency
ultimately means energy and resource efficiency, which will
improve quality of life – especially in urban settings – for all
end users. Customer Experience and New Business Models
related big data scenarios are entirely based on better serving
the end user of energy. However, both scenarios need
personalized data in higher resolution, geo- or time-tagged.
We have been discussing the real big data value in cross-
combining such variety in data, which on the downside can
make pseudonymization or even anonymization ineffective in
protecting the identity and behavioral patterns of individuals
or business patterns and strategies of companies. New
business models based monetizing once collected data, with
currently unclear regulations, leaves end users entirely
transparent, uninformed, and unprotected against secondary
use of their data for purposes they might not agree with, e.g.
insurance classification, credit rating, etc.
Reverse transparency is at the top of the wish list of data-
literate end users: End users need practical access to
information on what data is used by whom for what purpose in
an easy-to-use, manageable way. Rules and regulations are
needed for granting such transparency for end users; data
protection laws that are minimally consistent, but also allow
individualization for maximal privacy are required.
Data access, exchange, and sharing needs, hence, apply
on both business and end users. In today’s complex electricity
markets, there is almost no scenario where all required data
for answering a business or engineering question comes from
one department’s databases. Nonetheless, most of the
currently installed advanced metering infrastructures have a
lock-in of the acquired energy usage data in the utilities’
billing systems. The lock-in makes it cumbersome to use the
energy data for other valuable analytics. Such silos have
traditional roots from the recent era where most of European
utilities were vertically integrated companies. Also, the
amount of data to be exchanged was much less, so that
interfaces, protocols, and processes for data exchange have
been rather rudimentary.
The analyzed business user and end user needs, as well as
the different types of data sharing needs directly translate into
technical and non-technical requirements. Some of which are
discussed in the following. The technical requirements are
summarized under requirements for a so-called big data
refinery pipeline that shall satisfy the need to utilize entire
data value chain and generate smart data:
B. Technical Requirements for a Big Data Refinery Pipeline
– “Analytics Inside”
The big data refinery pipeline for energy businesses
consists of three main phases: data acquisition, data
management, and data usage. Data analytics, as indicated by
business user needs, is implicitly required within all steps, and
is not a separate phase as depicted in Fig. 2: During data
acquisition, analytics supports the data cleansing and ensures
data quality. For sensor data, analytics can enable efficient
event-specific data compression and increase data quality
through anomaly detection at acquisition time. For data usage,
business and engineering questions need to be formulated by
business users and translated into suitable models. These
models among other aspects imply which types of mass
storage and data handling are cost-efficient for the specific
purpose.
So, each step of the pipeline is setup to refine the data, but
the methods of refinement vary depending on the data type.
Such connectedness with the sources of data may be the main
differentiator of big data in the energy business from big data
usage scenarios in the online businesses: If data acquisition
and analytics do not deliver actionable information with
available options fast enough then the value of data may even
be negative.
1) Data Acquisition Requirements
In the data acquisition phase, security and privacy or
confidentiality policies need to be applied; the data should be
checked for quality features, such as missing data or
implausible data, e.g. in time series. Depending on whether
data is generated during an automated process, or by a human,
e.g. repair crew in the field or customer call agent, data is
highly structured or unstructured; it is acquired as continuous
streams of high frequency samples or sent irregularly. The
methods for processing and analyzing the data, acquired in
very different ways and formats, would reach from content
analytics and natural language processing for unstructured
data to signal and cross-correlation analysis on structured
topological time series data.
The great differentiator of big data scenarios in traditional
sectors such as energy, especially in Europe, is the concern
that data should be collected for a known purpose. This may
sound conservative at first. And with current technological
capabilities, it is very restrictive in practice. However, it
actually gives way to formulate a central requirement that
will drive Europe to thrive both technologically as well as in
terms of digital rights on individual and business level:
Dynamic configurability of data access about which data
can be collected for what purpose in what granularity and time
span and location must be given along the entire intelligent
infrastructure of electrification. The configuration or the
effects thereof must be easily comprehensible for the data
owners. The configuration must be dynamic in the sense that
service providers are able to receive the data in the required
Fig. 2 Cyber-physical systems as in the energy sector require a big data refinery pipeline that eemploys
analytics at every step of data processing to provide “smart data”
granularity and with the allowed privacy and confidentiality
protection settings defined – on demand.
Privacy and confidentiality preserving data analytics are
required to enable the service provider to retrieve the
knowledge without violating the agreed upon granularity of
data or the allowed privacy or confidentiality settings.
2) Data Management & Required Architectural Patterns
Different types of data management may be considered for
the same data source type depending on for what purpose it
will be used: Graph databases can yield faster results for geo-
spatial analysis, key-value stores may prove most efficient for
feature discovery purposes in time series. Regarding the
generation of value from a variety of data sources the data
storage step needs to enable the cost-efficient integration of
the different data sources in an extensible way. Flexible
abstractions, multiple layers of abstractions for data and
analytical models will be required for the data storage step.
Lightweight semantic data models that represent the
multiple links within various data sources, such as Linked
Data, may provide such cost-efficient abstractions, especially
when the data domain is complex and highly interlinked. For
high-dimensional data, multi-dimensional data structures such
as tensors and space-filling curves may be required to support
the design of more efficient analytics algorithms, which are
capable of exploring multiple relations at once. These are only
some examples of a few methods from machine learning,
semantic data technologies, information extraction, and
distributed computing. What will be required is a flexible
toolbox, an analytics engine, which scales with the increasing
business need to gain more value from ever increasing
amounts and variety of data without having to change the
entire analytics application:
Abstraction from the actual big data infrastructure is
required to enable (a) ease of use and (b) extensibility and
flexibility. The analyzed use cases have such diverse
requirements that there is no single big data analytics platform
that will empower the future utilities business with insights
from massive amounts of available data. Adaptive data and
system models are needed so that new knowledge extracted
from domain analytics or the ever changing circumstances of
the system can be redeployed into the data analytics
framework without disrupting the analytics applications. The
analytics engine is required to accomplish the plug-in such
adaptive models.
a) Schema on Read and the Active Archive Pattern
Active archive, sometimes also called Data Lake, allows
the cost-efficient storage of all data in its raw form. Raw data
is transformed as needed by the business users on demand,
which is called “schema on read.” Distributed data
management and massively parallel processing principle
combined with performant commodity hardware makes the
generation of a data schema on read feasible. An active
archive allows one to get to know the data and cost-efficiently
explore business opportunities.
b) Lambda Architecture
The lambda architecture is based on a set of principles for
architecting scalable big data systems that allow running ad-
hoc queries on large datasets. It consists of multiple layers
dealing with volume and variety, via an active archive – as
well as velocity, via stream data computing.
Fast and even real-time analytics is required to support
decisions, which need to be made in ever shorter time spans.
Speed in big data technologies is especially determined by the
solution architecture. However, in smart grid settings near
real-time dynamic control over events also requires insights to
be gained at the source of the data near the events.
c) In-field Analytics
The lambda architecture is described from an enterprise
perspective. In cyber-physical systems, such as industrial
automation or energy automation, field devices represent
important data acquisition as well as computing resources. In-
field analytics extends the lambda architecture by
encompassing the data acquisition layer, in order to deliver
faster insights for industrial businesses.
3) Data Usage Requirements
Although data usage is the last phase in the refinement
process, the business and engineering questions formulated in
this phase are the starting point for choosing the technology
stack. This phase represents the reason for setting up a
particular connection through the pipeline to the required data
sources.
In the industrial domain of energy, analytics for data
usage, reaches from descriptive analytics such as dashboards,
to predictive and prescriptive analytics such as forecasting for
portfolio management or simulation of interconnected power
systems and extraction of operational intelligence on what
actions to take in real-time or predictive manner. Industrial
data usage requires considerably more precision than data
usage in online data businesses. The business value generation
is versatile, too, since it not only covers the provisioning of an
infrastructure asset but also the usage of that asset in B2B as
well as B2C settings.
Data interpretability must be assured without the constant
involvement of domain experts. Also at anytime the results
must be traceable. Expert and domain know-how must be
blended into the data management and analytics. Self-
describing smart devices are one part of the requirement.
Semantic and adaptive data models are the other part.
C. Non-technical Requirements
In the interviews, workshops and online research several
non-technical requirements were encountered repeatedly,
which are listed in the following:
Investment in communication and connectedness:
Broadband communication or ICT in general, needs to be
widely available across all Europe and alongside the energy
infrastructure such that real-time data access is available.
A digitally united European Union: data silos are not just a
technicality: (a) the current implementation of the unbundling
prevents departments within a utility from exchanging and
cross-analyzing data freely; (b) EU member states all have
different data related regulations. European stakeholders
require reliable minimally consistent rules and regulation
regarding digital rights and regulations.
A better breeding ground for start-ups and start-up culture
is required, especially for techno-economic paradigm shifts
like big data and the spreading digitization, when also the
ways of business making widely deviate from business-as-
usual. Regarding energy and mobility related start-ups,
certainly their incubation includes more than just financial
investments but also a controlled but allowed new approach to
data access. Without this freedom for exploration and
experimentation innovation has little chance, unless of course
the aforementioned techniques privacy preserving analytics
are feasible. Incubators for energy and mobility data start-ups
are emerging in Europe [14].
Open data in this regard is a great opportunity, however,
standardization is required. The current data models, data
representation, as well as protocols require a practical
migration path to simpler state-of-the-art standards capable of
handling the volume, velocity, and variety of data. These
standards will also enable the growth of data ecosystems with
collaborative data mining, shareable granularity of data and
accompanying techniques that prevent de-anonymization.
Skilled people: Programming and statistical tools need to
be part of engineering education until big data technology
becomes more business user-friendly or in case this becomes
the “new normal:” Users need to deal with programmatic
handling, i.e., R or MapReduce are both programmatic
approaches instead of a graphical user interface or SQL
interface due to the adaptability that volume, velocity, and
variety of data demand. Traditional data analysts need to grasp
the distributed computing paradigm, e.g. how to design
algorithms that run on massively parallel systems, i.e., how to
move algorithms to data, or engineer entirely new breed of
algorithms.
IV. CONCLUSION
The energy sector, from an infrastructure perspective as
well as from resource efficiency and quality of life
perspectives, is very important for Europe. The high quality of
the energy infrastructure and global competitiveness also
needs to be maintained with respect to the digital
transformation and big data potentials.
The analysis of the available data sources in energy as well
as their use cases in the different categories for big data value:
operational efficiency, customer experience, and new business
models, helped in identifying the industrial needs and
requirements for big data technologies. Already in the
discussion of these requirements, it becomes clear that a mere
utilization of existing big data technologies as employed by
the online data businesses will not be sufficient. Domain- and
device-specific adaptations for use in electrical cyber-physical
systems are necessary. Innovation regarding privacy and
confidentiality preserving data management and analysis is a
primary concern of all energy stakeholders. Without satisfying
the need for privacy and confidentiality there will always be
regulatory uncertainty and uncertainty regarding user
acceptance of a new data-driven offering.
Among the energy stakeholders, there is a common sense
that “big data” is not enough: The increasing intelligence
embedded in the infrastructures is also able to analyze data to
some extent as to deliver “smart data.” This seems to be
necessary, since the analytics involved will require much more
elaborate algorithms than for analyzing click streams.
Additionally, the stakes in big data scenarios are very high,
since the optimization opportunities at the end are within
critical infrastructures.
These requirements represent the starting point for the
development of the big data roadmap for the European energy
sector. For this purpose the ongoing consultations with the
technical working groups on the big data refinery pipeline for
the energy sector will be continuously cross-checked with
sector representatives and against current and future
developments of the big data trend in the energy sector.
V. REFERENCES
[1] EU Project BIG – Big Data Public Private Forum. [Online]. Available:
http://www.big-project.eu/
[2] R.G. McGrath, “The End of Competitive Advantage: How to Keep
Your Strategy Moving as Fast as Your Business,” in Harvard Business
Press Books, 2013.
[3] G.Fournier, H. Hinderer, D. Schmid, R. Seign, and M. Baumann.
(2012). The new mobility paradigm: Transformation of value chain and
business models. [Online]. Available: http://run.unl.pt/bitstream/
10362/ 10154/1/FournierBaumann9-40.pdf
[4] Entelios. [Website]. www.entelios.com
[5] EnergyDeck. [Website]. www.energydeck.com
[6] Next Kraftwerke GmbH. [Website]. www.next-kraftwerke.de
[7] C. Schroeder. (2012, August 16). Energy Revolution Hiccups: Grid
Instability Has Industry Scrambling for Solutions. [Online]. Available:
http://www.spiegel.de/international/germany/instability-in-power-grid-
comes-at-high-cost-for-german-industry-a-850419.html
[8] S. Mechs, S. Lamparter, J. Peschke, J.P. Mueller. “Efficient
Identification of Energy-Optimal Switching and Operating Sequences
for Modular Factory Automation Systems,” Recent Trends in Applied
Artificial Intelligence, Lecture Notes in Computer Science Volume
7906, 2013, pp 202-211.
[9] T. Racciatti. (2014, March 21). Manufacturing The Internet Of Things.
[Online]. Available: http://www.mbtmag.com/articles/2014/03/
manufacturing-internet-things
[10] NERC.2010. Real-Time Application of Synchrophasors for Improving
Reliability. Technical Report. Available: http://www.nerc.com/docs/
oc/rapirtf/RAPIR final 101710.pdf, retrieved on 2013-05-17.
[11] Picard, M.-L. (2013, June 26). A Smart Elephant for A Smart-Grid:
(Electrical) Time-Series Storage And Analytics Within Hadoop.
[Online]. Available: http://www.teratec.eu/library/pdf/forum/2013/
Pr%C3%A9sentations/A3_03_Marie_Luce_Picard_EDF_FT2013.pdf
[12] C.O. Heyde, R. Krebs, O. Ruhle, Z.A. Styczynski. “Dynamic voltage
stability assessment using parallel computing,” In proceeding of: Power
and Energy Society General Meeting, 2010 IEEE.
[13] GTM Research. (December 2012). The Soft Grid 2013-2020: Big Data
& Utility Analytics for Smart Grid. [Online]. Available: www.sas.com/
news/analysts/Soft_Grid_2013_2020_Big_Data_Utility_Analytics_Sm
art_Grid.pdf
[14] SiliconRepublic. (2014, April 7). Energy and transportation start-up to
launch in Berlin, August 2014. [Online]. Available: www.silicon
republic.com/start-ups/item/36404-energy-and-transportation-s

More Related Content

What's hot

Market Analysis and Business Plan for Residential Energy Efficiency Services
Market Analysis and Business Plan for Residential Energy Efficiency ServicesMarket Analysis and Business Plan for Residential Energy Efficiency Services
Market Analysis and Business Plan for Residential Energy Efficiency ServicesAlbert Graells Vilella
 
NEW BUSINESS MODELS & DIGITALIZATION IN THE ENERGY SECTOR
NEW BUSINESS MODELS & DIGITALIZATION IN THE ENERGY SECTORNEW BUSINESS MODELS & DIGITALIZATION IN THE ENERGY SECTOR
NEW BUSINESS MODELS & DIGITALIZATION IN THE ENERGY SECTORArjun Reghu
 
Executive summary Utilities opportunity and betterment
Executive summary Utilities opportunity and bettermentExecutive summary Utilities opportunity and betterment
Executive summary Utilities opportunity and bettermentSagar Zilpe
 
EURELECTRIC Views on Demand-Side Participation
EURELECTRIC Views on Demand-Side ParticipationEURELECTRIC Views on Demand-Side Participation
EURELECTRIC Views on Demand-Side Participationdavidtrebolle
 
The Future Direction of Smart Grid by 2050 in India
The Future Direction of Smart Grid by 2050 in IndiaThe Future Direction of Smart Grid by 2050 in India
The Future Direction of Smart Grid by 2050 in Indiaijtsrd
 
Challenges, issues and opportunities for the development of smart grid
Challenges, issues and opportunities for the development of smart grid Challenges, issues and opportunities for the development of smart grid
Challenges, issues and opportunities for the development of smart grid IJECEIAES
 
Energy harvesting and related energy storage devices
Energy harvesting and related energy storage devicesEnergy harvesting and related energy storage devices
Energy harvesting and related energy storage devicespranaliparab
 
Branding of Energy Efficiency – Task 20 of the IEA DSM Programme
Branding of Energy Efficiency – Task 20 of the IEA DSM ProgrammeBranding of Energy Efficiency – Task 20 of the IEA DSM Programme
Branding of Energy Efficiency – Task 20 of the IEA DSM ProgrammeIEA DSM Implementing Agreement (IA)
 
[Smart Grid Market Research] The Optimized Grid - Zpryme Smart Grid Insights
[Smart Grid Market Research] The Optimized Grid - Zpryme Smart Grid Insights[Smart Grid Market Research] The Optimized Grid - Zpryme Smart Grid Insights
[Smart Grid Market Research] The Optimized Grid - Zpryme Smart Grid InsightsZpryme Research & Consulting, LLC
 
report on the Study Of Technology System Of Self-Healing Control In Smart Dis...
report on the Study Of Technology System Of Self-Healing Control In Smart Dis...report on the Study Of Technology System Of Self-Healing Control In Smart Dis...
report on the Study Of Technology System Of Self-Healing Control In Smart Dis...Yuvraj Singh
 
Capgemini ses - smart meter valuation model (gr)
Capgemini   ses - smart meter valuation model (gr)Capgemini   ses - smart meter valuation model (gr)
Capgemini ses - smart meter valuation model (gr)Gord Reynolds
 
In tech demand-management_and_wireless_sensor_networks_in_the_smart_grid
In tech demand-management_and_wireless_sensor_networks_in_the_smart_gridIn tech demand-management_and_wireless_sensor_networks_in_the_smart_grid
In tech demand-management_and_wireless_sensor_networks_in_the_smart_gridAWe Shinkansen
 
Electric Power Industry In Transition
Electric Power Industry In TransitionElectric Power Industry In Transition
Electric Power Industry In TransitionHIMADRI BANERJI
 
Switching Perspective: Creating New Business Models for a Changing Energy In...
Switching Perspective:  Creating New Business Models for a Changing Energy In...Switching Perspective:  Creating New Business Models for a Changing Energy In...
Switching Perspective: Creating New Business Models for a Changing Energy In...IBM Energy & Utilties
 
Eeb presentation june 2013
Eeb presentation june 2013Eeb presentation june 2013
Eeb presentation june 2013Sylvia Wower
 

What's hot (19)

Market Analysis and Business Plan for Residential Energy Efficiency Services
Market Analysis and Business Plan for Residential Energy Efficiency ServicesMarket Analysis and Business Plan for Residential Energy Efficiency Services
Market Analysis and Business Plan for Residential Energy Efficiency Services
 
Suciu ATOM2016
Suciu ATOM2016Suciu ATOM2016
Suciu ATOM2016
 
Mo sg china_vf
Mo sg china_vfMo sg china_vf
Mo sg china_vf
 
NEW BUSINESS MODELS & DIGITALIZATION IN THE ENERGY SECTOR
NEW BUSINESS MODELS & DIGITALIZATION IN THE ENERGY SECTORNEW BUSINESS MODELS & DIGITALIZATION IN THE ENERGY SECTOR
NEW BUSINESS MODELS & DIGITALIZATION IN THE ENERGY SECTOR
 
Executive summary Utilities opportunity and betterment
Executive summary Utilities opportunity and bettermentExecutive summary Utilities opportunity and betterment
Executive summary Utilities opportunity and betterment
 
EURELECTRIC Views on Demand-Side Participation
EURELECTRIC Views on Demand-Side ParticipationEURELECTRIC Views on Demand-Side Participation
EURELECTRIC Views on Demand-Side Participation
 
Smart Grid and Consumers
Smart Grid and ConsumersSmart Grid and Consumers
Smart Grid and Consumers
 
Smart grid: technology and market evidence
Smart grid: technology and market evidenceSmart grid: technology and market evidence
Smart grid: technology and market evidence
 
The Future Direction of Smart Grid by 2050 in India
The Future Direction of Smart Grid by 2050 in IndiaThe Future Direction of Smart Grid by 2050 in India
The Future Direction of Smart Grid by 2050 in India
 
Challenges, issues and opportunities for the development of smart grid
Challenges, issues and opportunities for the development of smart grid Challenges, issues and opportunities for the development of smart grid
Challenges, issues and opportunities for the development of smart grid
 
Energy harvesting and related energy storage devices
Energy harvesting and related energy storage devicesEnergy harvesting and related energy storage devices
Energy harvesting and related energy storage devices
 
Branding of Energy Efficiency – Task 20 of the IEA DSM Programme
Branding of Energy Efficiency – Task 20 of the IEA DSM ProgrammeBranding of Energy Efficiency – Task 20 of the IEA DSM Programme
Branding of Energy Efficiency – Task 20 of the IEA DSM Programme
 
[Smart Grid Market Research] The Optimized Grid - Zpryme Smart Grid Insights
[Smart Grid Market Research] The Optimized Grid - Zpryme Smart Grid Insights[Smart Grid Market Research] The Optimized Grid - Zpryme Smart Grid Insights
[Smart Grid Market Research] The Optimized Grid - Zpryme Smart Grid Insights
 
report on the Study Of Technology System Of Self-Healing Control In Smart Dis...
report on the Study Of Technology System Of Self-Healing Control In Smart Dis...report on the Study Of Technology System Of Self-Healing Control In Smart Dis...
report on the Study Of Technology System Of Self-Healing Control In Smart Dis...
 
Capgemini ses - smart meter valuation model (gr)
Capgemini   ses - smart meter valuation model (gr)Capgemini   ses - smart meter valuation model (gr)
Capgemini ses - smart meter valuation model (gr)
 
In tech demand-management_and_wireless_sensor_networks_in_the_smart_grid
In tech demand-management_and_wireless_sensor_networks_in_the_smart_gridIn tech demand-management_and_wireless_sensor_networks_in_the_smart_grid
In tech demand-management_and_wireless_sensor_networks_in_the_smart_grid
 
Electric Power Industry In Transition
Electric Power Industry In TransitionElectric Power Industry In Transition
Electric Power Industry In Transition
 
Switching Perspective: Creating New Business Models for a Changing Energy In...
Switching Perspective:  Creating New Business Models for a Changing Energy In...Switching Perspective:  Creating New Business Models for a Changing Energy In...
Switching Perspective: Creating New Business Models for a Changing Energy In...
 
Eeb presentation june 2013
Eeb presentation june 2013Eeb presentation june 2013
Eeb presentation june 2013
 

Similar to Rusitschka_SmartDataInElectrificationValueChain_BIG2014_V1

Paper: The Internet of Things is transforming the energy and utilities indust...
Paper: The Internet of Things is transforming the energy and utilities indust...Paper: The Internet of Things is transforming the energy and utilities indust...
Paper: The Internet of Things is transforming the energy and utilities indust...Ericsson
 
Actionable Intelligence: Transforming Utilities
Actionable Intelligence: Transforming UtilitiesActionable Intelligence: Transforming Utilities
Actionable Intelligence: Transforming UtilitiesEricsson
 
Facilitating cross-sectoral exchange of energy data
Facilitating cross-sectoral exchange of energy dataFacilitating cross-sectoral exchange of energy data
Facilitating cross-sectoral exchange of energy dataSaidh KESSACI
 
Horizon Scan: ICT and the future of utilities
Horizon Scan: ICT and the future of utilitiesHorizon Scan: ICT and the future of utilities
Horizon Scan: ICT and the future of utilitiesEricsson
 
2009 doe smart grid system report
2009 doe smart grid system report2009 doe smart grid system report
2009 doe smart grid system reportGrant Millin
 
Smart revenue mgt. makes utilities smarter interview feb 2015
Smart revenue mgt. makes utilities smarter   interview feb 2015Smart revenue mgt. makes utilities smarter   interview feb 2015
Smart revenue mgt. makes utilities smarter interview feb 2015Iraklis Pilatos
 
Beyond Brick and Mortar: Advanced Technology Platforms and Processes Power Sm...
Beyond Brick and Mortar: Advanced Technology Platforms and Processes Power Sm...Beyond Brick and Mortar: Advanced Technology Platforms and Processes Power Sm...
Beyond Brick and Mortar: Advanced Technology Platforms and Processes Power Sm...Cognizant
 
smart meter technical seminar report
smart meter technical seminar reportsmart meter technical seminar report
smart meter technical seminar reportravi kant
 
Rusitschka_Big Data meets Smart City-V0.19
Rusitschka_Big Data meets Smart City-V0.19Rusitschka_Big Data meets Smart City-V0.19
Rusitschka_Big Data meets Smart City-V0.19Sebnem Rusitschka
 
report on the Study Of Technology System Of Self-Healing Control In Smart Dis...
report on the Study Of Technology System Of Self-Healing Control In Smart Dis...report on the Study Of Technology System Of Self-Healing Control In Smart Dis...
report on the Study Of Technology System Of Self-Healing Control In Smart Dis...Yuvraj Singh
 
GIS Based Power Distribution System: A Case study for the Junagadh City
GIS Based Power Distribution System: A Case study for the Junagadh CityGIS Based Power Distribution System: A Case study for the Junagadh City
GIS Based Power Distribution System: A Case study for the Junagadh Cityijsrd.com
 
Revue de presse IoT / Data du 26/03/2017
Revue de presse IoT / Data du 26/03/2017Revue de presse IoT / Data du 26/03/2017
Revue de presse IoT / Data du 26/03/2017Romain Bochet
 
IoT for the energy sector
IoT for the energy sectorIoT for the energy sector
IoT for the energy sectorIET India
 
Eurec agency
Eurec agencyEurec agency
Eurec agencyAhmad Eid
 
Smart Grid Data Centers Distributed & ICTs Sustainability on Generation Energ...
Smart Grid Data Centers Distributed & ICTs Sustainability on Generation Energ...Smart Grid Data Centers Distributed & ICTs Sustainability on Generation Energ...
Smart Grid Data Centers Distributed & ICTs Sustainability on Generation Energ...IJMTST Journal
 
Innovative technologies and challenges in the field of smart grid
Innovative technologies and challenges in the field of smart gridInnovative technologies and challenges in the field of smart grid
Innovative technologies and challenges in the field of smart gridIAEME Publication
 
AN INVESTIGATION OF THE ENERGY CONSUMPTION BY INFORMATION TECHNOLOGY EQUIPMENTS
AN INVESTIGATION OF THE ENERGY CONSUMPTION BY INFORMATION TECHNOLOGY EQUIPMENTSAN INVESTIGATION OF THE ENERGY CONSUMPTION BY INFORMATION TECHNOLOGY EQUIPMENTS
AN INVESTIGATION OF THE ENERGY CONSUMPTION BY INFORMATION TECHNOLOGY EQUIPMENTSijcsit
 
01012010 demand response
01012010 demand response 01012010 demand response
01012010 demand response miguelazo84
 

Similar to Rusitschka_SmartDataInElectrificationValueChain_BIG2014_V1 (20)

Paper: The Internet of Things is transforming the energy and utilities indust...
Paper: The Internet of Things is transforming the energy and utilities indust...Paper: The Internet of Things is transforming the energy and utilities indust...
Paper: The Internet of Things is transforming the energy and utilities indust...
 
Actionable Intelligence: Transforming Utilities
Actionable Intelligence: Transforming UtilitiesActionable Intelligence: Transforming Utilities
Actionable Intelligence: Transforming Utilities
 
Facilitating cross-sectoral exchange of energy data
Facilitating cross-sectoral exchange of energy dataFacilitating cross-sectoral exchange of energy data
Facilitating cross-sectoral exchange of energy data
 
Horizon Scan: ICT and the future of utilities
Horizon Scan: ICT and the future of utilitiesHorizon Scan: ICT and the future of utilities
Horizon Scan: ICT and the future of utilities
 
2009 doe smart grid system report
2009 doe smart grid system report2009 doe smart grid system report
2009 doe smart grid system report
 
Smart revenue mgt. makes utilities smarter interview feb 2015
Smart revenue mgt. makes utilities smarter   interview feb 2015Smart revenue mgt. makes utilities smarter   interview feb 2015
Smart revenue mgt. makes utilities smarter interview feb 2015
 
Beyond Brick and Mortar: Advanced Technology Platforms and Processes Power Sm...
Beyond Brick and Mortar: Advanced Technology Platforms and Processes Power Sm...Beyond Brick and Mortar: Advanced Technology Platforms and Processes Power Sm...
Beyond Brick and Mortar: Advanced Technology Platforms and Processes Power Sm...
 
smart meter technical seminar report
smart meter technical seminar reportsmart meter technical seminar report
smart meter technical seminar report
 
Rusitschka_Big Data meets Smart City-V0.19
Rusitschka_Big Data meets Smart City-V0.19Rusitschka_Big Data meets Smart City-V0.19
Rusitschka_Big Data meets Smart City-V0.19
 
report on the Study Of Technology System Of Self-Healing Control In Smart Dis...
report on the Study Of Technology System Of Self-Healing Control In Smart Dis...report on the Study Of Technology System Of Self-Healing Control In Smart Dis...
report on the Study Of Technology System Of Self-Healing Control In Smart Dis...
 
GIS Based Power Distribution System: A Case study for the Junagadh City
GIS Based Power Distribution System: A Case study for the Junagadh CityGIS Based Power Distribution System: A Case study for the Junagadh City
GIS Based Power Distribution System: A Case study for the Junagadh City
 
CBI energy conference: Jon Bentley
CBI energy conference: Jon BentleyCBI energy conference: Jon Bentley
CBI energy conference: Jon Bentley
 
Revue de presse IoT / Data du 26/03/2017
Revue de presse IoT / Data du 26/03/2017Revue de presse IoT / Data du 26/03/2017
Revue de presse IoT / Data du 26/03/2017
 
IoT for the energy sector
IoT for the energy sectorIoT for the energy sector
IoT for the energy sector
 
Eurec agency
Eurec agencyEurec agency
Eurec agency
 
Smart Grid Data Centers Distributed & ICTs Sustainability on Generation Energ...
Smart Grid Data Centers Distributed & ICTs Sustainability on Generation Energ...Smart Grid Data Centers Distributed & ICTs Sustainability on Generation Energ...
Smart Grid Data Centers Distributed & ICTs Sustainability on Generation Energ...
 
Innovative technologies and challenges in the field of smart grid
Innovative technologies and challenges in the field of smart gridInnovative technologies and challenges in the field of smart grid
Innovative technologies and challenges in the field of smart grid
 
AN INVESTIGATION OF THE ENERGY CONSUMPTION BY INFORMATION TECHNOLOGY EQUIPMENTS
AN INVESTIGATION OF THE ENERGY CONSUMPTION BY INFORMATION TECHNOLOGY EQUIPMENTSAN INVESTIGATION OF THE ENERGY CONSUMPTION BY INFORMATION TECHNOLOGY EQUIPMENTS
AN INVESTIGATION OF THE ENERGY CONSUMPTION BY INFORMATION TECHNOLOGY EQUIPMENTS
 
Transformer Smart Grid
Transformer Smart GridTransformer Smart Grid
Transformer Smart Grid
 
01012010 demand response
01012010 demand response 01012010 demand response
01012010 demand response
 

Rusitschka_SmartDataInElectrificationValueChain_BIG2014_V1

  • 1. Smart Data along the Value Chain of Electrification User Needs and Requirements Analysis for a European Big Data Roadmap in the Energy Sector Sebnem Rusitschka Siemens AG Corporate Technology Business Analytics and Monitoring Sector Forum Lead for Energy and Transportation in EU BIG Munich, Germany Abstract—Massive amounts of data await the energy sector stakeholders once the digital transformation has reached a tipping point. In this paper, we are providing an overview of the big data market and competition arenas, which emerge with the increasing feasibility of data-driven scenarios in the energy sector. From the use cases within these scenario categories of Operational Efficiency, Customer Loyalty, and New Business Models we derived user needs and requirements particular to the European energy sector. We conclude with a discussion of the findings and an outlook on the further work of defining a big data value roadmap for infrastructure- and resource-centric sectors in Europe. Index Terms—Data systems, Big Data, Smart grids, Energy management. I. OVERVIEW Whilst some are still hang up on whether big data is hype or not – some have acknowledged that the masses and variety of data in the industrial businesses is a consequence of digitization and automation along their value chains. In the past year we have interviewed stakeholders and researched trends regarding big data in the energy and transportation sector within the European project BIG – Big Data Public Private Forum [1]. The other partners in the consortium analyzed yet other sectors, as well as technological advances from the original big data domains. Together we are looking into cross-sectorial similarities, the different maturity levels, even cross-sectorial scenarios. Most importantly we have analyzed user needs and requirements, and researched how current big data technologies – and regulatory frameworks – need to be adapted and further developed to give industrial businesses the cutting edge advantage in the digital transformation of their businesses. With the analysis of market and technological trends across infrastructure- and resource-centric industries such as energy and transportation with their similar enablers, drivers and constraints to big data, we set the stage for defining what big data means in these traditional sectors. It is evident that big data value potential will not be confined in clear cut industrial boundaries, but rather big data opportunity and competition can also come from other industries: Business models compete with business models in so-called arenas [2] irrespective of industry. Fig. 1 depicts the emergence of these new arenas around energy infrastructure, energy supply, and energy as well as mobility services at the end user side. These new arenas are filling with players reaching from car manufacturers to data-driven startups; electricity retailers and power generation companies are redefining their positioning. We look at the big picture, with energy sourcing from Oil & Gas to Renewables to Decentralized Generation. We observed trends within sectors that are the biggest energy consumers in Europe, i.e., transportation, manufacturing, and households. The short cut that electrification represents for energy efficiency is significant. The enabling technology backbone is the digitization and automation along the value chain of electrification: the smart infrastructure, i.e., smart grid. The “well-to-wheel” energy inefficiency in transportation is indicative: losses amount to 70 percent either during fossil power generation, transmission, and distribution for the electrification of vehicles or when driving and idling at traffic lights with petrol vehicles [3]. In light of these inefficiencies, the integration of renewable energy sources and the full- electric vehicle is a perfect technology match. Given that the technological challenges are overcome, this match not only enables zero-emission transportation, but electric cars that are This work has received funding from the European Union’s Seventh Framework Programme for research, technological development, and demonstration under grant agreement no 318062. Fig. 1 Competition in data-driven scenarios in the energy sector is no longer confined within industries but takes place in new "arenas"
  • 2. parked most of the time also represent a viable storage option for the peak supply from Renewables, which occur at noon via solar and at night via wind. This new arena, “mobility services,” in which electricity and transportation meet, for example, becomes populated by car manufacturers who provide infrastructure or even energy supply [3] alongside the incumbent energy sector players. The entrance of new players also transforms these traditional segments into new arenas. New players enable the efficient balance of energy supply and demand through data-driven services providing demand response [4], energy savings [5], and power [6]. Positive regulatory drivers are significant: Data-driven virtual power plant operation becomes lucrative through the direct marketing potential of the small but numerous distributed generation sources. Technological trends promise to disburden some of the negative regulatory side effects, such as the higher prices due to the current subventions for Renewables. Especially, the manufacturing sector is pressured by high electricity prices [7]. At the same time, industrial automation leads to digital factories, which are enabled to automate according to energy efficiency criteria [8]. The new arena, “energy services,” becomes increasingly more active with energy data-driven startups that offer savings and demand response programs primarily offering services to industrial consumers. Incumbent energy suppliers acknowledge the shift in consumer demand as well as entirely data-driven business actors do [9]. These energy services will eventually reach the mass market of households, utilizing resources such as smart thermostats or entire home automation systems [9]. All of these arenas as depicted in Fig. 1 have in common the typical characteristics that generally define big data: Volume of available data from digitization and automation along the infrastructure as well as at the supply and demand side: Phasor measurement units (PMUs) deliver up to 100 Hz resolution GPS-synchronized data on grid parameters – a current SCADA system with traditional RTUs, which poll data every 2-4 seconds, has to process less than one per cent of the data volume of the entire PMU data accumulating in the same area [10]. Smart metering results in 3,000-fold increase in data when 15 minutes intervals are measured [11]; the software deployment for the smart infrastructure additionally generates data in forms of error logs, etc. Velocity of data, i.e. the rate at which data is produced and consumed is also increasing [12], since the balancing of energy supply and demand as well as the provision of capacity becomes increasingly more real-time due to the added uncertainties through Renewables, decentralization, and active or mobile consumption. Variety of data arises due to the diversification of energy consumption and generation as well as consumerization and liberalization. Market communication alone results in the exchange of a variety of data also coming from new source and roles. With the always connected end user, online sources such as social media as well as smart phones add to this inherent variety in directly and indirectly-related data sources [13]. In our approach to analyzing the energy sector towards its need for handling big data and the requirements towards the usage of big data technology, we collected and discussed various use cases with the above characteristics as briefly described in Section II. The analysis of user needs and requirements of the energy sector on the basis of these use cases and scenarios are detailed in Section III. Section IV concludes the paper with a discussion of the findings and an outlook on the planned roadmapping activities. II. DATA-DRIVEN SCENARIOS IN THE ENERGY SECTOR The big data use cases we collected in interviews, workshops, and online research are holding immense potential for improvement and growth for business. In order to be able to analyze industrial user needs and requirements, it is also important to understand what big data really means in the energy sector and how the value generation is affected by the very nature of cyber-physical systems that are at the core of the digitized infrastructure businesses. Thus, in a second top down approach we clustered the use cases with similar prerequisites into application scenarios that are congruent with inherent business needs: a) Operational efficiency subsumes all use cases that involve improvements in maintenance and operations in real- time or a predictive manner based on the data which comes from infrastructure, stations, assets, and end users of energy. Technology vendors, who develop the sensorization, i.e. digitization and automation, of the infrastructure are the main enablers. The market demand for such enhanced technologies is increasing, because it also helps the businesses in energy to better manage risk: The complexity pan-European interconnected electricity markets with the integration of Renewables and liberalization of electricity trading requires more visibility of the underlying system and of the energy flows in (near) real-time. As a rule of thumb – anything with the adjective “smart” falls into this category: smart grid, smart metering; smart cities, smart (oil, gas) fields. Some examples of big data use cases in operational efficiency are: (i) Predictive and real-time analysis of disturbances in power systems and cost-effective countermeasures; (ii) Operational capacity planning, monitoring and control systems for energy supply and networks, potentially with dynamic pricing or (iii) Optimizing multimodal networks in energy as well as transportation especially in urban settings, such as city logistics or eCar- sharing for which the energy consumption and feed-in in the transportation hubs could be cross-optimized with the logistics. All of the scenarios in this category have the main big challenge of breaking up silos: be it across departments within vertically integrated companies or across organizations along the value chain of electrification, which now are liberalized and oftentimes have conflicting objectives. The big data use cases in the operational efficiency scenario require seamless integration of data, communication and analytics across a variety of data sources, which are owned by different stakeholders. b) Customer Loyalty: Understanding big data opportunities regarding customer needs and wants is especially interesting for companies in liberalized, consumerized markets such as electricity, where entry barriers for new players as well as the margins are decreasing.
  • 3. Customer loyalty and continuous service improvement is what enables energy players to grow in these markets. There are, however, also B2B use cases mainly from technology vendors, but also from energy data or mobility data start-ups. Some examples of using big data to improve customer experience are: (i) Continuous service improvement and product innovation, e.g. individualized tariff offerings based on detailed customer segmentation using smart meter or device level consumption or feed-in data; (ii) Predictive lifecycle management of assets, i.e. data from machines & devices in energy combined with enterprise resource planning and engineering data to offer services such as intelligent on- demand spare-parts logistics, for example; or (iii) Industrial demand-side management, which allows for energy efficient production and hence increases competitiveness of manufacturing businesses. Big challenge is handling confidentiality and privacy of private and business customers whilst getting to know and anticipate their needs. Data originator, data owner, and data users are different stakeholders, who need to collaborate and share data to realize these application scenarios. c) New Business Models revolve around monetizing available data sources and existing data services in entirely different ways. There are quite a few cases in which data sources or analysis results from one sector represent insights for stakeholders of another sector. The analysis of energy and mobility data start-ups shows that there is a whole new way of generating business value if the resources are owned by the end user. Then the business is entirely customer- and service- oriented; whereas the infrastructures of energy and transportation with their existing stakeholders are utilized as part of the service. We call these the intermediary business models. Energy consumer segment profiles, such as prosumer profiles for PVs, CHPs; or actively managed demand side profile, etc., from metering service providers could also be offered for smaller energy retailers, network operators or utilities who can benefit from improvements on the standard profiles of energy usage but do not otherwise have access to high resolution energy data of their own customers yet. The big challenge is the unclear regulation around the secondary use of energy data. Additionally, one of the other scenarios needs to be realized ensuring the digitization of most parts of the infrastructure and the sourcing of end usage data. The connected end user is the minimal prerequisite for the consumer focused new business models. III. USER NEEDS & REQUIREMENTS IN THE EUROPEAN ENERGY SECTOR A. User Needs Business user needs can be derived from the above categories of business needs: Ease of use: Big data technologies employ new paradigms and mostly offer programmatic access only, e.g. R the statistical programming language or MapReduce, a framework for programming massively parallel task execution. Without software development skills and a deep understanding of distributed computing paradigm as well as application of data analytics algorithms within such distributed environments, scalable analytics of mass, streaming data is currently not possible. Semantics of correlations and anomalies that can be discovered and visualized via big data analytics need to be made accessible. Currently only domain and data experts together can interpret the data outliers, business users are often left with guesswork when looking at data analytics results. Veracity of data needs to be guaranteed before it is used in smart grid applications, such as distribution automation or billing of variable tariffs. Because the increase in data that will be used for these applications will be magnitudes bigger, simple rules or manual plausibility checks no longer are applicable. Smart Data often is used by industrial stakeholders emphasizing that a business user needs refined data – not raw data (big data). In cyber-physical system as opposed to online businesses, there is information and communication technology (ICT) embedded in the entire system instead of only in the enterprise IT backend. Especially infrastructure operators have the opportunity to pre-process data in the field and aggregate levels or distribute the intelligence for data analytics along the entire installed ICT infrastructure to make best use of computing and communication resources to deal with volume and velocity of mass sensor data. The need for smart data is driven by the need for cost-effective usage of all installed ICT resources along the infrastructure. Decision support or automation becomes a core need, as the pace and structure of business is changing. European grid operators today need to intervene almost daily to prevent potentially large-scale blackouts, e.g. due to integration of Renewables or liberalized markets. Business users need more than the information that something is wrong. Visualizations can be extremely useful, but the question of what needs to be done remains to be answered either in real-time or in advance of an event, i.e. in a predictive manner. Strong need for scalable, advanced analytics, hence, will push the envelope of state-of-the-art, such as smart metering data analytics [11]: e.g. Segmentation based on load curves, forecasting on local areas, scoring for non-technical losses, pattern recognition within load curves, predictive modeling, etc. End users needs are not directed towards big data or energy data as is. However, big data scenarios in energy offer many improvements for the end users: Operational Efficiency ultimately means energy and resource efficiency, which will improve quality of life – especially in urban settings – for all end users. Customer Experience and New Business Models related big data scenarios are entirely based on better serving the end user of energy. However, both scenarios need personalized data in higher resolution, geo- or time-tagged. We have been discussing the real big data value in cross- combining such variety in data, which on the downside can make pseudonymization or even anonymization ineffective in protecting the identity and behavioral patterns of individuals or business patterns and strategies of companies. New business models based monetizing once collected data, with
  • 4. currently unclear regulations, leaves end users entirely transparent, uninformed, and unprotected against secondary use of their data for purposes they might not agree with, e.g. insurance classification, credit rating, etc. Reverse transparency is at the top of the wish list of data- literate end users: End users need practical access to information on what data is used by whom for what purpose in an easy-to-use, manageable way. Rules and regulations are needed for granting such transparency for end users; data protection laws that are minimally consistent, but also allow individualization for maximal privacy are required. Data access, exchange, and sharing needs, hence, apply on both business and end users. In today’s complex electricity markets, there is almost no scenario where all required data for answering a business or engineering question comes from one department’s databases. Nonetheless, most of the currently installed advanced metering infrastructures have a lock-in of the acquired energy usage data in the utilities’ billing systems. The lock-in makes it cumbersome to use the energy data for other valuable analytics. Such silos have traditional roots from the recent era where most of European utilities were vertically integrated companies. Also, the amount of data to be exchanged was much less, so that interfaces, protocols, and processes for data exchange have been rather rudimentary. The analyzed business user and end user needs, as well as the different types of data sharing needs directly translate into technical and non-technical requirements. Some of which are discussed in the following. The technical requirements are summarized under requirements for a so-called big data refinery pipeline that shall satisfy the need to utilize entire data value chain and generate smart data: B. Technical Requirements for a Big Data Refinery Pipeline – “Analytics Inside” The big data refinery pipeline for energy businesses consists of three main phases: data acquisition, data management, and data usage. Data analytics, as indicated by business user needs, is implicitly required within all steps, and is not a separate phase as depicted in Fig. 2: During data acquisition, analytics supports the data cleansing and ensures data quality. For sensor data, analytics can enable efficient event-specific data compression and increase data quality through anomaly detection at acquisition time. For data usage, business and engineering questions need to be formulated by business users and translated into suitable models. These models among other aspects imply which types of mass storage and data handling are cost-efficient for the specific purpose. So, each step of the pipeline is setup to refine the data, but the methods of refinement vary depending on the data type. Such connectedness with the sources of data may be the main differentiator of big data in the energy business from big data usage scenarios in the online businesses: If data acquisition and analytics do not deliver actionable information with available options fast enough then the value of data may even be negative. 1) Data Acquisition Requirements In the data acquisition phase, security and privacy or confidentiality policies need to be applied; the data should be checked for quality features, such as missing data or implausible data, e.g. in time series. Depending on whether data is generated during an automated process, or by a human, e.g. repair crew in the field or customer call agent, data is highly structured or unstructured; it is acquired as continuous streams of high frequency samples or sent irregularly. The methods for processing and analyzing the data, acquired in very different ways and formats, would reach from content analytics and natural language processing for unstructured data to signal and cross-correlation analysis on structured topological time series data. The great differentiator of big data scenarios in traditional sectors such as energy, especially in Europe, is the concern that data should be collected for a known purpose. This may sound conservative at first. And with current technological capabilities, it is very restrictive in practice. However, it actually gives way to formulate a central requirement that will drive Europe to thrive both technologically as well as in terms of digital rights on individual and business level: Dynamic configurability of data access about which data can be collected for what purpose in what granularity and time span and location must be given along the entire intelligent infrastructure of electrification. The configuration or the effects thereof must be easily comprehensible for the data owners. The configuration must be dynamic in the sense that service providers are able to receive the data in the required Fig. 2 Cyber-physical systems as in the energy sector require a big data refinery pipeline that eemploys analytics at every step of data processing to provide “smart data”
  • 5. granularity and with the allowed privacy and confidentiality protection settings defined – on demand. Privacy and confidentiality preserving data analytics are required to enable the service provider to retrieve the knowledge without violating the agreed upon granularity of data or the allowed privacy or confidentiality settings. 2) Data Management & Required Architectural Patterns Different types of data management may be considered for the same data source type depending on for what purpose it will be used: Graph databases can yield faster results for geo- spatial analysis, key-value stores may prove most efficient for feature discovery purposes in time series. Regarding the generation of value from a variety of data sources the data storage step needs to enable the cost-efficient integration of the different data sources in an extensible way. Flexible abstractions, multiple layers of abstractions for data and analytical models will be required for the data storage step. Lightweight semantic data models that represent the multiple links within various data sources, such as Linked Data, may provide such cost-efficient abstractions, especially when the data domain is complex and highly interlinked. For high-dimensional data, multi-dimensional data structures such as tensors and space-filling curves may be required to support the design of more efficient analytics algorithms, which are capable of exploring multiple relations at once. These are only some examples of a few methods from machine learning, semantic data technologies, information extraction, and distributed computing. What will be required is a flexible toolbox, an analytics engine, which scales with the increasing business need to gain more value from ever increasing amounts and variety of data without having to change the entire analytics application: Abstraction from the actual big data infrastructure is required to enable (a) ease of use and (b) extensibility and flexibility. The analyzed use cases have such diverse requirements that there is no single big data analytics platform that will empower the future utilities business with insights from massive amounts of available data. Adaptive data and system models are needed so that new knowledge extracted from domain analytics or the ever changing circumstances of the system can be redeployed into the data analytics framework without disrupting the analytics applications. The analytics engine is required to accomplish the plug-in such adaptive models. a) Schema on Read and the Active Archive Pattern Active archive, sometimes also called Data Lake, allows the cost-efficient storage of all data in its raw form. Raw data is transformed as needed by the business users on demand, which is called “schema on read.” Distributed data management and massively parallel processing principle combined with performant commodity hardware makes the generation of a data schema on read feasible. An active archive allows one to get to know the data and cost-efficiently explore business opportunities. b) Lambda Architecture The lambda architecture is based on a set of principles for architecting scalable big data systems that allow running ad- hoc queries on large datasets. It consists of multiple layers dealing with volume and variety, via an active archive – as well as velocity, via stream data computing. Fast and even real-time analytics is required to support decisions, which need to be made in ever shorter time spans. Speed in big data technologies is especially determined by the solution architecture. However, in smart grid settings near real-time dynamic control over events also requires insights to be gained at the source of the data near the events. c) In-field Analytics The lambda architecture is described from an enterprise perspective. In cyber-physical systems, such as industrial automation or energy automation, field devices represent important data acquisition as well as computing resources. In- field analytics extends the lambda architecture by encompassing the data acquisition layer, in order to deliver faster insights for industrial businesses. 3) Data Usage Requirements Although data usage is the last phase in the refinement process, the business and engineering questions formulated in this phase are the starting point for choosing the technology stack. This phase represents the reason for setting up a particular connection through the pipeline to the required data sources. In the industrial domain of energy, analytics for data usage, reaches from descriptive analytics such as dashboards, to predictive and prescriptive analytics such as forecasting for portfolio management or simulation of interconnected power systems and extraction of operational intelligence on what actions to take in real-time or predictive manner. Industrial data usage requires considerably more precision than data usage in online data businesses. The business value generation is versatile, too, since it not only covers the provisioning of an infrastructure asset but also the usage of that asset in B2B as well as B2C settings. Data interpretability must be assured without the constant involvement of domain experts. Also at anytime the results must be traceable. Expert and domain know-how must be blended into the data management and analytics. Self- describing smart devices are one part of the requirement. Semantic and adaptive data models are the other part. C. Non-technical Requirements In the interviews, workshops and online research several non-technical requirements were encountered repeatedly, which are listed in the following: Investment in communication and connectedness: Broadband communication or ICT in general, needs to be widely available across all Europe and alongside the energy infrastructure such that real-time data access is available. A digitally united European Union: data silos are not just a technicality: (a) the current implementation of the unbundling prevents departments within a utility from exchanging and cross-analyzing data freely; (b) EU member states all have different data related regulations. European stakeholders require reliable minimally consistent rules and regulation regarding digital rights and regulations.
  • 6. A better breeding ground for start-ups and start-up culture is required, especially for techno-economic paradigm shifts like big data and the spreading digitization, when also the ways of business making widely deviate from business-as- usual. Regarding energy and mobility related start-ups, certainly their incubation includes more than just financial investments but also a controlled but allowed new approach to data access. Without this freedom for exploration and experimentation innovation has little chance, unless of course the aforementioned techniques privacy preserving analytics are feasible. Incubators for energy and mobility data start-ups are emerging in Europe [14]. Open data in this regard is a great opportunity, however, standardization is required. The current data models, data representation, as well as protocols require a practical migration path to simpler state-of-the-art standards capable of handling the volume, velocity, and variety of data. These standards will also enable the growth of data ecosystems with collaborative data mining, shareable granularity of data and accompanying techniques that prevent de-anonymization. Skilled people: Programming and statistical tools need to be part of engineering education until big data technology becomes more business user-friendly or in case this becomes the “new normal:” Users need to deal with programmatic handling, i.e., R or MapReduce are both programmatic approaches instead of a graphical user interface or SQL interface due to the adaptability that volume, velocity, and variety of data demand. Traditional data analysts need to grasp the distributed computing paradigm, e.g. how to design algorithms that run on massively parallel systems, i.e., how to move algorithms to data, or engineer entirely new breed of algorithms. IV. CONCLUSION The energy sector, from an infrastructure perspective as well as from resource efficiency and quality of life perspectives, is very important for Europe. The high quality of the energy infrastructure and global competitiveness also needs to be maintained with respect to the digital transformation and big data potentials. The analysis of the available data sources in energy as well as their use cases in the different categories for big data value: operational efficiency, customer experience, and new business models, helped in identifying the industrial needs and requirements for big data technologies. Already in the discussion of these requirements, it becomes clear that a mere utilization of existing big data technologies as employed by the online data businesses will not be sufficient. Domain- and device-specific adaptations for use in electrical cyber-physical systems are necessary. Innovation regarding privacy and confidentiality preserving data management and analysis is a primary concern of all energy stakeholders. Without satisfying the need for privacy and confidentiality there will always be regulatory uncertainty and uncertainty regarding user acceptance of a new data-driven offering. Among the energy stakeholders, there is a common sense that “big data” is not enough: The increasing intelligence embedded in the infrastructures is also able to analyze data to some extent as to deliver “smart data.” This seems to be necessary, since the analytics involved will require much more elaborate algorithms than for analyzing click streams. Additionally, the stakes in big data scenarios are very high, since the optimization opportunities at the end are within critical infrastructures. These requirements represent the starting point for the development of the big data roadmap for the European energy sector. For this purpose the ongoing consultations with the technical working groups on the big data refinery pipeline for the energy sector will be continuously cross-checked with sector representatives and against current and future developments of the big data trend in the energy sector. V. REFERENCES [1] EU Project BIG – Big Data Public Private Forum. [Online]. Available: http://www.big-project.eu/ [2] R.G. McGrath, “The End of Competitive Advantage: How to Keep Your Strategy Moving as Fast as Your Business,” in Harvard Business Press Books, 2013. [3] G.Fournier, H. Hinderer, D. Schmid, R. Seign, and M. Baumann. (2012). The new mobility paradigm: Transformation of value chain and business models. [Online]. Available: http://run.unl.pt/bitstream/ 10362/ 10154/1/FournierBaumann9-40.pdf [4] Entelios. [Website]. www.entelios.com [5] EnergyDeck. [Website]. www.energydeck.com [6] Next Kraftwerke GmbH. [Website]. www.next-kraftwerke.de [7] C. Schroeder. (2012, August 16). Energy Revolution Hiccups: Grid Instability Has Industry Scrambling for Solutions. [Online]. Available: http://www.spiegel.de/international/germany/instability-in-power-grid- comes-at-high-cost-for-german-industry-a-850419.html [8] S. Mechs, S. Lamparter, J. Peschke, J.P. Mueller. “Efficient Identification of Energy-Optimal Switching and Operating Sequences for Modular Factory Automation Systems,” Recent Trends in Applied Artificial Intelligence, Lecture Notes in Computer Science Volume 7906, 2013, pp 202-211. [9] T. Racciatti. (2014, March 21). Manufacturing The Internet Of Things. [Online]. Available: http://www.mbtmag.com/articles/2014/03/ manufacturing-internet-things [10] NERC.2010. Real-Time Application of Synchrophasors for Improving Reliability. Technical Report. Available: http://www.nerc.com/docs/ oc/rapirtf/RAPIR final 101710.pdf, retrieved on 2013-05-17. [11] Picard, M.-L. (2013, June 26). A Smart Elephant for A Smart-Grid: (Electrical) Time-Series Storage And Analytics Within Hadoop. [Online]. Available: http://www.teratec.eu/library/pdf/forum/2013/ Pr%C3%A9sentations/A3_03_Marie_Luce_Picard_EDF_FT2013.pdf [12] C.O. Heyde, R. Krebs, O. Ruhle, Z.A. Styczynski. “Dynamic voltage stability assessment using parallel computing,” In proceeding of: Power and Energy Society General Meeting, 2010 IEEE. [13] GTM Research. (December 2012). The Soft Grid 2013-2020: Big Data & Utility Analytics for Smart Grid. [Online]. Available: www.sas.com/ news/analysts/Soft_Grid_2013_2020_Big_Data_Utility_Analytics_Sm art_Grid.pdf [14] SiliconRepublic. (2014, April 7). Energy and transportation start-up to launch in Berlin, August 2014. [Online]. Available: www.silicon republic.com/start-ups/item/36404-energy-and-transportation-s