1. WHITE PAPER
LEVERAGING BIG DATA AND REAL-TIME
ANALYTICS TO ACHIEVE SITUATIONAL
AWARENESS FOR SMART GRIDS
PART ONE IN VERSANT’S SCALABLE SITUATIONAL
AWARENESS SERIES
By Bert Taube, Director of Energy and Smart Grid Solutions
Versant Corporation
Sponsored by Versant Corporation
Versant Corporation U.S. Headquarters
255 Shoreline Dr. Suite 450, Redwood City, CA 94065
www.versant.com +1 650-232-2400
2. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 2 of 20
EXECUTIVE SUMMARY:
WHY THE PURPOSE OF BIG DATA IS REALLY
SITUATIONAL AWARENESS
The concept of Big Data has been around for more than a decade,
and its potential to transform the effectiveness, efficiency, and
profitability of virtually any enterprise has been well documented.
Yet, despite the concept of Big Data being well-defined, and the
general enormity of its opportunity well-understood, the means to
effectively leverage Big Data and realize its promised benefits still
eludes many.
Big Data’s remaining challenge to realizing these benefits comes in
two parts. The first is to understand that the true purpose of
leveraging Big Data is to take action - to make more accurate
decisions, more quickly. We call this situational awareness, an idea
that is quite self-explanatory. Regardless of industry or
environment, situational awareness means having an
understanding of what you need to know, have control of, and
conduct analysis for in real-time to identify anomalies in normal
patterns or behaviors that can affect the outcome of a business or
process. If you have these things, making the right decision in the
right amount of time in any context becomes much easier.
Although the term of situational awareness itself is fairly recent, the
concept has roots in the history of military theory - it is recognizable
in Sun Tzu's The Art of War, for instance. The term itself, can also
be traced to World War I, where it was recognized as a crucial
component for crews in military aircraft. Before being widely
adopted by human factors scientists in the 1990s, the term was first
used by United States Air Force (USAF) fighter aircrews returning
from war in Korea and Vietnam. Today, the concept of situational
awareness has been expanded to a variety of areas such as air
traffic control, nuclear power plant operation, vehicle operation, and
anesthesiology. Defining the parameters of situational awareness
for any industry is not simple, and thus surmounting Big Data’s
other remaining challenge of creating new approaches to data
management and analysis to support these needs is also no small
feat. Achieving situational awareness used to be much easier
because data volumes were smaller, and new data was created at
a slower rate, which meant our worlds were defined by a much
smaller amount of information. But new data is now created at a
hugely exponential rate, and therefore any data management and
analysis system that is built to provide situational awareness today
must also be able to do so tomorrow. Thus, the imperative for any
enterprise is to create systems that manage Big Data and provide
scalable situational awareness.
Table of Contents
Section 1: The Smart
Grid Network’s Unique
Big Data Demands
Section 2: Utilities’
Current Data Analytics
and Management
Methods, and the Need
for Change
Section 3:
Requirements of Data
Analytics Systems to
Create Scalable
Situational Awareness
for Smart Grids
Section 4: Versant and
The Object-Oriented
DB’s Pedigree, and
their Value for Smart
Grid Situational
Awareness
Section 5: How To
Apply Object-Oriented
Data Management and
Analysis Technologies
in Smart Grids
Section 6: Benefits of
Scalable Situational
Awareness through
Object-Oriented Data
Management and
Analysis for Smart
Grids
Conclusion
3. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 3 of 20
The utilities industry is in particular need of scalable situational
awareness so that it can realize benefits for a wide range of
important functions that are critical for enabling Smart Grid
paradigms. Scalable situational awareness for utilities means
knowing where power is needed, and where it can be taken from,
to keep the grid stable. When power flow is not well understood,
the resulting consequences can quite literally leave utilities and
their customers in the dark: a fitting-though-ironic analogy
considering the goal of awareness. .
Utilities can learn much about how to achieve scalable situational
awareness from other industries, most notably building
management and telecommunications, which have learned to deal
with Big Data’s complexity and scale well.
This white paper will describe the diversity of utility data, the criteria
which define a big energy data problem, and how utility business
value can be built through increased situational awareness. The
various problems that arise from storing the large number of data
types in smart grid systems using traditional software technologies
(data historians and relational databases) will be summarized.
Overcoming these problems can be accomplished by applying
NoSQL-based data management and analytics solutions like
Versant’s, which can be seamlessly integrated with object-oriented
languages. A power outage example will be utilized to demonstrate
how Versant’s NoSQL solutions have enabled grid reliability and
situational awareness, and how they can help utilities achieve other
critical business goals.
SECTION 1: THE SMART GRID NETWORK’S
UNIQUE BIG DATA DEMANDS
In 2001, Doug Laney, now an analyst with Gartner, created what
has become the widely-accepted three-dimensional definition of
Big Data, also known as the Three V’s of Big Data: volume,
velocity, and variety.
Laney’s definition, though, was created for the paradigm of a
typical business, such as manufacturing, where profitability is often
achieved by the minimization of fixed assets, where work in
progress is measured in days, weeks, or months, and real-time
data collection and analysis are often not critical to ensure the
profitability of the organization. The value chain for manufacturing
almost always crosses company boundaries. However, in the utility
industry, there are vertically-integrated and deregulated variants
that have to act exactly the same, increasing complexity.
4. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 4 of 20
In this environment, the acquisition of real-time data can both be
costly and can seriously impact the bottom line. This adds more
dimensions to the utility industry’s Big Data needs as enterprises
must not only deal with data’s volume, velocity, and variety
challenges, but also with two new V’s: validity and veracity.
Validity - Information in the utility environment often has a “shelf
life” and therefore may only be needed for storage and evaluation
for a fixed period of time. The questions of when to archive or
dispose of data become relevant given the cost of storing large
volumes of data.
Veracity – This fifth variable recognizes that data is not perfect, and
that achieving “perfect” data carries a cost. Utilities must consider
two questions: 1) how good must the data be to achieve the
necessary level of accurate analysis, and 2) at what point does the
cost of correcting the data exceed the benefit of obtaining it?
The utility industry’s time scales vary over 15 orders of magnitude
due to the unique diversity of sensors and critical business
processes, and often at much faster intervals than other industries,
which, when trying to create scalable situational awareness,
impacts all five V’s of the industry’s Big Data pressures.
Image Caption: Data from Utilities’ devices and sensors has an extraordinarily broad range
of relevant time durations for which they are valuable to the business, from milliseconds, to
decades.
Analyzing huge volumes of data that span multiple orders of time-
scale magnitude falls short of traditional data-management
technologies’ abilities. Traditional methods of data management,
such as relational databases (RDB) or time-serialized databases,
may not have the capability to capture the causal effects of years
5. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 5 of 20
or decades of events that may occur in a millisecond or
microsecond range, and therefore cannot meet the real-time smart
grids’ scalable situational awareness needs. Additionally, such an
array of devices and processes create an especially-wide variety of
data types and formats that must be considered when making any
decision, and thus for enabling scalable situational awareness.
A Word on Synchrophasors: Smart Grid Big Data’s Best Friend
(and Worst Enemy…)
Utilities already have considerable challenges with regards to the
speed, complexity, and variety of their data, and many would argue
they also are already facing true data volume problems. But if a
consensus can’t be reached apropos on utilities’ data volume
challenges, it is certain they will face them in the future as they
pursue the goals of the smart grid.
Currently, most utilities are in smart grid deployment mode, and
many investor-owned utilities have entered the implementation
phase as described by the Public Utilities Commissions’ smart grid
strategic plans. By 2020, there will be a large number of real-world,
full-scale smart grid deployments. The achievement of this goal will
be driven largely by a massive increase in the number of installed
phasor measurement units (PMUs), or synchrophasors, in the
power grid.
Synchrophasors are considered one of the most important
measuring devices for smart grids, and their continued adoption is
expected to increase data volumes for utilities by 700 to 800
percent by the year 2020:
Image Caption: Expected Data Volumes from PMU Deployments
Assumption : 15 installed PMUs at the time of North America’s largest Blackout in 2003
(Western Part of the United States)
6. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 6 of 20
SECTION 2: UTILITIES' CURRENT DATA
ANALYTICS AND MANAGEMENT METHODS, AND
THE NEED FOR CHANGE
Utilities have primarily used the same two types of databases to
manage and analyze their data for the last 30 years, despite
neither of them ever being ideal for these critical tasks.
The relational database (RDB) rose to prominent use by utilities
during the seventies when storage media was very expensive. The
main advantage that the RDB was that storage costs could be
minimized, but at the expense of needing to write a lot of
proprietary code to describe the relationships between data
(commonly known as a JOIN).
Data historian technologies, sometimes referred to as streaming
data stores, rose to similar prominence for storing utilities’ time-
serialized data. These, essentially, were another form of the RDB,
in this case optimized for storing data with a time-stamp, which
emphasized reduction in storage costs at the price of not being
able to easily correlate it with important variables other than just
time.
The shortcomings of these technologies have become especially
apparent as the need to conduct analysis across multiple data
types, formats, and domains has become more important for
enabling scalable situational awareness for the smart grid. The
failure of traditional databases like RDBs to scale well in the face of
rising data volumes, complexity, and speed has been well proven,
with alternative technologies often outperforming them by more
than ten-fold in benchmarking tests.
What are needed instead are data management technologies that
are optimized for analysis rather than constraints like speed and
storage space. Ideally, these technologies would also be built much
like the grid itself, with classes of assets that have natural, pre-
defined relationships between them. These capabilities are readily
found in proven object-oriented databases (ODB) and emerging
NoSQL technologies like those that Versant offers, and which
indeed have been deployed across multiple industries with similar
data challenges with great effect.
The ODB has been used for years in the telecommunications,
transportation, building management, and many other industries to
track and analyze large numbers of data types and classes. Unlike
relational or serialized databases, ODB’s offer seamless integration
with object-oriented languages. This means that the objects’
application descriptions translate directly to the database objects
7. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 7 of 20
themselves, making analysis much easier and faster, and thus
supporting the goals of scalable situational awareness.
Additionally, query is used for optimization based on use cases and
the business application of the data, not as the sole means of
accessing and manipulating the underlying data. There is no
application code needed to manage the connectivity between
objects or how they are mapped to the underlying database
storage. ODBs use and store object identity directly, bypassing the
need for the CPU and memory-intensive, set-based join operations
used by RDBs.
SECTION 3: REQUIREMENTS OF DATA
ANALYTICS SYSTEMS TO CREATE SCALABLE
SITUATIONAL AWARENESS FOR SMART GRIDS
The underlying data management and analytics solutions required
to provide scalable situational awareness for smart grids must have
five key characteristics: Flexibility, Interoperability through
Connectivity, a Control Network, it must use Open, Standards-
Based Data Management Technologies, and it must support
Scalable Data Analysis.
Flexibility - Unlike many industries, power delivery is notoriously
variable, with daily, weekly, and annual variations due to variability
in customer load, generation dispatch, delivery system outages,
and other reasons. This variability has challenged the industry to
discern patterns that can be used to identify abnormal conditions
and anomalies that spur critical decisions-making processes.
Versant’s technologies can deal with data that looks at voltage and
current rate data just as easily as any other type of data from any
other industry. By embedding a variety of different data object
models to capture the different energy data types, as well as its
corresponding sample rates, object-oriented programming allows
for an integrated data management and analytics concept. It
creates the necessary flexibility to deal with the challenging
characteristics of Big energy Data in real-time. Fast and reliable
data retrieval, suitable data formats for data analysis, one object-
oriented programming language (for DDL and DML), connectivity
between objects without application code, direct use and storage of
object identities, and advanced, as well as traditional data
management, features merged together represent critical values of
a fully-integrated object-oriented data management and analytics
solution. This is what gives you the situational awareness that is
needed for utilities: understanding the immediate value of making a
decision to solve an abnormality in normal data patterns within a
relevant time frame.
8. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 8 of 20
Interoperability and Connectivity – The smart grid of the future will
be a massive collection of devices, sensors, actuators, and
systems, all of them creating ever-larger data volumes and ever-
greater analytics complexity. In this form, the smart grid is a hugely
complex network that must have full accessibility of all these
devices and sensors. Central to enabling this is Internet
connectivity, something again that Versant’s technologies have
proven highly capable of by managing data and analysis for many
global telecommunications service providers.
Control Network - Not only is collecting all of the data that smart
grid sensors and devices produce a challenge, but all of these
devices must be fully communicative, interconnected, and, critically
controllable. The decisions made based on having full situational
awareness must be rapidly translated in to the functioning grid,
which, like enabling interoperability, requires a single, cohesive
control system enabled heavily through Internet connectivity.
Open, Standards-Based Data Management Systems – A network
as complex, variable, and fast-moving as the smart grid requires
billions of devices, sensors, and machines. It is impossible to
expect that any one data management technology vendors’
systems will be used across every grid application and scenario.
But more to the point, smart grids will be integral to the everyday
life of billions of people, so as new technologies are developed and
adopted over time the smart grid must be able to adjust and
change the data management systems to meet new requirements.
To enable this, utilities must leverage open system architectures
across five specific areas to permit ease of adoption and avoid
costly vendor lock-in:
Network Infrastructure: Includes protocol, routers, media
type, IT connectivity, etc.
Control Devices: Heavily-utilized devices that produce,
consume, and manipulate data, as well as control and
monitor the energy grid network.
Network Management and Diagnostic Tools: Enable
configuration, commission, and maintenance for the
system.
Human-Machine Interface (HMI): Includes the visualization
tools through which users and managers obtain a view into
the system, including both PC software and instrumentation
panels.
Enterprise/IT Level Interface: Connects the control network
into the data network. No gateways other than open
systems standards-based routers and IT-based data
exchange mechanisms are used.
9. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 9 of 20
A critical sixth factor is the object-oriented data management
system itself, which must also be considered part of this open
standards-based architecture. The ODB represents the
configuration database for the complete network of the grid, storing
the configuration profile data of every device participating in the
open, fully interoperable and integrated control network, and
enabling effective communication and control between them all.
Scalable Data Analysis – Utilities will face immense data volume
increases over the next several years, making the job of ensuring
the validity and veracity of data analysis ever harder. Open
architectures and data management technologies will play a pivotal
role in enabling data analysis that scales to these new volume
demands. These systems must not only be capable of dynamically
scaling to account for and manage increased data complexity, but
also sheer volume as new types of devices are deployed on the
grid network.
Versant’s abilities to scale for both extreme data volumes and
analytics complexity have been proven in large network
environment deployments like telecommunications and building
energy management. These industries are directly relevant to the
smart grid because of their scale and complexity challenges, as
well as the nature of the data itself. Technologies like the Versant
Object Database (VOD) and the Versant Java Persistence API
(Versant JPA) have been deployed by dozens of
telecommunications network operators and equipment vendors,
and in more than 150,000 buildings around the world.
As explained more fully in the next section, the Verite Group, for
example, uses Versant’s object-oriented technologies to power the
company’s new Netscope IP session packet reconstruction
application, allowing it to easily scale to meet the needs of Verite’s
customers, which include some of the world’s largest
telecommunications network operators. Likewise, for China
Telecom, who has one of the world’s largest subscriber bases, the
tremendous challenge of developing a database to store and
quickly access subscriber records at the rate of up to 480,000
queries and 1,000 update transactions per second was drastically
simplified with Versant.
SECTION 4: VERSANT AND THE OBJECT-
ORIENTED DB’S PEDIGREE, AND THEIR VALUE
FOR SMART GRID SITUATIONAL AWARENESS
Relational databases rose to fame in the utility industry when,
compared to what they are today, storage was expensive and data
complexity and volumes were low. The need to avoid substantial
10. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 10 of 20
cost was understandably paramount at a time when the grid was
already relatively large compared to other similar networks and
would have therefore multiplied storage costs.
However, that did not stop other industries from finding more
effective data analytics and management technologies. In other
industries where storage concerns were not quite as much of a
concern initially, yet managing extreme data complexity was of the
utmost importance, such as building management systems, the
object-oriented database quickly became the go-to choice. And
even for the world’s original Big Data industry, telecommunications,
where managing and analyzing massive data volumes on par with
those the utilities industry will soon face, the ODB has been the de-
facto data management technology for decades.
Indeed, several other industries that have multi-faceted,
interdependent network environments make for great starting
points from which the utilities industry can learn to apply scalable
situational awareness. Most notable of these are data
communications networks like those of telecommunications,
geospatial environments, building control networks, and power
transmission networks.
The concept of centralized power transmission system operation
implemented through Independent System Operators (ISOs) has
proven critical in North America, as well as Europe, to increase
bulk power grid reliability, improve planning, enhance technology,
and guarantee energy market pricing. Implementing advanced
information management systems in the control room of an ISO
plays a crucial part in this concept.
NoSQL-based, object-oriented data management solutions
empower control room systems operated by the French ISO Rte,
the French Security Agency CNES, as well as the service
coordination center CORESO shared by three European ISOs
(Rte, Elia, National Grid). The following table summarizes where
object-oriented programming is applied to forecast day-ahead
electricity consumption, monitor power transmission system
security, or analyze and simulate steady-state grid performance. It
explains the critical role of object-oriented data management
solutions as they support effective, near real-time information
management for large, complex data management in bulk power
grids:
11. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 11 of 20
Customer French National
Security Center
(CNES)
French Transmission
System Operator
(Rte)
Technical Coordination
Service Center (Coreso;
Rte+Elia+National Grid)
Name of
Control Room
System
PRELUDE ARCADE CONVERGENCE
Purpose of
Control Room
System
Forecast of Day-
Ahead Electricity
Consumption for the
French National
Security Center
(CNES) to monitor
and guarantee daily
Electricity Supply-
Demand Balance
Monitoring of French
Power Transmission
System Security by
French National
Security Center
(CNES)
Analysis and Simulation of
steady-state Performance of
French, Belgian and British
Power Transmission Systems
to guarantee Reliability of
electric Power Supply
The complexity of building management data models and
management needs are similar to those of smart grids.
Transforming buildings to become more “green” has been a
primary concern for years, and requires making them more
intelligent, energy efficient, and cost effective. Buildings’ energy
consumption typically represents 30 percent of its total operating
costs. A 30 percent reduction in a building’s energy use can yield a
five percent increase in total operating income, making reductions
in operating costs from increased energy efficiency one of the
strongest selling points for green buildings.
Making a building “green” means making every system that uses
natural resources to support humans more efficient. In addition to
energy consumption, this means transforming the way a building
uses water, air, and many more resources, which requires that
eight key systems are completely integrated via a data
management system according to the key characteristics outlined
in section 3 of this paper: Fire, security, access, energy, lighting,
lifts, communications, 24/7 monitoring, and HVAC. And again, as in
section 3, the Internet is also critical for enabling the connectivity
needed to link all these systems.
12. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 12 of 20
These many different systems create levels of data management
complexity that actually surpass those of the smart grid.
Additionally, like the energy grid, buildings are also still
transforming as part of the “green” push, with new and more
sensors being added continually. And, actually, buildings are ahead
of utilities in this regard already having made major strides in
transforming from entities where each of these systems and
devices are islands of information to open-architecture-based,
intelligent buildings where components and control networks are
integrated, fully interoperable, and where controllers have full
situational awareness of the building’s abilities to support the
humans inside it.
As of 2009, Echelon’s LonWorks Network Systems (LNS) building
management technologies have been deployed in about 50 million
intelligent network devices across 1 million individual networks. The
LNS management system is powered by Versant’s object-oriented
data management solutions, enabling rapid data management and
analysis across all of these devices and sensors. Further, LonMark
has over 400 members using the LonWorks platform, and the
platform is an international open standard, ISO/IEC 14908-1,
making it a perfect example of the kind of open, standards-based
systems that smart grids require.
By using these open, object-oriented control systems to enable full
situational awareness of the building’s status, users reported:
About a six months advantage in time-to-market for being
able to declare the building “green” and intelligent
Several years of saved developer time
Enablement of more powerful modeling features
A significant increase in productivity for maintaining and
enhancing the code base that defines the underlying
database and data models
An inherently more secure network
Energy efficiency in these buildings was also significantly
improved. The Balanced Office Building in Germany, for example,
reduced HVAC and lighting costs by 80 percent. The United States
National Resource Defense Council offices in California reduced
total energy use by 45 percent. And the Philips HELIO Lighting
Control Application, which is deployed throughout the world by
many large companies, uses the Echelon system to help reduce
companies’ energy costs by up to 50 percent by creating a data
management model that lets lighting systems be automatically
controlled depending on inputs from other systems and sensors
like HVAC.
13. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 13 of 20
Enabling this degree of automation and interoperability is a form of
demand response capabilities, a major portion of utilities’ overall
demand-side management goals for the smart grid. And as
buildings represent 39 percent of primary energy use in the United
States, it is even more critical that utilities’ smart grid data
management systems interoperate with the open, standards based
building management systems like Versant’s and Echelon’s if they
are to achieve situational awareness.
Where buildings differ in their data management needs from
utilities is in the scale of their data volumes and velocities.
Buildings have roughly 72 object classes and 32,000 device
instances per building control network, whereas power grids deal
with data volumes on a scale more than ten-fold larger, with more
than 700 object classes. Additionally, buildings’ sensors and
systems need to function in the fastest instances at the second or
minute level for things like motion sensors and temperature
controls. But grid sensors must function to deliver effective
predictive analysis in an extremely wide time range from
milliseconds to decades. This essentially means that data
management systems for utilities must be able to respond to
analysis queries and deliver real interoperability on demand and in
true real-time.
The telecommunications industry examples referenced at the end
of section 4 of this paper reveal the ability of object-oriented
database technologies to perform extraordinarily well under
extreme data volume and velocity conditions. The Versant Object
Database has been successfully tested by China Telecom to
process up to 1,000,000 queries per second, earning it the
operators’ title of best Object Database technology after testing its
scalability against a range of other database technologies. During
head-to-head benchmarking between the Versant Object Database
(VOD) and other databases (ORM, XML), Verite Group’s VOD-
powered Netscope application processed data captures twice as
fast with minimal computational resource needs.
Open, standards-based data management technologies have
discernibly more pedigree than alternative solutions in terms of
scaling to meet the volume, variety, velocity, veracity, and validity
needs of Big Data. And these abilities are not mutually exclusive.
After conducting extensive benchmark tests against several
different database technologies, both NoSQL and relational, under
multi-dimensional scenarios at different scales and speeds,
Versant’s object-based technologies again outperformed other
technologies by up to 1000 percent, and in 80 percent of the tests.
14. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 14 of 20
Image Caption: PolePosition benchmark testing shows how Versant’s technologies radically
outperform other data management solutions when processing Complex Concurrency
operations, which are highly relevant for utilities’ multi-faceted Big Data needs.
SECTION 5: HOW TO APPLY OBJECT-ORIENTED
DATA MANAGEMENT AND ANALYSIS
TECHNOLOGIES IN SMART GRIDS
As defined, the purpose of building open, object-oriented data
management systems that provide scalable situational awareness
for the smart grid is to identify anomalies in normal patterns in real-
time so that accurate decisions can be made, and the appropriate
actions taken, to positively affect the outcome of a business or
process.
One of the best examples of how Versant’s NoSQL - based data
15. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 15 of 20
management and analytics technology can address utilities‘ smart
grid situational awareness challenges stems from the infamous
U.S. blackout that occurred on August 14, 2003.
In 2006, the United States Department of Energy found that it was
not for lack of action that the outage happened. Plenty of decisions
were made, but, as the report also found, the blackout was due, in
part, to “lack of awareness of deteriorating conditions,” making it
impossible to know if the decisions that were being made were
accurate, and the actions being taken helpful. The report
recommended that a real-time measurement system and
computer-based operational and management tools be developed.
Within the guidance of the Electric Power Research Institute
(EPRI), research was begun to determine how such a system could
be created to connect the multiple utility providers’ grids and
provide the situational awareness to prevent such a wide-scale
problem from occurring again. To prove that it could be done, EPRI
and Versant created a working demonstration technology
architecture using sizeable simulated data samples from the hours
leading up to the blackout. By configuring the system according to
the requirements and open, standards-based architectures outlined
in this paper, the demo was able to provide advanced warning that
a problem event was nearing, based on the calculation of the
phase angular difference and its trend as measured from the data
in the synchrophasors. The following diagrams show how the
demonstration technology indicates the divergence well in advance
of the actual blackout:
Image Caption: There were three time windows where situational awareness would have
given sufficient time to adequately respond.
16. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 16 of 20
Image Caption: A screen shot from the actual demonstration interface shows that the phase
angular difference trend had fallen outside the acceptable range, and indicated that action
was needed, more than 35 minutes prior to the blackout.
The data management and analysis system architecture of the
demo was based on Versant’s new and fully-integrated software
development solution that has been applied in other industries:
Image Caption – Graphic representation of Versant’s fully integrated
software development toolkit.
Versant’s platform allows for an automated means to ingest,
manage, and analyze large, complex data volumes in real-time.
The approach is based on object-oriented programming, allowing
data to be modeled as objects and classified into object classes.
This produces the right match with the nature of data generated by
network topologies like those of the smart grid. The platform’s
ability to model data entities as schema according to unified
modeling language (UML) also corresponds with the UML
reference model for the electric utility established in the IEC CIM
17. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 17 of 20
standard, where interoperability between all network devices used
in a Smart Grid is also specified. Adherence to these standards
allows for a stochastic topological model to be established between
the devices via network configuration models and their associated
real-time data.
Versant’s development framework can provide solutions that
support the following objectives of the energy sector’s smart grid
initiatives:
1. Development, storage, retrieval, and management of
network configuration models like the PMU measurement
network or the smart meter measurement network with
NoSQL database technology while still leveraging industry
standard APIs.
2. Evolving the afore-mentioned models for suitability in
analytical methods for optimal and scalable situational
awareness.
3. Data ingestion from multiple sources, including
Hadoop/MapReduce, real-time streaming data monitors,
such as Twitter, utility data sensor networks, and
transactional data sources like those in the financial
services industry, and data virtualization sources, such as
classic ETL system data. The ingested data can have
further context added through semantic enrichment from
external data sources offering information on weather,
existing assets, and communication network performance.
4. Build a real-time model that depicts connectivity while
continuously ingesting new data to provide absolutely
current situational awareness.
5. Implement architectural scale patterns commonly found in
NoSQL technology to support both scale-up on modern n-
core process architectures, and horizontal scale-out
patterns in partitioned systems to manage utilities’ growing
data volumes.
6. Support NoSQL architectural patterns to enable real-time
data analysis for situational-awareness through in-database
analytics. The Versant In-Database Analytic Framework
embraces a number of key analytic features that can be
applied to identify patterns and reveal critical factors for
controlling power grid stability under multiple simultaneous
events.
Based on the availability of network configuration data, as well as
real-time data from smart meters, PMUs, or SCADA systems,
Versant can model and manage device configurations and ingest
and analyze real-time data. As a result, Versant has the capability
to provide the necessary platforms and interfaces for numerical
18. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 18 of 20
methods that allow for simulation and optimization at the scale of
utilities’ smart grid needs.
SECTION 6: BENEFITS OF SCALABLE
SITUATIONAL AWARENESS THROUGH OBJECT-
ORIENTED DATA MANAGEMENT AND ANALYSIS
FOR SMART GRIDS
In addition to being able to prevent large-scale and serious
problems like the August 2003 blackout, providing scalable
situational awareness via the technologies outlined in this paper
can realize the following key business benefits for utilities.
Asset Management –Routine maintenance and repairs to power
lines and other grid infrastructure account for a substantial portion
of utilities’ ongoing costs. With a sophisticated data management
system that enables advanced analytics like the one described in
this paper, fault locations can be more precisely identified and
characterized before a truck is even sent to fix it. This can also
allow utilities to determine if a truck and crew is needed to fix a
problem at all, resulting in immediate cost savings.
Power Usage Efficiency/Quality - Demand response and real-time
pricing are top priorities for utilities, but most only talk about it in
terms of meter data management and measuring kilowatt hours
used to enable more precise billing and energy flow. But power
quality at the residential and commercial building level needs much
more attention. If the data needed to measure this is built in to the
underlying data management model this can allow utilities to have
a much better understanding of power demand and supply
balance. But because each building is different, a database that
can scale with complexity and easily interoperate with buildings
systems is key.
Faster Smart Grid Roll-Out – The object-oriented approaches
outlined in this paper and exemplified by the demonstration
architecture created with EPRI enable the creation of a modular,
toolkit-like approach that enables shorter deployment time, much
like the benefits realized by the users of Echelon’s LNS system.
Open standards-based systems like these make the technologies
and applications understandable and more easily useable by a
broader group of people.
19. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 19 of 20
CONCLUSION
The potential power of scalable situational awareness through
object-oriented data management for the smart grid is very
substantial. Utilities are faced with the simultaneously large
challenges and opportunities of Big Data, which make achieving
scalable situational awareness harder, but also more important and
rewarding. By turning virtually every piece of utilities’ infrastructure
in to a sensor, and making them fully interoperable and controllable
with object-oriented data management technologies, utilities can
prevent outages, mitigate potential threats to the network, and
realize a range of other important business benefits. The energy
industry would do well to learn from other industries that have
already conquered the challenges of Big Data, and apply the
lessons they have learned to turn the current grid in to a smart grid.
REFERENCES AND ACKNOWLEDGEMENTS
The author would like to thank Paul Myrda, Dr. John Simmins,
Scott Sternfeld and Dr. Gerald Gray from the Electric Power
Research Institute, as well as Barry Haaser from LonMark
International, and Volker John, Andreas Renner, Vishal Bagga,
Victor Dreiling, Yaniv Schwerin, and Torben Rebhan from Versant’s
R&D team for their invaluable assistance and contributions to
various aspects of this paper’s knowledge base. He would also like
to express his sincere appreciation for the insightful comments
given by Robert Greene and Dr. Robert Brammer.
About Versant
Versant Corportation
(Nasdaq:VSNT) is an
industry leader in
building specialized
NoSQL data
management systems
to enable the r real-time
enterprise. Using the
Versant Database
Engine, enterprises can
handle complex
information in
environments that
demand high
performance,
concurrency, and
availability, significantly
cut hardware and
administration costs,
speed and simplify
development, and
deliver products with a
strong competitive
edge. Versant's
solutions are deployed
in over 150,000
installations across a
wide array of industries,
including
telecommunications,
energy, financial
services, transportation,
manufacturing, and
defense. For more than
20 years, Versant has
been a trusted partner
of Global 2000
companies such as
Ericsson, Verizon,
Siemens, and Financial
Times, as well as the
U.S. Government. For
more information, call
650-232-2400 or visit
www.versant.com.
20. Leveraging Big Data and Real-Time Analytics to
Achieve Situational Awareness for Smart Grids
Page 20 of 20
BIBLIOGRAPHY
1. Doug Laney, 3-D Data Management: Controlling Data
Volume, Velocity and Variety, February 2001
2. Codd, E.F. (1970). “A Relational Model of Data for Large
Shared Data Banks”. In: Communications of the ACM 13
(6): 377-387
3. Jeffery Taft, Paul de Martini, Leonardo von Prellwitz. “Utility
Data Management and Intelligence”. Cisco White Paper,
May 2012.
4. Alexandra von Meier (California Institute for Energy and
Environment – CIEE). “Electric Power Systems – A
Conceptual Introduction”. IEEE Wiley. 2006.
5. Don v. Dollen and Bert Taube. “Advanced Computational
Techniques for Situational Awareness and Analytics in
Power Grids.” (in cooperation with J. Simmins, P. Myrda, G.
Gray, S. Sternfeld, V. Bagga). Redwood City: Presentation
at Smart Grid Update's Conference “Data Management and
Analytics for Utilities”, June 2012.
6. Bert Taube. “How Object-Oriented Data Management
enables Smart Energy Control in Buildings.” Santa Clara:
Presentation at ConnectivityWeek, May 2012.
7. Paul Myrda, John Simmins and Bert Taube. “Advanced
Utility Analytics with Object-Oriented Database
Technology”. Accepted Paper to be presented at CIGRE
Forum in Lisbon, April 2013.
KEYWORDS
Situational awareness, Big Data, big utility data challenges,
scalability, smart grid, NoSQL-based data management and
analytics, energy data categories (telemetry, oscillography, usage
data, event messages, meta data), key characteristics of energy
data, synchrophasor (PMU), blackout, real-time utility enterprise,
five characteristics of Big Utility Data (volume, velocity, variety,
validity, veracity), back-end data systems integration, connectivity
and interoperability, control network, open standards, smart
energy-efficient buildings, analysis and simulation of grid
performance, power consumption forecast, grid security.