This document provides an overview of applications of big data in electrical energy systems. It discusses big data technology, power systems components, characteristics of big data such as volume, velocity, variety and veracity. It describes analyzing big data and various sources of data in power systems from intelligent electronic devices like smart meters, phasor measurement units and SCADA systems. It also discusses the role of big data in power systems and some applications like predictive maintenance, fault detection and renewable energy forecasting.
2. Applications of Big Data in Electrical Energy System
Page 2
List Of Contents Page no
1.Introduction 04
2.Big Data Technology 05
3.Power Systems 06
4.Characteristics of Big Data 07
5.Analysing Big Data 12
6.Sources of Data in Power System 14
7.Intelligent Electronic Devices 16
8.Role of Big Data in Power Systems 21
9.Advantages and Disadvantages 23
10.Applications 25
11.Conclusion 29
12.References 30
3. Applications of Big Data in Electrical Energy System
Page 3
List of Figures
Description Page no.
FIG.2.1: Big Data 05
FIG.3.1: Power Systems 06
FIG.4.1: Characteristics of Big Data 07
FIG.4.2: Volume 07
FIG.4.3: Velocity 08
FIG.4.4: Variety 09
FIG.4.5: Veracity 11
FIG.5.1: Analyzing of Big Data 12
FIG.7.1: Smart Meter 17
FIG.7.2: Phasor Measurement Unit 18
FIG.7.3: SCADA System 19
FIG.8.1: Big Data Monitoring System 21
FIG.10.1: Big Data in Education 25
FIG.10.2: Big Data in Health Care 26
FIG.10.3: Big Data in Retail Industry 26
FIG.10.4: Big Data in Media and Entertainment 27
FIG.10.6 Big Data in travel industry 27
4. Applications of Big Data in Electrical Energy System
Page 4
1. INTRODUCTION
With the fast development of digital technology and cloud computing, more and more data
are produced through digital equipment and sensors, such as smart phones, computers, advanced
measuring infrastructures, etc., as well as through human activities and communications. For
instance, the size of data on the internet is now measured in exabytes and zettabytes. Rational,
effective and efficient analysis of these data brings huge value and benefit to our daily life and
company activities. However, the collected data are mounting at an exponential growth, and the
structure of them is also becoming much more complicated. The processing and analysis method
of these large volume data is a new challenge but opportunity at the beginning of this century
with the concept of “big data”. Although big data is a newly-appeared term, the concept of
discovering valuable information from massive collected data in commercial operation as aiding
knowledge for business decision has already been proposed in 1989 by Howard Dresner as
“business intelligence” (BI). The trend of internet revolution and ubiquitous information
acquisition devices successfully reduce the cost of data collection, while the huge amount and
complex structure challenge the capability of traditional data analytics techniques.
In power grid, the traditional fossil fuels are facing the problem of depletion and the de-
carbonization demands the power system to reduce the carbon emission. Smart grid and super
grid are effective solutions to accelerate the pace for electrification of human society with high
penetration of renewable energy sources. Although the rising awareness of sustainable
development have become the impetus to the utilization of renewable energy sources, the
intermittent characteristics of wind and photovoltaic energies bring huge challenges to the safe
and stable operation in a low inertia power system. The data analytics based renewable energy
forecasting methods are a hot research topic for a better regulation and dispatch planning in such
cases. Traditional electricity meters in distribution systems only produce a small amount of data
which can be manually collected and analyzed for billing purpose. While the huge volume of
data collected from two-way communication smart grids at different time resolutions in
nowadays need advanced data analytics to extract valuable information not only for billing
information but also the status of the electricity network. For example, the high-resolution user
consumption data can also be used for customer behavior analysis, demand forecasting and
energy generation optimization. Predictive maintenance and fault detection based on the data
analytics with advanced metering infrastructure are more crucial to the security of power system.
Thus, the great progress of information and communication technology (ICT) provides a new
vision for engineers to perceive and control the traditional electrical system and makes it smart.
5. Applications of Big Data in Electrical Energy System
Page 5
2. BIG DATA TECHNOLOGY
Big data technology is an emerging technology which applies to data sets where data size is
so large and common data-related technology tools are hard to capture, manage, and operate
under multi-limits. Common data-related technology use a consistent, single, traditional data
management framework, which is only suitable for data with low diversity and volume. If
coming from diverse data sets, data will not be correlated in space and time, and it is hard for
those data to get correlated to a unified and generalized power system model. Therefore,
traditional data-related technology will no longer guarantee reliable data management, operation,
etc. In this way, big data technology provides integrated architectures for complex power grid,
effective data analysis to aware the unfavorable situation in advance, and various data processing
methods.
The big data evolution includes multiple stages known as megabyte (MB) to gigabyte (GB)
from the 1970 s to 1980 s, GB to terabyte (TB) from the late 1980 s to 1990, TB to petabyte (PB)
from the late 1990 s to around 2011 and PB to exabyte introduced in 2011.Based on various
definitions of big data, the current big data can be compared to traditional data in order to clarify
the characteristics of big data. At first, the total volume of traditional data is about GBs, whereas
in emerging big data it is around TBs or even PBs and continuously updating. The second
difference is based on the consideration that while the structure of traditional data is typical and
it is simply storable, big data contains several classified structures. Then, the rate of processing
and analyzing datasets corresponding to the rate of data generation is known as the big data
velocity characteristic. There are several real times or near real time applications that need very
fast processing of big data in the form of data streams.
Fig.2.1: Big Data
6. Applications of Big Data in Electrical Energy System
Page 6
3. POWER SYSTEMS
The power system is a network which consists generation, distribution and transmission
system. It uses the form of energy (like coal and diesel) and converts it into electrical energy.
The power system includes the devices connected to the systems like the synchronous generator,
motor, transformer, circuit breaker, conductor, etc.
The power plant, transformer, transmission line, substations, distribution line, and
distribution transformer are the six main components of the power system. The power plant
generates the power which is step-up or step-down through the transformer for transmission. The
transmission line transfers the power to the various substations. Through substation, the power is
transferred to the distribution transformer which step-down the power to the appropriate value
which is suitable for the consumers.
Fig.3.1: Power System
The main Components of power systems are:
• Supplies
• Loads
• Conductors
• Capacitors and reactors
• Power electronics
7. Applications of Big Data in Electrical Energy System
Page 7
4.CHARACTERISTICS OF BIG DATA
The characteristics of big data is defined by five major elements. These are Volume, Velocity,
Variety, Veracity and Value. These are 5Vs of big data.
Fig.4.1: Characteristics of Big Data
VOLUME
Big data has large volume of data. This is a consequence of data being built by large
number of deployed smart metering devices and a wide-range level of measurement. This
presents also new opportunities to detail analysis, but also a challenging task to store and
processing of data. Volume is the amount of data generated that must be understood to make
data-based decisions. A text file is a few kilobytes, a sound file is a few megabytes while a full-
length movie is a few gigabytes.
Fig.4.2: Volume
8. Applications of Big Data in Electrical Energy System
Page 8
EXAMPLE
• Amazon handles 15 million customers click stream user data per day to recommend
products.
VELOCITY
This refers to speed for collecting, processing and usage of big data of systems. Time
intervals of data collection should be as short as possible to obtain data in details. Processing
time should be short to decrease response time to an acceptable level. Power systems are
required to work in real-time or almost real-time manner for known reasons. An increased
number of data sources both machine and human generated drive velocity.
Fig.4.3: Velocity
EXAMPLE
• 72 hours of video are uploaded to YouTube every minute this is the velocity. Extremely
high velocity of data is another major big data characteristic.
VARIETY
This stands for having to consider different types of data. Structured or unstructured data
may be in consideration for a system due to different levels of recording and at different time.
Since standardization is reduced, processing algorithms have to adapt to this situation.
9. Applications of Big Data in Electrical Energy System
Page 9
Fig.4.4: Variety
STRUCTURED DATA
It is typically found in tables with columns and rows of data. The intersection of the row
and the column in a cell has a value and is given a “key,” which it can be referred to in queries.
Because there is a direct relationship between the column and the row, these databases are
commonly referred to as relational databases. A retail outlet that stores their sales data (name of
person, product sold, amount) in an Excel spreadsheet or CSV file is an example of structured
data.
EXAMPLE
A Product table in a database is an example of Structured Data
Product_id Product_name Product_price
1 Pen $5.95
2 Paper $8.95
10. Applications of Big Data in Electrical Energy System
Page 10
<product>
<name>Pen </name>
<price>$7.95</price>
</product>
<product>
<name>Paper </name>
<price>$8.95</price>
</product>
SEMI-STRUCTURED DATA
It also has an organization, but the table structure is removed so the data can be more easily
read and manipulated. XML files or an RSS feed for a webpage are examples of semi-structured
data.
EXAMPLE
UNSTRUCTURED DATA
Unstructured data generally has no organizing structure, and Big Data technologies use
different ways to add structure to this data. Typical example of unstructured data is, a
heterogeneous data source containing a combination of simple text files, images, videos etc.
EXAMPLE
Output returned by ‘Google Search‘
VERACITY
This represents uncertainty of data. Especially in power systems, sometimes generation,
sometimes demand and mostly both of them is related to natural events. Overall system of
processing of data must assure to regard this veracity. Fig 1. shows major elements of big data
concept.
11. Applications of Big Data in Electrical Energy System
Page 11
Fig.4.5: Veracity
12. Applications of Big Data in Electrical Energy System
Page 12
5. ANALYSING BIG DATA
The big data system architecture can be presented as a value chain consisting of four
phases: 1) generation; 2) acquisition; 3) storage; and 4) processing. During these four phases,
the raw data generated by various data sources are processed and changed into useful
information for different control and management purposes.
Fig.5.1: Analyzing of Big Data
DATA GENERATION
This phase refers to the big data generation processes and the sources of big data with
various types, characteristics, and origins.
DATA ACQUISITION
This phase concerns big data aggregation to obtain and classify the resulted information
for further phases. Data collection, transmission, and preprocessing are the most significant
13. Applications of Big Data in Electrical Energy System
Page 13
aspects of this phase.
1.Data collection is the process of raw data retrieval from data generation sources.
Specially, sensors and measurement instruments are considered as the main data sources in
SGs
2.Data transmission refers to the data transfer into data centers (DCs) for storage and
process using communication infrastructures.
3.Data preprocessing, an important aspect of data acquisition process, is considered to
integrate 1and cleanse errors and eliminate data redundancy and compress.
DATA STORAGE
This phase aims to store and manage big data for further processes and applications.
DATA ANALYSIS
This phase facilitates the analytic approaches to analyze the gathered data by using inspection
and modeling methods to prepare classified and extracted information from the collected raw
data. This phase is the most important stage in big data system aiming at useful information
extraction for further decision-making processes. Big data analytics can be categorized into
descriptive, predictive, and prescriptive analytics based on the depth of analysis. Big data
analysis methods can be classified into data visualization, statistical analysis, machine learning
(ML), and DM.
14. Applications of Big Data in Electrical Energy System
Page 14
6. SOURCES OF DATA IN POWER SYSTEM
Integrating telecommunication, automation and electric network control, smart grid
requires reliable real-time data processing. To support this requirement and benefit future
analysis and decision making, huge amount of historic data should be well fetched and stored
in a reasonable time budget. In addition, there are various sources that huge amount of data
can be generated through diverse measurements acquired by Intelligent Electronic Devices
(IEDs) in the smart grid:
• Data from power utilization habits of users
• Data from Phasor Measurement Units (PMUs) for situation awareness
• Data from energy consumption measured by the widespread smart meters
• Data from energy market pricing and bidding collected by Automated Revenue Metering
(ARM) system
• Data from management, control and maintenance of device and equipment in the electric
power generation, transmission and distribution in the grid
• Data from operating utilities, like financial data and large data sets which are not directly
obtained through the network measurement.
The data volume of electric power industry rises up sharply with the comprehensive
construction of intelligent, digital, informatized power grid. Construction and operation
of power grid are the core business of electric power enterprises. These enterprises’ big
data can be divided into three classes, including power grid operation data, customer data
and enterprise management data. Power grid operation data comes from all aspects of
power generation, transmission, substation, distribution, utilization and scheduling, such
as power flow data, grid operation key indicators, the devices detecting or monitoring
data, etc. Customer data includes electricity consumption behavior, business expansion
information and other related marketing data. Enterprise management data consists of
human, financial and material resources management data. According to the different
data sources, electric power big data can be divided into power grid data and external
data two categories.
15. Applications of Big Data in Electrical Energy System
Page 15
Electric power enterprises big data belongs to power grid data. Electric power big
data has many resources and mainly comes from the following information collection and
management systems: supervisory control and data acquisition.
(SCADA) system, energy management system (EMS), wide area measurement system
(WAMS), operation management system (OMS), automatic dispatching system,
distribution management system (DMS), electricity consumption information collection
system, advanced metering infrastructure (AMI), tele-meter reading
(TMR), power quality monitoring system, marketing business system, 95598 customer
service system, electricity trading platform, wind power and photovoltaic power
prediction system, production management system (PMS), management information
system, enterprise resource planning (ERP), geographic information system (GIS),
weather forecast system (WFS) and so on.
The existing information collection and management systems are relatively isolated
because these systems come from different departments and the data management of
different departments is isolated. The electric power big data exists problems of multi-
source heterogeneity, information redundancy, different time granularity, inconsistency
of statistical model and uneven data quality. These problems will pose
challenges to the integrated management, analysis and processing of electric power
big data.
16. Applications of Big Data in Electrical Energy System
Page 16
7. INTELLIGENT ELECTRONIC DEVICES
1.SMART METERS
A smart meter is an electronic device that records consumption of electrical energy and
communicates the information to the electricity supplier for monitoring and billing. Smart meters
typically record energy hourly or more frequently, and report at least daily. Smart meters enable
two-way communication between the meter and the central system. Such an advanced metering
infrastructure (AMI) differs from automatic meter reading (AMR) in that it enables two-way
communication between the meter and the supplier. Communications from the meter to the
network may be wireless, or via fixed wired connections such as power line carrier (PLC).
IN-HOME DISPLAY YOU'LL BE ABLE TO READ
1.How much energy you’re using in near real time.
2.How much energy was used in the last hour, week and month (and what it cost).
3.Whether your electricity use is high, medium or low.
4.Updates in near real time for electricity and every half hour for gas.
Advantages
1. No need to submit meter readings.
2. You can closely track your usage and spend.
3. Accurate bills- no more estimates.
4. Highlights faulty appliances.
5. Greater selection of tariffs on offer.
Disadvantages
1.Older smart meters become “dumb” once you switch.
2.In-Home Display may be inaccurate.
17. Applications of Big Data in Electrical Energy System
Page 17
7.1 : Smart Fig Meter
2. PHASOR MEASUREMENT UNIT (PMU)
A phasor measurement unit (PMU) is a device used to estimate the magnitude and phase
angle of an electrical phasor quantity (such as voltage or current) in the electricity grid using a
common time source for synchronization. PMUs are capable of capturing samples from a
waveform in quick succession and reconstructing the phasor quantity, made up of an angle
measurement and a magnitude measurement. PMUs can also be used to measure the frequency in
the power grid. A typical commercial PMU can report measurements with very high temporal
resolution in the order of 30-60 measurements per second.
1. Power system automation, as in smart grids
2. Load shedding and other load control techniques such as demand response mechanisms to
manage a power system. (i.e. Directing power where it is needed in real-time)
3. Increase the reliability of the power grid by detecting faults early, allowing for isolation of
operative system, and the prevention of power outages.
4. Increase power quality by precise analysis and automated correction of sources of system
degradation.
18. Applications of Big Data in Electrical Energy System
Page 18
5. Wide area measurement and control through state estimation, in very wide area super grids,
regional transmission networks, and local distribution grids.
6. Phasor measurement technology and synchronized time stamping can be used for Security
improvement through synchronized encryptions like trusted sensing base. Cyber-attack
recognition by verifying data between the SCADA system and the PMU data.
7. Distribution State Estimation and Model Verification. Ability to calculate impedances of
loads, distribution lines, verify voltage magnitude and delta angles based on mathematical state
models.
8. Event Detection and Classification. Events such as various types of faults, tap changes,
switching events, circuit protection devices. Machine learning and signal classification methods
can be used to develop algorithms to identify these significant events.
9. Microgrid applications––islanding or deciding where to detach from the grid, load and
generation matching, and resynchronization with the main grid.
APPLICATIONS
1.Oscillation detection.
2.Frequency stability monitoring.
3.Voltage stability monitoring.
Fig7.2: Phasor Measurement Unit
19. Applications of Big Data in Electrical Energy System
Page 19
3.SCADA
A SCADA system is a common industrial process automation system which is used to
collect data from instruments and sensors located at remote sites and to transmit data at a central
site for either monitoring or controlling purpose. The collected data from sensors and instruments
is usually viewed on one or more SCADA host computers that are located at the central site.
Based on the information received from the remote stations, automated or operator-driven
supervisory commands can be pushed to remote station control devices, which are often referred
to as field devices.
As the power system deals with power generation, transmission and distribution sectors,
monitoring is the main aspect in all these areas. Thus, the SCADA implementation of power
system improves the overall efficiency of the system for optimizing, supervising and controlling
the generation and transmission systems. SCADA function in the power system network
provides greater system reliability and stability for integrated grid operation.
Fig7.3: SCADA System
systems are crucial for industrial organizations since they help to maintain efficiency, process
data for smarter decisions, and communicate system issues to help mitigate downtime.
The basic SCADA architecture begins with programmable logic controllers (PLCs) or remote
terminal units (RTUs). PLCs and RTUs are microcomputers that communicate with an array of
20. Applications of Big Data in Electrical Energy System
Page 20
objects such SCADA as factory machines, HMIs, sensors, and end devices, and then route the
information from those objects to computers with SCADA software. The SCADA software
processes, distributes, and displays the data, helping operators and other employees analyze the
data and make important decisions.
For example, the SCADA system quickly notifies an operator that a batch of product is
showing a high incidence of errors. The operator pauses the operation and views the SCADA
system data via an HMI to determine the cause of the issue. The operator reviews the data and
discovers that Machine 4 was malfunctioning. The SCADA system’s ability to notify the
operator of an issue helps him to resolve it and prevent further loss of product.
APPLICATIONS
1. Control industrial processes locally or at remote locations.
2. Monitor, gather, and process real-time data.
3. Directly interact with devices such as sensors, valves, pumps, motors, and more through
human-machine interface (HMI) software.
4. Record events into a log file.
21. Applications of Big Data in Electrical Energy System
Page 21
8. ROLE OF BIG DATA IN POWER SYSTEMS
The data obtained from PMUs, smart meters, sensors and other IEDs have opened up a
plethora of chances, such as predictive analytics, real-time vulnerability assessment, theft
detection, demand-side-management, economic dispatch, energy trading, etc. Big data can help
improve the smart grid management to a higher level. For example, by enhancing the
accessibility of a customer's electricity consumption data, the demand response will be expanded
and energy efficiency will be improved. Similarly, analyzing data obtained from PMUs and
IEDs will help to improve customer service, prevent outages, maximize safety and ensure
service reliability. Furthermore, electric utilities are using prediction data analytics for estimating
several parameters which are helpful to operate the smart grid in an efficient, economical and
reliable way. For instance, whether the excess energy is available from renewable sources can be
predicted through accessing the ability of the smart grid to transmit it.
Fig.8.1: Big Data Monitoring System
22. Applications of Big Data in Electrical Energy System
Page 22
At the same time, calculating equipment downtime, accessing power system failures, and
managing unit commitment can also be done effectively in high correlation with integrating
distributed generation. Thus, the grid management state will be enhanced, the efficiency will be
improved and the robustness of generation and deployment will be ensured. With the integration
of various distributed generating sources, refined forecasting, load planning, and unit
commitment are more convenient to avoid inefficient energy transmitting or dispatching extra
generation.
The Role of Big Data Analytics in Smart Grid Communications Because a Smart Grid can
be described as a huge sensor network, with a lot of intelligent devices, the growth in the number
of devices will produce a considerable amount of measured data. How to quantify and to analyze
these data to enhance grid operation arises as one big concern. Advances of the Smart Grid
promise to give operators and utilities a better understanding of customer behavior, demand
consumption, weather forecast, power outages, and failures. However, it is vital to quantify the
volume of sampled data to take advantage of them. Therefore, this chapter aims to characterize
and to evaluate the emerging growth of data in communications network applied to Smart Grid
scenario. A future active distribution system will serve as an example to demonstrate the data
requirements for monitoring and controlling the grid.
23. Applications of Big Data in Electrical Energy System
Page 23
9. ADVANTAGES AND DISADVANTAGES
ADVANTAGES
POWER GENERATION MANAGEMENT
According to the existing literature, the power generation management is one of the
important application areas of big data analytics in SGs. The big data analytics is applicable to
the power generation planning and operation to facilitate decision-making processes. The
applications of big data in power generation planning, economic load dispatch, power generation
and storage systems performance and efficiency analyses, and power grid optimization and cost
minimization are the outstanding aspects of studies in this scope.
RENEWABLE ENERGY RESOURCES AND MICROGRID MANAGEMENT
The big data analytics is a promising technology able to improve the forecasting,
management, and processing procedures of renewable energy resources (RERs) integration and
microgrid management issues in SGs.
LOAD FORECASTING
Load modeling and forecasting represent remarkable applications of big data
POWER SYSTEM MONITORING, CONTROL, PROTECTION, AND MANAGEMENT
The AMI represent the main source of the data in SGs. The data obtained from PMUs, smart
meters, sensors, and other IEDs, referred to as control signals, can be used for different purposes,
such as customer service enhancement, predictive analytics, fault diagnosis and outages
prevention, real-time vulnerability assessment, theft and cyber-attacks detection and safety, and
security and reliability improvement. System monitoring including events and state estimation
based on AMI and smart devices, wide area situational awareness, islanding detection,
oscillation detection, real -time rotor angle monitoring.
DISADVANTAGES
CYBER-SECURITY
The cyber-security is a concept dealing with the security issues including availability, integrity,
privacy, auditability, authentication, authorization, confidentiality, and trust within the SG.
24. Applications of Big Data in Electrical Energy System
Page 24
10. APPLICATIONS
I.BIG DATA IN EDUCATION
Big data is the key to shaping the future of the people and has the power to transform the
education for better. It is not only rekindling academic skills but also the non-academic ones
such as inter-personal skills. It is providing assistance in evaluating the performance of both the
teachers as well as the students.
Fig.10.1: Big data in education
Following are some of the fields in the education industry that have been transformed by big
data changes:
1. Customized and Dynamic Learning Programs
2. Reframing Course Material
3. Grading Systems
4. Career Prediction
II.BIG DATA IN HEALTHCARE
Big data and healthcare are an ideal match. It complements the healthcare industry better than
anything ever will. The amount of data the healthcare industry has to deal with is unimaginable.
Following are some of the ways in which big data has contributed to healthcare:
1.Big data reduces costs of treatment since there is less chances of having to perform
unnecessary diagnosis.
2. It helps in predicting outbreaks of epidemics and also in deciding what preventive
measures could be taken to minimize the effects of the same.
25. Applications of Big Data in Electrical Energy System
Page 25
3.It helps avoid preventable diseases by detecting them in early stages. It prevents them from
getting any worse which in turn makes their treatment easy and effective.
4.Patients can be provided with evidence-based medicine which is identified and prescribed after
doing research on past medical results.
Fig.10.2. Big data in healthcare
III.BIG DATA IN RETAIL INDUSTRY
Even a minute detail about any customer has now become significant for them. They are now
closer to their customers than they have ever been. This empowers them to provide customers
with more personalized services and predict their demands in advance.
Fig.10.3: Big data in retail industry
26. Applications of Big Data in Electrical Energy System
Page 26
IV.BIG DATA IN MEDIA AND ENTERTAINMENT
Keeping a customer pleased is a lifelong journey for the media and entertainment industry. They
must offer their customers with new content to keep them engaged with their firm. The
recommendation engine plays an important role here as well.
1.Predicting the interests of audiences
2.Optimized or on-demand scheduling of media streams in digital media distribution
platforms
3. Getting insights from customer reviews
4.Effective targeting of the advertisements
Fig.10.4: Big data in media and entertainment
V.BIG DATA IN TELECOM INDUSTRY
With the ever-increasing popularity of smart phones, it has flooded the telecom industry with
massive amounts of data. And this data is like a goldmine, telecom companies just need to know
how to dig it properly. Through big data companies are able to provide the customers with
smooth connectivity.
VI.BIG DATA IN TRAVEL INDUSTRY
Through big data, travel companies are now able to offer more customized travelling experience.
They are now able to understand their customer’s requirements in a much-enhanced way.
Fig.10.6: Big data in travel industry
27. Applications of Big Data in Electrical Energy System
Page 27
VII. BIG DATA IN BANKING SECTOR
1.Misuse of credit/debit cards
2.Venture credit hazard treatment
3.Business clarity
4.Customer statistics alteration
5.Risk mitigation
28. Applications of Big Data in Electrical Energy System
Page 28
11. CONCLUSION
Big data analytics will equip electric utilities with the solution for the analysis of huge
amount of data but there are also processes where conventional methods will still be useful for
answering questions. However, conventional models have to be updated with new technologies
and techniques, either using a service approach for an analytics platform or developing a
dedicated platform. While exploring the latter approach, we found that developing an analytical
platform requires time, knowledge and resources to define analytical models; to design,
implement and maintain the platform; and to develop analytics tools. In so doing, knowledge and
experience of a multi-disciplinary team is required, including information technology staff, data
scientists, and utility engineers. Apache Hadoop framework, its ecosystem and Apache Spark are
open-source tools that provide all functionality to develop big data analytics applications.
However, complementary software tools have to be developed in order to implement big data
solutions, mainly at the extremes of the flow of information, integration and visualization stages.
The integration stage is facilitated when the utility has already been modelled for interoperability
based on common information model and enterprise service bus. At the visualization stage,
further development of tools is needed in order to facilitate the implementation of applications to
visualize analytical results. In addressing analytics and data processing requirements of the
power utilities, a number of applications have been suggested in this paper. The analytical
platform in conjunction with these applications should capable of processing data in batch,
streaming and interactively. Although the value of big data analytics depends on the ability to
design analytic models and an iterative process to extract valuable insights and situational
awareness to support decision making, this has to be complemented with engineering for
improving electric utility processes.
29. Applications of Big Data in Electrical Energy System
Page 29
12. REFERENCES
[1] Zhang, Y., Huang, T., Bompard, E. F., “Big data analytics in smart grids a review”,
Energy Informatics, pp. 1-8, 2018
[2] Abhisek, U., Zivanovic, R., “Automated Analysis of Power Systems Disturbance Records:
Smart Grid Big Data Perspective”, IEEE Innovative Smart Grid Technologies,2015.
[3] Sagiroglu, S., Terzi, R., Canbay, Y., Colak, I., “Big Data Issues in Smart Grid Systems”,
IEEE International Conference on Renewable Energy Research and Applications
(ICRERA),2016.
[4] Günther, W. A., Mehrizi, M. H. R., Huysman, M., Feldberg, F., “Debating big data: a
literature review on realizing value from big data”, Journal of Strategic Information
Systems, 26:191–209, 2017.
[5] Yu, N., Shah, S., Johnson, R., Sherick, R., Hong, M.,Loparo, K., “Big Data Analytics in
Power Distribution Systems” IEEE Power & Energy Society Innovative Smart Grid
Technologies Conference, 2016.
[6] Lai, C. S., Lai, L. L., “Application of Big Data in Smart Grid”, IEEE International
Conference on Systems, Man, and Cybernetics, 2015.
[7] “Big data in the cloud: Converging technologies”, IT Center, Intel, 2015.
[8] http://en.wikipedia.org/wiki/Quantum_computing, 2015.
[9] https://www.smartgrid.gov/the_smart_grid/smart_gri d.html, 2019.
[10] Feng,G.“Frameworks for Big Data Integration, Warehousing, and Analytics”, chapter 4 in
“Big Data Application in Power Systems”, edited by Arghandeh, R. and Zhou, Y.,
Elsevier, 2018.
[11] Liu Q., Cui L., Chen H., “Key technologies and applications of internet of things”,
Computer Science 37 (6), 2010.
[12] Yu, N., Shah, S., Johnson, R., Sherick, R., Hong, M. &Loparo, K., “Big Data Analytics in
Power Distribution Systems”, IEEE Power & Energy Society Innovative Smart Grid
Technologies Conference, 2016.
[13] Peppanen, J., Reno, M. J., Broderick, R. J., Grijalva, S., “Distribution System Model
Calibration With Big Data From AMI and PV Inverters”, IEEE Transactions on Smart
Grid,