The electric grid has evolved from linear generation and delivery to a complex mix of renewables, prosumer-generated electricity, and electric vehicles (EVs). Smart meters are generating loads of data. As a result, traditional forecasting models and technologies can no longer adequately predict supply and demand. Extreme weather, an aging infrastructure, and the burgeoning worldwide population are also contributing to increased outage frequency. In oil and gas, commodity pricing pressures, resulting workforce reductions, and the need to reduce failures, automate workflows, and increase operational efficiencies are driving operators to shift analytics initiatives to advanced data-driven applications to complement physics-based tools. While sensored equipment and legacy surveillance applications are generating massive amounts of data, just 2% is understood and being leveraged. Operationalizing it along with external datasets enables a shift from time-based to condition-based maintenance, better forecasting and dramatic reductions in unplanned downtime. The session includes plenty of real-world anecdotes. For example, how an electric power holding company reduced the time it took to investigate energy theft from six months to less than one hour, producing theft leads in minutes and an expected multi-million dollar ROI. How a global offshore contract drilling services provider implemented an open source IIoT solution across its fleet of assets in less than a year, enabling remote monitoring, predictive analytics and maintenance. Key takeaways: • How are new processes for data collection, storage and democratization making it accessible and usable at scale? • Beyond time series data, what other data types are important to assess? • What advantage are open source technologies providing to enterprises deploying IIoT? • Why is collaboration important across industrial verticals to increase IIoT open source adoption? Speaker Kenneth Smith, General Manager, Energy, Hortonworks

1. 1 © Hortonworks Inc. 2011–2018. All rights reserved.
Kenneth Smith – General Manager, Energy & Utilities and Oil & Gas
Wade Salazar - Senior Solutions Engineer, Hortonworks
IIoT & Predictive Analytics: Solving
for Disruption in O&G and E&U

2. 2 © Hortonworks Inc. 2011–2018. All rights reserved.
Industrial IoT Market Opportunity Estimates
“In other words, the industrial internet will be worth more than twice the consumer internet”
https://www.forbes.com/sites/louiscolumbus/2016/11/27/roundup-of-
internet-of-things-forecasts-and-market-estimates-2016/#ad68a67292d5

3. 3 © Hortonworks Inc. 2011–2018. All rights reserved.
OT / IT Convergence – Must Occur to Achieve Business Improvement
Source: IBM

4. 4 © Hortonworks Inc. 2011–2018. All rights reserved.
Internal Challenges – The Missing Middle
Source: Accenture
In many companies, a breach—
the missing middle—is evident
in multiple dimensions:
between the data available and
disparate systems used; from
the lack of end-to-end
integration across processes or
workflows; and between
corporate strategies and
analytics efforts at functional
and departmental levels. With
this gap, energy companies
struggle for a complete and
timely assessment of the
impact of operational decisions
on corporate performance.
Likewise, corporate entities are
unable to factor in day-to-day
field operations in their
objective setting and planning
decisions.

5. 5 © Hortonworks Inc. 2011–2018. All rights reserved.
Why Open Source for IIoT?
• Community driven innovation to develop an end-to-end OPEN SOURCE IoT data platform for
“industrials”
• It’s not just about time-series data; it’s the ability to collect, manage, and analyze all
pertinent structured & unstructured data sets related to an industrial asset, operation,
process, piece of equipment, etc. in in addition to time-series
• Enables OT/IT/ET convergence to build descriptive, predictive, & prescriptive applications
• Cost effective storage and parallel processing of large data sets
• An open source IIoT platforms allow operators to maintain control over their data and
analytics vs. a ”closed” OEM’s IIoT product telling them when their own equipment needs
replacing
• An open IIoT platform is applicable across all asset intensive industries with “moving metal”;
oil & gas, utilities, mining, manufacturing, automotive, transportation, agriculture, etc.
• Future Proof - Open source eliminates vendor lock-in and de-risks adoption
• “Data is not a competitive advantage. It’s the algorithms you build to analyze your data
that will differentiate you from your competitors.”

6. 6 © Hortonworks Inc. 2011–2018. All rights reserved.
Is the Energy Industry Ready to Embrace an Open Model?
http://www.lockheedmartin.com/us/news/press-releases/2016/january/160114-mst-us-exxonmobil-awards-
lockheed-martin-next-generation-refining-and-chemical-facility-automation-system-contract.html
ExxonMobil representatives
express frustration when
observing step change
improvements in adjacent
industries enabled by open
technologies. Those adjacent
industries have deployed
significantly higher function
software that have lowered
lifecycle cost and delivered
higher return on investment.
The explosive growth of technologies driven by the Internet of Things (IoT) including
cloud computing, mobile computing, embedded computing, and consumer electronics
makes it obvious that the mainstream industrial automation industry can deliver
more value with the adoption of an open, multi-vendor platform approach.
http://www.automation.com/automati
on-news/article/exxonmobil-to-build-
next-generation-multi-vendor-
automation-architecture

7. 7 © Hortonworks Inc. 2011–2018. All rights reserved.
Upstream O&G Companies Digital Technology Focus & Investments?
Source: https://www.accenture.com/us-en/insight-2017-upstream-oil-gas-digital-trends-survey
Source: DZone
With a modern IIoT and cloud platform underlying the
next generation of applications and analytics, the oil and
gas industry can move beyond just doing the same thing
faster or cheaper and adopt new levels of productivity.
Mostly seeing “bridging the gap”
brownfield use-cases

8. 8 © Hortonworks Inc. 2011–2018. All rights reserved.
E&U Industry Data & Analytics Investment by Line of Business
While the trend toward more open source technologies as part of a
utility’s analytics strategy exists across the board, there are some
differences when looking at the use of open source in large utilities (over 1
million customers) versus small utilities (50,000 to 1 million customers).
For instance, large utilities are more than three times as likely to depend
on Hadoop data storage to a moderate, large or very large extent.
SAS Survey: Utility analytics in 2017: Aligning data and analytics with business strategy

9. 9 © Hortonworks Inc. 2011–2018. All rights reserved.
Connected Data Platforms Enables IIoT in Energy & Utilities
Source: https://www.cm-collaborative-tech.com/wp-content/uploads/2016/11/Smart-grid-A-1.jpg
Predictive MaintenanceFraud DetectionExternal Sources
(Weather, Social
Media, GPS, etc.)
Single View of Customer

10. 10 © Hortonworks Inc. 2011–2018. All rights reserved.
Data Acquisition

11. 11 © Hortonworks Inc. 2011–2018. All rights reserved.
Time Series Data is Emerging as the Fastest Growing Type of Data

12. 12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Title Goes HereThe Data Management Challenge for Energy Companies
Data is Siloed
• Datasets are dispersed
and difficult to access
• Insights are based
sourced from the silos
are narrow & incomplete
• Historical data (years) is not
easily accessible and often
takes longer than expected to
extract
• Data required to solve real
problems comes from several
different sources (lab systems,
product scheduling) and
requires significant manual
effort to pull the data together
Considerable effort
to leverage
• Lack of connectivity to
proper data analytics tools
• Python
• MATLAB
• SAS
• R
• Inability to find data and a
reliance on “tribal
knowledge” and previous
engineer’s spreadsheets to
find tags and queries
Data Analytics
Missing
Lack of Resources
Inability to Leverage BOTH OT and IT Data Sources
• Time dependence limits
how data sets can be
processed
• Point solutions are
almost always closed
source, locking users
into closed ecosystems
• Do not scale with
exponential data growth

13. 13 © Hortonworks Inc. 2011–2018. All rights reserved.
Highest Value Data
Always on, always connected devices generate a
constant stream of data related to the operations of
industrial businesses
These datasets contain:
• What events occurred
• Why and event occurred, or not
• Quantification of an event’s impact
These datasets go by many names:
• “SCADA Data”
• “Control System Data”
• “Historian Data”
• “Machine Data”
• “Measurement Logs”
• “Telemetry”
How are my …
People?
Processes?
Equipment?
Lots of misnomers

14. 14 © Hortonworks Inc. 2011–2018. All rights reserved.
Stepwise Approach to the Challenge
Remote Field or Manufacturing Site
RDBMS & EDW
Files / Other Unstructured Data
Video
IoT Gateways
WITSML
SCADA, DCS, PLC, RTU, Historians
Location 1
Data Consumers
Data
Marts
Analytics,
Statics &
Science
Visualization
& Dashboards

15. 15 © Hortonworks Inc. 2011–2018. All rights reserved.
Data Lakes Address Part of the Problem
Field or Manufacturing Site
RDBMS & EDW
Files / Other Unstructured Data
Video
IoT Gateways
WITSML
SCADA, DCS, PLC, RTU, Historians
Location 1
Data Consumers
Data
Marts
Analytics,
Statics &
Science
Visualization
& Dashboards

16. 16 © Hortonworks Inc. 2011–2018. All rights reserved.
Open Connected Platform Approach Addresses the End to Challenge
Field or Manufacturing Site
RDBMS & EDW
Files / Other Unstructured Data
Video
IoT Gateways
WITSML
SCADA, DCS, PLC, RTU, Historians
Location 1
Data Consumers
Data
Marts
Analytics,
Statics &
Science
Visualization
& Dashboards

17. 17 © Hortonworks Inc. 2011–2018. All rights reserved.
Instrumentation
 Commonly only output is
electrical signals
 Integration with sensors
requires specialized
hardware
 serial bus, or wireless are
increasingly available
Challenges in Accessing Data in the ICS Landscape
Control Systems
 Data is transmitted via
proprietary vendor specific
protocols
 Direct Integration with
control systems requires
protocol translation/parsing
for each platform family
Nifi’s is a toolbox of connectors
 Ingest text files and interrogate REST APIs
using built in connectors
 Connect to industry standard protocols like
OPC UA with custom processors
 Build your own
Existing ICS Components
PLC, RTU & DCS
Open Source Tools
Governance
&Integration
Security
Operations
Data Access
Data Management
Process Historians &
OPC Servers
 Data is typically available via
programmatic access such
as OPC, API or SQL
 There is almost always an
option to create text files

18. 18 © Hortonworks Inc. 2011–2018. All rights reserved.
Actual Use Case Results

19. 19 © Hortonworks Inc. 2011–2018. All rights reserved.
Typical Goals for an Industrial Analytics Practice
• Data democratization ( broad simple access )
• Event processing – create events or react to variables (e.g. pump
overheat, weather, emission)
• Forecasting / Prediction - Predict the most likely value
• Event Correlation – Measure the coincidence of two things? Measure the
likeness of events or periods of time?
• Impute missing values - What are the most likely values of missing data?
• Anomaly detection – Find “out of normal” events in a series, based on a
model of expected behavior

20. 20 © Hortonworks Inc. 2011–2018. All rights reserved.
Time Series Analytics for Power Generation Anomaly Detection
 Two week engagement – no direct knowledge of existing systems
 Two days were able to isolate problem down from 5000 potential
causes to 19 using standard data science algorithms
 Company investigated findings and found a valve was installed
backwards causing plant to shutdown
 Plant failure hasn’t occurred since, saving millions of dollars in
unplanned shutdowns
 VP of Engineering – “I never thought we would see a solution like
this”

21. 21 © Hortonworks Inc. 2011–2018. All rights reserved.
Vertically Integrated Utility’s Data Journey
Accelerating Revenue Protection with an Open Analytics Platform
 One of the largest electric power holding companies in the US that supplies electricity to approximately 7.4 million
customers and operates natural gas distribution services serving more than 1.5 million customers.
 Revenue Protection Use Case: Protect revenue from theft, malfunctioning meters, and misconfigured meters.
 Why HDP: The only cost effective platform able to do parallel / multi-node analytics on large data sets.
 Currently have loaded 200 Billion rows of meter data across 80 nodes of HDP growing to 1.4 Trillion by 2020 from all of
their service areas.
 Previous energy theft data science process: Predictive model was run on a laptop 1x per week for 10K accounts at a time
and produced 100 leads weekly for investigation. At that rate, it would have taken them 6 months to process one state’s
data (all states/enterprise data would take much longer)
 Current process: Leveraging HDF & HDP to ingest, process, store, and analyze 5 minute meter data from Itron Open Way
 Realized business value from the Revenue Protection use-case $17.5M in 2017, goal of $30M for 2018.
 Other use-case include predictive equipment maintenance on nuclear power & solar generation, “Next Best Action”
program for cross-selling opportunities on goods and service, amongst others.

22. 22 © Hortonworks Inc. 2011–2018. All rights reserved.
Using HDP and HDF for Industrial IoT – Rowan Companies
Requirement – A New Business Model:
• Fluid and flexible data platforms that can quickly integrate raw data
and deliver actionable intelligence to people and processes
• Ability to operate when network connectivity with a data center or
the shore is intermittent, latent and provide minimal bandwidth
• Analysis of large volumes of data and avoid data being stranded and
out of reach for analysts and support teams.
• Move from an operations posture of reacting and suffering from
unnecessary downtime, equipment failures, efficiency losses, and
safety risks
• Bring the data that increases the collective expertise available to
support safer and more efficient operations
Solution and Outcomes – New Sources of Value:
• HDF aggregates, prioritizes, compresses and encrypts control system
data before sending it over a 64 kb/sec satellite link to the data
center in real-time
• Data from top drives, BOPs and other equipment is in HDP and every
data consumer from data scientist to BI users can be serviced from
their tool of choice
• With predictive analytics and maintenance forecasting, Rowan
expects to reduce downtime and alleviate future troubleshooting
trips to the rigs.
• Rowan will be able to comply with the important BSEE regulations
going into effect in 2019.

23. 23 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Title Goes HereFrom Refinery to Enterprise Level Analytics
Problem: Refinery-level analytics sub-optimizes performance
 Analytics performed at each refinery in Excel spreadsheets
 Missed opportunities for optimization based on larger data sets
Solution: Centralize data in Manufacturing Data Lake for analytics
 Ingest data from each refinery using with HDF into centralized Data Lake
 Initial data set was over 1 million data tags, grew to 6 million
 Data Types: Time series, raw materials, quality results, SAP work order
data, etc.
Benefits: Enterprise level-analytics to optimize performance
 ROI Analysis
 $106 million in cost savings per year
 20X ROI annually
Oil Refining
Multinational Oil & Gas
Company
Core Use Cases
• Blend Monitoring
• Corrosion Prediction
• Analyzer Reliability Analytics
• Heat Exchanger
Performance Analytics
• Inferential Models Analytics

24. 24 © Hortonworks Inc. 2011–2018. All rights reserved.
Open source is a way to enable a group of
collaborative people to further their
individual interests while contributing back to
the community for the common good.
Open source

25. 25 © Hortonworks Inc. 2011–2018. All rights reserved.
Questions? How can we
help you get started?