Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Oil & Gas Big Data use cases

2,815 views

Published on

Presented at Houston Hadoop Meetup by Kenneth Smith and Wade Salazar: Hortonworks upstream use cases.

Published in: Software

Oil & Gas Big Data use cases

  1. 1. 1 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Powering Data Driven Transformations Hortonworks in Oil & Gas
  2. 2. 2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved DATA AT RESTDATA IN MOTION ACTIONABLE INTELLIGENCE Modern Data Applications PERISHABLE INSIGHTS HISTORICAL INSIGHTS INTERNET OF ANYTHING Hortonworks DataFlow Hortonworks Data Platform Hortonworks Delivers Connected Data Platforms… Capturing perishable insights from data in motion Ensuring rich, historical insights on data at rest
  3. 3. 3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Field Data Capture Office or Datacenter Hortonworks Industrial Data Analytics Platform – In Practice WITSML Files / Other Unstructured Data Video IoT Gateways PLC / RTU SCADA, DCS, Historians Central HDP Cluster Hive Central HDF Cluster NiFi Kafka Storm Streaming Options HBase Solr YARN HDFS Location 1 NiFi Location n NiFi Data Center Data Ingestion Framework End users DATA IN MOTION – HDF DATA AT REST – HDP HDF Edge (MiNiFi + NiFi) § Reliable collection § Small footprint § Edge processing § Data provenance § Integrates with core policies HDF Core (NiFi with Streaming) § Processing at larger scale § Distributed stream processing HDP § Security and data governance § Monitoring, management, operations § Applications § Analytics Structured Data Sets
  4. 4. 4 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Oil & Gas Industry Selected Use Cases
  5. 5. 5 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Cyber Security Analytics G&G Analytics Yield Analytics Defect Detection Assess Physical Threats Chemicals and Refinery R&D Product Design M & A Competitive Analysis Geo Asset Supply Chain Pipeline Surveillance Leased Sites Business Unit Leak Detection Mfg Process Customers Basin Records Mgmt Legacy Offloads Historical Archive Device Data Ingest Rapid Reporting Digital Protection Well Planning Real-Time Operations Risk Modeling IoT Pre-Drill Analytics Route Optimization Reduce QHSE Incidents Trading Surveillance Equipment Predictions Equipment Predictions EDW Archive Data as a Service eDiscovery Connected Well Seismic Analytics Public Data Capture
  6. 6. 6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved What à Storing years worth of log files from all of their wells was very costly in legacy O&G systems and their EDW à Analysts didn’t have access to log data stored in legacy systems and couldn’t use common analytic applications to perform data science, build forecast models, predictive use-cases, etc. à Existing platforms didn’t have the functionality to combine log data with data from other systems like auction history, production data, seismic data, geolocation & perforation data. Why à Active archive broadens access to well log data which is otherwise only available to specialized software à Serves as a foundational data set for future use cases where log data can be easily joined as part of other well and formation analysis or data science à Acceleration of geological and geophysical workflows and process automation How à Using HDP analysis can now store all of their historical log files at a fraction of the cost in addition joining that to well data from other systems à Analysts now have all log files in a single location with the ability to build “synthetic log files” comprised of data from multiple sources and perform analytics on combined data sets from siloed applications. Why Hortonworks? Data Discovery Blog post: http://hortonworks.com/blog/big-data-on-a-budget-in-oil-gas/ LAS Active Archive & Analytics
  7. 7. 7 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Field Data Capture Office or Datacenter LAS Active Archive & Analytics – Reference Architecture WITSML Files / Other Unstructured Data Video IoT Gateways PLC / RTU SCADA, DCS, Historians Central HDP Cluster Hive Central HDF Cluster NiFi Kafka Storm Streaming Options HBase Solr YARN HDFS Location 1 NiFi Location n NiFi Data Center Data Ingestion Framework End users DATA IN MOTION – HDF DATA AT REST – HDP HDF Edge (MiNiFi + NiFi) § Reliable collection § Small footprint § Edge processing § Data provenance § Integrates with core policies HDF Core (NiFi with Streaming) § Processing at larger scale § Distributed stream processing HDP § Security and data governance § Monitoring, management, operations § Applications § Analytics Structured Data Sets
  8. 8. 8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Blog post: http://hortonworks.com/blog/industrial-iot-offshore-drilling-solution-delivered-less-90-days/ Real-time Drilling & Production Why Hortonworks? Data Discovery What à Stream critical telemetry of drilling, completions and production process into big data platform à Drilling typically done in remote parts of the world, where connectivity is intermittent, latent and provide minimal bandwidth. à Typical remote monitoring technologies do not perform well in these conditions, therefore large volumes of data end up stranded and out of reach for analyst and support teams Why à Remote drilling operations operate reactively and suffer from unnecessary downtime, equipment failures, efficiency losses, and safety risks. à Provides Industrial Control System datasets in format comprehensible to traditional analytical techniques à BSEE (the governing body for offshore drilling in the U.S.) proposed new regulations that require offshore drillers to monitor safety critical equipment in real-time and archive the data at an offshore facility How à Hortonworks and Kepware delivered an Industry IIoT Solution in less than 90 days for acquiring and storing time series measurements and related safety equipment data from an operated drill ship using Kepware IoT Gateway, HDF & HDP. à Key data consumption patterns planned include KPI dashboards, condition-based monitoring and maintenance, event-based surveillance, and traditional BI reporting; ensuring safer more efficient offshore operations.
  9. 9. 9 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Field Data Capture Office or Datacenter Real-time Drilling & Production – Reference Architecture WITSML Files / Other Unstructured Data Video IoT Gateways PLC / RTU SCADA, DCS, Historians Central HDP Cluster Hive Central HDF Cluster NiFi Kafka Storm Streaming Options HBase Solr YARN HDFS Location 1 NiFi Location n NiFi Data Center Data Ingestion Framework End users DATA IN MOTION – HDF DATA AT REST – HDP HDF Edge (MiNiFi + NiFi) § Reliable collection § Small footprint § Edge processing § Data provenance § Integrates with core policies HDF Core (NiFi with Streaming) § Processing at larger scale § Distributed stream processing HDP § Security and data governance § Monitoring, management, operations § Applications § Analytics Structured Data Sets
  10. 10. 10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved What à Model based failure prevention based on equipment telemetry and asset patterns à Every form of artificial lift is prone to failure when subjected to different types of reservoir conditions à Problems like scale, low static bottom hole pressure, overload and rod string failure are the result of machines encountering gas, sand, H2S and CO2 corrosion and other subsurface conditions Why à Increased production through fewer unplanned well interventions à Decreased unplanned downtime and cost for recovering failed equipment from downhole How à Collect streaming data using HDF, store all current and historical patterns of data usage including failure histories à NOTE: this can be developed manually or through partner like OspreyData Why Hortonworks? Predictive Analytics Production Optimization
  11. 11. 11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Field Data Capture Office or Datacenter Production Optimization – Reference Architecture WITSML Files / Other Unstructured Data Video IoT Gateways PLC / RTU SCADA, DCS, Historians Central HDP Cluster Hive Central HDF Cluster NiFi Kafka Storm Streaming Options HBase Solr YARN HDFS Location 1 NiFi Location n NiFi Data Center Data Ingestion Framework End users DATA IN MOTION – HDF DATA AT REST – HDP HDF Edge (MiNiFi + NiFi) § Reliable collection § Small footprint § Edge processing § Data provenance § Integrates with core policies HDF Core (NiFi with Streaming) § Processing at larger scale § Distributed stream processing HDP § Security and data governance § Monitoring, management, operations § Applications § Analytics Structured Data Sets
  12. 12. 12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Single View of an Asset Why Hortonworks? Single View / Data Discovery What à Data sources and applications used by asset team resided in silos with limited visibility across OT & IT systems. à Operations team having to rely on IT to generate reports from enterprise systems that weren’t timely and required manual data integration using spreadsheets à Extract data from relational data stores that underpin line of business apps (Wellview, OFM etc) Why à Provide a single environment for exploring current and historical conditions, KPIs and well economics à Using the solution the asset team has been able to identify potential well failures 4X-5X faster than before How à Using HDP and a common visualization application, the 360 View solution was built in three months combining OT data sources like SCADA, maintenance logs, & unstructured data (text files, emails) with ERP data, geospatial, and external data from midstream partners, and weather data. à The solution is fairly self-service that can be modified by the operations team with limited involvement from IT.
  13. 13. 13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Field Data Capture Office or Datacenter Single View of an Asset – Reference Architecture WITSML Files / Other Unstructured Data Video IoT Gateways PLC / RTU SCADA, DCS, Historians Central HDP Cluster Hive Central HDF Cluster NiFi Kafka Storm Streaming Options HBase Solr YARN HDFS Location 1 NiFi Location n NiFi Data Center Data Ingestion Framework End users DATA IN MOTION – HDF DATA AT REST – HDP HDF Edge (MiNiFi + NiFi) § Reliable collection § Small footprint § Edge processing § Data provenance § Integrates with core policies HDF Core (NiFi with Streaming) § Processing at larger scale § Distributed stream processing HDP § Security and data governance § Monitoring, management, operations § Applications § Analytics Structured Data Sets
  14. 14. 14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved About Hortonworks Customer Momentum à ~1400 customers (as of October 2016) à ~400 customers added in 2015 à Publicly traded on NASDAQ: HDP The Leader in Connected Data Platforms à Hortonworks Data Flow for data in motion à Hortonworks Data Platform for data at rest à Powering new modern data applications Partner for Customer Success à Leader in open-source community, focused on innovation to meet enterprise needs à Unrivaled support subscriptions Founded in 2011 Original 24 Architects, Developers, Operators of Hadoop from Yahoo! 1000+ E M P L O Y E E S 1500+ E C O S Y S T E M PA R T N E R S

×