Splunk for Monitoring and Diagnostics Breakout Session
•
1 like•958 views
Splunk for Monitoring and Diagnostics is a presentation about using Splunk software to gain real-time insights from industrial machine data. The document discusses how Splunk can be used to collect, index, enrich, search, analyze, and report on data from industrial IoT sensors, equipment, and systems. It provides examples of how Splunk has helped companies in oil/gas, manufacturing, and other industries improve operations, maintenance, safety and security by turning their machine data into business value. The presentation includes a demo of Splunk's capabilities for industrial use cases.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Splunk for Monitoring and Diagnostics Breakout Session
Similar to Splunk for Monitoring and Diagnostics Breakout Session (20)
More from Splunk
More from Splunk (20)
Recently uploaded
Recently uploaded (20)
Splunk for Monitoring and Diagnostics Breakout Session
- 1. Splunk for Monitoring and Diagnostics Gaining real-time insights into industrial operations Manish Jiandani Director Solutions Marketing
- 2. 2 Safe Harbor Statement During the course of this presentation,we may make forward looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described orto includeany suchfeatureor functionalityina futurerelease.
- 3. IoT Opportunity and Impact Energy savings and labor efficiency Reduction in energy consumption Savings from smarter infrastructure Savings from improved routing and navigation Streamline operations and reduced cost Savings from reduced care for patients $11.0 M $25.0 M $10.0 M $200+ M $1.8 M $1.0 + B 10-20%* $800 B* 20%* $170 B* $470 B* $560 B* Healthcare Oil & Gas LogisticsBuilding Management TransportationManufacturing 3 Source – The Internet of things: Mapping the value beyond the hype – McKinsey Global Institute Economic impact through 2025*
- 4. 4 IT and OT Data is Machine Data Sensors, Historians RTU,PLC,HMI, Pumps, HVAC, Drills, Pipelines, Conveyor Belts, Transformers, Generators, UPS, Telematics, Turbines, Fuel Cells, Telemedicine, Windmills, Valves, GPS, RFID, Hypervisor, Web Servers, Email, Messaging, Clickstreams, Mobile, Telephony, IVR, Databases, Sensors, Telematics, Storage, IT OT
- 5. 5 Industrial Data Contains Critical Insights 05/27/2014T10:24:17GMT applicationId="safetyObs" eventType="safety" assetID="CV1002384-1045" employeeId="114635" jobSite="PLEC-2014-GC" observationId="184568-451124-256" observation="Control Valve handle extracted to manual position. No lockout/tagout or other tag visible. Process is running." observationCriticality="5" imageId="PLEC-2014-GC-184568-451124-256" imageUri="https://mybucket.s3.amazonaws.com/PLEC-2014-GC-184568- 451124-256.png" 1543541, workorder, bsic, 78544, pipefitting, CV1002384, "install manual bleed bypass", 04/13/2014, 05/21/2014, 25663, complete 05/22/2014 03:17:31 Tag="CV1002384.ValvePos" Value=”50" Quality=“Good” 05/22/2014 03:17:46 Tag="CV1002384.ValveCmd" Value=”100" Quality=“Good” 05/22/2014 03:19:22 Tag="CV1002384.ValveCmd" Value=”100" Quality=“Good” 05/22/2014 03:19:27 Tag="CV1002384.ValvePos" Value=”50" Quality=“Bad” Sources Alarms and Events Work Order Sensor Data
- 6. 6 05/27/2014T10:24:17GMT applicationId="safetyObs" eventType="safety" assetID=" " employeeId="114635" jobSite="PLEC-2014-GC" observationId="184568-451124-256" observation="Control Valve handle extracted to manual position. No lockout/tagout or other tag visible. Process is running." observationCriticality="5" imageId="PLEC-2014-GC-184568-451124-256" imageUri="https://mybucket.s3.amazonaws.com/PLEC-2014-GC-184568- 451124-256.png" 1543541, workorder, bsic, , pipefitting, , "install manual bleed bypass", 04/13/2014, , 25663, complete Tag=" .ValvePos" Value=” " Quality=“Good” 05/22/2014 03:17:46 Tag="CV1002384.ValveCmd" Value=”100" Quality=“Good” 05/22/2014 03:19:22 Tag="CV1002384.ValveCmd" Value=”100" Quality=“Good” 05/22/2014 03:19:27 Tag="CV1002384.ValvePos" Value=”50" Quality=“ Industrial Data Contains Critical Insights Sources Alarms and Events Work Order Sensor Data Asset ID Technician Completed MTBF Eval Alert Asset ID Asset ID
- 7. 7 7 Make machine data accessible, usable and valuable to everyone. 7
- 8. 8 HA/DR Admin Data Security Apps SDKs/APIsScale Collect Data Index Data Enrich Data Search & Explore Analyze & Predict Report & Visualize Alert & Action 8 Fully Integrated Enterprise Platform
- 9. 9 Turning Machine Data Into Business Value 9 Platform for Machine Data Application Delivery IT Operations Security, Compliance and Fraud Business Analytics Industrial Data and Internet of Things
- 10. Platform for Machine Data Splunk for IoT Monitoring and Diagnostics Security, Safety & Compliance Preventative Maintenance Asset Lifecycle Management
- 11. 1 Remote Freight Train Monitoring Energy Efficiency Calculations Driving Strategy Recommendations Over $1 Billion Saved
- 12. Improving SCADA Operations and Security 95%Improvement in Incident Response Time Analyze 51K miles of pipeline data from servers and OT networks Improved pipeline safety and availability through higher application uptime Increase regulatory compliance
- 13. Robot Analytics to Improve Supply Chain Throughout 4%Increased Throughput per Distribution Center Aggregate machine data from robots Failure pattern detection and reporting Preventative maintenance scheduling
- 14. 14 Benefits Ensure equipment in the field is operating as intended Monitor and avoid unplanned downtime Perform better root-cause analysis Reduce costs and optimize processes
- 15. 15 IoT and Industrial Machine Data DevelopVisualize PredictAlertSearch Engineers Data Analysts Security Analysts Business Users Native Inputs TCP, UDP, Logs, Scripts, Wire, Mobile SDKs and APIs Java, JS, C#, Python, Ruby, PHP Modular Inputs MQTT, AMQP, COAP, REST, JMS HTTP Event Collector Token Authenticated JSON Real-time Technology Partnerships Kepware, ThingWorx, Cisco, Palo Alto Maintenance Info Asset Info Data Stores External Lookups/Enrichment OT Industrial Assets IT Consumer and Mobile Devices
- 16. 16 Splunk’s IoT and Industrial Partner Ecosystem SDKs UI Ingest and Platforms IoT and ICS SecurityAdvanced Analytics and ML Custom User Interfaces Services and Delivery
- 17. 17 Splunk and Kepware Exploration and Production Operations Enterprise Data Environment - Splunk > Enterprise - Splunk > Cloud OPC DA OPC UA OPC HDA Splunk Universal Forwarder Local Data Collection - SCADA - HMI
- 18. 18 Best Practices Build Baselines of Asset Performance Find Seasonality in Your Operations Monitor Trends and KPIs Identify Anomalies and Outliers Enrich Operational Data with External Sources
- 19. Demo
- 20. Splunk at Enterprise Products Partners, L.P.
- 21. 21 Enterprise Product Partners L.P.
- 23. 2 Critical ICS Endpoints Engineering Workstations Control System Communication Embedded Devices HMI Historian Controllers
- 24. 24 Why Splunk?
- 25. 25 Our Adoption Journey Security Operations Business Analytics …
- 26. 26 Central Pipeline Monitoring Center
- 27. 27 Data Sources Schneider Electric DNA Application Logs Proprietary SCADA Application Logs AutoSol AES Poller Data Logs Palo Alto Networks Data Symantec Antivirus Logs Windows Event Logs DB Connect Configuration Files
- 29. 29 Get Started More Info: Splunk.com/IoT Download Splunk Download Kepware or Modular Inputs Download other Splunk Apps (MQTT, COAP, Kepware Explorer) Visit Splunk Answers
- 30. Thank You
Editor's Notes
- Splunk safe harbor statement.
- Splunk customers are realizing tremendous value across multiple industries and use cases. From Fortune 100 to small shops, enterprises, service providers and government agencies are improving service levels, reduce IT operations costs, mitigate security risks and drive new levels of operational visibility. As they gain new visibility into their real-time and historical machine data, Splunk’s customers are finding answers and solving the most challenging issues facing IT and the business.
- What is this machine data, and why is it a big deal? Well, it’s one of the fastest growing, most complex and most valuable segments of data. All the webservers, applications, network devices, mobile devices, sensors – all of the technology infrastructure running your enterprise – generates massive streams of data, in an array of unpredictable formats that are difficult to process and analyze by traditional methods or in a timely manner. Why is this “machine data” valuable? Because it contains a trace - a categorical record - of user behavior, cyber-security risks, application behavior, service levels, fraudulent activity and customer experience. Characteristics of machine data – the four V’s - the last two are the most interesting / challenging.
- Let’s take a peek at what machine data looks like. Safety is #1 priority for you and in this example, we see here excerpts from 3 typically siloed operational technology systems: Safety Observation Reporting System Computerized Maintenance and Management System (CMMS) Alarm Logs from the Plant Control or SCADA system.
- Let’s take a peek at what machine data looks like. Safety is #1 priority for you and in this example, we see here excerpts from 3 typically siloed operational technology systems: Safety Observation Reporting System Computerized Maintenance and Management System (CMMS) Alarm Logs from the Plant Control or SCADA system.
- At Splunk, our mission is to make machine data accessible, usable and valuable to everyone. And this overarching mission is what drives our company and product priorities.
- Splunk provides an open, fully integrated platform. That means you can collect, index, analyze, report and predict on machine-generated data from a single product. It’s enterprise-ready with high availability and disaster recovery features, role-based access control and scales to index hundreds of terabytes per day. It’s an open platform with over 500 Splunk Apps available and allows for custom development.
- Splunk products are being used for data volumes ranging from gigabytes to hundreds of terabytes per day. Splunk software and cloud services reliably collects and indexes machine data, from a single source to tens of thousands of sources. All in real time. Once data is in Splunk Enterprise, you can search, analyze, report on and share insights form your data. The Splunk Enterprise platform is optimized for real-time, low-latency and interactivity, making it easy to explore, analyze and visualize your data. This is described as Operational Intelligence. The insights gained from machine data support a number of use cases and can drive value across your organization.
- Customers are using Splunk solutions to collect and correlate data from control systems, sensors, mobile devices and IT systems for a variety of Industrial Data and IoT use cases. These use cases include operational efficiency, predictive maintenance, industrial cybersecurity and asset analytics. Splunk is a great place to collect and analyize this data. I think you will see a greater value from IoT data than existing sources. Let’s hear how EnerNoc uses Splunk.
- New York Air Brake’s Train Dynamic Systems Division is using Splunk to manage inter-train forces, the “slinky factor” inherent in large freight trains with 6 inches of flex between cars. With splunk, they are able to produce insight and reports allowing the owners of the locomotives they manage to better train the engineers, and better manage the acceleration and braking of the trains throughout thousand mile journeys. Managing this data with Splunk, they can produce 5-10% fuel savings for customers. For their largest customers this can mean a billion dollars in savings a year.
- Enterprise Product Partners is using Splunk to monitor and manage their critical Industrial Control System infrastructure. This infrastructure powers 51000 miles of some of the most critical hardware in the world – oil pipelines. By using Splunk enterprise and partner solutions from companies like Palo Alto Networks, EPP is able to better monitor and manage the availability of the applications and hardware in their environment, and are able to react more quickly to the unexpected but inevitable downtime in a system this large and complex. PHIMSA regulations require that you react to critical application downtime almost immediately – and EPP is using Splunk to satisfy this requirement. Since starting with Splunk, they have seen tremendous improvement in their response time.
- Visuals need to be worked on
- There are many free add-ons and Apps for Splunk software that simplify the connection and collection of data from both industrial systems and the Internet of Things. These include: Rest API Modular Input: Poll local and remote REST APIs and index the responses. Amazon Kinesis Modular Input: Index data from Amazon Kinesis, a fully managed service for real-time streaming data. Apache Kafka Modular Input: Index messages from Apache Kafka messaging brokers, including clusters managed by Zookeeper. DB Connect 2: Integrate structured data sources with your Splunk real-time machine data collection. Universal Forwarder for Linux (ARM – Raspberry Pi): Dedicated Splunk package for Linux and ARM based systems where data needs to be collected directly from embedded devices such as the Raspberry Pi. MQTT Modular Input: Index messages from MQTT, a machine-to-machine connectivity protocol, by subscribing Splunk software to MQTT Broker Topics. AMQP Modular Input: Index data from message queues provided by AMQP brokers. JMS Modular Input: Poll and index message queues and topics from messaging queues and topics, including MQTT messages, provided by message providers, including TibcoEMS, Weblogic JMS and ActiveMQ. Protocol Data Inputs: Recieve data via a number of different data protocols such as TCP , TCP(s) ,HTTP(s) PUT/POST/File Upload , UDP , Websockets , SockJS. Splunk App for Stream: Capture, filter and index real-time streaming wire data and network events. COAP Modular Input: Index messages from a COAP (Constrained Application Protocol) Server. SNMP Modular Input: Collect data by polling SNMP attributes and catching SNMP traps from datacenter infrastructure devices providing cooling and power distribution. In addition, Splunk has a powerful ecosystem of technology partners. Kepware Technologies – Connects Splunk software with thousands of industrial devices communicating on over a hundred proprietary industrial protocols. Stream real-time data to Splunk from industrial control systems, including SCADA. Carvoyant – Connected car platform, integration with Splunk software allows enterprises to monitor their automobile fleets, including geo-location, engine parameters and diagnostics. B&B SmartWorx – Intelligent sensors and gateways. Integration with Splunk (Splunk App) will include sensor data collection (via MQTT), and gateway and sensor network diagnostics and cyber security. Bluvision– Intelligent beacons. Integration with Splunk (Splunk App) will include beacon data collection (via Websockets). Powerful retail applications. ThingWorx (PTC) – The leading IoT Application Development Platform. Seamless data exchange between ThingWorx applications and Splunk Enterprise and Splunk Cloud, and ThingWorx customers can access Splunk search and analytics through the ThingWorx mashup builder. Buddy.com – Cloud services for connected devices. Integration (Splunk App) will allow Splunk to stream data from any device connected to the buddy platform. Octoblu (Citrix) – IoT developer platform. Has created libraries that allow any Octoblu-enabled device to stream its data to Splunk software and allows those same devices to use Splunk search and analytics to inform their own decisions and logic Red Balloon Security – Security platform for the defense of embedded systems in the enterprise (IP Phones, Printers, switches and routers, etc). Uses proprietary firmware level protection and appliance-based endpoint monitoring, and is integrating (Splunk TA/ES Compliant CIM) with Splunk software to allow Enterprise Security monitoring of threats to embedded enterprise devices. Bayshore Networks – Content-aware cyber security platform for industrial networks. Is integrating (Splunk TA/ES Compliant CIM) with Splunk software to allow Enterprise Security monitoring of threats to SCADA and other industrial networks. Foxguard Solutions– Cyber security and compliance solutions for industrial networks built with Splunk. NERC-CIP compliance specialists. UltraElectronics-3eti – Cyber security platform for industrial networks. Building ES compliant TA to allow collection and analysis of security relevant ICS data in Splunk. Distrix – Software defined networking for industrial networks and the internet of things. Simplifies connectivity and delivers and enhances data over extremely complex networks. Distrix’s SDN supports Splunk to Splunk communication, and can enhance other data, including timestamping and meta-data enrichment, for ingestion in Splunk. Prelert – Anomaly detection app for Splunk Enterprise. Valuable app for management of sensors and devices where rapid identification of anomalies in sensor readings or operations are critical. Predikto – Leverages the power of Predictive Analytics enabling organizations to use their data to predict future asset failures. N3N – Custom, advanced user interfaces for Splunk specializing in isometric views of industrial facilities. R Project App – harness the power of R statistical processing language directly from Splunk interfaces and search processing language. D3.js – Data driven documents for powerful user experiences. HTML5 – Advanced web interfaces and applications for browser and mobile based user experience.
- Midstream Energy Services Provider Serves producers and consumers of natural gas, crude oil, refined products and petrochemicals Manages approximately 50,000 miles of pipeline across U.S.
- SCADA: Supervisory Control and Data Acquisition PLCs: Programmable Logic Controllers RTUs: Remote Terminal Units Servers Switches Routers Desktops
- Previous issues with visibility and issue investigations were long and tedious Needed to manage tens of thousands of diverse legacy field devices Existing in-house solutions and vendor built tools made it impossible to correlate across systems PHMSA requirements were strict and uptime is critical
- Discovered Splunk Enterprise while looking at security solutions Realized Splunk Enterprise could also solve operational challenges Conduct operational investigations Proactive Alerting Increase overall pipeline visibility
- Endpoint messaging Timeouts Leak Detection Correlate sensor data with other SCADA data