Unifying IT Operations with Splunk's Operational Intelligence Platform
•
4 likes•1,694 views
Splunk Live Philadelphia Break Out Session with Jon Rooney, Director, Splunk
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (7)
Similar to Unifying IT Operations with Splunk's Operational Intelligence Platform
Similar to Unifying IT Operations with Splunk's Operational Intelligence Platform (20)
More from Splunk
More from Splunk (20)
Recently uploaded
Recently uploaded (20)
Unifying IT Operations with Splunk's Operational Intelligence Platform
- 1. Unifying IT Operations Jon Rooney Director, Solutions Marketing Splunk
- 2. Session Agenda 1 2 3 4 IT Operational Intelligence with Splunk An overview of Splunk Apps Apps deep-dive with demos Getting started
- 3. Escalating IT Complexity… SERVERS STORAGE NETWORKING VITUALIZATION INFRASTRUCTURE APPLICATIONS PACKAGED APPLICATIONS CUSTOM APPLICATIONS Identity VPN IP Phone HR Email Finance App Svr DB Web Svr SaaS/PaaS IaaS
- 4. … Plaguing IT Operations SERVERS STORAGE NETWORKING VITUALIZATION INFRASTRUCTURE APPLICATIONS PACKAGED APPLICATIONS CUSTOM APPLICATIONS Identity VPN IP Phone HR Email Finance App Svr DB Web Svr SaaS/PaaS IaaS Complex, silo-based technologies Disconnected and outdated point solutions Reactive brute-force problem resolution Over 80% of time on maintaining not innovating
- 5. Industry Leading Platform for Machine Data Any Machine Data Online Services Web Services Servers Security GPS Location Storage Desktops Networks Packaged Applications Custom ApplicationsMessaging Telecoms Online Shopping Cart Web Clickstream s Databases Energy Meters Call Detail Records Smartphones and Devices RFID Datacenter Private Cloud Public Cloud Enterprise Scalability Search and Investigation Proactive Monitoring Operational Visibility Real-time Business Insights Operational Intelligence
- 6. Industry Leading Platform for Machine Data Any Machine Data Online Services Web Services Servers Security GPS Location Storage Desktops Networks Packaged Applications Custom ApplicationsMessaging Telecoms Online Shopping Cart Web Clickstream s Databases Energy Meters Call Detail Records Smartphones and Devices RFID Datacenter Private Cloud Public Cloud Enterprise Scalability Search and Investigation Proactive Monitoring Operational Visibility Real-time Business Insights Operational Intelligence Any amount, any location, any source Schema- on-the-fly Universal indexing No back-end RDBMS No need to filter data
- 7. Splunk : Platform For IT Operational Intelligence Apps and Add-ons Accelerate Value From Machine Data API SDKs UI Server, Storage, Network Server Virtualization Operating Systems Infrastructure Applications Business Applications Cloud Services XenApp XenDesktop Other Monitoring Ticketing/Help Desk Web Intelligence No rigid schemas– Add in data from any other source Custom Applications Stream
- 8. Apps Provide Deep Insights By Role Find and resolve problems fast in individual technology areas Exchange Admin Service Health Performance Message tracking VMware/Win/L inux Admin Infrastructure Health Performance Anomalies/Outliers Storage Admin Infrastructure Health Performance Anomalies/Outliers
- 9. End to End Correlation With Splunk Enterprise Reduce Costs: Consolidate tools, eliminate silos, find root cause faster! Exchange Admin Linux/Win Admin Network Admin Applications Admin Line of Business User Application Support VMware/Linux/ Win Admin Security Admin Storage Admin IT Management
- 10. Powerful Cross-Tier Operational Analytics Harness IT data for business decision-making Data driven decisions across the enterprise Forecasting and planning Root cause analysis Proactive alerting User/Usage analytics Change monitoring Security and forensics
- 11. Splunk Apps Accelerate Insights
- 12. Splunk For Operating Systems Proactive Monitoring Operational Analytics End-to-End Visibility Get instant insight into infrastructure health OS metrics for performance, capacity & resource allocation analyses Scale and correlate across all tiers of your technology stack
- 13. Splunk For Virtualization and Storage Proactive Monitoring Operational Analytics End-to-End Visibility Real-time actionable insights into problem spots and health issues Real-time & historical insights into performance, security, capacity, forecasting and change tracking Scalable Big Data solution for holistic visibility across all technology tiers
- 14. Splunk For Infrastructure & Business Applications Keep the Business Running Increase Productivity Access to Intelligence Proactively monitor the one service that all other systems actively depend on Analyze, report & monitor via simple dashboards and decrease troubleshooting time Get detailed information on irregular activities affecting security policies or SLA
- 16. Splunk App for Exchange Service Uptime Instant visibility into email service health across all infrastructure components Capacity Planning In depth reporting on capacity usage, anomalies, & trends/forecast “I know at-a-glance if Exchange has a problem” “I know what resources are being used, by whom and at what rate” Message Tracking End to end visibility into message delivery across heterogeneous email components “I know exactly when & where there is a problem with messages delivery” Operational Analytics Security events reporting, threat detection, reputation monitoring, change tracking, user behavior “I can get a rich set of reports on my messaging environment”
- 17. Splunk App for VMware Real-time Insights Instant visibility into VMware environment health across all infrastructure components Capacity Planning In depth reporting on capacity usage, anomalies, & trends/forecast “I know at-a-glance if I have a problem in VMware” “I know what resources are being used, by whom and at what rate” Cross Tier Visibility End to end trouble- shooting across VMware, storage, OS and applications “I know exactly which infrastructure tier has a problem” Operational Analytics Security events reporting, change tracking, usage reporting, planning and forecasting “I can use virtualization data for multiple uses”
- 18. Splunk App for NetApp Data ONTAP Central Proactive Monitoring Visualize storage infrastructure health, configuration and important issues instantly Comprehensive Operational Analytics Use storage data metrics for performance, capacity and resource allocation analyses End-to-End Visibility Scale and correlate across all tiers of your technology stack
- 19. Splunk App for Stream Wire Data Enriches Machine Data Insights Added visibility forIT operations,security and the business Efficient, cloud readywiredata capture Usercontrols,filters forspecific data desire Simple, customizable deployment forfast time to value
- 20. DEMO
- 21. The Splunk Developer Platform Gain Application Intelligence Increase the speed and efficiency of application development and testing lifecycle Integrate and Extend Splunk Extend Splunk into other applications using the SDKs, programmatic control over search commands and data ingestion Build Splunk Apps Build Splunk apps with flexible UI, custom dashboard and data visualizations, using the Splunk Web Framework
- 22. Customers Leading The Way
- 23. Copyright © 2014 Splunk Inc. Why Enterprises Use Splunk for IT Operations Increased Uptime to 99.9% Availability Reduced MTTR from 2-3 days to few minutes Improved Margins by protecting millions in ad-revenue Consolidated Tools by retiring 27 monitoring solutions Optimized Capacity by saving $500K in SW, HW & licenses Drives Innovation with usage analytics on product features
- 24. Industry Leading Platform For Machine Data Machine Data: Any Location, Type, Volume Online Services Web Services Servers Security GPS Location Storage Desktops Networks Packaged Applications Custom ApplicationsMessaging Telecoms Online Shopping Cart Web Clickstream s Databases Energy Meters Call Detail Records Smartphones and Devices RFID On- Premises Private Cloud Public Cloud Platform Support (Apps / API / SDKs) Enterprise Scalability Universal Indexing Answer Any Question Developer Platform Report and analyze Custom dashboards Monitor and alert Ad hoc search
- 25. INDUSTRY LEADING PRODUCTS Full-featured platform for real-time operational intelligence Splunk Enterprise as a cloud service Explore, analyze, visualize data in Hadoop and NoSQL data stores
- 26. Self-Managed Sign Up for the Service Full app, SDK, API, platform support 1-Click Launch on AWS Splunk Enterprise and Hunk AMIs AMAZON MACHINE IMAGES (AMI) AVAILABLE EVERYWHERE * Available in USA, Canada
- 27. Copyright © 2014 Splunk Inc. Splunk provides a platform for IT and the business to gain visibility, insights and intelligence from all machine data Strong ecosystem of apps deliver end-to-end operational visibility enabling IT to reduce costs, consolidate tools and eliminate silos Splunk delivers Operational Intelligence allowing IT to go beyond maintenance to enabling business insights and growth
- 28. Easy to Get Started http://splunkbase.splunk.com 3. Start Splunking1. Download for free* 2. Accelerate data collection and correlation * Free 60-day trial for premium Apps
- 29. Copyright © 2014 Splunk Inc. Thank you
Editor's Notes
- There has been an explosion of growth of IT data center technologies, IoT mobile, distributed apps, virtualization. What this brought is increased efficiency and utilization, however at the same time there was escalating IT complexity. <click>
- Lots of disparate and complex and siloed based solutions If you need to find a solution to a problem you maybe need to get a war room ready, finger pointing and trying to debug an issue in production environment. You maybe using hours and hours trying to find a solution. Often times it is a brute force approach when you need to restart the system, so brute-force approach is something used. So IT is no longer spending time on innovating but losing valuable time on keeping the lings on or fighting fires.
- Splunk Enterprise is fully featured, platform for collecting, searching, monitoring and analyzing machine data and getting operational intelligence. You can monitor both real-time (as the data is streaming) and historical data. Splunk collects machine data securely and reliably from wherever it’s generated in any formant. It stores and indexes the data in real time in a centralized location and protects it with role-based access controls. You can troubleshoot your network problems and investigate security incidents in minutes (not hours or days). Monitor your end-to-end infrastructure to avoid service degradation or outages. Gain real-time visibility and critical insights into customer experience, transactions and behavior.
- <click>We don’t require you to understand your data and have predefined schema and requirements. You don’t need to have expensive custom connecters to get data into Splunk. We have our own map reduce based high speed data index and retrieval mechanism. We can index the data from any part of your infrastructure. We scale from a single server to petabytes of data and you can use commodity x86 hardware. And you can store data in the cloud as well if you don’t want to manage your Splunk instance. So what you can start getting into the core of the problem, If you have a system that does not have proactive capabilities you can do that with Splunk Enterprise. And expand from there into security, capacity planning applications management – truly big gold mine of use cases from your data. And our customers once they star to gain that operational visibility they evolve to getting deeper insights from your data. No database in the backend as we apply schema on the fly. You need raw data to be able to re-use it. We are creating intelligence on top of the data therefore easy scaling.
- Over the last couple of years Splunk has evolved from an engine for machine data to a platform for machine data – nothing is a testimony of this more than our Apps store apps which range from plugins and templates to full fledged apps that help you collect, analyze and harness data from every layer of your technology stack. These apps are built by our customers, technology partners such as Cisco, Netapp, ExtraHop and our splunk employees. We are a platform as it is very easy to get data into Splunk and out of Splunk. We are complementing other solutions in the data center Two important things to remember: If a logo you have doesn't show up here, Splunk still doesn't’t limit you – you can always index data from that technology – Splunk extensions simply help you accelerate the process. We provide a full featured REST API and a variety of SDKs that help you build your own custom apps for technologies and insights custom to your business. This is to help you create a specific interface to your data in special format and development languages your team is used to. Lastly, each of the Splunk extensions is not comparable to point solutions in every silo, simply because your data from each silo is more valuable when in context of other data from other technology tiers. Splunk apps simply help you get to the point faster where you can see correlations and comparisons of machine data ACROSS silos.
- With Apps, you can accelerate insights into specific issue or a problem area. For example if you are focusing on Exchange, you want to understand what is the service health are messages going through, do I have any security issues. If you are a Virtualization or storage admin, you want to understand what is going on with your infrastructure, Am I forecasting proper resources for capacity growth? How are my applications affected by storage latency? Do I have enough storage capacity? Our Apps can provide you with these insights since we have visibility into specific siloes.
- So now, no mater what is your administrative area, you want to have cross-tier insights across the environment. How many times you have had complaints from applications guys that there is a big latency on the storage side or as a virtualization admin, you may need to allocate additional resources add more CPU cores to boost user performance and applications. Or as OS admin, you see that your OS is showing correct storage utilization but you still have application running slow. This is because each one of the IT professionals are looking into the isolated tools. They do not have insight into other siloes. That is what our apps deliver and is a core functionality of Splunk as platform. If you have an Exchange running on top of VMware/hypervisor, Windows, over Cisco ACI and with attached storage. You can use Splunk as a platform to help you get insight into how your business service is performing. It is central and easy for Splunk because we look at this just as another data source
- And what can you use this data for? You have specific individual business need. Splunk is flexible and in you have a performance issue you can move into the root cause analysis of that problem. And more into proactive monitoring. You want to understand how your users are interacting with your website and which content is popular you can do that with Splunk. You can forecast and plan for enterprise growth you can do that. Understand insider threat or security breaches – We have an App for that?. The key point I would like to ask you to remember is that Splunk enables you to make informed analytics driven decisions across your enterprise.
- Let’s take a closer look at few of the apps we are highlighting here. We will mention few Splunk supported Apps. We are investing in these apps and provide full support for them.
- One of our most popular apps are Windows and Linux and Unix Apps. And if you have thousand of servers out there deployed we have added functionality in these apps to let you easily monitor infrastructure. Primary with OS you want smooth operation and at a glance visibility into infrastructure health. Proactive alerts and understand CPU and memory consumption for processes running in the environment.
- Another set of Apps which are very popular are Virtualization and storage apps. VMware is hugely popular app and it is one of our few that are premium. We support other virtualization apps such as apps for hyper-v, Xen App, Hyper-V, and when we talk about storage NetApp App is on top of the list. So what these apps allow you to do is again at a glance insight into your infrastructure. They provide operational analytics with insights into performance capacity forecasting and finally ability to view and cross-correlate across technology tiers.
- Another set of extremely popular apps are Splunk Apps for Exchange, Active directory and Amazon Web Services. Why? Because you can get correlated insights across multiple tiers. What are the benefits to you? First keep the service running smoothly. For example, how is the message volume dependent on the health of your network or do you have enough CPU resources. Since you do have all the data located in one location this directly brings increased productivity. Also, security policies can be enforced as you have insights into how your users are interacting with the applications.
- Let’s take a look at deeper dive, ]
- One of our premium Apps is Splunk App for Microsoft Exchange. How many of you use Exchange? How many of you administer Exchange? In order to keep the Exchange up and running you need to understand several different components – from Active Directory, Windows Infrastructure all the way to Network environment and user information. And what do you think is the biggest variable in managing the exchange environment – I will give you a hint it starts with U…? So as Exchange Admin, you need to understand operations and tracking message across different component. One of the biggest financial institution that has one of the biggest deployments of Exchange in the world uses our app to track how the messages are going through. You need to understand that your environment is up and running. Who is using resources and how much, also how are you growing and for the chargeback information? Also capacity planning performance? Is my storage becoming a bottleneck or is it a network issue affecting messages going through. Want to make sure that messages are flowing from point A to point B. Also you need to understand security and reputation monitoring. So imaging there is a Friday afternoon all of sudden IT director is trying to find what is the needed capacity to support a mobile component of exchange.
- Now let’s move on to Splunk App for VMware. Another hugely popular premium App. This App gives you deep and granular insights into VMware infrastructures. It collects performance metrics at 20 sec granularity, tasks, events, inventory, topology, ESXi logs, indexes it and provides you intelligence on top of this data. And you get operational analytics across multiple use cases. Example: Restart of the VM was not restarting properly and VC was keeping up process that caused restart to fail. Because this customer utilized the Vmware App, they were collecting tasks and events data, they were able to get to the root cause of the problem, for every event of restart they were able to understand what are the associated tasks. This was a simple use case. Another example would be to understand across all of your VMware environment is to see that all of the patches are on the same version so you are not having a security breach. You want to have which snapshots are consuming specific amounts of memory, you can do that with VMware App. The huge differentiator in Splunk is that we are creating a built in correlation with storage systems. Now you can see what is the impact of performance of attached data stores do you get the data into the app. You can drill down directly into the NetApp data if you want to get further insights from this tier. This is really important as now as a virtualization admin, you do not need to depend on other teams to get you data, the information is readily available. Also we are building integrations with other storage systems such as VNX and others. So you are on the right track to get cross tiers visibility not only across virtual and physical but across various hybrid storage infrastructures. Optional Example: Chevron uses it in the following use case: The hypervisor alone does not have insights into memory consumption and for example what is happening into the OS cash. if you plan capacity based on what hypervisor tells you that, it is actually inaccurate. In Chevron’s use case, they collect WMI metrics using Windows app, they use VMware app to collect hypervisor metrics and based on this they understand what is the memory consumption, they apply standard deviation on top of that data and they figure out how can they plan for the capacity? Possible question: How do we get the data in - we have appliance like solution that has DCN we ship it as virtual appliance or you can put it on a physical box. DCN makes API calls into your vCenter server, collecting it and sending it into the Splunk indexer tier. We collect the data at 20s granularity. And for performance metrics we are pinging vCenter every 3 min. For ESXi logs, you are sending the syslog type data to Splunk and for vCenter log, there is an Application which is vCenter Add-on (very lightweight)
- Central Proactive Monitoring of NetApp Data ONTAP Systems The Splunk App for NetApp Data ONTAP provides central visibility into the operational health of your NetApp Data ONTAP storage systems. It enables real-time and historical visibility into the most important metrics of NetApp filers such as latency, and IOPS of your storage environment to get an accurate picture for trending and storage pattern analysis. Without deploying multiple storage monitoring instances, visualize configuration and performance of your entire Data ONTAP storage systems in both 7-Mode and clustered mode with quick drill-downs to specific sub-systems including cluster, aggregate, volume, disk and other events. Operational Analytics Gain comprehensive operational analytics for your NetApp filers, to speed up troubleshooting efforts and prevent application performance or user experience issues caused by increased latency or insufficient storage resource allocation. Utilizing pre-packaged reports, proactively plan your NetApp storage allocation, identify trends and get warnings and error messages about important events such as storage resource consumption issues, anomalies or outliers for quick navigation to problem areas. End-to-End IT Operational Visibility Combine your NetApp storage data with data from all other technology tiers such as, virtual applications, OS, networks and servers to gain a complete, central view of KPIs across your entire enterprise. With the Big Data scale provided by Splunk Enterprise, analyze and correlate performance, capacity, error and security information across all your technology tiers for faster problem resolution and more accurate capacity and resource planning. Reclaim underutilized resources and replenish over-utilized capacity for optimal cost control. .
- Moving on into Splunk App for VMware and NetApp
- Let’s mention Splunk as a developer platform. Our most successful customers extended utilization of the Splunk Enterprise as developer platform. This is when they can start getting deeper insights into their applications and in devOps environment they can speed up application deployments and apps testing lifecycle. In addition to REST Api we offer SDKs many examples in the popular languages such as Python, Java, Csharp, Ruby etc that help you integrate and utilize splunk functionalities directly into yur environment. You can export data out of Splunk, or you can create your own apps we provide you the web framework to create your own apps . 00U
- We have many customers – over the last quarter we have added 500 more. And now More than 7,900 enterprises, government agencies, universities and service providers in 100 countries use Splunk software to deepen business and customer understanding,
- Here we see the benefits customers are getting from Splunk they presented their use cases in the last 8 months on Splunk Lives Quest Diagnostics – within one hour time there are processing one million of dollars worth of revenue. In their web facing application they were reliant on manual Perl and Unix information to actually look at that data. After they introduced Splunk Enterprise, they were able to monitor that data from applications availability and infrastructure and increased up time to 99.9% Safeway they have many monitoring tools in their environment. They use Splunk as a centralized platform to monitor that data. They saved huge amount of money by consolidating their monitoring tools and retiring 27 tools and Splunk is deployed as centralized monitor across their data. DirectTV: They Introduced NFL streaming service (2011) . They did not realize the impact on the infrastructure. They ended up spending ~150k to beef up the servers and that still did not help. Post Splunk they were able to improve their margins because now they were able to dynamically adapt their infrastructure to the service they are offering.
- What you just saw was the industry leading platform for machine data – Splunk Enterprise. Splunk Enterprise is at use in over [7,900] organizations worldwide – representing 14 major sectors – telecoms, retail, financial services, federal and local government, education, and so on. And what you get out of of your data is up to you. Not only you need to understand your applications, infrastructures and go beyond that into innovative use case where you analyze business impact of your data and many more.