Copyright © 2013 Splunk, Inc.
Splunk Overview &
Splunk for VMware
Presenter Name
Presenter Title
Agenda
What Is Splunk?
Splunk Value in Virtualized Datacenters
Splunk App for VMware Intro
What Customers Are Saying
Demo/...
Make machine data accessible, usable
and valuable to everyone.
3
Customer
Facing Data
Outside the
Datacenter
Applications
Web logs
Log4J, JMS, JMX
.NET events
Code and scripts
Networking
...
Splunk Collects and Indexes Any Machine Data
Customer
Facing Data
Outside the
Datacenter
Applications
Web logs
Log4J, JMS,...
Splunk Turns Machine Data into
Operational Intelligence
Search and
Investigate
Proactive
Monitoring
Operational
Visibility...
ASingleSolutionforOperationalIntelligence
Real-time Visibility
• Live dashboards
• Event correlation
• Monitoring and aler...
Splunk Delivers Value Across the Enterprise
8
Web
Intelligence
Application
Mgmt
Security
&
Compliance
IT
Operations
Busine...
Education
Healthcare
Technology
Energy and Utilities
Manufacturing
Telecommunications
Cloud and Online Services
Government...
Splunk In Virtualized
Datacenters
The Virtualized Datacenter Problem
End user
devices
End user
devices
End user
devices
End user
devices
Virtualized
Servers...
How Does Splunk Help?
End user
devices
End user
devices
End user
devices
End user
devices
Virtualized
Servers
Storage
Netw...
Splunk App for
VMware
Splunk App for VMware
Complete Operational Visibility Into VMware Environments
• Collects and persists logs and
performanc...
Why Use the Splunk App for VMware?
• Harness virtualization layer data
– Special forwarders for ESX/ESXi and
VC
– Comprehe...
Unlock the Value of your Virtualization Data
How It Works
Splunk App
for VMware
>
Splunk Add-on
for vCenter
>
Splunk
UF/LF
vCenter
server
Provides:
Dashboards, Views,
...
Data Volume
18
• 1 Forwarder Appliance is required per 20-30 physical ESX hosts –we
don’t charge for the app or the applia...
What Differentiates
The Splunk App for
VMware
19
End to End Visibility
20
Splunk used to correlate the business data (Users
, Usage) with the IT/Infrastructure data
Unders...
One Splunk – Many Uses
21
Using Splunk for VMware gets
us our data in one place, for
many uses: capacity
planning, event
m...
Detailed History For Analysis &
Troubleshooting
22
I love that I can track virtual
machines in my environment as
they move...
Benefits of the Splunk Approach
1. Collects all the data without interfering with the normal operation of VC – Data how
yo...
Why Splunk Over Everyone Else?
You don’t know what data you will need till you need it
– Every other tool goes through VC ...
What Customers Are Saying
25
“Thank you for bypassing VC – it resolves a huge bottleneck in our
environment”.
“VC shows me...
Why Use the Splunk App for VMware
Isn’t this the data I can see in VC?
NO
• VC has only 5 minute summaries of performance ...
Why Use the Splunk App for VMware
VCOps tells me what to do and “predicts” the health of my
environment
VCOps is virtualiz...
Screenshots
29
Track any metric historically as the VM moves
from host to host
30
View any tasks
performed/changes made to the
host or vms
31
Detailed host and VM
metrics including disk
latencies, IO, storage
queue depths..
Configurable Capacity Reporting
32
Cluster level/Host level/VM level
Log Browsing
33
Datastore Capacity Reporting
34
How much storage is being taken up by
snapshots?
How much are my datastores over-
provisio...
Thank You
Backup Slides
What is vSphere?
Virtualization Layer –Abstracts and Pools Server, Storage, Networking
Installs on servers, managed by one...
What Could Possibly Go Wrong?
Nothing.
We run 1 VM per host, attached to local disk.
No Problems.
CPU Memory Network Stora...
Upcoming SlideShare
Loading in …5
×

Splunk for vmware virtualization customer presentation

804 views

Published on

Splunk for VMWARE Application

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
804
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
31
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Splunk is already a leader in the big data space – specifically massive machine data. Splunk’s mission has been and still is to “Make machine data accessible, usable and valuable to everyone.”
  • Splunk is the leading enterprise solution for managing and analyzing machine data. It provides a unified way to organize and to extract actionable insights from the massive amounts of machine data generated across diverse sources.One person can download and implement Splunk in hours, rather than having a team of people take months or even years to deploy a solution. You can connect to your data in a few clicks and create powerful dashboards with a few more. Key capabilities:Splunk collects machine data securely and reliably from wherever it’s generated. Splunk stores and indexes all of the data in real time in a centralized location and protects it with role-based access controls. Splunk turns your machine data into a NoSQL data fabric that can be searched, browsed, navigated, analyzed and visualized. This enables IT professionals businesses to solve a wide range of mission-critical problems, all without the inherent limitations of traditional approaches.Search and analyze live streaming and terabytes of historically indexed data from one place. Splunk automatically monitors your data for trends and specific patterns of activity or behavior. Then notifies the people that need to know immediately.Powerful search, drilldown and reporting capabilities meet the needs of novice users and expert analysts alike. Easy-to-create dashboards put critical insights from your machine data into the hands of the people who need it.
  • Search and investigate: Find and fix problems dramatically fasterProactive monitoring: Automatically monitor to identify issues, problems and attacksOperational visibility: Gain end-to-end visibility to track and deliver on IT KPIs and make better-informed IT decisionsReal-time business insights: Gain real-time insight from operational data to make better-informed business decisionsCombining and correlating machine data with business data provides unique business insights. Watching the consumption of new online services by channel or demographics. Combining telecoms call records with tariff databases to get a real time view of revenue and 3rd party charges. There is a diverse set of cases where surfacing machine data provides operational intelligence to the business. And the lead times to get to this intelligence is dramatically less than other solutions. Months to a few days in many cases.
  • To deliver Operational Intelligence requires handling three primary workloads from within the same system.Providing real-time visibility of live data, including correlating transactions and events across multiple sources, monitoring against thresholds and alerting, tracking against SLAs, etc.Enabling powerful navigation of the data to get to “the needle in the haystack” – to troubleshoot and identify root cause and to perform incident investigations.Providing the ability to analyze historical (as well as live streaming) data – to identify trends and patterns, to prove compliance, etc.Supporting these three workloads in the same system delivers value across the organization. Specific dashboards can provide meaningful information for different users and roles – from the server room to the boardroom, so the value of Operational Intelligence can be recognized deep within the organization.
  • More than 5,200 users in over 90 countries have purchased the enterprise license of Splunk. This includes a majority of the Fortune 100. Enterprises, service providers and government agencies in 78 countries use Splunk to improve service levels, reduce IT operations costs, mitigate security risks and drive new levels of operational visibility.As they gain new visibility into their real-time and historical machine data, Splunk’s customers are finding answers and solving the most challenging issues facing IT and the business.
  • Virtualization separates applications from the hardware they run onMakes it easier to share resources
  • Customers start by using Splunk Enterprise to address one specific solution area. Then they leverage it and their machine data to solve other pressing problems over time.Consequently, Splunk Enterprise has many critical uses across IT and the business: Application Management: provide end-to-end visibility across distributed infrastructures; troubleshoot across application environments; monitor for performance degradation; trace transactions across distributed systems and infrastructure.Development: accelerate development and test cycles; support advanced development methodologies like agile, continuous; integrate enterprise applications with SDKs and a robust API; build enterprise applications that leverage Splunk software.Infrastructure and Operations Management: proactively monitor across IT silos to ensure uptime; rapidly pinpoint and resolve problems; report on SLAs/track SLAs of service providers.Security and Compliance: provide rapid incident response, real-time correlation and in-depth monitoring across data sources; statistical analysis for advance pattern detection and threat defense.Web and Business Analytics: gain visibility and intelligence on customers, services and transactions; identify trends and patterns in real time; fully understand the impact of new product features on back-end services.
  • Aggregated metrics report at host level or virtual machine levelPer instance metrics report by individual devices attached to the hosts or virtual machines.Example:
  • Understand how much resources each customer consume (CPU, Memory, Network, etc …) and when.Customer can have more then 1 VM or environment , splunk help us aggregate the date easily and look at the customer level usageSLA DashboardsMeasure service level Analyze and present statistics according to business guidelines
  • Peter Cole from melbourne IT cant wait to get Splunk App for VMware deployed across his environment. Some of the big benefits he gets from it:Find where storage is way over provisioned, clean up snapshots where they are taking up space, find errors in logs related to storageFind out what happened when in the environment, for troubleshooting, issue diagnosis, security reporting and moreUnderstand service levels of virtual machines in detail during performance/load testing
  • Rapid Troubleshooting and AnalysisDiscovery Communications, the world's largest non-fiction media company, uses Splunk to monitor application and operating system logs and events. The Splunk App for VMware enhances their operational visibility by giving them access to their virtualization layer data. With Splunk Discovery Communications gets an immediate understanding of virtualization layer failures and receives alerts before there is a full-blown impact on operations."I love that I can track virtual machines in my environment as they move from host to host.I can now identify the root cause of issues or errors" -Matthew Cluver, Network Operations Analyst, Discovery Communications.When asked which views of the app he likes – he liked them all!
  • Splunk for vmware virtualization customer presentation

    1. 1. Copyright © 2013 Splunk, Inc. Splunk Overview & Splunk for VMware Presenter Name Presenter Title
    2. 2. Agenda What Is Splunk? Splunk Value in Virtualized Datacenters Splunk App for VMware Intro What Customers Are Saying Demo/Screenshots 2
    3. 3. Make machine data accessible, usable and valuable to everyone. 3
    4. 4. Customer Facing Data Outside the Datacenter Applications Web logs Log4J, JMS, JMX .NET events Code and scripts Networking Configurations syslog SNMP netflow Databases Configurations Audit/query logs Tables Schemas Virtualization & Cloud Hypervisor Guest OS, Apps Cloud Linux/Unix Configuration s syslog File system ps, iostat, top Windows Registry Event logs File system sysinternals Logfiles Configs Messages Traps Alerts Metrics Scripts TicketsChanges Click-stream data Shopping cart data Online transaction data Manufacturing, logistics … CDRs & IPDRs Power consumption RFID data GPS data Splunk Collects and Indexes Any Machine Data 4
    5. 5. Splunk Collects and Indexes Any Machine Data Customer Facing Data Outside the Datacenter Applications Web logs Log4J, JMS, JMX .NET events Code and scripts Networking Configurations syslog SNMP netflow Databases Configurations Audit/query logs Tables Schemas Virtualization & Cloud Hypervisor Guest OS, Apps Cloud Linux/Unix Configuration s syslog File system ps, iostat, top Windows Registry Event logs File system sysinternals Logfiles Configs Messages Traps Alerts Metrics Scripts TicketsChanges Click-stream data Shopping cart data Online transaction data Manufacturing, logistics … CDRs & IPDRs Power consumption RFID data GPS data •Any amount, any location, any source Noupfrontschema Nocustomconnectors NoRDBMS Noneedtofilter/forward 5
    6. 6. Splunk Turns Machine Data into Operational Intelligence Search and Investigate Proactive Monitoring Operational Visibility Real-time Business Insights Proactive Reactive Machine Data Universe 6
    7. 7. ASingleSolutionforOperationalIntelligence Real-time Visibility • Live dashboards • Event correlation • Monitoring and alerting • Performance issues • Transaction levels • SLA tracking ThreePrimaryCapabilities Historical Analytics • Baseline and thresholds • Trending • Operational insights • Historical patterns • Compliance reports Single Data Store Single UI Across Use Cases Search / Navigation • Data drilldown • “Needle in a haystack” • Root cause analysis / troubleshooting • Incident investigations 7
    8. 8. Splunk Delivers Value Across the Enterprise 8 Web Intelligence Application Mgmt Security & Compliance IT Operations Business Analytics
    9. 9. Education Healthcare Technology Energy and Utilities Manufacturing Telecommunications Cloud and Online Services Government Retail Financial Services and Insurance Media Travel and Leisure Proven at 5,200+ Customers in 90+ Countries 9 Over Half the Fortune 100
    10. 10. Splunk In Virtualized Datacenters
    11. 11. The Virtualized Datacenter Problem End user devices End user devices End user devices End user devices Virtualized Servers Storage Networking Connection Broker Active Directory /LDAP Virtual Machines Networking Security Shared Resources = Shared Problems No ability to link user or application level issues with hardware or hypervisor problems No way to go back in time and recreate environment state
    12. 12. How Does Splunk Help? End user devices End user devices End user devices End user devices Virtualized Servers Storage Networking Connection Broker Active Directory /LDAP Virtual Machines Networking Security Talks to every technology in your stack Correlates data across the different tiers – find causal links Built for Big Data - Visualize, analyze, trend all your data at large scale
    13. 13. Splunk App for VMware
    14. 14. Splunk App for VMware Complete Operational Visibility Into VMware Environments • Collects and persists logs and performance metrics directly from ESX/i hosts , avoiding the VC bottleneck • Integrates data with VC topology information • Collects and persists tasks & events from VC to get complete visibility into actions • Pre-built dashboards and views to showcase initial use cases APP OS VM VMware vSphere VM Physical Layer ServersStorage Network Devices APP OS VMware vCenter Server(VC)
    15. 15. Why Use the Splunk App for VMware? • Harness virtualization layer data – Special forwarders for ESX/ESXi and VC – Comprehensive data collection at scale – Initial visualizations to navigate the data The Power of Splunk – Mash up data with all other technology tiers – Correlate, analyze, visualize – Monitor and alert in real time – Adaptive reporting and dash boarding
    16. 16. Unlock the Value of your Virtualization Data
    17. 17. How It Works Splunk App for VMware > Splunk Add-on for vCenter > Splunk UF/LF vCenter server Provides: Dashboards, Views, Field Extractions From ESX/i Host: Host Inventory / Hierarchy, Time, Performance*, a nd Log Data From VC: VC Logs, Time Data Splunk virtual FA for VMware > VMware Perl SDK From VC: VC Inventory / Hierarchy, Tasks, and Events Data * Performance data at 20 s granularity
    18. 18. Data Volume 18 • 1 Forwarder Appliance is required per 20-30 physical ESX hosts –we don’t charge for the app or the appliances • Each host typically generates 400MB-1 GB of data per day – Includes logs, tasks & events, inventory & hierarchy , “aggregated” metrics data – “Per instance” metrics collection is an additional ~1 GB – this is turned off by default – Configuration choices for data types available
    19. 19. What Differentiates The Splunk App for VMware 19
    20. 20. End to End Visibility 20 Splunk used to correlate the business data (Users , Usage) with the IT/Infrastructure data Understand resource/usage and cost per customer Monitor the entire environment from server, storage, network, hypervisors, custom cloud back-end for possible SLA issues, trouble spots and more We have deep visibility and correlation across all tiers of our cloud infrastructure – giving us not only ongoing monitoring of key datacenter statistics, but also giving us business visibility into customer experience and usage” “ Elad Gotfrid, Manager of IT
    21. 21. One Splunk – Many Uses 21 Using Splunk for VMware gets us our data in one place, for many uses: capacity planning, event monitoring, performance analysis, security monitoring and more.. “ ” Helps retain a definitive record of what happened in our environment Analyze and trend performance as well as user activities very easily Useful for both operational monitoring, capacity usage, performance metrics and for security monitoring Peter Cole Technical Lead, ITS Operations
    22. 22. Detailed History For Analysis & Troubleshooting 22 I love that I can track virtual machines in my environment as they move from host to host.I can now identify the root cause of issues or errors. Matthew Cluver Network Operations Analyst “ ” Splunk already used for operating system and applications event monitoring & analysis For the first time, they have insight into granular virtualization layer data – helps solve problems immediately
    23. 23. Benefits of the Splunk Approach 1. Collects all the data without interfering with the normal operation of VC – Data how you want it, when you want it “We really wanted to get our business units off of VC for reporting, this is a great way to serve up the data they need” -Melbourne IT 2. Persists the data at scale, hard to do with any other tool “Its comforting to know the data is in Splunk for me to go back and resolve any issue” -Nancy Kafer, Homesteader’s Life Insurance 3. Enables correlation with application data, network data, storage data “Splunk helps me correlate application performance issues with hypervisor level sharing issues”. -Major Mobile Technology Provider
    24. 24. Why Splunk Over Everyone Else? You don’t know what data you will need till you need it – Every other tool goes through VC - only has access to 5 min summaries of data, Splunk collects 20 second granularity – Most don’t even incorporate log data 24 “Splunk already paid for itself, it helped us find an issue with SCSI resets on storage that we wouldn’t have found otherwise” -Commerzbank Splunk isn't JUST for virtualization – it is for everything – Can be used for many use cases– capacity, configuration monitoring, security, compliance etc – Can be used with any other technology in the stack “We initially wanted something to aggregate the logs, but now we use the detailed data from the VMs and the hosts to optimize resource allocation in our environment” -Major Energy Company
    25. 25. What Customers Are Saying 25 “Thank you for bypassing VC – it resolves a huge bottleneck in our environment”. “VC shows me information for now, but I cant see what yesterday looked like or what this time last month looked like” “I really like the changed events tracking, let’s me see exactly what people are doing in the environment”
    26. 26. Why Use the Splunk App for VMware Isn’t this the data I can see in VC? NO • VC has only 5 minute summaries of performance data which hides problems rather than surface them • VC summarizes this data further after 12 hours, you cannot go back to the level of detail • Even the data it does retain is hard to report on • Try constructing a dashboard in VC that has VM CPU metrics, tasks associated with the VM, host storage metrics, tasks associated with the host, metrics reported inside the VM
    27. 27. Why Use the Splunk App for VMware VCOps tells me what to do and “predicts” the health of my environment VCOps is virtualization only – we complement it for everything else The VCOps sku that lets you integrate in data from non-VMware sources is 4 times more expensive – integrating other data sources is default in Splunk VCOps predictive algorithms are black box – you can’t teach it what you already know and it takes a while to learn Does not have reporting Limited scale
    28. 28. Screenshots
    29. 29. 29 Track any metric historically as the VM moves from host to host
    30. 30. 30 View any tasks performed/changes made to the host or vms
    31. 31. 31 Detailed host and VM metrics including disk latencies, IO, storage queue depths..
    32. 32. Configurable Capacity Reporting 32 Cluster level/Host level/VM level
    33. 33. Log Browsing 33
    34. 34. Datastore Capacity Reporting 34 How much storage is being taken up by snapshots? How much are my datastores over- provisioned by?
    35. 35. Thank You
    36. 36. Backup Slides
    37. 37. What is vSphere? Virtualization Layer –Abstracts and Pools Server, Storage, Networking Installs on servers, managed by one or many vCenter Servers Applications running inside virtual machines have no knowledge of who else they are sharing compute, storage or networking capacity with Not just CPU/Memory/Storage virtualization : – Resource aggregation – Automated load balancing – High Availability APP OS VM VMware vSphere VM Physical Layer ServersStorage Network Devices APP OS VMware vCenter Server(VC)
    38. 38. What Could Possibly Go Wrong? Nothing. We run 1 VM per host, attached to local disk. No Problems. CPU Memory Network Storage HA agent crashes, behaves badly DRS causes too many vMotion s vMotion fail to execute Contention among VMs for resources manifests as application performance problems Unauthorized changes by unaware people Hypervisor functionality failures Reality

    ×