Uploaded bySplunk
Getting Started with Splunk Enterprise
Splunk Enterprise is a software platform for searching, monitoring, and analyzing machine-generated big data, such as logs, metrics, and mobile data. The presentation provided an overview of Splunk Enterprise capabilities including: live demonstrations of installing Splunk, searching data, creating dashboards and alerts. It also covered Splunk deployment architectures for scaling from single instances to distributed environments supporting hundreds of terabytes per day.
More Related Content
Similar to Getting Started with Splunk Enterprise
Getting Started with Splunk Enterprise
- 1. Copyright © 2015Splunk Inc. Getting Started with Splunk Enterprise Jon Nussbaum Sr. Splunk Sales Engineer
- 2. 2 Agenda 1. Splunk EnterpriseOverview 2. Using Splunk (Live Demonstration/Walkthrough) • Installing & Onboard Data • Searching • Field Extraction • Dashboards • Alerting • Analytics • Pivot Interface 3. Splunk Deployment Architecture 4. Splunk Communities, Q&A
- 3. 3 Safe Harbor Statement Duringthe course of this presentation, we may make forward-looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward- looking statements made in this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward-looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release. 3
- 4. 8 THE Industry LeadingPlatform For Machine Data Machine Data: Any Location, Type, Volume Online Services Web Services Servers Security GPS Location Storage Desktops Networks Packaged Applications Custom ApplicationsMessaging Telecoms Online Shopping Cart Web Clickstreams Databases Energy Meters Call Detail Records Smartphones and Devices RFID On- Premises Private Cloud Public Cloud Platform Support (Apps / API / SDKs) Enterprise Scalability Universal Indexing Answer Any Question Developer Platform Report and analyze Custom dashboards Monitor and alert Ad hoc search Universal Machine Data Platform No backend database Schema-on-the-fly No need to filter data Quick time to value Agile statistics and reporting Real-time architecture
- 5. 9 IT Service Intelligence ITS I Mainframe Data VMware Platform forMachine Data Splunk also offers a wealth of apps to address many use cases Exchange PCISecurity DB Connect MobileForwarders Syslog / TCP / Other Sensors & Control Systems Rich Ecosystem of Apps Stream Passionate and Vibrant Community 750 1000 free apps on Splunkbase.com Packet Analysis (Wire Data) - App Response Time - Detect unauthorized access Import & Correlate external DB data - 3rd party tools - Enrich data already in Splunk Place Splunk search & analytics on top of Hadoop/noSQL cluster Mobile Application Performance Management (APM) - App Crashes - User Experience
- 6. Installing & Using Splunk (LiveDemonstration & Walk-along)
- 7. 11 1. 2. 3. 4. Getting Started Download Install &Start Forward Data Search Databases Networks Servers Virtual Machines Smart phones and Devices Custom Applications Security WebServer Sensors Four steps:
- 8. 12 1. Download SplunkEnterprise - http://www.splunk.com/en_us/download/splunk-enterprise.html – Or google “splunk download” -> Download Splunk Enterprise for Free 1. Download Tutorial Data – http://www.splunkbook.com , 3rd link under “Related Links” Downloading Splunk Enterprise + Tutorial Data
- 9. 13 Connect to WiFi:“SplunkLiveDownloads” – PW= splunk16 For Windows: 10.10.10.254 username=Admin password=splunk16 – Download the MSI that maps to your architecture under the Windows directory – Install the MSI – Start Splunk from Windows Services after you’ve installed. – Download tutorialdata.zip For OSX: Finder -> Connect to Server -> cifs://10.10.10.254 User=Admin Password=splunk16 – Download tarball from the OSX Directory – mkdir /opt/splunk_live_sd – cd /opt/splunk_live_sd ; tar –xzvf ~/Downloads/splunk-6.3.3-f44afce176d0-darwin-64.tgz – /opt/splunk_live_sd/splunk/bin/splunk start – Download tutorialdata.zip For Linux: You tell me! MD5’s – MD5 (splunk-6.3.3-f44afce176d0-x64-release.msi) = 40fbac74c3113a28884463b671569138 – MD5 (splunk-6.3.3-f44afce176d0-x86-release.msi) = 61960b0c42b523afc929fafb553be1c6 – MD5 (splunk-6.3.3-f44afce176d0-Linux-x86_64.tgz) = 6431246b114aed0d79677e3c04f78b97 – MD5 (splunk-6.3.3-f44afce176d0-darwin-64.tgz) = 5a2bf750b67860b64f58ff32855e92fd Downloading Splunk Enterprise + Tutorial Data
- 10. Let’s get ourhands dirty! Install & Initial Setup
- 11. 15 Download the samplefile, follow this link and save the file to your desktop, IMPORT THE ZIP FILE, not individual files within it: http://www.splunkbook.com (sample data is located under ‘related links’ section) Log into Splunk – http://127.0.0.1:8000 username=admin password=changeme To add the file to Splunk: – Click Add Data – Click Upload files from my computer. – Drag and drop you sample data zip file. – Add a new Index – Review and Finish. Getting Data into Splunk We will import sample web ecommerce store events
- 12. 16 Search All YourMachine Data Search all your data Results right away Schema on the fly • Search across real-time and historical data • Over 135 search commands built in • See results instantly Data ParsingQueue Parsing Pipeline • Source, event typing • Character set normalization • Line breaking • Timestamp identification • Regex transforms Indexing Pipeline Real- time Buffer Raw data Index Files Real-time Search Process Monitor Input IndexQueue TCP/UDP Input Scripted Input Splunk Index
- 13. Time to usethe magnifying glass
- 14.
- 15. What If Iwant to create my own field extraction? Like what Web Browser Is used?
- 16.
- 17.
- 18. 23 Actionable Alerting Alerts • Createalerts based on any search • Customize content and format of email alerts • Provide context • Highlight next steps • Enable custom workflows • Trigger an action • SMS alert • SNMP trap • Other
- 19. 24 Data Analysis Over 135different analytic commands – Transaction – Anomaly Detection – Prediction Geospatial Visualization – Visualizes metric variance across a customizable geographic area Single Value Display – At-a-glance, single-value indicators with useful context 2
- 20. 25 Alternative to SearchLanguage: Pivot Interface • Drag-and-drop interface enables any user to analyze data • Create complex queries and reports without learning search language • Click to visualize any chart type; reports dynamically update when fields change Select fields from data model Time window All chart types available in the chart toolbox Save report to share Pivot
- 21.
- 22. 27 Single Instance orDistributed? Single environment Distributed Environment Recommended Specs: 6X2 Core CPUs/12GB RAM/800+ IOPs A Splunk install can be one or all roles…
- 23. 28 Scales to Hundredsof TBs/Day Enterprise-class Scale, Resilience and Interoperability Collect machine data from thousands sources via Splunk forwarders Compress and store data on Splunk Indexers Initiate searches and visualize results via Search Heads Forwarders Indexer Search Head
- 24. 29 Scalability & HighAvailability Forwarders load balance across Indexers Indexed data can be replicated across peers and different physical sites Search Heads can be Clustered to eliminate single point of failure and handle large search loads
- 25. 30 Over 900 Apps@ http://apps.splunk.com 3
- 26. 31 Time to startSPLUNKING!!! Documentation – http://www.splunk.com/base/Documentation Technical Support – http://www.splunk.com/support Videos – http://www.splunk.com/videos Education – http://education.splunk.com Community – http://answers.splunk.com • Splunk Book – http://splunkbook.com Where do I go for help?
- 27. Copyright © 2015Splunk Inc. • September 26-29, 2016 • The Disney Swan and Dolphin, Orlando • 5000+ IT & Business Professionals • 3 days of technical content • 165+ sessions • 3 days of Splunk University • Sept 24-26, 2016 • Get Splunk Certified for FREE! • Get CPE credits for CISSP, CAP, SSCP • Save thousands on Splunk education! • 80+ Customer Speakers • 35+ Apps in Splunk Apps Showcase • 75+ Technology Partners • 1:1 networking: Ask The Experts and • Security Experts, Birds of a Feather and Chalk Talks • NEW hands-on labs! • Expanded show floor, Dashboards Control Room & Clinic, and MORE! .conf2016: The 7th Annual Splunk Worldwide Users’ Conference
- 28.
- 29. 34 Queries used insession All errors events: – index=* sourcetype=access_combined_wcookie status != 200 Hits by uri_path (perfect for a pie chart): – index=* sourcetype=access_combined_wcookie | stats count by uri_path Timechart of error by HTTP code: – index=* sourcetype=access_combined_wcookie status != 200| timechart count by status limit=10 Timechart of errors by URL: – index=* sourcetype=access_combined_wcookie status != 200 | timechart count by uri_path Charting options for timechart
- 30. 35 Queries used insession Long Transactions index=* sourcetype=access_combined_wcookie | transaction maxspan=1m JSESSIONID clientip | table clientip duration | where duration>10 | sort –duration Anomalies: index=* sourcetype=access_combined_wcookie status!=200 |timechart count | anomalies index=* sourcetype=access_combined_wcookie status!=200 | timechart count | anomalous values Prediction/Projection: index=* sourcetype=access_combined_wcookie status!=200 | timechart count | predict count index=* sourcetype=access_combined_wcookie status!=200 | timechart span=2h count | predict future_timespan=100 count Sample query to return all available geo values for an IP: index=* sourcetype=access_combined_wcookie | iplocation prefix=iploc_ allfields=true clientip | fields iploc_* clientip Sample query to map the location by city: index=* sourcetype=access_combined_wcookie | iplocation allfields=true clientip | geostats count by City Leveraging Chloropleth feature in Splunk 6.3 to map by country: index=* sourcetype=access_combined_wcookie | iplocation allfields=true clientip | eval featureId=Country |stats count by featureId | geom geo_countries NOTE: If some queries don’t work, there might be some foreign characters. Try typing the query
- 31. 36 Field Extractions 1. Performa search for a specific sourcetype 2. Expand an event of interest and select ‘Extract Fields’ 3. From Field Extraction UI, select ‘regular expression’ 4. Highlight the field of interest, and define a field name 5. Add a non-matching event (from the non-matching events tab) 6. Add desired text to match 7. Go through remaining steps
Editor's Notes
- #5 At the same time, organizations are creating and capturing more data every day. Most of that data is machine data coming from webservers, network devices, sensors, mobile devices – all of the technology infrastructure running an enterprise. This data is valuable because it contains a categorical record - of user behavior, cyber-security risks, application behavior, service levels, and customer experience. At the same time, these massive streams of data come in an array of unpredictable formats that are difficult to process and analyze in a timely manner by traditional methods.
- #6 Unlike traditional structured data or multi-dimensional data, machine data is non-standard, highly diverse, dynamic and high volume. You’ll notice that machine data events are also typically time-stamped – or time-series data. Take this example of purchasing a product on your tablet or smartphone: the purchase transaction fails, you call the call center and then tweet about your experience. All these events are captured as they occur in the machine data generated by the different systems supporting these different interactions. Each of the underlying systems can generate millions of machine data events daily that can be very difficult to understand using traditional methods.
- #7 When we look more closely at the data we see that it contains valuable information – customer id, order id, time waiting on hold, twitter id … what was tweeted. If you can correlate and visualize related events across these disparate sources, you can build a picture of activity, behavior and experience.
- #8 And that’s exactly what Splunk empowers organizations to do. Correlating high volume machine data from disparate data sources in real-time without the need to transform the data so organizations can make more informed decisions at the pace of their business.
- #9 All of this is accomplished with: No backend database No custom connectors Without filtering data – no need to filter data for questions that you have now, index all your data to support questions that you’ll have in the future. Without knowing the questions before hand. While Providing a quick time to value With agile statistics and reporting All in real-time
- #10 The Splunk platform consists of multiple products and deployment models to fit your needs. At the core we have the universal machine data platform that underpins our four key technology offerings. These include: Splunk Enterprise – for on-premise deployments Splunk Cloud – Fully managed service with 100% SLA and all the capabilities of Splunk Enterprise…in the Cloud (currently only available in North America) Hunk – for analytics on data in Hadoop Splunk Mint – to get insights into data from Mobile devices The products can pull in data from virtually any source to support multiple use cases. On top of deployments for Splunk Enterprise and Splunk Cloud, Splunk Apps extend and simplify deployments by providing pre-packaged content designed for specific use cases and data types. There is a rich ecosystem of apps that come from the Splunk community. And there are premium apps that are developed and fully supported by Splunk. You see those at the top here. For the rest of the presentation, we’ll focus on explaining Splunk Enterprise.
- #12 It only takes minutes to download and install Splunk on the platform of your choice, bringing you fast time to value. Once Splunk has been downloaded and installed the next step is to get data into a Splunk instance. The data then becomes searchable from a single place! Since Splunk stores only a copy of the raw data, searches won’t affect the end devices data comes from. Having a central place to search your data not only simplifies things, it also decreases risk since a user doesn’t have to log into the end devices. Splunk can be installed on a single small instance, such as a laptop, or installed on multiple servers to scale as needed. The ability to scale from a single desktop to an enterprise is another of our key differentiators. When installed on multiple servers the functions can be split up to meet any performance, security, or availability requirements.
- #17 Allows you to search all your data in one place in real time. The search interface operates very similar to doing a search on any web search engine. Any user can become powerful very quickly with preexisting knowledge of using a search engine; however, Splunk has created over 100 commands (135 published) to make analyzing the data quicker and easier. <This is a great time to start a demo and show the search language if giving a demo> The schema on the fly approach is a key differentiator with Splunk.
- #19 Applying a schema at the last possible moment allows for the greatest flexibility when asking questions of your data. Splunk enterprise will atomically extract the values from the fields in events. If the data source is updated and new fields are added or the format of the events change Splunk does not need to re-index the data. Sometimes the raw event doesn’t contain useful enough information and it needs to be enriched.
- #24 Alerts are triggered when certain conditions are met by the results of the search upon which it is based. Alerts can be based on both historical and real-time searches. When an alert is triggered, it performs an alert action. This action can be the sending of the alert information to a designated set of email addresses, or the posting of the alert information to an RSS feed. Alerts can also be set up to run a custom script when they are triggered. You can base these alerts on a wide range of threshold and trend-based scenarios, including empty shopping carts, brute force firewall attacks, and server system errors.
- #25 Splunk now offers a new set of visualization and analytics features that are targeted to help address user challenges of big data analysis. Enhanced Anomaly detection: Helps you rapidly discover events that merit further investigation Geospatial mapping: Lets you visualize and better communicate results using geographic or custom-defined areas Single value display: Gives you "at-a-glance" indicators and relevant contextual data for war-room displays and management discussions Sample commands: TRANSACTION index=* sourcetype=access_combined_wcookie | transaction JSESSIONID clientip index=* sourcetype=access_combined_wcookie | transaction maxspan=1m JSESSIONID clientip | table clientip duration | where duration>10 | sort -duration ANOMALY index=* sourcetype=access_combined_wcookie status!=200 | timechart count | anomalies index=* sourcetype=access_combined_wcookie status!=200 | timechart count | anomalous values PREDICT index=* sourcetype=access_combined_wcookie status!=200 | timechart count | predict count index=* sourcetype=access_combined_wcookie status!=200 | timechart span=2h count | predict future_timespan=100 count
- #26 The Pivot interface enables non-technical and technical users alike to quickly generate sophisticated charts, visualizations and dashboards using simple drag and drop and without learning the Search Processing Language (SPL). Users can access different chart types from the Splunk toolbox to easily visualize their data different ways. Queries using the Pivot interface are powered by underlying “data models” which define the relationships in Machine Data.
- #29 Splunk Forwarders are lightweight components which collect Machine data throughout your environment. Forwarder deployment is highly customizable, you can have the forwarder remotely collect data or place the forwarder locally on hundreds of thousands of devices as some of our customers do. Forwarders automatically load-balance their collected machine data across a pool of Indexers, which scale horizontally on commodity hardware to adjust to your growing pool of Machine Data. Search Heads initiate map-reduced searches across the indexer tier, combine and return the results to the Splunk console or your interface of choice. Like Indexers, Search Heads can scale horizontally to meet your needs on commodity hardware.
- #31 More than 600 are available for download on our Website. These Apps significantly reduce the time to value and make it easy for customers to extend their visibility across common sources or use cases.
- #33 We’re headed to the East Coast! 2 inspired Keynotes – General Session and Security Keynote + Super Sessions with Splunk Leadership in Cloud, IT Ops, Security and Business Analytics! 165+ Breakout sessions addressing all areas and levels of Operational Intelligence – IT, Business Analytics, Mobile, Cloud, IoT, Security…and MORE! 30+ hours of invaluable networking time with industry thought leaders, technologists, and other Splunk Ninjas and Champions waiting to share their business wins with you! Join the 50%+ of Fortune 100 companies who attended .conf2015 to get hands on with Splunk. You’ll be surrounded by thousands of other like-minded individuals who are ready to share exciting and cutting edge use cases and best practices. You can also deep dive on all things Splunk products together with your favorite Splunkers. Head back to your company with both practical and inspired new uses for Splunk, ready to unlock the unimaginable power of your data! Arrive in Orlando a Splunk user, leave Orlando a Splunk Ninja! REGISTRATION OPENS IN MARCH 2016 – STAY TUNED FOR NEWS ON OUR BEST REGISTRATION RATES – COMING SOON!