SplunkLive! Zürich 2014 Beginner Workshop: Getting started with Splunk


Published on

Published in: Software, Technology
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

SplunkLive! Zürich 2014 Beginner Workshop: Getting started with Splunk

  1. 1. Copyright © 2013 Splunk Inc. Zürich, 8. April , 2014 Technical Workshops Getting Started User Training Getting Started User Training Workshop Holger Sesterhenn Senior Sales Engineer
  2. 2. Agenda • Getting Started with Splunk • Search • Alert • Dashboard • Deployment and Integration • Community • Help & Questions 2
  3. 3. Getting Started With Splunk
  4. 4. IT Operations Security and Compliance Digital Intelligence App Dev and App Mgmt. Developer Platform (REST API, SDKs) Business Analytics Industrial Data and Internet of Things Small Data. Big Data. Huge Data. Splunk Delivers Value Across IT and the Business
  5. 5. Install Splunk Splunk Home • WIN: Program FilesSplunk • Other: /opt/splunk (Applications/splunk) Start Splunk • WIN: Program FilesSplunkbinsplunk.exe start (services start) • *NIX: /opt/splunk/bin/splunk start www.splunk.com/download • 32 or 64 Bit? • Indexer or Universal Forwarder?
  6. 6. Splunk Licenses Free Download Limits Indexing to 500MB/day • Enterprise Trial License expires after 60 days • Reverts to Free License Features Disabled in Free License • Multiple user accounts and role-based access controls • Distributed search • Forwarding to non-Splunk Instances • Deployment management • Scheduled saved searches and alerting • Summary indexing Other License Types • Enterprise, Forwarder, Trial
  7. 7. Default installation on: http://localhost:8000 7 Splunk Web Basics Browser Support • Firefox 10.x and latest • Internet Explorer 7, 8, 9 and 10 • Safari (latest) • Chrome (latest) Index data • Add data • Getting Started App • Install an App (Splunk for Windows, *NIX)
  8. 8. 8 Splunk Web Basics continued… Splunk Home • Provides Interactive portal to the Apps & data. • Includes a search bar and three panels: 1 – Apps 2 – Data 3 - Help Splunk Apps • Splunk Home  Find more apps • Provide different contexts for your data out of sets of views, dashboards, and configurations • Default Search App • You can create your own!
  9. 9. Optional: add some test data Download the sample file, follow this link and save the file to your desktop, then unzip: http://www.splunkbook.com (Using Splunk Book) To add the file to Splunk: – From the Welcome screen, click Add Data. – Click From files and directories on the bottom half of the screen. – Select Skip preview. – Click the radio button next to Upload and index a file. – Click Save. 9
  10. 10. Best Practice Suggestion: Create an individual Index based on sourcetype. • Easier to re-index data if you make a mistake. • Easier to remove data. • Easier to define permissions and data retention. 10
  11. 11. Search Basics
  12. 12. Search app – Summary viewcurrent view global stats app navigation time range picker Selecting Data Summary: • Host • Source • Sourcetype start search search box
  13. 13. Searching 13 Search > * Select Time Range • Historical, custom, or real-time Select Mode • Smart, Fast, Verbose Using the timeline • Click events and zoom in and out • Click and drag over events for a specific range
  14. 14. 14 Everything is searchable Everything is searchable • * wildcards supported • Search terms are case insensitive • Booleans AND, OR, NOT – Booleans must be uppercase – Implied AND between terms – Use () for complex searches • Quote phrases fail* fail* nfs error OR 404 error OR failed OR (sourcetype=access_*(500 OR 503)) "login failure"
  15. 15. Example Search: 15
  16. 16. Search Assistant 16 Contextual Help - advanced type-ahead History - search - commands Search Reference - short/long description - examples suggests search terms updates as you type shows examples and help toggle off / on
  17. 17. Searches can be managed as asynchronous processes Jobs can be • Scheduled • Moved to background tasks • Paused, stopped, resumed, finalized • Managed • Archived • Cancelled Job Management Modify Job Settings pause finalize delete 17
  18. 18. Search Commands 18 Search > error | head 1 Search results are “piped” to the command Commands for: • Manipulating fields • Formatting • Handling results • Reporting
  19. 19. Over 130 Commands! 19 splunk.com > Documentation > Search Reference abstract accum addcoltotals addinfo addtotals af analyzefields anomalies anomalousvalue append appendcols ar associate audit autoregress bin bucket chart cluster collect common contingency convert correlate counttable crawl ctable dbinspect dedup delete delta diff discretize erex eval eventcount eventstats excerpt extract file fillnull folderize format gentimes head highlight iconify input inputcsv inputlookup iplocation join kmeans kv kvform loadjob localize localop lookup macro makecontinuous makemv maketable map metadata multikv mvcombine mvexpand nomv outlier outlierfilter outputcsv outputlookup outputtext overlap rangemap rare regex relevancy rename replace reverse run savedsearch savedsplunk script scrub selfjoin sendemail set sichart sirare sistats sitimechart sitop slc stash strcat streamstats sumindex summaryindex tail test timechart top transaction transam trendline typeahead typelearner typer uniq untable xmlkv xmlunescape xpath xyseries http://www.splunk.com/base/Documentation/latest/SearchReference/SearchCheatsheet
  20. 20. Field Extraction Fun
  21. 21. Fields 21 Default fields • host, source, sourcetype, linecount, etc. • View on left panel in search results or all in field picker Where do fields come from? • Pre-defined by sourcetypes • Automatically extracted key-value pairs • User defined
  22. 22. Sources, Sourcetypes, Hosts • Host - hostname, IP address, or name of the network host from which the events originated • Source - the name of the file, stream, or other input • Sourcetype - a specific data type or data format 2 2
  23. 23. 23 Tagging and Event Typing Eventtypes for more human-readable reports • to categorize and make sense of mountains of data • punctuation helps find events with similar patterns Search > eventtype=failed_login instead of Search > “failed login” OR “FAILED LOGIN” OR “Authentication failure” OR “Failed to ………………authenticate user” Tags are labels • apply ad-hoc knowledge • create logical divisions or groups • tag hosts, sources, fields, even eventtypes Search > tag=web_servers instead of Search > host=“apache1.splunk.com” OR host=“apache2.splunk.com” OR …………….host=“apache3.splunk.com”
  24. 24. Extract Fields 24 Interactive Field Extractor • generate PCRE • editable regex • preview/save
  25. 25. Extract Fields 25 Interactive Field Extractor • generate PCRE • editable regex • preview/save props.conf [mysourcetype] REPORT-myclass = myFields transforms.conf [myFields] REGEX = ^(w+)s FORMAT = myFieldLabel::$1 Configuration File • manual field extraction • delim-based extractions Rex Search Command ... | rex field=_raw "From: (?<from>.*) To: (?<to>.*)"
  26. 26. Saved Search & Alert Basics
  27. 27. Saved Searches 27 Leverage Searches for future Insights! • Reports • Dashboards • Alerts • Eventtypes Add a Time Range Picker • Preset • Relative • Real-time • Date-Range • Date & Time Range • Advanced
  28. 28. Create Alerts 28 Scheduled or Real-Time • Define Time Ranges • Conditions • Thresholds
  29. 29. Alerting Continued… 29 Searches run on a schedule and fire an alert • Example: Run a search for “Failed password” every 15 min over the last 15 min and alert if the number of events is greater than 10 Searches are running in real-time and fire an alert • Example: Run a search for “Failed password user=john.doe” in a 1 minute window and alert if an event is found
  30. 30. Alerting Actions 30 • Send email • RSS • Execute a script • Track Alert Details
  31. 31. Report & Dashboard Wackiness
  32. 32. Reporting 32 results of any search Define your Search and set your time range, accelerate you search and more Choose the type of chart (line, area, column, etc) and other formatting options Build reports from
  33. 33. Reporting Examples 33 • Use wizard or reporting commands (timechart, top, etc) • Build real-time reports with real-time searches • Save reports for use on dashboards
  34. 34. Dashboards 34 Create dashboards from search results
  35. 35. Dashboard Examples 35
  36. 36. Manager Settings 36 For All of that Cool Stuff You Just Created (and more!) • Permissions • Saved Searches/Reports • Custom Views • Distributed Splunk • Deployment Server • License Usage….
  37. 37. Deployment and Integration
  38. 38. Splunk Has Four Primary Functions 38 • Searching and Reporting (Search Head) • Indexing and Search Services (Indexer) • Local and Distributed Management (Deployment Server) • Data Collection and Forwarding (Forwarder) A Splunk install can be one or all roles…
  39. 39. Getting Data Into Splunk 39 Agent and Agent-less Approach for Flexibility perf shell code Mounted File Systems hostnamemount syslog TCP/UDP WMI Event Logs Performance Active Directory syslog compatible hosts and network devices Unix, Linux and Windows hosts Windows hosts Custom apps and scripted API connections Local File Monitoring log files, config files dumps and trace files Windows Inputs Event Logs performance counters registry monitoring Active Directory monitoring virtual host Windows hosts Scripted Inputs shell scripts custom parsers batch loading Agent-less Data Input Splunk Forwarder
  40. 40. Understanding the Universal Forwarder 40 Forward data without negatively impacting production performance. Scripts Universal Forwarder Deployment Logs ConfigurationsMessages Metrics Central Deployment Management Monitor files, changes and the system registry; capture metrics and status. Universal Forwarder Regular (Heavy) Forwarder Monitor All Supported Inputs ✔ ✔ Routing, Filtering, Cloning ✔ ✔ Splunk Web ✔ Python Libraries ✔ Event Based Routing ✔ Scripted Inputs ✔
  41. 41. Horizontal Scaling 41 Load balanced search and indexing for massive, linear scale out. Forwarder Auto Load Balancing Distributed Search
  42. 42. Multiple Datacenters 42 Headquarters London Hong Kong Tokyo New York Distributed Search Index and store locally. Distribute searches to datacenters, networks & geographies.
  43. 43. High Availability, On Commodity Servers and Storage 43 As Splunk collects data, it keeps multiple identical copies If indexer fails, incoming data continues to get indexed Indexed data continues to be searchable Easy setup and administration Data integrity and resilience without a SAN Index Replication Splunk Universal Forwarder Pool Constant Uptime
  44. 44. Service Desk Event Console SIEM Send Data to Other Systems 44 Route raw data in real time or send alerts based on searches.
  45. 45. Integrate External Data 45 LDAP, AD Watch Lists CRM/ER P CMDB Correlate IP addresses with locations, accounts with regions Extend search with lookups to external data sources.
  46. 46. Integrate Users and Roles 46 Problem Investigation Problem Investigation Problem Investigation Save Searches Share Searches LDAP, AD Users and Groups Splunk Flexible Roles Manage Users Manage Indexes Capabilities& Filters NOT tag=PCI App=ERP … Map LDAP & AD groups to flexible Splunk roles. Define any search as a filter. Integrate authentication with LDAP and Active Directory.
  47. 47. Centralized Licensing Management 47 Problem Investigation Groups, Stacks, and Pools for Enterprise Deployments
  48. 48. Deployment Monitoring 48 Keep Tabs On Your Splunk Enterprise Deployment ForwardersIndexersSourcetypesLicenses
  49. 49. Support and Community
  50. 50. Support Through the Splunk Community 50 Browse and share Apps from Splunk, Partners and the Community apps.splunk.com Splunkbase Community-driven knowledge exchange and Q&A answers.splunk.com 5 tracks, more than 40 sessions, the smartest Splunk users together conf.splunk.com .conf2014
  51. 51. Where to Go for Help 51 • Documentation – http://www.splunk.com/base/Documentation • Technical Support – http://www.splunk.com/support • Videos – http://www.splunk.com/videos • Education – http://www.splunk.com/goto/education • Community – http://answers.splunk.com – http://apps.splunk.com • Splunk Book – http://splunkbook.com
  52. 52. Thank you November 12st, 2012 Technical Workshops Getting Started User Training