SlideShare a Scribd company logo
1 of 125
© 2017 SPLUNK INC.© 2017 SPLUNK INC.
Machine Data Workshop 101
Beyond the Basics
© 2017 SPLUNK INC.
Splunk Approach to Machine Data
SQL Search
Schema at Write Schema at Read
Traditional Splunk
ETL Universal Indexing
Volume Velocity Variety
UnstructuredStructured
RDBMS
© 2017 SPLUNK INC.
Industry Leading Platform For Machine Data
Custom
dashboards
Report and
analyze
Monitor
and alert
Developer
Platform
Ad hoc
search
On-Premises
Private Cloud
Public
Cloud
Storage
Online
Shopping Cart
Telecoms
Desktops
Security
Web
Services
Networks
Containers
Web
Clickstreams
RFID
Smartphones
and Devices
Servers
Messaging
GPS
Location
Packaged
Applications
Custom
Applications
Online
Services
DatabasesCall Detail
Records
Energy MetersFirewall
Intrusion
Prevention
Platform Support (Apps / API / SDKs)
Enterprise Scalability
Universal Indexing
Machine Data: Any Location, Type, Volume Answer Any Question
Any Amount, Any Location, Any Source
Schema
on-the-fly
Universal
indexing
No
back-end
RDBMS
No need
to filter
data
© 2017 SPLUNK INC.
The Splunk Portfolio
Rich Ecosystem of
Apps & Add-Ons
Splunk Premium
Solutions
Mainframe
Data
Relational
Databases
MobileForwarders
Syslog/
TCP
IoT
Devices
Network
Wire Data
Hadoop
Platform for Operational Intelligence
© 2017 SPLUNK INC.
▶ Non-Traditional Data Sources
▶ Data Enrichment
▶ Level Up on Search and Reporting Commands
▶ Data Models and Pivot
▶ Custom Visualizations and the Web Framework
Agenda
© 2017 SPLUNK INC.
Workshop Setup
© 2017 SPLUNK INC.
Download Splunk or Sign Up For Splunk Cloud
www.splunk.com > Free Splunk > Splunk Enterprise or Splunk Cloud
SHOW
1
2
3
© 2017 SPLUNK INC.
▶ Box > access_datasample_last4h.log
▶ Box > http_status.csv
Download Data Sample and Lookup
https://splunk.box.com/v/MD101Workshop
SHOW
© 2017 SPLUNK INC.
▶ Browser: http://localhost:8000
▶ Default username/password is admin/changeme
Index Data SampleSHOW
1
2
© 2017 SPLUNK INC.
Index Data SampleSHOW
3
2
1
4
5
© 2017 SPLUNK INC.
Index Data SampleSHOW
1
2
© 2017 SPLUNK INC.
Index Data SampleSHOW
1
2
You will need to refresh
the search after a few
moments for all events
to show up
© 2017 SPLUNK INC.
12.130.60.4 - - [18/Sep/2014 05:26:50:193] "GET
/product.screen?product_id=AV-CB-01&JSESSIONID=SD8SL4FF8ADFF5
HTTP 1.1" 200 3221
"http://www.myflowershop.com/category.screen?category_id=BOUQUET
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"634
▶ Keyword searching
▶ Interesting fields sourcetype=access_combined
▶ Field extractions + why are they important
• IFX | rex | auto kv through app logging best practices
▶ Filters
Quick UI/Search OrientationSHOW
clientip method url
bytes xfered
status return code
user agent
© 2017 SPLUNK INC.
▶ Data discovery
▶ Group like events
▶ Save as event type
▶ Create alert
Pattern DetectionSHOW
Back to
Slides
© 2017 SPLUNK INC.
Non-Traditional
Data Sources
© 2017 SPLUNK INC.
▶ Network Inputs
▶ HTTP Event Collector
▶ Log Event Alert Action
▶ Splunk Stream
▶ Scripted Inputs
▶ Database Inputs
▶ Splunk ODBC Driver
▶ Modular Inputs
▶ zLinux Forwarder
▶ MINT
▶ Non-Splunk Datastores
Non-Traditional Data Sources
© 2017 SPLUNK INC.
▶ Captures events from log files in real time
▶ Runs scripts to gather system metrics,
connect to APIs and databases
▶ Listens to syslog and gathers Windows events
▶ Universally indexes any data format so it
doesn’t need adapters
Traditional Data Sources
Windows
• Registry
• Event logs
• File system
• sysinternals
Linux/Unix
• Configurations
• Syslog
• File system
• Ps, iostat, top
Virtualization
• Hypervisor
• Guest OS
• Guest Apps
Applications
• Web logs
• Log4J, JMS, JMX
• .NET events
• Code and scripts
Databases
• Configurations
• Audit/query logs
• Tables
• Schemas
Network
• Configurations
• syslog
• SNMP
• netflow
© 2017 SPLUNK INC.
▶ Collect data over any UDP or TCP port
• Some devices only send data over a network port
▶ Best Practice: use syslog-ng or rsyslog
• Offers persistence
• Categorizes data by host
Network Inputs
© 2017 SPLUNK INC.
▶ Collect data over HTTP or HTTPS directly to Splunk
• Application Developer focus – few lines of code in app to send data
▶ HEC Features Include:
• Token-based, not credential based
• Indexer Acknowledgements – guarantees data indexing
• Raw and JSON formatted event payloads
• SSL, CORS (Cross-Origin Resource Sharing), and Network Restrictions
HTTP Event Collector (HEC)
© 2017 SPLUNK INC.
▶ Use Splunk alerting to index a custom log event
• Splunk searchable index of custom alert events
▶ Configurable Features Include:
• Host
• Source
• Sourcetype
• Index
• Event text – construct the exact syntax of the log event, including
any text, tokens, or other information
Log Event Alert Action
© 2017 SPLUNK INC.
Wire Data Enhances the Platform
for Operational Intelligence
Efficient, Cloud-Ready Wire
Data Collection
Simple Deployment Supports
Fast Time to Value
The Splunk Stream
Log Files Configurations Wire Data Alerts Metrics Scripts Changes Tickets
Sensors Security Custom
Applications
Networks Databases Servers Smartphones
and Devices
Web
Services
Virtual
Machines
© 2017 SPLUNK INC.
Solution Area Contextual Data Wire Data Enriched View
Application Management application logs,
monitoring data,
metrics, events
protocol conversations on
database performance, DNS
lookups, client data,
business transaction
paths…
Measure application
response times, deeper
insights for root-cause
diagnostics, trace tx paths,
establish baselines…
IT Operations application logs,
monitoring data,
metrics, events
payload data including
process times, errors,
transaction traces, ICA
latency, SQL statements,
DNS records…
Analyze traffic volume,
speed and packets to
identify infrastructure
performance issues,
capacity constraints,
changes; establish
baselines…
Stream = Better Insights for *
© 2017 SPLUNK INC.
Solution Area Contextual Data Wire Data Enriched View
Security app + infra logs, monitoring
data, events
protocol identification,
protocol headers, content
and payload information,
flow records
Build analytics and context
for incident response, threat
detection, monitoring and
compliance
Digital Intelligence website activity,
clickstream data, metrics
browser-level customer
interactions
Customer Experience –
analyze website and application
bottlenecks to improve customer
experience and online revenues
Customer Support (online,
call center) – faster root cause
analysis and resolution of customer
issues with website or apps
Stream = Better Insights for *
© 2017 SPLUNK INC.
▶ Send data to Splunk via a custom script
• Splunk indexes anything written to stdout
• Splunk handles scheduling
• Supports shell, Python scripts, WIN batch, PowerShell
• Any other utility that can format and stream data
Scripted Inputs
Streaming Mode
• Splunk executes script and indexes stdout
• Checks for any running instances
Write to File Mode
• Splunk launches script which produces
output file, no need for external scheduler
• Splunk monitors output file
© 2017 SPLUNK INC.
▶ Alternative to file-base or network-based inputs
▶ Stream data from command-line tools, such as vmstat and iostat
▶ Poll a web service, API or database and process the results
▶ Reformat complex or binary data for easier parsing into events and fields
▶ Maintain data sources with slow or resource-intensive startup procedures
▶ Provide special or complex handling for transient or unstable inputs
▶ Scripts that manage passwords and credentials
▶ Wrapper scripts for command line inputs that contain special characters
Use Cases for Scripted Inputs
© 2017 SPLUNK INC.
▶ DB Connect provides reliable, scalable,
real-time integration between Splunk and
traditional relational databases
• Create value with structured data
• Enrich search results with additional business context
• Easily import data for deeper analysis
• Integrate multiple DBs concurrently
• Simple set-up, non-invasive and secure
Database Inputs
DB CONNECT
JRE
JDBC
DATABASE DRIVER
DATABASE
© 2017 SPLUNK INC.
▶ DB Connect App
• Real-time, scalable integration with relational DBs
• Browse and navigate schemas and tables before data import
• Reliable scheduled import
• Seamless installation and UI configuration
• Supports connection pooling and caching
▶ “Tail” tables or import entire tables
• Detect and import new/updated rows using timestamps or unique IDs
▶ Supports many RDBMS flavors
• AWS RDS Aurora, AWS RedShift, IBM DB2 for Linux, Informix, MemSQL, MS SQL, MySQL,
Oracle, PostgreSQL, SAP SQL Anywhere (aka Sybase SA), Sybase ASE and IQ, Teradata
Configure Database Inputs
© 2017 SPLUNK INC.
▶ Interact with, manipulate and visualize machine data in Splunk Enterprise using
business software tools
▶ Leverage analytics from Splunk alongside Microsoft Excel, Tableau Desktop or
Microstrategy Analytics Desktop
▶ Industry-standard connectivity to Splunk Enterprise
▶ Empowers business users with direct and secure access to machine data
▶ Combine machine data with structured data for better operational context
Splunk ODBC Driver
© 2017 SPLUNK INC.
ODBC: How it Works
Splunk AdminAnalyst
Step 3:
Business Analyst uses Microsoft Excel, Tableau or Mocrostrategy to access
Data Models and saved searches and retrieve machine data from Splunk Enterprise
Step 2:
Splunk Admin authors Data Models or saved
searches in Splunk Enterprise
Step 1:
Business Analyst communicates data
requirements to Splunk Admin
Analyst
REQUIREMENTS
Saved Searches
or Data Models
Tableau or MS Excel
or Microstrategy ODBC Driver
© 2017 SPLUNK INC.
▶ Create your own custom inputs
• Scripted input with structure and intelligence
• First class citizen in the Splunk management interface
• Appears under Settings > Data Inputs
▶ Benefits over simple scripted input
• Instance control: launch a single instance or multiple instances
• Input validation
• Support multiple platforms
• Stream data as text or XML
• Secure access to mod input scripts via REST endpoints
Modular Inputs
© 2017 SPLUNK INC.
▶ Twitter
• Stream JSON data from a Twitter source to Splunk using Tweepy
▶ Amazon S3 Online Storage
• Index data from the Amazon S3 online storage web service
▶ Java Messaging Service (JMS)
• Poll message queues and topics through JMS Messaging API
• Talks to multiple providers: MQSeries (Websphere MQ), ActiveMQ,
TibcoEMS, HornetQ, RabbitMQ, Native JMS, WebLogic JMS, Sonic MQ
▶ Splunk Windows Inputs
• Retrieve WIN event logs, registry keys, perfmon counters
Example Modular Inputs
© 2017 SPLUNK INC.
More Modular Inputs
© 2017 SPLUNK INC.
▶ Easily collect and index data on IBM mainframes
▶ Collect application and platform data
▶ Download as new Forwarder distribution for s390x Linux
zLinux Forwarder
© 2017 SPLUNK INC.
Deliver Better
Performing, More
Reliable Apps
End-to-End
Performance and
Capacity Insights
Deliver Real-Time
Omni-Channel
Analytics
Extend Operational Intelligence
to Mobile Apps
© 2017 SPLUNK INC.
▶ Improve user retention by quickly
identifying crashes and performance
issues
▶ Establish whether issues are caused
by an app or the network(s)
▶ Correlate app, OS and device type
to diagnose crash and network
performance issues
Monitor App Usage and Performance
© 2017 SPLUNK INC.
▶ Hunk Archive functionality
moves under Splunk Enterprise
as Data Roll
▶ Hunk searching of third party
data is rebranded as Splunk
Analytics for Hadoop
▶ Pricing model stays the same
as Hunk - no new SKU
HUNK > Splunk Analytics for Hadoop
Hadoop
Clusters
Splunk Analytics for
Hadoop Add-on
© 2017 SPLUNK INC.
▶ Build custom streaming resource
libraries
▶ Search and analyze data from other
data stores in Splunk
▶ In partnership with leading NoSQL
vendors
▶ Use in conjunction with DB Connect
for relational database lookups
Connect to NoSQL and Other Data Stores
Splunk Analytics for Hadoop Add-on
STREAMING ERP
© 2017 SPLUNK INC.
▶ Rolls historical data into
existing Hadoop distribution
▶ Reduces storage up to 80%*
▶ Retains Splunk search
capability
with performance tradeoffs
▶ Integrated, zero-cost option
of Splunk Enterprise
* Achieved by reducing Splunk performance optimization data
Hadoop Data Roll
Amazon EMR
on S3
Hadoop
Clusters
Leverage existing Hadoop
Datastore to reduce TCO
© 2017 SPLUNK INC.
▶ Enables seamless use of almost the
entire Splunk stack on data
▶ Automatically handles MapReduce
▶ Technology is patent pending
Virtual Indexes
© 2017 SPLUNK INC.
Data Enrichment
© 2017 SPLUNK INC.
▶ Tags – categorize and add meaning to data
▶ Field Aliases – simplify search and correlation
▶ Calculated Fields – shortcut complex/repetitive computations
▶ Event Types – group common events and share knowledge
▶ Lookups – augment data with additional external fields
Agenda
© 2017 SPLUNK INC.
▶ Adds inline meaning/context/specificity to raw data
▶ Used to normalize metadata or raw data
▶ Simplifies correlation of multiple data sources
▶ Created in Splunk
▶ Transferred from external sources
What is Data Enrichment?
© 2017 SPLUNK INC.
▶ Add meaning/context/specificity to raw data
▶ Labels describing team, category, platform, geography
▶ Applied to field-value combination
▶ Multiple tags can be applied for each field-value
▶ Case sensitive
Tags
© 2017 SPLUNK INC.
Create TagsSHOW
© 2017 SPLUNK INC.
Search events with tag in any field
Search events with tag in a specific field
Search events with tag using wildcards
Find the Web Servers
Tags in Action
tag=webserver
tag::host=webserver
tag=web*
Tag the host
as webserver
Tag the sourcetype
as web
1
2
3
4
5
SHOW
Back to
Slides
© 2017 SPLUNK INC.
▶ Normalize field labels to simplify search and correlation
▶ Apply multiple aliases to a single field
• Example: Username | cs_username | User  user
• Example: c_ip | client | client_ip  clientip
▶ Processed after field extractions + before lookups
▶ Can apply to lookups
▶ Aliases appear alongside original fields
Field Aliases
© 2017 SPLUNK INC.
Re-Label Field to Intuitive Name
Create Field Alias
SHOW
1
2
3
© 2017 SPLUNK INC.
Create field alias of clientip = customer
Search events in last 15 minutes, find
customer field
Field alias (customer) and original field
(clientip) are both displayed
Search using an Intuitive Field Name
Field Alias in Action
sourcetype=access_combined
SHOW
1
2
3
© 2017 SPLUNK INC.
▶ Shortcut for performing
repetitive/long/complex
transformations using eval
command
▶ Based on extracted or discovered
fields only
▶ Do not apply to lookup or
generated fields
Calculated Fields
1
2
3
3
© 2017 SPLUNK INC.
Compute Kilobytes from Bytes
Create Calculated Field
SHOW
1
2
3
© 2017 SPLUNK INC.
Create kilobytes = bytes/1024
Search events in last 15 minutes for kilobytes
and bytes
Search Using Kilobytes instead of Bytes
Calculated Fields in Action
SHOW
Back to
Slides
1
2
sourcetype=access_combined
© 2017 SPLUNK INC.
▶ Classify and group common events
▶ Capture and share knowledge
▶ Based on search
▶ Use in combination with fields and tags to define event topography
Event Types
© 2017 SPLUNK INC.
▶ Best Practice: Use punct field
• Default metadata field describing event structure
• Built on interesting characters: ",;-#$%&+./:=?@'|*nr"(){}<>[]^! »
• Can use wildcards
Create Event Types
event punct
####<Jun 3, 2014 5:38:22 PM MDT> <Notice>
<WebLogicServer> <bea03> <asiAdminServer>
<WrapperStartStopAppMain> <>WLS Kernel<>
<> <BEA-000360> <Server started in
RUNNING mode>
####<_,__::__>_<>_<>_<>_<>_
<>_
172.26.34.223 - - [01/Jul/2005:12:05:27 -0700]
"GET /trade/app?action=logout HTTP/1.1" 200
2953
..._-_-_[:::_-]_"_?=_/."__
© 2017 SPLUNK INC.
Show punct for sourcetype=access_combined
Pick a punct, then wildcard it after the timestamp
Add NOT status=200
Save as “bad” event type + Color:red + Priority:1
(shift reload in browser to show coloring)
Classify Events as Known Bad
Create Event Type
sourcetype="access_combined" punct="..._-_-_[//_:::]*" NOT status=200
SHOW
Back to
Slides
1
2
3
4
eventtype=bad
© 2017 SPLUNK INC.
Lookups to Enrich Raw Data
CRM/
ERP
External Data Sources
Data goes in
Create additional fields
from the raw data with
a lookup to an external
data source
Insight comes out
Watch
Lists
LDAP
AD
CMDB
© 2017 SPLUNK INC.
▶ Augment raw events with additional fields
• Provide context or supporting details
▶ Translate field values to more descriptive data
• Example: add text descriptions for error codes, IDs
• Example: add contact details to user names or IDs
• Example: add descriptions to HTTP status codes
▶ File-based or scripted lookups
Lookups
© 2017 SPLUNK INC.
Convert a Code into a Description
Configure a Static Lookup
1. Upload/create table
2. Assign table to lookup object
3. Map lookup to data set
SHOW
© 2017 SPLUNK INC.
Get the lookup from the Splunk Wiki (save to .csv file)
http://wiki.splunk.com/Http_status.csv
Lookup table files > Add new
• Name: http_status.csv (must have .csv file extension)
• Upload: <path to .csv>
Verify lookup was created successfully
1. Create HTTP Status TableSHOW
1
2
3
| inputlookup http_status.csv
© 2017 SPLUNK INC.
Lookup definitions > Add new
• Name: http_status
• Type: File-based
• Lookup file: http_status.csv
Invoke the lookup manually
2. Add Lookup DefinitionSHOW
sourcetype=access_combined | lookup http_status
status OUTPUT status_description
1
2
© 2017 SPLUNK INC.
Automatic lookups > Add new
• Name: http_status (cannot have spaces)
• Lookup table: http_status
• Apply to: sourcetype = access_combined
• Lookup input field: status
• Lookup output field: status_description
Verify lookup is invoked automatically
3. Configure Automatic LookupSHOW
1
2
© 2017 SPLUNK INC.
▶ Temporal lookups for time-based lookups
• Example: Identify users on your network based on their IP address and the
timestamp in DHCP logs
▶ Use search results to populate a lookup table
• … | outputlookup <tablename|filename>
▶ Call an external command or script
• Python scripts only
• Example: DNS lookup for IP  Host
▶ Create a lookup table using a relational database
• Review matches against a database column or SQL query
Fancy Lookups
© 2017 SPLUNK INC.
▶ Creating and Managing Alerts (Job Inspector)
▶ Macros
▶ Workflow Actions
More Data Enrichment
© 2017 SPLUNK INC.
BREAK
15 MINUTES
© 2017 SPLUNK INC.
Level Up on Search &
Reporting Commands
© 2017 SPLUNK INC.
▶ Doing more with basic search commands
▶ Advanced search commands
▶ Doing more with basic reporting commands
Agenda
© 2017 SPLUNK INC.
Search Syntax Components
© 2017 SPLUNK INC.
Anatomy of a Search
Disk
© 2017 SPLUNK INC.
▶ top – limit
▶ rare – same options as top
▶ timechart – parameters
▶ stats – functions (sum, avg, list, values, sparkline)
▶ sort – inline ascending or descending
▶ addcoltotals
▶ addtotals
Doing More with Basic Search Commands
© 2017 SPLUNK INC.
Workshop Notes for Presenter
Tip #6:
In the next section, after each search, have the participants
save the search as a dashboard panel. At the end of the workshop,
they will have a living document of the workshop exercises to reference
later. A complete version of this dashboard is packaged as an app.
It is uploaded to the Box folder as a leave behind.
© 2017 SPLUNK INC.
... | rare limit=20 clientip
... | top limit=20 clientip
▶ Commands have parameters or qualifiers
▶ Top and rare have similar syntax
▶ Each search command has its own syntax – show inline help
Find Most and Least Active Customers
Using the top + rare Commands
SHOW
IPs with the
most visits
IPs with the
least visits
© 2017 SPLUNK INC.
... | stats count by clientip | sort + count
... | stats count by clientip | sort - count
▶ Sort inline descending or ascending
The Number of Customer Requests
Using the sort Command
SHOW
Number of requests by
customer - descending
Number of requests by
customer - ascending
© 2017 SPLUNK INC.
▶ Show Search Command Reference Docs
• Functions for eval + where
• Functions for stats + chart and timechart
▶ Invoke a function
▶ Rename inline
... | stats sum(bytes) as totalbytes by clientip | sort - totalbytes
... | stats sum(bytes) by clientip | sort - sum(bytes)
Determine Total Customer Payload
Using functions + rename command
SHOW
Total payload by
customer - descending
Total payload by
customer - ascending
© 2017 SPLUNK INC.
▶ List all values of a field
▶ List only distinct values of a field
Observe Customer Activity
Using the list + values Functions
... | stats values(action) by clientip
... | stats list(action) by clientip
SHOW
Activity by customer
Distinct actions by
customer
© 2017 SPLUNK INC.
▶ Show distinct actions and cardinality of each action
Analyze Customer Activity
Combine list + values Functions
sourcetype=access_combined
| stats count(action) as value by clientip, action
| eval pair=action + " (" + value + ")"
| stats list(pair) as values by clientip
SHOW
© 2017 SPLUNK INC.
... | stats sum(bytes) as totalbytes, avg(bytes) as avgbytes, count as
totalevents by clientip | addcoltotals totalbytes, totalevents
▶ Add columns
▶ Sum specific columns
Building a Table of Customer Activity
Add Columns and Sum Columns
... | stats count by clientip, action
SHOW
2 cols: clientip + action
Sum totalbytes and
totalevents columns
© 2017 SPLUNK INC.
... | stats sum(bytes) as totalbytes, sum(other) as totalother
by clientip | addtotals fieldname=totalstuff
Building a Table of Customer Activity
Sum Across Rows
SHOW
Sum totalbytes and
totalevents columns
A better example:
physical memory + virtual
memory = total memory
© 2017 SPLUNK INC.
... | stats sparkline(count) as trendline sum(bytes) by clientip
Trend Individual Customer Activity
Sparklines in Action
... | stats sparkline(count) as trendline by clientip
SHOW
In context of larger
event set
Inline in tables
Back to
Slides
© 2017 SPLUNK INC.
Advanced Search Commands
Command Short Description Hints
transaction Group events by a common field value. Convenient, but resource intensive.
cluster Cluster similar events together. Can be used on _raw or field.
associate Identifies correlations between fields. Calculates entropy btn field values.
correlate Calculates the correlation between different fields.
Evaluates relationship of all fields
in a result set.
contingency Builds a contingency table for two fields.
Computes co-occurrence, or % two fields
exist in same events.
anomalies Computes an unexpectedness score for an event.
Computes similarity of event (X) to a
set of previous events (P).
anomalousvalue
Finds and summarizes irregular, or uncommon,
search results.
Considers frequency of occurrence or
number of stdev from the mean.
© 2017 SPLUNK INC.
▶ Sew events together + creates duration + eventcount
View Customer Activity by Session
Using the transaction Command
... | transaction JSESSIONID | table JSESSIONID, action, product_id
SHOW
Group by JSESSIONID
© 2017 SPLUNK INC.
▶ Predict over time
▶ Chart Overlay with and without streamstats
▶ Maps with iplocation + geostats
▶ Single value
▶ Metered visuals with gauge
Doing More with Basic Reporting Commands
© 2017 SPLUNK INC.
▶ Predict future values using lower/upper bounds – single and multiple series
Predict Website Traffic
Using the predict Command
... | timechart count as traffic | predict traffic
SHOW
© 2017 SPLUNK INC.
Compare Browsing vs. Buying Activity
Simple Chart Overlay
SHOW
sourcetype=access_combined (action=view OR action=purchase)
| timechart span=10m count(eval(action="view")) as Viewed,
count(eval(action="purchase")) as Purchased
© 2017 SPLUNK INC.
Map Customer Activity Geographically
Geolocation in Action
SHOW
... | iplocation clientip | geostats count by clientip Combine IP lookup with
geo mapping
© 2017 SPLUNK INC.
Display a Simple Count of Events
Single Value in Action
SHOW
... | stats count
© 2017 SPLUNK INC.
Display Counts Using Gauges
Single Value, Radial and Filler Gauges in Action
SHOW
... | stats count | gauge count 10000 20000 30000 40000 50000
Back to
Slides
© 2017 SPLUNK INC.
BREAK
15 MINUTES
© 2017 SPLUNK INC.
Data Model and Pivot
© 2017 SPLUNK INC.
▶ What is a data model?
▶ Build a data model
▶ Pivot Interface
▶ Accelerate a data model
Agenda
© 2017 SPLUNK INC.
Pivot
Enables non-technical users to build complex
reports without the search language
Powerful Analytics Anyone Can Use
Data
Model
Provides more meaningful representation of
underlying raw machine data
Analytics
Store
Acceleration technology delivers up to 1000x
faster analytics over Splunk 5
© 2017 SPLUNK INC.
▶ Data Model
• Describes how underlying
machine data is represented and
accessed
• Defines meaningful relationships
in the data
• Enables single authoritative view
of underlying raw data
Define Relationships in Machine Data
Hierarchical object view of underlying data
Add constraints to filter out events
© 2017 SPLUNK INC.
▶ High Performance
Analytics Store
• Automatically collected
− Handles timing issues, backfill…
• Automatically maintained
− Uses acceleration window
• Stored on the indexers
− Peer to the buckets
• Fault tolerant collection
Transparent Acceleration
Check to enable
acceleration of data model
Time window of data
that is accelerated
© 2017 SPLUNK INC.
▶ Pivot
• Drag-and-drop interface enables
any user to analyze data
• Create complex queries and
reports without learning search
language
• Click to visualize any chart type;
reports dynamically update when
fields change
Easy-to-Use Analytics
All chart types available in
the chart toolbox
Select fields from
data model
Time window
Save report to share
© 2017 SPLUNK INC.
▶ Defines least common denominator for a data domain
▶ Standard method to parse, categorize, normalize data
▶ Set of field names and tags by domain
▶ Packaged as Data Models in a Splunk App
• Domains: security, web, inventory, JVM,
performance, network sessions, and more
• Minimal setup to use Pivot interface
Common Information Model (CIM) App
© 2017 SPLUNK INC.
Apps > Find More Apps >
Search: “Common Information Model”
Install free
Show fields for web + Web Data Model
Download CIM AppSHOW
Back to
Slides
1
2
3
4
© 2017 SPLUNK INC.
http://docs.splunk.com/Documentation/Splunk/latest/PivotTutorial/WelcometothePivotTutorial
Data Model & Pivot Tutorial
© 2017 SPLUNK INC.
Custom Visualizations and
the Web Framework Toolkit
© 2017 SPLUNK INC.
▶ Custom Visualizations
▶ Developer Platform
▶ Resources
Agenda
© 2017 SPLUNK INC.
▶ Native charts and maps
• Bar / Line / Area charts
• Bubble / Scatter plots
• Gauges
• Maps
• Single Value Displays
• Tables
▶ Generalized to fit use cases
across many different areas
▶ Can be customized to some
extent to cover specific use cases
Native Visualizations In Splunk
© 2017 SPLUNK INC.
▶ Many use cases require a more
specific visualization
▶ Specific custom appearance
▶ Represent data where native
visualizations are not suitable
• You can Splunk everything!
• We won’t be able to predict every possible
use case
• Still uses SPL to drive visualizations
Custom Visualizations FTW!
© 2017 SPLUNK INC.
▶ Platform extensibility framework and API
▶ Targeted at internal and external
developers with web development / JS
skills and basic knowledge of the
Splunk platform
▶ Developers can make use of any third party
libraries (d3.js, three.js, highcharts.js, etc…)
that run in the browser*
* with minor adjustments, and if third party license permits such use
Custom Visualizations
© 2017 SPLUNK INC.
▶ Packaged as an app!
▶ Installed like any other app
▶ Users can search for
visualizations on Splunkbase
and directly in the product
Custom Visualizations For Admins
In-productInstallation
© 2017 SPLUNK INC.
▶ Choose from potentially dozens of installed
visualizations!
▶ Appears as a first-class citizen alongside
native visualizations
• Looks and works just like packaged native
visualizations
▶ Customize functionality and appearance of
the visualization without touching any code,
straight from the UI
SPL Example provided as you hover
over each visualization option.
Custom Visualizations How-to
© 2017 SPLUNK INC.
New Splunk Visualizations
Multiple use cases across IT, security, IoT, and business analytics
Treemap
Sankey
Diagram
Punchcard Calendar
Heat Map
Parallel
Coordinates
Bullet GraphLocation
Tracker
Horseshoe
Meter
Machine Learning
Charts
Timeline
Horizon Chart
© 2017 SPLUNK INC.
Box Plot
3D scatter plot
New Partner/Community Visualizations
Wordcloud
Donut Chart
Heat Map
© 2017 SPLUNK INC.
New Partner/Community Visualizations
Geo
Heatmap
Custom Cluster Map
Clustered
Single
Value Map
Missile Map
© 2017 SPLUNK INC.
The Splunk Enterprise Platform
Collection
Indexing
Search Processing Language
Core Functions
Inputs, Apps, Other Content
Content
Core Engine
User and Developer Interfaces
Core Engine
User and Developer Interfaces
Content
Web Framework
SDK
Rest API
© 2017 SPLUNK INC.
Developer Platform
What’s Possible with the
Splunk Enterprise Platform?
Power
Mobile Apps
Log
Directly
Extract
Data
Customer
Dashboards
Integrate
BI Tools
Integrate Platform
Services
© 2017 SPLUNK INC.
Web Framework Toolkit
© 2017 SPLUNK INC.
SDKs
Powerful Platform for Enterprise Developers
Developers Can Customize and Extend
Rest API
Web Framework Java
JavaScript
Python
Simple XML
JavaScript
HTML5
Data Models
Search Extensibility
Modular Inputs
Ruby
C#
PHP
Extend and Integrate SplunkBuild Splunk Apps
© 2017 SPLUNK INC.
Splunk Software for Developers
GAIN APPLICATION
INTELLIGENCE
INTEGRATE AND
EXTEND SPLUNK
BUILD SPLUNK
APPS
© 2017 SPLUNK INC.
A Wealth of Splunk Apps
Over 1,300 apps available on the Splunk apps site
Server, Storage,
Network
Server Virtualization Operating Systems
Custom
Applications
Business
Applications
Cloud Services
App Performance
MonitoringTicketing/ and Other
Web Intelligence
Mobile
Applications
Stream
API
SDKs UI
© 2017 SPLUNK INC.
▶ Interactive, cut/paste examples from popular source repositories:
D3, GitHub, jQuery
▶ Splunk 6.x Dashboard Examples App
https://apps.splunk.com/app/1603
▶ Custom SimpleXML Extensions App
https://apps.splunk.com/app/1772
▶ Splunk Web Framework Toolkit App
https://apps.splunk.com/app/1613
Example Advanced Visualizations
© 2017 SPLUNK INC.
Resources
© 2017 SPLUNK INC.
▶ http://docs.splunk.com
▶ Official Product Docs
▶ Wiki and community topics
▶ Updated daily
▶ Can be printed to .PDF
Splunk Documentation
© 2017 SPLUNK INC.
▶ http://answers.splunk.com
▶ Community driven
▶ Splunk supported
▶ Knowledge exchange
▶ Q & A
Splunk Answers
© 2017 SPLUNK INC.
▶ Recommended for Users
• Using Splunk
• Searching & Reporting
▶ Recommended for UI/Dashboard Developers
• Developing Apps
▶ Instructor-Led Courses
• Web
• Onsite
Splunk Education
© 2017 SPLUNK INC.
Delivered Globally:
Online, Classroom,
Self-Paced
15 Free
Getting Started Videos
Get Splunk Certified
in 5 Days
20 Classes
For more information: splunk.com/education
Knowledge is Power
Splunk Education
© 2017 SPLUNK INC.
Become a Splunk Expert
Knowledge is Power
Using Splunk
Searching and Reporting with Splunk
Creating Splunk Knowledge Objects
Splunk Administration
Architecting and Deploying Splunk
Developing Apps with Splunk
Splunk Architect Certification Lab
© 2017 SPLUNK INC.
Splunk Education for Security
Knowledge is Power
Using Splunk
Searching and Reporting with Splunk
Creating Splunk Knowledge Objects
Using the Splunk App for
Enterprise Security
Splunk Administration
Architecting and Deploying Splunk
Administering the Splunk App
for Enterprise Security
© 2017 SPLUNK INC.
Splunk Education for IT Service Intelligence
Knowledge is Power
Using Splunk
Searching and Reporting with Splunk
Creating Splunk Knowledge Objects
Splunk Administration
Implementing IT Service Intelligence
© 2017 SPLUNK INC.
Course Topics
• Overview of ITSI features
• ITSI architecture and deployment
• Installing ITSI
• Designing and implementing services
and entities
• Configuring correlation searches and
notable events
• Creating deep dive pages
• Creating glass tables
• ITSI troubleshooting
Splunk Education for IT Service Intelligence
Knowledge is Power
© 2017 SPLUNK INC.
Splunk Education for IT Service Intelligence
Knowledge is Power
Course Topics
• Overview of ITSI features
• ITSI architecture and deployment
• Installing ITSI
• Designing and implementing services and entities
• Configuring correlation searches and notable
events
• Creating deep dive pages
• Creating glass tables
• ITSI troubleshooting
PREREQUISITES | 13.5 hour course
Using Splunk, Searching and Reporting with Splunk,
Creating Splunk Knowledge Objects, Splunk Administration
© 2017 SPLUNK INC.
Q&A
Please use HipChat Room: [Title of Room]
to Submit Your Questions to eStaff
© 2017 SPLUNK INC.
Get Started Fast!
splunk.com/education
© 2017 SPLUNK INC.© 2017 SPLUNK INC.
Thank You

More Related Content

What's hot

Splunk workshop-Service Intelligence
Splunk workshop-Service IntelligenceSplunk workshop-Service Intelligence
Splunk workshop-Service IntelligenceSplunk
 
Splunk Discovery Indianapolis - October 10, 2017
Splunk Discovery Indianapolis - October 10, 2017Splunk Discovery Indianapolis - October 10, 2017
Splunk Discovery Indianapolis - October 10, 2017Splunk
 
Hitchhikers Guide to Service Intelligence
Hitchhikers Guide to Service IntelligenceHitchhikers Guide to Service Intelligence
Hitchhikers Guide to Service IntelligenceSplunk
 
Machine Data 101
Machine Data 101Machine Data 101
Machine Data 101Splunk
 
Splunk Ninjas: New Features and Search Dojo
Splunk Ninjas: New Features and Search DojoSplunk Ninjas: New Features and Search Dojo
Splunk Ninjas: New Features and Search DojoSplunk
 
SplunkLive! Zurich 2017 - Data Obfuscation in Splunk Enterprise
SplunkLive! Zurich 2017 - Data Obfuscation in Splunk EnterpriseSplunkLive! Zurich 2017 - Data Obfuscation in Splunk Enterprise
SplunkLive! Zurich 2017 - Data Obfuscation in Splunk EnterpriseSplunk
 
Machine Data 101
Machine Data 101Machine Data 101
Machine Data 101Splunk
 
SplunkLive! Zurich 2017 - Getting Started with Splunk Enterprise
SplunkLive! Zurich 2017 - Getting Started with Splunk EnterpriseSplunkLive! Zurich 2017 - Getting Started with Splunk Enterprise
SplunkLive! Zurich 2017 - Getting Started with Splunk EnterpriseSplunk
 
Splunk Discovery Day Milwaukee 9-14-17
Splunk Discovery Day Milwaukee 9-14-17Splunk Discovery Day Milwaukee 9-14-17
Splunk Discovery Day Milwaukee 9-14-17Splunk
 
SplunkLive! London 2017 - DevOps Powered by Splunk
SplunkLive! London 2017 - DevOps Powered by SplunkSplunkLive! London 2017 - DevOps Powered by Splunk
SplunkLive! London 2017 - DevOps Powered by SplunkSplunk
 
SplunkLive! Zurich 2017 - How to Design, Build and Map IT and Business Servic...
SplunkLive! Zurich 2017 - How to Design, Build and Map IT and Business Servic...SplunkLive! Zurich 2017 - How to Design, Build and Map IT and Business Servic...
SplunkLive! Zurich 2017 - How to Design, Build and Map IT and Business Servic...Splunk
 
SplunkLive! London 2017 - Getting Started with Splunk IT Service Intelligence
SplunkLive! London 2017 - Getting Started with Splunk IT Service IntelligenceSplunkLive! London 2017 - Getting Started with Splunk IT Service Intelligence
SplunkLive! London 2017 - Getting Started with Splunk IT Service IntelligenceSplunk
 
The Hitchhiker's Guide to Service Intelligence Workshop
The Hitchhiker's Guide to Service Intelligence WorkshopThe Hitchhiker's Guide to Service Intelligence Workshop
The Hitchhiker's Guide to Service Intelligence WorkshopSplunk
 
SplunkLive! Zurich 2017 - Advanced Analytics / Machine Learning
SplunkLive! Zurich 2017 - Advanced Analytics / Machine LearningSplunkLive! Zurich 2017 - Advanced Analytics / Machine Learning
SplunkLive! Zurich 2017 - Advanced Analytics / Machine LearningSplunk
 
Machine Data 101
Machine Data 101Machine Data 101
Machine Data 101Splunk
 
Splunk workshop-2017-Power-of-SPL
Splunk workshop-2017-Power-of-SPLSplunk workshop-2017-Power-of-SPL
Splunk workshop-2017-Power-of-SPLSplunk
 
Hitchhikers Guide to Service Intelligence
Hitchhikers Guide to Service IntelligenceHitchhikers Guide to Service Intelligence
Hitchhikers Guide to Service IntelligenceSplunk
 
Splunk Discovery Brussels - September 2017
Splunk Discovery Brussels - September 2017Splunk Discovery Brussels - September 2017
Splunk Discovery Brussels - September 2017Splunk
 
SplunkLive! London 2017 - Happy Apps, Happy Users
SplunkLive! London 2017 - Happy Apps, Happy UsersSplunkLive! London 2017 - Happy Apps, Happy Users
SplunkLive! London 2017 - Happy Apps, Happy UsersSplunk
 

What's hot (20)

Splunk
SplunkSplunk
Splunk
 
Splunk workshop-Service Intelligence
Splunk workshop-Service IntelligenceSplunk workshop-Service Intelligence
Splunk workshop-Service Intelligence
 
Splunk Discovery Indianapolis - October 10, 2017
Splunk Discovery Indianapolis - October 10, 2017Splunk Discovery Indianapolis - October 10, 2017
Splunk Discovery Indianapolis - October 10, 2017
 
Hitchhikers Guide to Service Intelligence
Hitchhikers Guide to Service IntelligenceHitchhikers Guide to Service Intelligence
Hitchhikers Guide to Service Intelligence
 
Machine Data 101
Machine Data 101Machine Data 101
Machine Data 101
 
Splunk Ninjas: New Features and Search Dojo
Splunk Ninjas: New Features and Search DojoSplunk Ninjas: New Features and Search Dojo
Splunk Ninjas: New Features and Search Dojo
 
SplunkLive! Zurich 2017 - Data Obfuscation in Splunk Enterprise
SplunkLive! Zurich 2017 - Data Obfuscation in Splunk EnterpriseSplunkLive! Zurich 2017 - Data Obfuscation in Splunk Enterprise
SplunkLive! Zurich 2017 - Data Obfuscation in Splunk Enterprise
 
Machine Data 101
Machine Data 101Machine Data 101
Machine Data 101
 
SplunkLive! Zurich 2017 - Getting Started with Splunk Enterprise
SplunkLive! Zurich 2017 - Getting Started with Splunk EnterpriseSplunkLive! Zurich 2017 - Getting Started with Splunk Enterprise
SplunkLive! Zurich 2017 - Getting Started with Splunk Enterprise
 
Splunk Discovery Day Milwaukee 9-14-17
Splunk Discovery Day Milwaukee 9-14-17Splunk Discovery Day Milwaukee 9-14-17
Splunk Discovery Day Milwaukee 9-14-17
 
SplunkLive! London 2017 - DevOps Powered by Splunk
SplunkLive! London 2017 - DevOps Powered by SplunkSplunkLive! London 2017 - DevOps Powered by Splunk
SplunkLive! London 2017 - DevOps Powered by Splunk
 
SplunkLive! Zurich 2017 - How to Design, Build and Map IT and Business Servic...
SplunkLive! Zurich 2017 - How to Design, Build and Map IT and Business Servic...SplunkLive! Zurich 2017 - How to Design, Build and Map IT and Business Servic...
SplunkLive! Zurich 2017 - How to Design, Build and Map IT and Business Servic...
 
SplunkLive! London 2017 - Getting Started with Splunk IT Service Intelligence
SplunkLive! London 2017 - Getting Started with Splunk IT Service IntelligenceSplunkLive! London 2017 - Getting Started with Splunk IT Service Intelligence
SplunkLive! London 2017 - Getting Started with Splunk IT Service Intelligence
 
The Hitchhiker's Guide to Service Intelligence Workshop
The Hitchhiker's Guide to Service Intelligence WorkshopThe Hitchhiker's Guide to Service Intelligence Workshop
The Hitchhiker's Guide to Service Intelligence Workshop
 
SplunkLive! Zurich 2017 - Advanced Analytics / Machine Learning
SplunkLive! Zurich 2017 - Advanced Analytics / Machine LearningSplunkLive! Zurich 2017 - Advanced Analytics / Machine Learning
SplunkLive! Zurich 2017 - Advanced Analytics / Machine Learning
 
Machine Data 101
Machine Data 101Machine Data 101
Machine Data 101
 
Splunk workshop-2017-Power-of-SPL
Splunk workshop-2017-Power-of-SPLSplunk workshop-2017-Power-of-SPL
Splunk workshop-2017-Power-of-SPL
 
Hitchhikers Guide to Service Intelligence
Hitchhikers Guide to Service IntelligenceHitchhikers Guide to Service Intelligence
Hitchhikers Guide to Service Intelligence
 
Splunk Discovery Brussels - September 2017
Splunk Discovery Brussels - September 2017Splunk Discovery Brussels - September 2017
Splunk Discovery Brussels - September 2017
 
SplunkLive! London 2017 - Happy Apps, Happy Users
SplunkLive! London 2017 - Happy Apps, Happy UsersSplunkLive! London 2017 - Happy Apps, Happy Users
SplunkLive! London 2017 - Happy Apps, Happy Users
 

Similar to Machine Data Workshop: Index, Search and Analyze Non-Traditional Data Sources

Machine Data 101 Workshop
Machine Data 101 Workshop Machine Data 101 Workshop
Machine Data 101 Workshop Splunk
 
Splunk workshop-Machine Data 101
Splunk workshop-Machine Data 101Splunk workshop-Machine Data 101
Splunk workshop-Machine Data 101Splunk
 
Machine Data 101 Hands-on
Machine Data 101 Hands-onMachine Data 101 Hands-on
Machine Data 101 Hands-onSplunk
 
Getting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseGetting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseSplunk
 
SplunkLive! London 2016 Splunk Overview
SplunkLive! London 2016 Splunk OverviewSplunkLive! London 2016 Splunk Overview
SplunkLive! London 2016 Splunk OverviewSplunk
 
Delivering New Visibility and Analytics for IT Operations
Delivering New Visibility and Analytics for IT OperationsDelivering New Visibility and Analytics for IT Operations
Delivering New Visibility and Analytics for IT OperationsGabrielle Knowles
 
SplunkLive Auckland - Operational Intelligence
SplunkLive Auckland - Operational IntelligenceSplunkLive Auckland - Operational Intelligence
SplunkLive Auckland - Operational IntelligenceSplunk
 
SplunkLive Wellington 2015 - Operational Intelligence
SplunkLive Wellington 2015 - Operational IntelligenceSplunkLive Wellington 2015 - Operational Intelligence
SplunkLive Wellington 2015 - Operational IntelligenceSplunk
 
Workshop splunk 6.5-saint-louis-mo
Workshop splunk 6.5-saint-louis-moWorkshop splunk 6.5-saint-louis-mo
Workshop splunk 6.5-saint-louis-moMohamad Hassan
 
Splunk Data Onboarding Overview - Splunk Data Collection Architecture
Splunk Data Onboarding Overview - Splunk Data Collection ArchitectureSplunk Data Onboarding Overview - Splunk Data Collection Architecture
Splunk Data Onboarding Overview - Splunk Data Collection ArchitectureSplunk
 
Reactive to Proactive: Intelligent Troubleshooting and Monitoring with Splunk
Reactive to Proactive: Intelligent Troubleshooting and Monitoring with SplunkReactive to Proactive: Intelligent Troubleshooting and Monitoring with Splunk
Reactive to Proactive: Intelligent Troubleshooting and Monitoring with SplunkSplunk
 
SplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding OverviewSplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding OverviewSplunk
 
SplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding OverviewSplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding OverviewSplunk
 
Microsoft Fabric Introduction
Microsoft Fabric IntroductionMicrosoft Fabric Introduction
Microsoft Fabric IntroductionJames Serra
 
Machine Data Is EVERYWHERE: Use It for Testing
Machine Data Is EVERYWHERE: Use It for TestingMachine Data Is EVERYWHERE: Use It for Testing
Machine Data Is EVERYWHERE: Use It for TestingTechWell
 
Splunk for IT Operations Breakout Session
Splunk for IT Operations Breakout SessionSplunk for IT Operations Breakout Session
Splunk for IT Operations Breakout SessionGeorg Knon
 
SplunkLive! - Splunk for IT Operations
SplunkLive! - Splunk for IT OperationsSplunkLive! - Splunk for IT Operations
SplunkLive! - Splunk for IT OperationsSplunk
 
Latest Updates to Splunk from .conf 2017 Announcements
Latest Updates to Splunk from .conf 2017 Announcements Latest Updates to Splunk from .conf 2017 Announcements
Latest Updates to Splunk from .conf 2017 Announcements Harry McLaren
 
Cisco UCS and Splunk Workshop
Cisco UCS and Splunk WorkshopCisco UCS and Splunk Workshop
Cisco UCS and Splunk WorkshopRobb Boyd
 

Similar to Machine Data Workshop: Index, Search and Analyze Non-Traditional Data Sources (20)

Machine Data 101 Workshop
Machine Data 101 Workshop Machine Data 101 Workshop
Machine Data 101 Workshop
 
Splunk workshop-Machine Data 101
Splunk workshop-Machine Data 101Splunk workshop-Machine Data 101
Splunk workshop-Machine Data 101
 
Machine Data 101 Hands-on
Machine Data 101 Hands-onMachine Data 101 Hands-on
Machine Data 101 Hands-on
 
Getting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseGetting Started with Splunk Enterprise
Getting Started with Splunk Enterprise
 
SplunkLive! London 2016 Splunk Overview
SplunkLive! London 2016 Splunk OverviewSplunkLive! London 2016 Splunk Overview
SplunkLive! London 2016 Splunk Overview
 
Delivering New Visibility and Analytics for IT Operations
Delivering New Visibility and Analytics for IT OperationsDelivering New Visibility and Analytics for IT Operations
Delivering New Visibility and Analytics for IT Operations
 
SplunkLive Auckland - Operational Intelligence
SplunkLive Auckland - Operational IntelligenceSplunkLive Auckland - Operational Intelligence
SplunkLive Auckland - Operational Intelligence
 
SplunkLive Wellington 2015 - Operational Intelligence
SplunkLive Wellington 2015 - Operational IntelligenceSplunkLive Wellington 2015 - Operational Intelligence
SplunkLive Wellington 2015 - Operational Intelligence
 
Workshop splunk 6.5-saint-louis-mo
Workshop splunk 6.5-saint-louis-moWorkshop splunk 6.5-saint-louis-mo
Workshop splunk 6.5-saint-louis-mo
 
Splunk Data Onboarding Overview - Splunk Data Collection Architecture
Splunk Data Onboarding Overview - Splunk Data Collection ArchitectureSplunk Data Onboarding Overview - Splunk Data Collection Architecture
Splunk Data Onboarding Overview - Splunk Data Collection Architecture
 
Reactive to Proactive: Intelligent Troubleshooting and Monitoring with Splunk
Reactive to Proactive: Intelligent Troubleshooting and Monitoring with SplunkReactive to Proactive: Intelligent Troubleshooting and Monitoring with Splunk
Reactive to Proactive: Intelligent Troubleshooting and Monitoring with Splunk
 
SplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding OverviewSplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding Overview
 
SplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding OverviewSplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding Overview
 
inmation Presentation
inmation Presentationinmation Presentation
inmation Presentation
 
Microsoft Fabric Introduction
Microsoft Fabric IntroductionMicrosoft Fabric Introduction
Microsoft Fabric Introduction
 
Machine Data Is EVERYWHERE: Use It for Testing
Machine Data Is EVERYWHERE: Use It for TestingMachine Data Is EVERYWHERE: Use It for Testing
Machine Data Is EVERYWHERE: Use It for Testing
 
Splunk for IT Operations Breakout Session
Splunk for IT Operations Breakout SessionSplunk for IT Operations Breakout Session
Splunk for IT Operations Breakout Session
 
SplunkLive! - Splunk for IT Operations
SplunkLive! - Splunk for IT OperationsSplunkLive! - Splunk for IT Operations
SplunkLive! - Splunk for IT Operations
 
Latest Updates to Splunk from .conf 2017 Announcements
Latest Updates to Splunk from .conf 2017 Announcements Latest Updates to Splunk from .conf 2017 Announcements
Latest Updates to Splunk from .conf 2017 Announcements
 
Cisco UCS and Splunk Workshop
Cisco UCS and Splunk WorkshopCisco UCS and Splunk Workshop
Cisco UCS and Splunk Workshop
 

More from Splunk

.conf Go 2023 - Data analysis as a routine
.conf Go 2023 - Data analysis as a routine.conf Go 2023 - Data analysis as a routine
.conf Go 2023 - Data analysis as a routineSplunk
 
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTVSplunk
 
.conf Go 2023 - Navegando la normativa SOX (Telefónica)
.conf Go 2023 - Navegando la normativa SOX (Telefónica).conf Go 2023 - Navegando la normativa SOX (Telefónica)
.conf Go 2023 - Navegando la normativa SOX (Telefónica)Splunk
 
.conf Go 2023 - Raiffeisen Bank International
.conf Go 2023 - Raiffeisen Bank International.conf Go 2023 - Raiffeisen Bank International
.conf Go 2023 - Raiffeisen Bank InternationalSplunk
 
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett .conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett Splunk
 
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär).conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)Splunk
 
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu....conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...Splunk
 
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever....conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...Splunk
 
.conf go 2023 - De NOC a CSIRT (Cellnex)
.conf go 2023 - De NOC a CSIRT (Cellnex).conf go 2023 - De NOC a CSIRT (Cellnex)
.conf go 2023 - De NOC a CSIRT (Cellnex)Splunk
 
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)Splunk
 
Splunk - BMW connects business and IT with data driven operations SRE and O11y
Splunk - BMW connects business and IT with data driven operations SRE and O11ySplunk - BMW connects business and IT with data driven operations SRE and O11y
Splunk - BMW connects business and IT with data driven operations SRE and O11ySplunk
 
Splunk x Freenet - .conf Go Köln
Splunk x Freenet - .conf Go KölnSplunk x Freenet - .conf Go Köln
Splunk x Freenet - .conf Go KölnSplunk
 
Splunk Security Session - .conf Go Köln
Splunk Security Session - .conf Go KölnSplunk Security Session - .conf Go Köln
Splunk Security Session - .conf Go KölnSplunk
 
Data foundations building success, at city scale – Imperial College London
 Data foundations building success, at city scale – Imperial College London Data foundations building success, at city scale – Imperial College London
Data foundations building success, at city scale – Imperial College LondonSplunk
 
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...Splunk
 
SOC, Amore Mio! | Security Webinar
SOC, Amore Mio! | Security WebinarSOC, Amore Mio! | Security Webinar
SOC, Amore Mio! | Security WebinarSplunk
 
.conf Go 2022 - Observability Session
.conf Go 2022 - Observability Session.conf Go 2022 - Observability Session
.conf Go 2022 - Observability SessionSplunk
 
.conf Go Zurich 2022 - Keynote
.conf Go Zurich 2022 - Keynote.conf Go Zurich 2022 - Keynote
.conf Go Zurich 2022 - KeynoteSplunk
 
.conf Go Zurich 2022 - Platform Session
.conf Go Zurich 2022 - Platform Session.conf Go Zurich 2022 - Platform Session
.conf Go Zurich 2022 - Platform SessionSplunk
 
.conf Go Zurich 2022 - Security Session
.conf Go Zurich 2022 - Security Session.conf Go Zurich 2022 - Security Session
.conf Go Zurich 2022 - Security SessionSplunk
 

More from Splunk (20)

.conf Go 2023 - Data analysis as a routine
.conf Go 2023 - Data analysis as a routine.conf Go 2023 - Data analysis as a routine
.conf Go 2023 - Data analysis as a routine
 
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
 
.conf Go 2023 - Navegando la normativa SOX (Telefónica)
.conf Go 2023 - Navegando la normativa SOX (Telefónica).conf Go 2023 - Navegando la normativa SOX (Telefónica)
.conf Go 2023 - Navegando la normativa SOX (Telefónica)
 
.conf Go 2023 - Raiffeisen Bank International
.conf Go 2023 - Raiffeisen Bank International.conf Go 2023 - Raiffeisen Bank International
.conf Go 2023 - Raiffeisen Bank International
 
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett .conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
 
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär).conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
 
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu....conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
 
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever....conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
 
.conf go 2023 - De NOC a CSIRT (Cellnex)
.conf go 2023 - De NOC a CSIRT (Cellnex).conf go 2023 - De NOC a CSIRT (Cellnex)
.conf go 2023 - De NOC a CSIRT (Cellnex)
 
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
 
Splunk - BMW connects business and IT with data driven operations SRE and O11y
Splunk - BMW connects business and IT with data driven operations SRE and O11ySplunk - BMW connects business and IT with data driven operations SRE and O11y
Splunk - BMW connects business and IT with data driven operations SRE and O11y
 
Splunk x Freenet - .conf Go Köln
Splunk x Freenet - .conf Go KölnSplunk x Freenet - .conf Go Köln
Splunk x Freenet - .conf Go Köln
 
Splunk Security Session - .conf Go Köln
Splunk Security Session - .conf Go KölnSplunk Security Session - .conf Go Köln
Splunk Security Session - .conf Go Köln
 
Data foundations building success, at city scale – Imperial College London
 Data foundations building success, at city scale – Imperial College London Data foundations building success, at city scale – Imperial College London
Data foundations building success, at city scale – Imperial College London
 
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...
 
SOC, Amore Mio! | Security Webinar
SOC, Amore Mio! | Security WebinarSOC, Amore Mio! | Security Webinar
SOC, Amore Mio! | Security Webinar
 
.conf Go 2022 - Observability Session
.conf Go 2022 - Observability Session.conf Go 2022 - Observability Session
.conf Go 2022 - Observability Session
 
.conf Go Zurich 2022 - Keynote
.conf Go Zurich 2022 - Keynote.conf Go Zurich 2022 - Keynote
.conf Go Zurich 2022 - Keynote
 
.conf Go Zurich 2022 - Platform Session
.conf Go Zurich 2022 - Platform Session.conf Go Zurich 2022 - Platform Session
.conf Go Zurich 2022 - Platform Session
 
.conf Go Zurich 2022 - Security Session
.conf Go Zurich 2022 - Security Session.conf Go Zurich 2022 - Security Session
.conf Go Zurich 2022 - Security Session
 

Recently uploaded

Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraDeakin University
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptxLBM Solutions
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Alan Dix
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxnull - The Open Security Community
 
Unlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power SystemsUnlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power SystemsPrecisely
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksSoftradix Technologies
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 

Recently uploaded (20)

Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning era
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptx
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptxVulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
 
Unlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power SystemsUnlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power Systems
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 

Machine Data Workshop: Index, Search and Analyze Non-Traditional Data Sources

  • 1. © 2017 SPLUNK INC.© 2017 SPLUNK INC. Machine Data Workshop 101 Beyond the Basics
  • 2. © 2017 SPLUNK INC. Splunk Approach to Machine Data SQL Search Schema at Write Schema at Read Traditional Splunk ETL Universal Indexing Volume Velocity Variety UnstructuredStructured RDBMS
  • 3. © 2017 SPLUNK INC. Industry Leading Platform For Machine Data Custom dashboards Report and analyze Monitor and alert Developer Platform Ad hoc search On-Premises Private Cloud Public Cloud Storage Online Shopping Cart Telecoms Desktops Security Web Services Networks Containers Web Clickstreams RFID Smartphones and Devices Servers Messaging GPS Location Packaged Applications Custom Applications Online Services DatabasesCall Detail Records Energy MetersFirewall Intrusion Prevention Platform Support (Apps / API / SDKs) Enterprise Scalability Universal Indexing Machine Data: Any Location, Type, Volume Answer Any Question Any Amount, Any Location, Any Source Schema on-the-fly Universal indexing No back-end RDBMS No need to filter data
  • 4. © 2017 SPLUNK INC. The Splunk Portfolio Rich Ecosystem of Apps & Add-Ons Splunk Premium Solutions Mainframe Data Relational Databases MobileForwarders Syslog/ TCP IoT Devices Network Wire Data Hadoop Platform for Operational Intelligence
  • 5. © 2017 SPLUNK INC. ▶ Non-Traditional Data Sources ▶ Data Enrichment ▶ Level Up on Search and Reporting Commands ▶ Data Models and Pivot ▶ Custom Visualizations and the Web Framework Agenda
  • 6. © 2017 SPLUNK INC. Workshop Setup
  • 7. © 2017 SPLUNK INC. Download Splunk or Sign Up For Splunk Cloud www.splunk.com > Free Splunk > Splunk Enterprise or Splunk Cloud SHOW 1 2 3
  • 8. © 2017 SPLUNK INC. ▶ Box > access_datasample_last4h.log ▶ Box > http_status.csv Download Data Sample and Lookup https://splunk.box.com/v/MD101Workshop SHOW
  • 9. © 2017 SPLUNK INC. ▶ Browser: http://localhost:8000 ▶ Default username/password is admin/changeme Index Data SampleSHOW 1 2
  • 10. © 2017 SPLUNK INC. Index Data SampleSHOW 3 2 1 4 5
  • 11. © 2017 SPLUNK INC. Index Data SampleSHOW 1 2
  • 12. © 2017 SPLUNK INC. Index Data SampleSHOW 1 2 You will need to refresh the search after a few moments for all events to show up
  • 13. © 2017 SPLUNK INC. 12.130.60.4 - - [18/Sep/2014 05:26:50:193] "GET /product.screen?product_id=AV-CB-01&JSESSIONID=SD8SL4FF8ADFF5 HTTP 1.1" 200 3221 "http://www.myflowershop.com/category.screen?category_id=BOUQUET "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"634 ▶ Keyword searching ▶ Interesting fields sourcetype=access_combined ▶ Field extractions + why are they important • IFX | rex | auto kv through app logging best practices ▶ Filters Quick UI/Search OrientationSHOW clientip method url bytes xfered status return code user agent
  • 14. © 2017 SPLUNK INC. ▶ Data discovery ▶ Group like events ▶ Save as event type ▶ Create alert Pattern DetectionSHOW Back to Slides
  • 15. © 2017 SPLUNK INC. Non-Traditional Data Sources
  • 16. © 2017 SPLUNK INC. ▶ Network Inputs ▶ HTTP Event Collector ▶ Log Event Alert Action ▶ Splunk Stream ▶ Scripted Inputs ▶ Database Inputs ▶ Splunk ODBC Driver ▶ Modular Inputs ▶ zLinux Forwarder ▶ MINT ▶ Non-Splunk Datastores Non-Traditional Data Sources
  • 17. © 2017 SPLUNK INC. ▶ Captures events from log files in real time ▶ Runs scripts to gather system metrics, connect to APIs and databases ▶ Listens to syslog and gathers Windows events ▶ Universally indexes any data format so it doesn’t need adapters Traditional Data Sources Windows • Registry • Event logs • File system • sysinternals Linux/Unix • Configurations • Syslog • File system • Ps, iostat, top Virtualization • Hypervisor • Guest OS • Guest Apps Applications • Web logs • Log4J, JMS, JMX • .NET events • Code and scripts Databases • Configurations • Audit/query logs • Tables • Schemas Network • Configurations • syslog • SNMP • netflow
  • 18. © 2017 SPLUNK INC. ▶ Collect data over any UDP or TCP port • Some devices only send data over a network port ▶ Best Practice: use syslog-ng or rsyslog • Offers persistence • Categorizes data by host Network Inputs
  • 19. © 2017 SPLUNK INC. ▶ Collect data over HTTP or HTTPS directly to Splunk • Application Developer focus – few lines of code in app to send data ▶ HEC Features Include: • Token-based, not credential based • Indexer Acknowledgements – guarantees data indexing • Raw and JSON formatted event payloads • SSL, CORS (Cross-Origin Resource Sharing), and Network Restrictions HTTP Event Collector (HEC)
  • 20. © 2017 SPLUNK INC. ▶ Use Splunk alerting to index a custom log event • Splunk searchable index of custom alert events ▶ Configurable Features Include: • Host • Source • Sourcetype • Index • Event text – construct the exact syntax of the log event, including any text, tokens, or other information Log Event Alert Action
  • 21. © 2017 SPLUNK INC. Wire Data Enhances the Platform for Operational Intelligence Efficient, Cloud-Ready Wire Data Collection Simple Deployment Supports Fast Time to Value The Splunk Stream Log Files Configurations Wire Data Alerts Metrics Scripts Changes Tickets Sensors Security Custom Applications Networks Databases Servers Smartphones and Devices Web Services Virtual Machines
  • 22. © 2017 SPLUNK INC. Solution Area Contextual Data Wire Data Enriched View Application Management application logs, monitoring data, metrics, events protocol conversations on database performance, DNS lookups, client data, business transaction paths… Measure application response times, deeper insights for root-cause diagnostics, trace tx paths, establish baselines… IT Operations application logs, monitoring data, metrics, events payload data including process times, errors, transaction traces, ICA latency, SQL statements, DNS records… Analyze traffic volume, speed and packets to identify infrastructure performance issues, capacity constraints, changes; establish baselines… Stream = Better Insights for *
  • 23. © 2017 SPLUNK INC. Solution Area Contextual Data Wire Data Enriched View Security app + infra logs, monitoring data, events protocol identification, protocol headers, content and payload information, flow records Build analytics and context for incident response, threat detection, monitoring and compliance Digital Intelligence website activity, clickstream data, metrics browser-level customer interactions Customer Experience – analyze website and application bottlenecks to improve customer experience and online revenues Customer Support (online, call center) – faster root cause analysis and resolution of customer issues with website or apps Stream = Better Insights for *
  • 24. © 2017 SPLUNK INC. ▶ Send data to Splunk via a custom script • Splunk indexes anything written to stdout • Splunk handles scheduling • Supports shell, Python scripts, WIN batch, PowerShell • Any other utility that can format and stream data Scripted Inputs Streaming Mode • Splunk executes script and indexes stdout • Checks for any running instances Write to File Mode • Splunk launches script which produces output file, no need for external scheduler • Splunk monitors output file
  • 25. © 2017 SPLUNK INC. ▶ Alternative to file-base or network-based inputs ▶ Stream data from command-line tools, such as vmstat and iostat ▶ Poll a web service, API or database and process the results ▶ Reformat complex or binary data for easier parsing into events and fields ▶ Maintain data sources with slow or resource-intensive startup procedures ▶ Provide special or complex handling for transient or unstable inputs ▶ Scripts that manage passwords and credentials ▶ Wrapper scripts for command line inputs that contain special characters Use Cases for Scripted Inputs
  • 26. © 2017 SPLUNK INC. ▶ DB Connect provides reliable, scalable, real-time integration between Splunk and traditional relational databases • Create value with structured data • Enrich search results with additional business context • Easily import data for deeper analysis • Integrate multiple DBs concurrently • Simple set-up, non-invasive and secure Database Inputs DB CONNECT JRE JDBC DATABASE DRIVER DATABASE
  • 27. © 2017 SPLUNK INC. ▶ DB Connect App • Real-time, scalable integration with relational DBs • Browse and navigate schemas and tables before data import • Reliable scheduled import • Seamless installation and UI configuration • Supports connection pooling and caching ▶ “Tail” tables or import entire tables • Detect and import new/updated rows using timestamps or unique IDs ▶ Supports many RDBMS flavors • AWS RDS Aurora, AWS RedShift, IBM DB2 for Linux, Informix, MemSQL, MS SQL, MySQL, Oracle, PostgreSQL, SAP SQL Anywhere (aka Sybase SA), Sybase ASE and IQ, Teradata Configure Database Inputs
  • 28. © 2017 SPLUNK INC. ▶ Interact with, manipulate and visualize machine data in Splunk Enterprise using business software tools ▶ Leverage analytics from Splunk alongside Microsoft Excel, Tableau Desktop or Microstrategy Analytics Desktop ▶ Industry-standard connectivity to Splunk Enterprise ▶ Empowers business users with direct and secure access to machine data ▶ Combine machine data with structured data for better operational context Splunk ODBC Driver
  • 29. © 2017 SPLUNK INC. ODBC: How it Works Splunk AdminAnalyst Step 3: Business Analyst uses Microsoft Excel, Tableau or Mocrostrategy to access Data Models and saved searches and retrieve machine data from Splunk Enterprise Step 2: Splunk Admin authors Data Models or saved searches in Splunk Enterprise Step 1: Business Analyst communicates data requirements to Splunk Admin Analyst REQUIREMENTS Saved Searches or Data Models Tableau or MS Excel or Microstrategy ODBC Driver
  • 30. © 2017 SPLUNK INC. ▶ Create your own custom inputs • Scripted input with structure and intelligence • First class citizen in the Splunk management interface • Appears under Settings > Data Inputs ▶ Benefits over simple scripted input • Instance control: launch a single instance or multiple instances • Input validation • Support multiple platforms • Stream data as text or XML • Secure access to mod input scripts via REST endpoints Modular Inputs
  • 31. © 2017 SPLUNK INC. ▶ Twitter • Stream JSON data from a Twitter source to Splunk using Tweepy ▶ Amazon S3 Online Storage • Index data from the Amazon S3 online storage web service ▶ Java Messaging Service (JMS) • Poll message queues and topics through JMS Messaging API • Talks to multiple providers: MQSeries (Websphere MQ), ActiveMQ, TibcoEMS, HornetQ, RabbitMQ, Native JMS, WebLogic JMS, Sonic MQ ▶ Splunk Windows Inputs • Retrieve WIN event logs, registry keys, perfmon counters Example Modular Inputs
  • 32. © 2017 SPLUNK INC. More Modular Inputs
  • 33. © 2017 SPLUNK INC. ▶ Easily collect and index data on IBM mainframes ▶ Collect application and platform data ▶ Download as new Forwarder distribution for s390x Linux zLinux Forwarder
  • 34. © 2017 SPLUNK INC. Deliver Better Performing, More Reliable Apps End-to-End Performance and Capacity Insights Deliver Real-Time Omni-Channel Analytics Extend Operational Intelligence to Mobile Apps
  • 35. © 2017 SPLUNK INC. ▶ Improve user retention by quickly identifying crashes and performance issues ▶ Establish whether issues are caused by an app or the network(s) ▶ Correlate app, OS and device type to diagnose crash and network performance issues Monitor App Usage and Performance
  • 36. © 2017 SPLUNK INC. ▶ Hunk Archive functionality moves under Splunk Enterprise as Data Roll ▶ Hunk searching of third party data is rebranded as Splunk Analytics for Hadoop ▶ Pricing model stays the same as Hunk - no new SKU HUNK > Splunk Analytics for Hadoop Hadoop Clusters Splunk Analytics for Hadoop Add-on
  • 37. © 2017 SPLUNK INC. ▶ Build custom streaming resource libraries ▶ Search and analyze data from other data stores in Splunk ▶ In partnership with leading NoSQL vendors ▶ Use in conjunction with DB Connect for relational database lookups Connect to NoSQL and Other Data Stores Splunk Analytics for Hadoop Add-on STREAMING ERP
  • 38. © 2017 SPLUNK INC. ▶ Rolls historical data into existing Hadoop distribution ▶ Reduces storage up to 80%* ▶ Retains Splunk search capability with performance tradeoffs ▶ Integrated, zero-cost option of Splunk Enterprise * Achieved by reducing Splunk performance optimization data Hadoop Data Roll Amazon EMR on S3 Hadoop Clusters Leverage existing Hadoop Datastore to reduce TCO
  • 39. © 2017 SPLUNK INC. ▶ Enables seamless use of almost the entire Splunk stack on data ▶ Automatically handles MapReduce ▶ Technology is patent pending Virtual Indexes
  • 40. © 2017 SPLUNK INC. Data Enrichment
  • 41. © 2017 SPLUNK INC. ▶ Tags – categorize and add meaning to data ▶ Field Aliases – simplify search and correlation ▶ Calculated Fields – shortcut complex/repetitive computations ▶ Event Types – group common events and share knowledge ▶ Lookups – augment data with additional external fields Agenda
  • 42. © 2017 SPLUNK INC. ▶ Adds inline meaning/context/specificity to raw data ▶ Used to normalize metadata or raw data ▶ Simplifies correlation of multiple data sources ▶ Created in Splunk ▶ Transferred from external sources What is Data Enrichment?
  • 43. © 2017 SPLUNK INC. ▶ Add meaning/context/specificity to raw data ▶ Labels describing team, category, platform, geography ▶ Applied to field-value combination ▶ Multiple tags can be applied for each field-value ▶ Case sensitive Tags
  • 44. © 2017 SPLUNK INC. Create TagsSHOW
  • 45. © 2017 SPLUNK INC. Search events with tag in any field Search events with tag in a specific field Search events with tag using wildcards Find the Web Servers Tags in Action tag=webserver tag::host=webserver tag=web* Tag the host as webserver Tag the sourcetype as web 1 2 3 4 5 SHOW Back to Slides
  • 46. © 2017 SPLUNK INC. ▶ Normalize field labels to simplify search and correlation ▶ Apply multiple aliases to a single field • Example: Username | cs_username | User  user • Example: c_ip | client | client_ip  clientip ▶ Processed after field extractions + before lookups ▶ Can apply to lookups ▶ Aliases appear alongside original fields Field Aliases
  • 47. © 2017 SPLUNK INC. Re-Label Field to Intuitive Name Create Field Alias SHOW 1 2 3
  • 48. © 2017 SPLUNK INC. Create field alias of clientip = customer Search events in last 15 minutes, find customer field Field alias (customer) and original field (clientip) are both displayed Search using an Intuitive Field Name Field Alias in Action sourcetype=access_combined SHOW 1 2 3
  • 49. © 2017 SPLUNK INC. ▶ Shortcut for performing repetitive/long/complex transformations using eval command ▶ Based on extracted or discovered fields only ▶ Do not apply to lookup or generated fields Calculated Fields 1 2 3 3
  • 50. © 2017 SPLUNK INC. Compute Kilobytes from Bytes Create Calculated Field SHOW 1 2 3
  • 51. © 2017 SPLUNK INC. Create kilobytes = bytes/1024 Search events in last 15 minutes for kilobytes and bytes Search Using Kilobytes instead of Bytes Calculated Fields in Action SHOW Back to Slides 1 2 sourcetype=access_combined
  • 52. © 2017 SPLUNK INC. ▶ Classify and group common events ▶ Capture and share knowledge ▶ Based on search ▶ Use in combination with fields and tags to define event topography Event Types
  • 53. © 2017 SPLUNK INC. ▶ Best Practice: Use punct field • Default metadata field describing event structure • Built on interesting characters: ",;-#$%&+./:=?@'|*nr"(){}<>[]^! » • Can use wildcards Create Event Types event punct ####<Jun 3, 2014 5:38:22 PM MDT> <Notice> <WebLogicServer> <bea03> <asiAdminServer> <WrapperStartStopAppMain> <>WLS Kernel<> <> <BEA-000360> <Server started in RUNNING mode> ####<_,__::__>_<>_<>_<>_<>_ <>_ 172.26.34.223 - - [01/Jul/2005:12:05:27 -0700] "GET /trade/app?action=logout HTTP/1.1" 200 2953 ..._-_-_[:::_-]_"_?=_/."__
  • 54. © 2017 SPLUNK INC. Show punct for sourcetype=access_combined Pick a punct, then wildcard it after the timestamp Add NOT status=200 Save as “bad” event type + Color:red + Priority:1 (shift reload in browser to show coloring) Classify Events as Known Bad Create Event Type sourcetype="access_combined" punct="..._-_-_[//_:::]*" NOT status=200 SHOW Back to Slides 1 2 3 4 eventtype=bad
  • 55. © 2017 SPLUNK INC. Lookups to Enrich Raw Data CRM/ ERP External Data Sources Data goes in Create additional fields from the raw data with a lookup to an external data source Insight comes out Watch Lists LDAP AD CMDB
  • 56. © 2017 SPLUNK INC. ▶ Augment raw events with additional fields • Provide context or supporting details ▶ Translate field values to more descriptive data • Example: add text descriptions for error codes, IDs • Example: add contact details to user names or IDs • Example: add descriptions to HTTP status codes ▶ File-based or scripted lookups Lookups
  • 57. © 2017 SPLUNK INC. Convert a Code into a Description Configure a Static Lookup 1. Upload/create table 2. Assign table to lookup object 3. Map lookup to data set SHOW
  • 58. © 2017 SPLUNK INC. Get the lookup from the Splunk Wiki (save to .csv file) http://wiki.splunk.com/Http_status.csv Lookup table files > Add new • Name: http_status.csv (must have .csv file extension) • Upload: <path to .csv> Verify lookup was created successfully 1. Create HTTP Status TableSHOW 1 2 3 | inputlookup http_status.csv
  • 59. © 2017 SPLUNK INC. Lookup definitions > Add new • Name: http_status • Type: File-based • Lookup file: http_status.csv Invoke the lookup manually 2. Add Lookup DefinitionSHOW sourcetype=access_combined | lookup http_status status OUTPUT status_description 1 2
  • 60. © 2017 SPLUNK INC. Automatic lookups > Add new • Name: http_status (cannot have spaces) • Lookup table: http_status • Apply to: sourcetype = access_combined • Lookup input field: status • Lookup output field: status_description Verify lookup is invoked automatically 3. Configure Automatic LookupSHOW 1 2
  • 61. © 2017 SPLUNK INC. ▶ Temporal lookups for time-based lookups • Example: Identify users on your network based on their IP address and the timestamp in DHCP logs ▶ Use search results to populate a lookup table • … | outputlookup <tablename|filename> ▶ Call an external command or script • Python scripts only • Example: DNS lookup for IP  Host ▶ Create a lookup table using a relational database • Review matches against a database column or SQL query Fancy Lookups
  • 62. © 2017 SPLUNK INC. ▶ Creating and Managing Alerts (Job Inspector) ▶ Macros ▶ Workflow Actions More Data Enrichment
  • 63. © 2017 SPLUNK INC. BREAK 15 MINUTES
  • 64. © 2017 SPLUNK INC. Level Up on Search & Reporting Commands
  • 65. © 2017 SPLUNK INC. ▶ Doing more with basic search commands ▶ Advanced search commands ▶ Doing more with basic reporting commands Agenda
  • 66. © 2017 SPLUNK INC. Search Syntax Components
  • 67. © 2017 SPLUNK INC. Anatomy of a Search Disk
  • 68. © 2017 SPLUNK INC. ▶ top – limit ▶ rare – same options as top ▶ timechart – parameters ▶ stats – functions (sum, avg, list, values, sparkline) ▶ sort – inline ascending or descending ▶ addcoltotals ▶ addtotals Doing More with Basic Search Commands
  • 69. © 2017 SPLUNK INC. Workshop Notes for Presenter Tip #6: In the next section, after each search, have the participants save the search as a dashboard panel. At the end of the workshop, they will have a living document of the workshop exercises to reference later. A complete version of this dashboard is packaged as an app. It is uploaded to the Box folder as a leave behind.
  • 70. © 2017 SPLUNK INC. ... | rare limit=20 clientip ... | top limit=20 clientip ▶ Commands have parameters or qualifiers ▶ Top and rare have similar syntax ▶ Each search command has its own syntax – show inline help Find Most and Least Active Customers Using the top + rare Commands SHOW IPs with the most visits IPs with the least visits
  • 71. © 2017 SPLUNK INC. ... | stats count by clientip | sort + count ... | stats count by clientip | sort - count ▶ Sort inline descending or ascending The Number of Customer Requests Using the sort Command SHOW Number of requests by customer - descending Number of requests by customer - ascending
  • 72. © 2017 SPLUNK INC. ▶ Show Search Command Reference Docs • Functions for eval + where • Functions for stats + chart and timechart ▶ Invoke a function ▶ Rename inline ... | stats sum(bytes) as totalbytes by clientip | sort - totalbytes ... | stats sum(bytes) by clientip | sort - sum(bytes) Determine Total Customer Payload Using functions + rename command SHOW Total payload by customer - descending Total payload by customer - ascending
  • 73. © 2017 SPLUNK INC. ▶ List all values of a field ▶ List only distinct values of a field Observe Customer Activity Using the list + values Functions ... | stats values(action) by clientip ... | stats list(action) by clientip SHOW Activity by customer Distinct actions by customer
  • 74. © 2017 SPLUNK INC. ▶ Show distinct actions and cardinality of each action Analyze Customer Activity Combine list + values Functions sourcetype=access_combined | stats count(action) as value by clientip, action | eval pair=action + " (" + value + ")" | stats list(pair) as values by clientip SHOW
  • 75. © 2017 SPLUNK INC. ... | stats sum(bytes) as totalbytes, avg(bytes) as avgbytes, count as totalevents by clientip | addcoltotals totalbytes, totalevents ▶ Add columns ▶ Sum specific columns Building a Table of Customer Activity Add Columns and Sum Columns ... | stats count by clientip, action SHOW 2 cols: clientip + action Sum totalbytes and totalevents columns
  • 76. © 2017 SPLUNK INC. ... | stats sum(bytes) as totalbytes, sum(other) as totalother by clientip | addtotals fieldname=totalstuff Building a Table of Customer Activity Sum Across Rows SHOW Sum totalbytes and totalevents columns A better example: physical memory + virtual memory = total memory
  • 77. © 2017 SPLUNK INC. ... | stats sparkline(count) as trendline sum(bytes) by clientip Trend Individual Customer Activity Sparklines in Action ... | stats sparkline(count) as trendline by clientip SHOW In context of larger event set Inline in tables Back to Slides
  • 78. © 2017 SPLUNK INC. Advanced Search Commands Command Short Description Hints transaction Group events by a common field value. Convenient, but resource intensive. cluster Cluster similar events together. Can be used on _raw or field. associate Identifies correlations between fields. Calculates entropy btn field values. correlate Calculates the correlation between different fields. Evaluates relationship of all fields in a result set. contingency Builds a contingency table for two fields. Computes co-occurrence, or % two fields exist in same events. anomalies Computes an unexpectedness score for an event. Computes similarity of event (X) to a set of previous events (P). anomalousvalue Finds and summarizes irregular, or uncommon, search results. Considers frequency of occurrence or number of stdev from the mean.
  • 79. © 2017 SPLUNK INC. ▶ Sew events together + creates duration + eventcount View Customer Activity by Session Using the transaction Command ... | transaction JSESSIONID | table JSESSIONID, action, product_id SHOW Group by JSESSIONID
  • 80. © 2017 SPLUNK INC. ▶ Predict over time ▶ Chart Overlay with and without streamstats ▶ Maps with iplocation + geostats ▶ Single value ▶ Metered visuals with gauge Doing More with Basic Reporting Commands
  • 81. © 2017 SPLUNK INC. ▶ Predict future values using lower/upper bounds – single and multiple series Predict Website Traffic Using the predict Command ... | timechart count as traffic | predict traffic SHOW
  • 82. © 2017 SPLUNK INC. Compare Browsing vs. Buying Activity Simple Chart Overlay SHOW sourcetype=access_combined (action=view OR action=purchase) | timechart span=10m count(eval(action="view")) as Viewed, count(eval(action="purchase")) as Purchased
  • 83. © 2017 SPLUNK INC. Map Customer Activity Geographically Geolocation in Action SHOW ... | iplocation clientip | geostats count by clientip Combine IP lookup with geo mapping
  • 84. © 2017 SPLUNK INC. Display a Simple Count of Events Single Value in Action SHOW ... | stats count
  • 85. © 2017 SPLUNK INC. Display Counts Using Gauges Single Value, Radial and Filler Gauges in Action SHOW ... | stats count | gauge count 10000 20000 30000 40000 50000 Back to Slides
  • 86. © 2017 SPLUNK INC. BREAK 15 MINUTES
  • 87. © 2017 SPLUNK INC. Data Model and Pivot
  • 88. © 2017 SPLUNK INC. ▶ What is a data model? ▶ Build a data model ▶ Pivot Interface ▶ Accelerate a data model Agenda
  • 89. © 2017 SPLUNK INC. Pivot Enables non-technical users to build complex reports without the search language Powerful Analytics Anyone Can Use Data Model Provides more meaningful representation of underlying raw machine data Analytics Store Acceleration technology delivers up to 1000x faster analytics over Splunk 5
  • 90. © 2017 SPLUNK INC. ▶ Data Model • Describes how underlying machine data is represented and accessed • Defines meaningful relationships in the data • Enables single authoritative view of underlying raw data Define Relationships in Machine Data Hierarchical object view of underlying data Add constraints to filter out events
  • 91. © 2017 SPLUNK INC. ▶ High Performance Analytics Store • Automatically collected − Handles timing issues, backfill… • Automatically maintained − Uses acceleration window • Stored on the indexers − Peer to the buckets • Fault tolerant collection Transparent Acceleration Check to enable acceleration of data model Time window of data that is accelerated
  • 92. © 2017 SPLUNK INC. ▶ Pivot • Drag-and-drop interface enables any user to analyze data • Create complex queries and reports without learning search language • Click to visualize any chart type; reports dynamically update when fields change Easy-to-Use Analytics All chart types available in the chart toolbox Select fields from data model Time window Save report to share
  • 93. © 2017 SPLUNK INC. ▶ Defines least common denominator for a data domain ▶ Standard method to parse, categorize, normalize data ▶ Set of field names and tags by domain ▶ Packaged as Data Models in a Splunk App • Domains: security, web, inventory, JVM, performance, network sessions, and more • Minimal setup to use Pivot interface Common Information Model (CIM) App
  • 94. © 2017 SPLUNK INC. Apps > Find More Apps > Search: “Common Information Model” Install free Show fields for web + Web Data Model Download CIM AppSHOW Back to Slides 1 2 3 4
  • 95. © 2017 SPLUNK INC. http://docs.splunk.com/Documentation/Splunk/latest/PivotTutorial/WelcometothePivotTutorial Data Model & Pivot Tutorial
  • 96. © 2017 SPLUNK INC. Custom Visualizations and the Web Framework Toolkit
  • 97. © 2017 SPLUNK INC. ▶ Custom Visualizations ▶ Developer Platform ▶ Resources Agenda
  • 98. © 2017 SPLUNK INC. ▶ Native charts and maps • Bar / Line / Area charts • Bubble / Scatter plots • Gauges • Maps • Single Value Displays • Tables ▶ Generalized to fit use cases across many different areas ▶ Can be customized to some extent to cover specific use cases Native Visualizations In Splunk
  • 99. © 2017 SPLUNK INC. ▶ Many use cases require a more specific visualization ▶ Specific custom appearance ▶ Represent data where native visualizations are not suitable • You can Splunk everything! • We won’t be able to predict every possible use case • Still uses SPL to drive visualizations Custom Visualizations FTW!
  • 100. © 2017 SPLUNK INC. ▶ Platform extensibility framework and API ▶ Targeted at internal and external developers with web development / JS skills and basic knowledge of the Splunk platform ▶ Developers can make use of any third party libraries (d3.js, three.js, highcharts.js, etc…) that run in the browser* * with minor adjustments, and if third party license permits such use Custom Visualizations
  • 101. © 2017 SPLUNK INC. ▶ Packaged as an app! ▶ Installed like any other app ▶ Users can search for visualizations on Splunkbase and directly in the product Custom Visualizations For Admins In-productInstallation
  • 102. © 2017 SPLUNK INC. ▶ Choose from potentially dozens of installed visualizations! ▶ Appears as a first-class citizen alongside native visualizations • Looks and works just like packaged native visualizations ▶ Customize functionality and appearance of the visualization without touching any code, straight from the UI SPL Example provided as you hover over each visualization option. Custom Visualizations How-to
  • 103. © 2017 SPLUNK INC. New Splunk Visualizations Multiple use cases across IT, security, IoT, and business analytics Treemap Sankey Diagram Punchcard Calendar Heat Map Parallel Coordinates Bullet GraphLocation Tracker Horseshoe Meter Machine Learning Charts Timeline Horizon Chart
  • 104. © 2017 SPLUNK INC. Box Plot 3D scatter plot New Partner/Community Visualizations Wordcloud Donut Chart Heat Map
  • 105. © 2017 SPLUNK INC. New Partner/Community Visualizations Geo Heatmap Custom Cluster Map Clustered Single Value Map Missile Map
  • 106. © 2017 SPLUNK INC. The Splunk Enterprise Platform Collection Indexing Search Processing Language Core Functions Inputs, Apps, Other Content Content Core Engine User and Developer Interfaces Core Engine User and Developer Interfaces Content Web Framework SDK Rest API
  • 107. © 2017 SPLUNK INC. Developer Platform What’s Possible with the Splunk Enterprise Platform? Power Mobile Apps Log Directly Extract Data Customer Dashboards Integrate BI Tools Integrate Platform Services
  • 108. © 2017 SPLUNK INC. Web Framework Toolkit
  • 109. © 2017 SPLUNK INC. SDKs Powerful Platform for Enterprise Developers Developers Can Customize and Extend Rest API Web Framework Java JavaScript Python Simple XML JavaScript HTML5 Data Models Search Extensibility Modular Inputs Ruby C# PHP Extend and Integrate SplunkBuild Splunk Apps
  • 110. © 2017 SPLUNK INC. Splunk Software for Developers GAIN APPLICATION INTELLIGENCE INTEGRATE AND EXTEND SPLUNK BUILD SPLUNK APPS
  • 111. © 2017 SPLUNK INC. A Wealth of Splunk Apps Over 1,300 apps available on the Splunk apps site Server, Storage, Network Server Virtualization Operating Systems Custom Applications Business Applications Cloud Services App Performance MonitoringTicketing/ and Other Web Intelligence Mobile Applications Stream API SDKs UI
  • 112. © 2017 SPLUNK INC. ▶ Interactive, cut/paste examples from popular source repositories: D3, GitHub, jQuery ▶ Splunk 6.x Dashboard Examples App https://apps.splunk.com/app/1603 ▶ Custom SimpleXML Extensions App https://apps.splunk.com/app/1772 ▶ Splunk Web Framework Toolkit App https://apps.splunk.com/app/1613 Example Advanced Visualizations
  • 113. © 2017 SPLUNK INC. Resources
  • 114. © 2017 SPLUNK INC. ▶ http://docs.splunk.com ▶ Official Product Docs ▶ Wiki and community topics ▶ Updated daily ▶ Can be printed to .PDF Splunk Documentation
  • 115. © 2017 SPLUNK INC. ▶ http://answers.splunk.com ▶ Community driven ▶ Splunk supported ▶ Knowledge exchange ▶ Q & A Splunk Answers
  • 116. © 2017 SPLUNK INC. ▶ Recommended for Users • Using Splunk • Searching & Reporting ▶ Recommended for UI/Dashboard Developers • Developing Apps ▶ Instructor-Led Courses • Web • Onsite Splunk Education
  • 117. © 2017 SPLUNK INC. Delivered Globally: Online, Classroom, Self-Paced 15 Free Getting Started Videos Get Splunk Certified in 5 Days 20 Classes For more information: splunk.com/education Knowledge is Power Splunk Education
  • 118. © 2017 SPLUNK INC. Become a Splunk Expert Knowledge is Power Using Splunk Searching and Reporting with Splunk Creating Splunk Knowledge Objects Splunk Administration Architecting and Deploying Splunk Developing Apps with Splunk Splunk Architect Certification Lab
  • 119. © 2017 SPLUNK INC. Splunk Education for Security Knowledge is Power Using Splunk Searching and Reporting with Splunk Creating Splunk Knowledge Objects Using the Splunk App for Enterprise Security Splunk Administration Architecting and Deploying Splunk Administering the Splunk App for Enterprise Security
  • 120. © 2017 SPLUNK INC. Splunk Education for IT Service Intelligence Knowledge is Power Using Splunk Searching and Reporting with Splunk Creating Splunk Knowledge Objects Splunk Administration Implementing IT Service Intelligence
  • 121. © 2017 SPLUNK INC. Course Topics • Overview of ITSI features • ITSI architecture and deployment • Installing ITSI • Designing and implementing services and entities • Configuring correlation searches and notable events • Creating deep dive pages • Creating glass tables • ITSI troubleshooting Splunk Education for IT Service Intelligence Knowledge is Power
  • 122. © 2017 SPLUNK INC. Splunk Education for IT Service Intelligence Knowledge is Power Course Topics • Overview of ITSI features • ITSI architecture and deployment • Installing ITSI • Designing and implementing services and entities • Configuring correlation searches and notable events • Creating deep dive pages • Creating glass tables • ITSI troubleshooting PREREQUISITES | 13.5 hour course Using Splunk, Searching and Reporting with Splunk, Creating Splunk Knowledge Objects, Splunk Administration
  • 123. © 2017 SPLUNK INC. Q&A Please use HipChat Room: [Title of Room] to Submit Your Questions to eStaff
  • 124. © 2017 SPLUNK INC. Get Started Fast! splunk.com/education
  • 125. © 2017 SPLUNK INC.© 2017 SPLUNK INC. Thank You

Editor's Notes

  1. The rise of big data has forced IT organizations to transition from a focus on structured, relational data, to accommodate unstructured data, driven by the volume, velocity and variety of today’s applications and systems. As the data has changed from structured data to unstructured data, the technology approach needs to change as well. When you don’t know what data types you’ll need to analyze tomorrow or what questions you need to ask in a week, flexibility becomes a key component of your technology decisions. The ability to index any data type, search across silos and avoid being locked into a rigid schema opens a new world of analytics and business insights to your organization. Schema at Read – Enables you ask any question of the deal Search – Enables rapid, iterative exploration of the data along with advanced analytics Universal Indexing – Enables you to ingest any type of machine data Horizontal scaling over commodity hardware enables big data analytics
  2. Splunk is the platform for machine data, it digests all machine data and allow users to quickly analyze their data and rapidly obtain insight. The platform was designed around the premise of being able to consume any machine data even if the format changes. A relational database would cannot effectively support constantly changing underlying schemas. Splunk solves this by creating a schema on the fly… Splunk Cloud is only available in the U.S. and Canada.
  3. The Splunk platform consists of multiple products to fit your needs. The products can pull in data from virtually any source to support multiple use cases. And we continue to invest heavily as new sources become available. There are 1,000s of apps built specifically to extend and simplify deployments by providing pre-packaged content designed for specific use cases and data types. Premium apps that are built by Splunk are available for Security, IT Service Intelligence and User Behavior Analytics .
  4. Will spend most of our time on the first 3 topics.
  5. Splunk’s mission statement is to make machine data accessible, useful and valuable to everyone. Splunk can take any machine data and automatically index it for fast searching. Because Splunk doesn’t use a database, there are no additional licenses, and most importantly, no pre-defined schema to limit how you use your information. Examples include the configuration files, syslog, Windows events and registry settings, as well as WMI. But the most important thing to note is how easy it is to get data into Splunk and make it useful.
  6. The Splunk Stream software captures real-time wire data from distributed infrastructures, including private, public and hybrid clouds with on-the-fly deployment and fine-grained filtering capabilities.
  7. Splunk DB Connect delivers reliable, scalable, real-time integration between Splunk Enterprise and traditional relational databases. With Splunk DB Connect, structured data from relational databases can be easily integrated into Splunk Enterprise, driving deeper levels of operational intelligence and richer business analytics across the organization. Organizations can drive more meaningful insights for IT operations, security and business users. For example, IT operations teams can track performance, outage and usage by department, location and business entities. Security professionals can correlate machine data with critical assets and watch-lists for: incident investigations, real-time correlations and advanced threat detection using the award-winning Splunk Enterprise. Business users can analyze service levels and user experience by customer in real-time to make more informed decisions.
  8. To address the needs of developers, operations and product management, you need Operational Intelligence for your mobile apps. This is what we call mobile intelligence. Mobile intelligence provides real-time insight on how your mobile apps are performing, and can correlate with and enhance Operational Intelligence.    Splunk software enables organizations to search, monitor, analyze and visualize machine-generated data from websites, applications, servers, networks, sensors and mobile devices. Splunk MINT helps organizations monitor mobile app usage and performance, gain deep visibility into mobile app transactions and accelerate development Deliver better performing, more reliable apps When a user has a problem with a mobile app, the issue could be isolated or spread across all app versions, handsets and OS types. With Splunk MINT, you can see issues with app performance or availability in real time. Bugs can be addressed quickly, and app developers can gain a head start in creating and delivering valuable app updates. Achieve End-to-End visibility When mobile apps fail, there are many potential sources of failure. With Splunk MINT, you can analyze overall transaction performance. And using Splunk MINT, you can correlate this data with information from back-end apps to gain detailed insight on transaction problems. As a result, operations can reduce MTTR and better anticipate future mobile app back-end requirements. Deliver real-time analytics Mobile apps give enterprises new ways of conducting digital business. With mobile app information in Splunk Enterprise, you can correlate usage and performance information— some call this omni-channel analytics—to better understand how users are engaging all aspects of your organization.
  9. Splunk MINT Express provides a dashboard that offers and at a glance view of Mobile app health and usage. This includes an overall index called “MobDex”, which provides a blended view of Application usage, crashes, engagement in and abandonment. The insight boxes provide top-level aggregated information, which you can click on to get more specific information, and context.
  10. Hadoop data roll is an option available to customers who would like to retain their historical Splunk data in their Hadoop data lake. This functionality used to be part of the Hunk product, but it now integrated within Splunk Enterprise and included with your license. It is compatible with most popular Apache Hadoop distributions as well as Amazon EMR running on S3 storage. The main benefit of Hadoop data roll is TCO reduction achieved by reducing the storage footprint and lower cost storage hardware. Additionally, your Hadoop applications will be able to use data that was originally indexed in Splunk. Reduction in storage footprint is achieved by reducing Splunk search optimization data that are primarily used to speed up ”needle in the haystack” type searches. The storage footprint reductions can range from 40-80%, depending on the characteristics of the underlying data. Once data is rolled into Hadoop, search, reporting, and analysis functionality within Splunk Enterprise is retained through virtual indexes, though with performance tradeoffs. Another consideration is that Splunk premium solutions like ES and ITSI don’t yet support use of data that has been rolled into Hadoop.
  11. Splunk MINT Express provides a dashboard that offers and at a glance view of Mobile app health and usage. This includes an overall index called “MobDex”, which provides a blended view of Application usage, crashes, engagement in and abandonment. The insight boxes provide top-level aggregated information, which you can click on to get more specific information, and context.
  12. The data for example may have a userid but you want to search on a name. Splunk’s lookup capability can enrich the raw data by adding additional fields at search time by. Some common use cases including event and error code description fields. Think “Page not Found” instead of “404”. Enriching your data can lead to entirely new insight. In the example shown, Splunk took the userid and looked up the name and role of the user from an HR database. Similarly, it determined the location of the failed log in attempt by correlating the IP address. Even though these fields don’t exist in the raw data, Splunk allows you to search or pivot on them at any time. You can also mask data. For example, you may want social security numbers to be replaced with all X’s for regular users but not masked for others. Removing data can also be useful, such as filtering PII, before writing it to an index in Splunk.
  13. Splunk 6 takes large-scale machine data analytics to the next level by introducing three breakthrough innovations: Pivot – opens up the power of Splunk search to non-technical users with an easy-to-use drag and drop interface to explore, manipulate and visualize data Data Model – defines meaningful relationships in underlying machine data and making the data more useful to broader base of non-technical users Analytics Store – patent pending technology that accelerates data models by delivering extremely high performance data retrieval for analytical operations, up to 1000x faster than Splunk 5 Let’s dig into each of these new features in more detail.
  14. Data Models are created using the Data Model Builder and are usually designed and implemented by users who understand the format and semantics of their indexed data, and who are familiar with the Splunk Search Processing Language (SPL). They define meaningful relationships in the data. Unlike data models in the traditional structured world, Splunk Data Models focus on machine data and data mashups between machine data and structured data. Splunk software is founded on the ability to flexibly search and analyze highly diverse machine data employing late-binding or search-time techniques for schematization (“schema-on-the-fly”). And Data Models are no exception. They define relationships in the underlying data, while leaving the raw machine data intact, and map these relationships at search time. They are therefore highly flexible and designed to enable users to rapidly iterate. Security is also a key consideration and data models are fully permissionable in Splunk 6.
  15. Data Models are created using the Data Model Builder and are usually designed and implemented by users who understand the format and semantics of their indexed data, and who are familiar with the Splunk Search Processing Language (SPL). They define meaningful relationships in the data. Unlike data models in the traditional structured world, Splunk Data Models focus on machine data and data mashups between machine data and structured data. Splunk software is founded on the ability to flexibly search and analyze highly diverse machine data employing late-binding or search-time techniques for schematization (“schema-on-the-fly”). And Data Models are no exception. They define relationships in the underlying data, while leaving the raw machine data intact, and map these relationships at search time. They are therefore highly flexible and designed to enable users to rapidly iterate. Security is also a key consideration and data models are fully permissionable in Splunk 6.
  16. Data Models are created using the Data Model Builder and are usually designed and implemented by users who understand the format and semantics of their indexed data, and who are familiar with the Splunk Search Processing Language (SPL). They define meaningful relationships in the data. Unlike data models in the traditional structured world, Splunk Data Models focus on machine data and data mashups between machine data and structured data. Splunk software is founded on the ability to flexibly search and analyze highly diverse machine data employing late-binding or search-time techniques for schematization (“schema-on-the-fly”). And Data Models are no exception. They define relationships in the underlying data, while leaving the raw machine data intact, and map these relationships at search time. They are therefore highly flexible and designed to enable users to rapidly iterate. Security is also a key consideration and data models are fully permissionable in Splunk 6.
  17. What does this platform look like? The platform consists of 2 layer: A core engine and an interface layer On top of the platform you can’t run a broad spectrum of content that supports use cases Use cases range from application mgmt. and IT operations, to ES and PCI compliance, to web analytics The core engine provides the basic services for real time data input, indexing and search as well alerting, large scale distributed processing and role based access The Interface layer consist of the basic UI for search, reporting and visualization – it contains developer interfaces, the REST API, SDKs and Web Framework The SDKs provide a convenient access to core engine services in a variety of programing language environments. The Web Framework enables developers to quickly create Splunk Apps by using the modern web programming paradigm including pre-built components, styles, templates, and reusable samples as well as supporting the development of custom logic, interactions, components, and UI. Developers can choose to program their Splunk App using Simple XML, JavaScript or Django (or any combination thereof). These programmatic interfaces allow you to eithe:r: extend Splunk integrate Splunk with other applications build completely new applications from scratch that require OI or analytical services that Splunk provides
  18. What have developers been building using Splunk Enterprise? Examples include the following: Run searches and retrieve Splunk data from existing Customer Service/Call Center applications (Comcast use case) Integrate Splunk data into existing BI tools and dashboard (Tableau, MS Excel) Build mobile applications with KPI dashboards and alerts powered by Splunk (Otto Group use case) Log directly to Splunk from remote devices (Bosch use cases) Build customer-facing dashboards powered by user-specific data in Splunk (Socialize, Hurricane Labs use cases) Programmatically extract data from Splunk for long-term data warehousing We hope this is just the beginning. We hope to open up a whole new world of enterprise apps.
  19. The Splunk Web Framework Toolkit is a resource to help developers learn how to build rich applications using the Splunk Web Framework. This app contains a collection of examples that show you how to use the components of the Web Framework. You will find documentation and reference information, along with code templates and additional components that you can use within your own Splunk Apps.
  20. BUILD SPLUNK APPS The Splunk Web Framework makes building a Splunk app looks and feels like building any modern web application.   The Simple Dashboard Editor makes it easy to BUILD interactive dashboards and user workflows as well as add custom styling, behavior and visualizations. Simple XML is ideal for fast, lightweight app customization and building. Simple XML development requires minimal coding knowledge and is well-suited for Splunk power users in IT to get fast visualization and analytics from their machine data. Simple XML also lets the developer “escape” to HTML with one click to do more powerful customization and integration with JavaScript.   Developers looking for more advanced functionality and capabilities can build Splunk apps from the ground up using popular, standards-based web technologies: JavaScript and Django. The Splunk Web Framework lets developers quickly create Splunk apps by using prebuilt components, styles, templates, and reusable samples as well as supporting the development of custom logic, interactions, components, and UI. Developers can choose to program their Splunk app using Simple XML, JavaScript or Django (or any combination thereof). EXTEND AND INTEGRATE SPLUNK Splunk Enterprise is a robust, fully-integrated platform that enables developers to INTEGRATE data and functionality from Splunk software into applications across the organization using Software Development Kits (SDKs) for Java, JavaScript, C#, Python, PHP and Ruby. These SDKs make it easier to code to the open REST API that sits on top of the Splunk Engine. With almost 200 endpoints, the REST API lets developers do programmatically what any end user can do in the UI and more. The Splunk SDKs include documentation, code samples, resources and tools to make it faster and more efficient to program against the Splunk REST API using constructs and syntax familiar to developers experienced with Java, Python, JavaScript, PHP, Ruby and C#. Developers can easily manage HTTP access, authentication and namespaces in just a few lines of code.   Developers can use the Splunk SDKs to: - Run real-time searches and retrieve Splunk data from line-of-business systems like Customer Service applications - Integrate data and visualizations (charts, tables) from Splunk into BI tools and reporting dashboards - Build mobile applications with real-time KPI dashboards and alerts powered by Splunk - Log directly to Splunk from remote devices and applications via TCP, UDP and HTTP - Build customer-facing dashboards in your applications powered by user-specific data in Splunk - Manage a Splunk instance, including adding and removing users as well as creating data inputs from an application outside of Splunk - Programmatically extract data from Splunk for long-term data warehousing Developers can EXTEND the power of Splunk software with programmatic control over search commands, data sources and data enrichment. Splunk Enterprise offers search extensibility through: - Custom Search Commands - developers can add a custom search script (in Python) to Splunk to create own search commands. To build a search that runs recursively, developers need to make calls directly to the REST API - Scripted Lookups: developers can programmatically script lookups via Python. - Scripted Alerts: can trigger a shell script or batch file (we provide guidance for Python and PERL). - Search Macros: make chunks of a search reuseable in multiple places, including saved and ad hoc searches.    Splunk also provides developers with other mechanisms to extend the power of the platform. - Data Models: allow developers to abstract away the search language syntax, making Splunk queries (and thus, functionality) more manageable and portable/shareable. - Modular Inputs: allow developers to extend Splunk to programmatically manage custom data input functionality via REST.
  21. Splunk Enterprise empowers developers with application intelligence across the entire product development lifecycle, from monitoring code check-ins and build servers, to pinpointing production issues in real-time and gaining valuable insights on application usage and user preferences. Splunk Enterprise is a robust platform that enables developers to integrate data and functionality from Splunk software into applications across the organization using Software Development Kits (SDKs) for Java, JavaScript, C#, Python, PHP and Ruby. Developers can extend the power of Splunk software with programmatic control over search commands, data sources and data enrichment. Developers can use the tools and languages they know to build Splunk apps with custom dashboards, flexible UI and custom data visualizations, using the Splunk Web Framework.
  22. Here are just some of the new Splunk Apps that have been delivered over the past year. Their goal is to make it easier to use Splunk for specific technologies and use cases – prepackaging inputs, field extractions, searches and visualizations. Highlight a few apps. These apps along with 100’s of others have been developed not only by Splunk but by partners, customers and members of the Splunk community.
  23. “After this workshop, if you want more information, all the product documentation is available online. The documentation is divided into several manuals. For reporting and dashboards you will likely be most interested in the User and Developer Manuals.”
  24. “For a more interactive approach to getting your questions addressed there is Splunk Answers. It is a web based Splunk community of Splunkers like you. Splunk employees are also regular experts on the site.”
  25. “It is not possible to cover everything you need to know about building reports and dashboards in 30-45 minutes. For more structured training with labs, consider Splunk education courses. These are available as instructor-led web-based courses or onsite if there is enough participants per class.”
  26. 15 Get Started Videos Select From X Classes Get Splunk Certified in 5 Days Delivered Online, Classroom, Self-Paced, Custom  
  27. Certification 8 Certification Tracks - Title list chart with checks Image / website
  28. Certification 8 Certification Tracks - Title list chart with checks Image / website
  29. Certification 8 Certification Tracks - Title list chart with checks Image / website
  30. Course Topics Overview of ITSI features ITSI architecture and deployment Installing ITSI Designing and implementing services and entities Configuring correlation searches and notable events Creating deep dive pages Creating glass tables ITSI troubleshooting
  31. Course Topics Overview of ITSI features ITSI architecture and deployment Installing ITSI Designing and implementing services and entities Configuring correlation searches and notable events Creating deep dive pages Creating glass tables ITSI troubleshooting
  32. Are you looking to learn, share, and participate with other Splunk users? Visit usergroups.splunk.com, search for <<City Name>>, and join the local user group to receive updates on upcoming meetings!