Besides seeing the newest features in Splunk Enterprise and learning the best practices for data models and pivot, we will show you how to use a handful of search commands that will solve most search needs. Learn these well and become a ninja.

1. Copyright © 2015 Splunk Inc.
Splunk Ninjas:
New Features and Search Dojo
Steve Hogan
Sr. Sales Engineer
shogan@splunk.com

2. Copyright © 2014 Splunk Inc.
Name: Intercontinental
Dallas Meetings
Access Code: AWS2015

3. Thanks to Our Sponsors

4. 4
Safe Harbor Statement
During the course of this presentation,we may make forward looking statements regarding future events
or the expected performance of the company. We caution you that such statements reflect our current
expectations and estimates based on factors currently known to us and that actual events or results could
differ materially. For important factors that may cause actual results to differ from those contained in our
forward-looking statements, please review our filings with the SEC. The forward-looking statements
made in this presentation are being made as of the time and date of its live presentation. If reviewed
after its live presentation, this presentation may not contain current or accurate information. We do not
assume any obligation to update any forward looking statements we may make. In addition, any
information about our roadmap outlines our general product direction and is subject to change at any
time without notice. It is for informational purposes only and shall not be incorporated into any contract
or other commitment. Splunk undertakes no obligation either to develop the features or functionality
described orto includeany suchfeatureor functionalityina futurerelease.

5. 5
Agenda
What’s new in 6.3
– Breakthrough Performance and Scale
– Advanced Analysis and Visualization
– High Volume Event Collection
– Enterprise-Scale Platform
Harness the power of search and dashboards
– Search Examples and dashboard tips.

6. 6
Splunk Enterprise & Cloud 6.3
Breakthrough
Performance & Scale
Doubles performance
and lowers TCO
Advanced Analysis
& Visualization
High Volume Event
Collection
Enterprise-Scale
Platform
Supports DevOps and IoT
data analysis at scale
Simplifies analysis of
large datasets
Enterprise
management and
integration

7. 7
Breakthrough Performance and Scale
Vertical scaling maximizes use of CPU power through:
– Indexer Parallelization
– Search Parallelization
Improved Search Performance and System Capacity through:
– Intelligent Job Scheduling
7

8. 0
20
40
60
80
100
120
Splunk 6.2 Splunk 6.3
(2 pipelines)
Splunk 6.3
(4 pipelines)
Indexer Parallelization: Cisco UCS Benchmark Preview
MB/sec
4X More Data Indexing
(Pure indexing)

9. Search Parallelization: Cisco UCS Benchmark Preview
0
4
8
12
16
20
24
seconds seconds
0
10
20
30
40
50
60
Splunk 6.2 Splunk 6.3
(2 pipelines)
Splunk 6.3
(4 pipelines)
Splunk 6.2 Splunk 6.3
(2 pipelines)
Splunk 6.3
(4 pipelines)
3X Faster Search
2X More Data
(8 concurrent searches)
(70 MB/sec indexing)
6X Faster Search Speed
(8 concurrent searches)

10. 10
Summary of Parallelization Settings
Setting Description Setting name / location
Default
Value
Max
Recmd
Value
Impact
Batch mode search
parallelization
Allows a batch mode search to open
additional search pipelines on each
indexer
limits.conf
batch_search_max_pipeline
1 2
Multiples the number of
search pipelines per batch
mode search per indexer.
Parallel
summarization for
data models
Allows the scheduler to run
concurrent data model acceleration
searches on the indexers.
datamodels.conf
acceleration.max_concurrent
2 2
Multiples the number of
scheduled acceleration
searches per data model per
indexer.
Parallel
summarization for
report accelerations
Allows the scheduler to run
concurrent report acceleration
searches on the indexers.
savedsearches.conf
auto_summarize.max_concurrent
1 2
Multiples the number of
scheduled acceleration
searches per search per
indexer.
Index parallelization
Allows concurrent data processing
pipelines on indexers and
forwarders.
server.conf
parallelIngestionPipelines
1 2
Multiples the number of
pipelines per indexer.
http://docs.splunk.com/Documentation/Splunk/latest/Capacity/Parallelization

11. 11
Intelligent Job Scheduling
• Adds better priority scoring and
search windows for much improved
saved search scheduling
• Reduces # of skipped searches
• Re-run failed searches during
downtime
11
6.3
6.2

12. 12
Splunk Enterprise & Cloud 6.3
Breakthrough
Performance & Scale
Doubles performance
and lowers TCO
Advanced Analysis
& Visualization
High Volume Event
Collection
Enterprise-Scale
Platform
Supports DevOps and IoT
data analysis at scale
Simplifies analysis of
large datasets
Enterprise
management and
integration

13. 13
Single Value Display
At-a-glance, single-value indicators with useful context
No JS coding / CSS styling necessary!
Configurable sparkline
Value rangemap, custom thresholds
Trend up/down - reversible
Great for Operation Centers and
War Rooms
13

14. 14
Anomaly Detection
Incorporates Z-Score, IQR & histogram methodologies in a single command
Detect and summarize anomalies
Return anomalous values and
outliers
3 Commands in one
Easy-to-use
Configurable threshold
1

15. 15
Choropleth maps
Visualize how a metric varies across a (custom) geographic area
50 States and World Countries built-in
3 Different Color Modes
– Sequential
– Divergent
– Categorical
Custom Polygon Definitions
– Use KMZs and also make your own!
– Shapester App!
Point-in Polygon lookups
15

16. 16
Demo

17. 17
Splunk Enterprise & Cloud 6.3
Breakthrough
Performance & Scale
Doubles performance
and lowers TCO
Advanced Analysis
& Visualization
High Volume Event
Collection
Enterprise-Scale
Platform
Supports DevOps and IoT
data analysis at scale
Simplifies analysis of
large datasets
Enterprise
management and
integration

18. curl -k https://<host>:8088/services/collector -H 'Authorization: Splunk <token>' -d
'{"event":"Hello Event Collector"}'
Applications IoT Devices
Agentless, direct data onboarding via a standard developer API
HTTP Event Collector

19. 20
Splunk Enterprise & Cloud 6.3
Breakthrough
Performance & Scale
Doubles performance
and lowers TCO
Advanced Analysis
& Visualization
High Volume Event
Collection
Enterprise-Scale
Platform
Supports DevOps and IoT
data analysis at scale
Simplifies analysis of
large datasets
Enterprise
management and
integration

20. 21
Distributed Management Console - II
New topology views, status, and alerting for Splunk deployments
• Visualizes Search Head/Indexer matrix
with KPI and performance overlays
• Search Head clustering replication
and scheduler views
• Forwarder views with status and
performance data
• Index and metadata storage utilization
• System health alerting
21

21. 22
Custom Alert Actions
Use Splunk Alerts to trigger & automate workflows
• Allows packaged integration with
third-party applications
• Simple admin/user configuration
• Developers can build, package, and
publish alert actions within an app
• Growing list of integrations available
22

22. 23
Other Notable Additions
23

23. 24
Demo

24. 25
Download the Overview App (6.3) & 6.x Dashboard Examples

25. Harness the Power of
Search

26. 27
search and filter | munge | report | cleanup
Search Processing Language
sourcetype=access*
| eval KB=bytes/1024
| stats sum(KB) dc(clientip)
| rename sum(KB) AS "Total MB" dc(clientip) AS "Unique Customers"

27. 28
Five Commands that will Solve Most Data Questions
eval - Modify or Create New Fields and Values
stats - Calculate Statistics Based on Field Values
eventstats - Add Summary Statistics to Search Results
streamstats - Cumulative Statistics for Each Event
transaction - Group Related Events Spanning Time

28. 30
Examples
• Calculation:
sourcetype=access*
|eval KB=bytes/1024
• Evaluation:
sourcetype=access*
| eval http_response = if(status == 200,
"OK", "Error”)
• Concatenation:
sourcetype=access*
| eval connection =
clientip.":".status_description
eval - Modify or Create New Fields and Values

29. 31
eval - Modify or Create New Fields and Values
Examples
• Calculation:
sourcetype=access*
|eval KB=bytes/1024
• Evaluation:
sourcetype=access*
| eval http_response = if(status == 200,
"OK", "Error”)
• Concatenation:
sourcetype=access*
| eval connection = clientip.":".port

30. 32
eval - Modify or Create New Fields and Values
Examples
• Calculation:
sourcetype=access*
|eval KB=bytes/1024
• Evaluation:
sourcetype=access*
| eval http_response = if(status == 200,
"OK", "Error”)
• Concatenation:
sourcetype=access*
| eval connection = clientip.":".port

31. 34
stats – Calculate Statistics Based on Field Values
Examples
• Calculate stats and rename
sourcetype=access*
| eval KB=bytes/1024
| stats sum(KB) AS “Total KB”
• Multiple statistics
sourcetype=access*
| eval KB=bytes/1024
| stats sum(KB) avg(KB)
• By another field
sourcetype=access*
| eval KB=bytes/1024
| stats sum(KB) avg(KB) by clientip

32. 35
stats – Calculate Statistics Based on Field Values
Examples
• Calculate stats and rename
sourcetype=access*
| eval KB=bytes/1024
| stats sum(KB) as “Total KB”
• Multiple statistics
sourcetype=access*
| eval KB=bytes/1024
| stats sum(KB) avg(KB)
• By another field
sourcetype=access*
| eval KB=bytes/1024
| stats sum(KB) avg(KB) by clientip

33. 36
stats – Calculate Statistics Based on Field Values
Examples
• Calculate statistics
sourcetype=access*
| eval KB=bytes/1024
| stats sum(KB) AS "Total KB”
• Multiple statistics
sourcetype=access*
| eval KB=bytes/1024
| stats avg(KB) sum(KB)
• By another field
sourcetype=access*
| eval KB=bytes/1024
| stats sum(KB) avg(KB) by clientip

34. 38
eventstats – Add Summary Statistics to Search Results
Examples
• Overlay Average
sourcetype=access*
| eventstats avg(bytes) AS avg_bytes
| timechart latest(avg_bytes) avg(bytes)
• Moving Average
sourcetype=access*
| eventstats avg(bytes) AS avg_bytes by date_hour
| timechart latest(avg_bytes) avg(bytes)
• By created field
sourcetype=access*
| eval http_response = if(status == 200, "OK", "Error”)
| eventstats avg(bytes) AS avg_bytes by http_response
| timechart latest(avg_bytes) avg(bytes) by http_response

35. 39
Examples
• Overlay Average
sourcetype=access*
| eventstats avg(bytes) AS avg_bytes
| timechart latest(avg_bytes) avg(bytes)
• Moving Average
sourcetype=access*
| eventstats avg(bytes) AS avg_bytes by date_hour
| timechart latest(avg_bytes) avg(bytes)
• By created field
sourcetype=access*
| eval http_response = if(status == 200, "OK", "Error”)
| eventstats avg(bytes) AS avg_bytes by http_response
| timechart latest(avg_bytes) avg(bytes) by http_response
eventstats – Add Summary Statistics to Search Results

36. 40
eventstats – Add Summary Statistics to Search Results
Examples
• Overlay Average
sourcetype=access*
| eventstats avg(bytes) AS avg_bytes
| timechart latest(avg_bytes) avg(bytes)
• Moving Average
sourcetype=access*
| eventstats avg(bytes) AS avg_bytes by date_hour
| timechart latest(avg_bytes) avg(bytes)
• By created field
sourcetype=access*
| eval http_response = if(status == 200, "OK", "Error”)
| eventstats avg(bytes) AS avg_bytes by http_response
| timechart latest(avg_bytes) avg(bytes) by http_response

37. 42
streamstats – Cumulative Statistics for Each Event
Examples
• Cumulative Sum
sourcetype=access*
| reverse
| streamstats sum(bytes) as bytes_total
| timechart max(bytes_total)
• Cumulative Sum by Field
sourcetype=access*
| reverse
| streamstats sum(bytes) as bytes_total by status
| timechart max(bytes_total) by status
• Moving Average
sourcetype=access*
| timechart avg(bytes) as avg_bytes
| streamstats avg(avg_bytes) AS moving_avg_bytes window=10
| timechart latest(moving_avg_bytes) latest(avg_bytes)

38. 43
streamstats – Cumulative Statistics for Each Event
Examples
• Cumulative Sum
sourcetype=access*
| timechart sum(bytes) as bytes
| streamstats sum(bytes) as cumulative_bytes
| timechart max(cumulative_bytes)
• Cumulative Sum by Field
sourcetype=access*
| reverse
| streamstats sum(bytes) as bytes_total by status
| timechart max(bytes_total) by status
• Moving Average
sourcetype=access*
| timechart avg(bytes) as avg_bytes
| streamstats avg(avg_bytes) AS moving_avg_bytes window=10
| timechart latest(moving_avg_bytes) latest(avg_bytes)

39. 44
streamstats – Cumulative Statistics for Each Event
Examples
• Cumulative Sum
sourcetype=access*
| timechart sum(bytes) as bytes
| streamstats sum(bytes) as cumulative_bytes
| timechart max(cumulative_bytes)
• Cumulative Sum by Field
sourcetype=access*
| reverse
| streamstats sum(bytes) as bytes_total by status
| timechart max(bytes_total) by status
• Moving Average
sourcetype=access*
| timechart avg(bytes) as avg_bytes
| streamstats avg(avg_bytes) AS moving_avg_bytes
window=10
| timechart latest(moving_avg_bytes) latest(avg_bytes)

40. 46
transaction – Group Related Events Spanning Time
Examples
• Group by Session ID
sourcetype=access*
| transaction JSESSIONID
• Calculate Session Durations
sourcetype=access*
| transaction JSESSIONID
| stats min(duration) max(duration) avg(duration)
• Stats is Better
sourcetype=access*
| stats min(_time) AS earliest max(_time) AS latest by JSESSIONID
| eval duration=latest-earliest
| stats min(duration) max(duration) avg(duration)

41. 47
transaction – Group Related Events Spanning Time
Examples
• Group by Session ID
sourcetype=access*
| transaction JSESSIONID
• Calculate Session Durations
sourcetype=access*
| transaction JSESSIONID
| stats min(duration) max(duration) avg(duration)
• Stats is Better
sourcetype=access*
| stats min(_time) AS earliest max(_time) AS latest by JSESSIONID
| eval duration=latest-earliest
| stats min(duration) max(duration) avg(duration)

42. 48
transaction – Group Related Events Spanning Time
Examples
• Group by Session ID
sourcetype=access*
| transaction JSESSIONID
• Calculate Session Durations
sourcetype=access*
| transaction JSESSIONID
| stats min(duration) max(duration) avg(duration)
• Stats is Better
sourcetype=access*
| stats min(_time) AS earliest max(_time) AS latest by JSESSIONID
| eval duration=latest-earliest
| stats min(duration) max(duration) avg(duration)

43. 49
Learn Them Well and Become a Ninja
eval - Modify or Create New Fields and Values
stats - Calculate Statistics Based on Field Values
eventstats - Add Summary Statistics to Search Results
streamstats - Cumulative Statistics for Each Event
transaction - Group Related Events Spanning Time
See many more examples and neat tricks at docs.splunk.com and answers.splunk.com

44. 50
Bonus DashboardSee Notes for xml

45. Questions?

46. Bonus Command

47. 53
cluster – Find Common and/or Rare Events
Examples
• Find the most common events
*
| cluster showcount=t t=0.1
| table cluster_count, _raw
| sort - cluster_count
• Select a field to cluster on
sourcetype=access*
| cluster field=bc_uri showcount=t
| table cluster_count bc_uri _raw
| sort -cluster_count
• Most or least common errors
index=_internal source=*splunkd.log* log_level!=info
| cluster showcount=t
| table cluster_count _raw
| sort -cluster_count

48. 54
cluster – Find Common and/or Rare Events
Examples
• Find the most common events
*
| cluster showcount=t t=0.1
| table cluster_count, _raw
| sort - cluster_count
• Select a field to cluster on
sourcetype=access*
| cluster field=bc_uri showcount=t
| table cluster_count bc_uri _raw
| sort -cluster_count
• Most or least common errors
index=_internal source=*splunkd.log* log_level!=info
| cluster showcount=t
| table cluster_count _raw
| sort -cluster_count

49. 55
cluster – Find Common and/or Rare Events
Examples
• Find the most common events
*
| cluster showcount=t t=0.1
| table cluster_count, _raw
| sort - cluster_count
• Select a field to cluster on
sourcetype=access*
| cluster field=bc_uri showcount=t
| table cluster_count bc_uri _raw
| sort -cluster_count
• Most or least common errors
index=_internal source=*splunkd.log* log_level!=info
| cluster showcount=t
| table cluster_count _raw
| sort -cluster_count

50. 56
Splunk Mobile App
EMBEDDING
OPERATIONAL
INTELLIGENCE
• Access dashboards and
reports
• Annotate dashboards and
share with others
• Receive push notifications
Native Mobile Experience

51. Thank You