Snehal Antani

1. Copyright © 2015 Splunk Inc.
Snehal Antani
CTO, Splunk

2. Commercial Intensity Market Speed
Earning a seat at the table

3. We should be
running the
business in real >me!
Why am I making
decisions on
last month’s data?

4. Data from the past RealAme Data StaAsAcal Forecast
T – a few days T + a few days
Security OperaAons Center
IT OperaAons Center
Business OperaAons Center
DescripAve
(BI Tools, Data Lakes) Grey space
PredicAve
(Models)
Insights from data-in-moAon

5. What Does Machine Data Look Like?
Sources
Order Processing
TwiGer
Care IVR
Middleware
Error
5

6. Machine Data Contains Cri>cal Insights
Customer ID Order ID
Customer’s Tweet
Time WaiAng On Hold
TwiSer ID
Product ID
Company’s TwiSer ID
Customer ID Order ID
Customer ID
Sources
Order Processing
TwiGer
Care IVR
Middleware
Error
6

7. Machine Data Contains Cri>cal Insights
Order ID
Customer’s Tweet
Time WaiAng On Hold
Product ID
Company’s TwiSer ID
Order ID
Customer ID
TwiSer ID
Customer ID
Customer ID
Sources
Order Processing
TwiGer
Care IVR
Middleware
Error
7

8. Security
Opera>ons
IT
Opera>ons
Business
Opera>ons
SAME DATA Of the
Asking different
Different PEOPLE
QUESTIONS

9. Blurring the Lines Between Digital & Physical
IoT WORKLOADS ENTERPRISE IT
WORKLOADS
Security
Ops
Center
IT Ops
Center
Business
Ops
Center
CLOUD WORKLOADS
ADVANCED CORRELATION
HYBRID DEPLOYMENT
SINGLE PANE OF GLASS
New points of failure
Valuable business data
New points of aSack
9

10. PLATFORM FOR MACHINE DATA
10
ANY MACHINE DATA
Online
Services
Web
Services
Servers
Security GPS
LocaAon
Storage
Desktops
Networks
Packaged
ApplicaAons
Custom
ApplicaAons Messaging
Telecoms
Online
Shopping
Cart
Web
Clickstreams
Databases
Energy
Meters
Call Detail
Records
Smartphones
and Devices
RFID
OPERATIONAL INTELLIGENCE
HA Indexes
and Storage
Search and
InvesAgaAon
ProacAve
Monitoring
OperaAonal
Visibility
Real-Ame
Business
Insights
Commodity
Servers

11. Detect. Decide. Act! Learn.
Web Logs
App Logs
Network Logs
Endpoint Logs
IdenAty Logs
Wearables
Vehicles
…
Industrial Control
SCADA
…
IT DATA
IOT DATA
OT DATA
Time
HOT WARM
Data Aging Policy
COOL COLD
Historical Near Real-Ame PredicAve
Data In-MoAon
T0
PREDICT ALERT SEARCH VISUALIZE ANALYZE DEVELOP Enrich ACT

12. Data Science Development Life Cycle
IT DATA
IOT DATA
OT DATA
Time T0
Analyze & Develop models
Develop
Back test
Deploy models at the edge
HOT WARM COOL COLD
Historical Near Real-Ame PredicAve
Data Lake
• Age Splunk data into HDFS
• Access Splunk data with your Hadoop tools (no data lock-in!)
• Integrate with your data science development lifecycle

13. Fast-feedback loop for acAonable commercial insights
Responsibly Move at Market Speed
Security, Compliance, Availability by design
BUSINESS LEADER DEVELOPERS
IT BUSINESS
APPLICATION
PLATFORM
HOW IS OUR:
• Security?
• Quality?
• Stability?
• Performance?
• Compliance?
HOW IS OUR:
• Market Launch?
• Feature Usage?
• MarkeAng Changes?
• PrioriAzaAon?
• Customer SaAsfacAon?
Failed Customer InteracAons
Compliance Response Time
Development Velocity

14. 2. Focus on the foundaAon
3. Ignore the “Eeyores”
1. Define your outcomes
• Development Velocity
• Failed Customer Interactions
• Compliance Response Time
App Teams
Programmable Enterprise
Hybrid
Cloud
Con>nuous
Delivery
Con>nuous
Insights
But how do you get there?
4. Religion vs. Candles
5. BeSer to be infamous than unknown
6. Free trapped capacity

15. Install App Servers
Install OS’
Install Physical Servers
Configure network
Configure security
Debug! Servers
Storage
Networking
Virtualization
Management
Security
Middleware
Patterns
Automated
Deployment
Applications
Principles
Fewer “moving parts” for speed & stability
Get out of the “IT integraAon business”
Challenge: People & processes, not tech

16. Dev/Test Cloud
Dev PaGern QA Prod PaGern
Production Cloud
QA Prod PaGern
Enterprise
Architecture
PaGern Library Developer Dev, QA, & Prod
provisioned in minutes
Speed, Consistency, & Repeatability with PaSerns

17. 2. ConAnuous Delivery
Sonar code quality Sta>c Security Scans
Build cloud
Developer
checks in code
Automated
Acceptance Tests
Dynamic Security
Scans
Test Cloud
“Chaos Monkey” tests
Automated
Deploy
=
=
X
X
Production Cloud
QA Prod PaGern
Automated
Deploy
PaGern Library

18. Real-Ame Dashboards Ensure Transparency
• Release when ready, not to a date!
• Best / worst developers
• Best / worst outsourced providers

19. • Visualize
• Analyze
• Take Ac>on
… quickly
… automaAcally
3. ConAnuous Insights: ProacAve, not reacAve
QA Prod PaGern

20. Building an Ops Center – lessons learned
Web CoE Middleware CoE DB CoE
CoE Specific KPI’s & Dashboards
Business TransacAons
KPI’s & Dashboards
IT Opera>ons Center
ü Web-MW-DB Logs
ü App Logs
ü Network logs
ü HW Infrastructure Logs
ü Endpoint Logs
ü IdenAty Logs
“Canary”
Thumbs up / down
“War Room”
CollaboraAve TroubleshooAng,
RCA, & Post-mortem
“Band-aids”
Document – Script –
Trust – Auto-execute
How we operated
“Con;nuous Improvement”
Develop more “band-aids”,
further instrument app code

21. Canary: Detect PotenAal Failed Customer InteracIons…

22. … and do something about it!

23. CISO
Enterprise-wide Data Fabric
IT DATA
IOT DATA
OT DATA
Mask fields
on ingest LoB 1 LoB 2
• Single data planorm
• Users only see what they should
• SensiAve data is masked on ingest and/or on search

24. InvesIgate the data in real-Ame
• Define StaAc schema
• ETL into Schema
• Enrich at write
• New data = new columns
• New quesAons = new columns
Ideal for asking the same
quesIon repeatedly
• Schema-on-the-fly
• Data in naAve format
• Enrich on read
• New data = no changes needed
• New quesAons = no changes needed
Ideal for invesIgaIon & exploraIon
150.128.102.148 - - [07/Aug/2014:00:59:52 +0000] "GET /images/
web/2009/banner.png HTTP/1.1" 200 52315 "http://
www.semicomplete.com/blog/articles/week-of-unix-tools/day-1-
sed.html" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/
537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36 vs.
Schema-on-write Schema-on-read

25. Establish a culture of ConInuous Improvement

26. From 20 Million Events to the 20 that Really MaSer
Notable Events
ConInuous
Queries
Opera;onal Analysts
InvesIgate & Remediate
Data Scien;sts analyze remediaIon's
and develop & refine models
2
3
5
Real->me Models
Batch Models
Model
Execu>on
Engine
(Spark/HDFS)
1
4
IT DATA
IOT DATA
OT DATA
Data In-
MoAon

27. Pivot & improve with
Con>nuous Insights
Business Leaders
idenAfy new
opportuniAes
Con>nuously delivered to market
… and Auditors are “happy”
Fast-feedback enables Market Speed

28. Listen to your data!

29. Synthesis: The Business Transforma>on Journey
Ability to Execute…”by design”
Con>nuously Improve
Ability to Commercialize
Disrupter,
not disrupted
Hybrid Cloud
ConAnuous Delivery
ConAnuous Insights
Programmable Enterprise
Transparency with data
Fast feedback loop
Protect the premium
Capture new revenue
Move at Market Speed
Operate with Commercial Intensity
Earn a seat at the table
Higher-value customer outcomes
Business model innovaAon
2
9
29

30. With a Strong IT foundaAon,
and the right culture…
Commercial Intensity
+ Market Speed =
Resources of an Enterprise,
Agility of a startup
Snehal Antani, CTO @ Splunk www.linkedin.com/in/snehalantani
@snehalantani