Splunk's Andi Mann addresses what he refers to as the real core of DevOps: increasing collaboration, communication, integration and delivery of better, faster software; the human side of DevOps, combined with the business impacts.

1. Andi Mann @AndiMann @Splunk
Copyright © 2016 Splunk Inc.
Data-Driven DevOps
Mining Machine Data for
‘Metrics that Matter’
Andi Mann @AndiMann @Splunk

2. Andi Mann @AndiMann @Splunk
CAMS – as close to prescriptive as DevOps gets

3. Andi Mann @AndiMann @Splunk3
BUT WHAT
SHOULD
YOU
MEASURE?

4. Andi Mann @AndiMann @Splunk
I’m working
super hard!!
That’s my
stapler.

5. Andi Mann @AndiMann @Splunk5
Yeah, but …
… what are you
achieving?
I’m gonna need you
to come in Sunday.

6. Andi Mann @AndiMann @Splunk
Gartner’s DevOps ‘Metrics that Matter’
Gartner Inc., Data-Driven DevOps: Use Metrics to Help Guide Your Journey, 29 May 2014 G00264319, Analyst(s): Cameron Haight | Tapati Bandopadhyay

7. Andi Mann @AndiMann @Splunk
IDC’s DevOps ‘Metrics that Matter’

8. Andi Mann @AndiMann @Splunk
Some DevOps Metrics that Might Matter
Culture
e.g.
• Retention
• Satisfaction
• Callouts
Process
e.g.
• Idea-to-cash
• MTTR
• Deliver time
Quality
e.g.
• Tests passed
• Tests failed
• Best/worst
Systems
e.g.
• Throughput
• Uptime
• Build times
Activity
e.g.
• Commits
• Tests run
• Releases
Impact
e.g.
• Signups
• Checkouts
• Revenue

9. Andi Mann @AndiMann @Splunk
But DevOps Is Always a Unique Journey
What Are Your
‘Metrics That
Matter’?

10. Andi Mann @AndiMann @Splunk
Every tool, every process, every
component, on-prem or off
One Constant:
Machine Data

11. Andi Mann @AndiMann @Splunk
Common Data Fabric
11
API
SDKs UI
Other Tools
Escalation/
Collaboration
Visibility Across the Whole Dev Lifecycle
Plan Code Build Test/QA Stage Release Config Monitor

12. Andi Mann @AndiMann @Splunk
Common Data Fabric
12
API
SDKs UI
Server, Storage.
N/W
Server
Virtualisation
Operating
Systems
Mobile
Applications
Cloud Services
Other Tools
Ticketing/Help
Desk
Custom
Applications
Visibility Across the Whole Ops Environment
API Services
Infrastructure
Applications

13. Andi Mann @AndiMann @Splunk
Use Machine Data To Identify ‘Waste’
Plan
Develop
(UI)
Develop
(Db)
Develop
(M’ware)
Develop
(Backend)
Security
Test
Monitor
Build
(Prod)
Architect
Secure/
Comply
DeployAccept
Unit
Test
Document
Cap Plan
Train
Feedback
Integration
Test
Configure
System
Test
Launch
CAB
Develop
(APIs)
Budget
Build
(Dev)
Mgmt/
Tooling
W
W
W
W
W
W
W
W
W
16 40 52 35 96 40 48 24 --8 2 5 6 8 2 12

14. Andi Mann @AndiMann @Splunk
Use Machine Data To Manage Testing and QA
• Release when
ready, not a date!
• Best / worst
developers
• Best / worst
providers
• Impact of new
code on ops
• Impact of new
code on biz

15. Andi Mann @AndiMann @Splunk
Use Machine Data To Enable Continuous
Improvement
Defect
Information
Capacity
Planning
Quality
Standards
Enhancement
Requests
Integration
Requirements
Acceptance
Metrics
Service Levels
and KPIs
Application Development Test and Acceptance Production
BuildCodePlan Test/QA Stage Release Config Monitor
Infrastructure
Dependencies

16. Andi Mann @AndiMann @Splunk
Use Machine Data To Accelerate Velocity
Pivot & improve with
Continuous Insights
Product Managers
identify new
opportunities
Continuously delivered to market
… and Auditors are “happy”

17. Andi Mann @AndiMann @Splunk
Use Machine Data To Improve Quality
Code quality scans Static security scans
White BoxDeveloper
checks in code
Automated
Acceptance Tests
Dynamic Security
Scans
Black Box
“Chaos Monkey”
tests
Test Fail:
Return
Test Fail:
Return
X
X
Production
QA Prod Pattern
QA Pattern Library
Test Pass:
Promote
Test Pass:
Promote to
Production
Pattern
library used
for test and
QA

18. Andi Mann @AndiMann @Splunk
Use Machine Data To Align Business Impact

19. Andi Mann @AndiMann @Splunk
Metrics that Matter Drive Better Feedback Loops
Improve
Application
Velocity
Improve
Application
Quality
Improve
Application
Impact

20. Andi Mann @AndiMann @Splunk
Copyright © 2016 Splunk Inc.
Andi Mann
@AndiMann
We make machine data accessible,
usable, and valuable to everyone