During Daniels presentation he showed you not only how to create meaningful metrics, but also how to develop a maturity matrix that makes sense to guide your journey.

1. DevOps
Metrics that matter

2. BritAn
Too many years in IT
Principal Consultant for Virtual
Clarity
ITSM-DevOps-Lean-AgileITSM
DevOps Institute trainer
Gamingworks GromIT associate
Active BCS & itSMFUK
Conference speaker
Blogs & webcasts
Daniel Breston
@DanielBreston

3. Just curious
?
Is ‘Gaming metrics’ something that happens where
you work?
No, our reports accurately reflect facts and actions
Yes we “game the numbers since we manage from
a distance
Not sure!
?
Does your business lifecycle match your IT lifecycle
from their point of view?
Yes, most of the time
Yes, some of the time
No not really

4. Business & Technology
Don’t measure me
on cost or
traditional IT
metrics, but on the
metrics of the
business.
Jim Fowler, CIO, GE Capital

5. Employee view of a KPI
“Tell me how you
measure me,
and I will tell you how I
will behave”
Eliyahu M. Goldratt

6. Measure, Metric, KPI
• Measure: a standard unit used to express the
size, amount, or degree of something
• Metric: A system or standard of measurement

7. KPI: Indicator of Performance that is KEY
An agreed measure that allows us to:
• Assess our performance against a baseline or expectation
• Immediately spot issues, problems or waste
• Enables a behaviour
• Let’s us continuously improve
• Is a Metric that Matters
• Links to a Value statement
Steve Bell” of LeanIT Strategies: “A useful metric that makes the invisible visible”

8. Measurement systems fail because
REASONS FOR FAILURE
Measure what is easy, not what is
important
Not created with people involved
Information is wrong or misleading
Reported too late to be actionable
Baseball bat
GOOD MEASUREMENTS
Minimize quantity
Maximize accuracy
Minimize effort
Minimize ambiguity
Maximise behaviours
Maximise being SMART

9.  9 
KPI: Does this make
sense?
∏=
=
n
i
i
E
EMSizeAPM
1
xx
∑=
+=
5
1
x01.0
j
jSFBE
where
Effort
kSLOC
kiloSourceLinesOfCo
de
Effort Multiplier: linear effect on effort
Scale Factor:
exponential effect on effort
Constants that can be
used to calibrate the formula
KPI: so what do you think?

10. Better, faster, safer value as often & early as possible
ITSM
AgileLean
DevOps
DevOps

11. DevOps: values & principles
• Focus on people
• Embrace change and experimentation
• Focus on people
• Embrace change and experimentation
CultureCulture
• “Continuous Delivery”
• “Infrastructure as Code”
• “Continuous Delivery”
• “Infrastructure as Code”
• Focus on producing value for the end user
• Small batch sizes
• Focus on producing value for the end user
• Small batch sizes
• Measure everything
• Show the improvement
• Measure everything
• Show the improvement
• Open information sharing
• Collaboration and communication
• Open information sharing
• Collaboration and communication
AutomationAutomation
LeanLean
MetricsMetrics
SharingSharing

12. DevOps Toll Gates
@VirtualClarity

13. KPI Design
An owner
Definition
Value Goal
Equation
Unit of measure
Frequency
Data location
Dependency
Most importantly – the required response when checked

14. ABCs of KPIs
Aligned
Provides clear line of sight between strategic goals and individual
targets and activities (cascaded targets)
Actionable
The person using the indicator should be capable of influencing and
improving it (accountability aligned with autonomy)
Balanced
Evaluates performance from multiple perspectives (leading and lagging;
customer and business; efficiency, effectiveness, quality, time, cost,
etc.)
Controllable Small number and owned
Consistent
Definitions, targets, data collection and analysis methods remain
constant from one period to the next (KPIs matrix and definitions)
Comparable
Measured against a historical baseline, a current plan, and a future
target (base, plan, and target)
KPI ABCs

15. 1. What is the
vision?
3. Where do
we want to
be?
2. Where are
we now?
4. How do we
get there?
5. Did we get
there?
6. How do we
keep the
momentum
going?
Business Vision,
mission, goals
and objectives
Baseline
Assessments
Measurable
targets
Service & process
improvement
Measurements
and metrics
Audits and
reviews
Image : © Crown copyright 2007. Reproduced with permission of OGC.
Agile ITSM

16. Sailing to an island
The journey normally takes 4 days
Day-1
Day-2
Day-3
Day-4
Analogy (or the way we really work??)

17. The captain checks his position after 4 days
Necessary and time consuming course correction is needed
Day-1
Day-2
Day-3
Day-4Day-5&6
(lost)
Oops to late!

18. The captain checks his position on a daily basis
Daily correction of the position
Partly
day-5
(lost)
Day-1
Day-2
Day-3
Day-4
DevOps: Frequency is KEY

19. The captain checks his position every half hour
Continuous and immediate correction with minimal time loss
On
time
Day-1
Day-2
Day-3
Day-4
An important difference between Lagging and Leading KPIs!
DevOps: better, faster, safer

20. KPI Tree
Quality
People
Productivity
Cost Safety
Strategy

21. DevOps metrics that matter
Customer Cost of delay Get it back

22. KPI tree example
Operations
Outage vital businessfunctions
- % & € (cost impact) Lost opportunities
-% & € (cost impact)
Release
Traced - % Categorized - % (risk based)
Decided - % Executed - % Stabilized– T
Process, procedures up to date & agreed %
Run
Availability- % Reliability– T
Maintainability– T Resilience– T
Process, procedures up to date & agreed %
Asset
Process, procedures up to date & agreed %
Major & Minor
Traceable % Planned % Documented %
Tested % Profiled % Approved %
Emergency
Documented % Profiled %
Tested % Approved %
Infrastructure
Traceable % Planned % Documented %
Tested % Profiled % Approved %
IMAC
In time % 1-st time right %
Online
Vitalbusiness Applicationsmonitored %
Failures in execution/performance%
Workorder
In time % 1-st time right %
Batch
Jobs under control %
Failures in execution %
Exceed time window %
Wintel
Fail over - % & T
DRP - % & T
Scale / Patch / upgrade compliance - % &T
SAP
Fail over - % & T
DRP - % & T
Scale / Patch / upgrade compliance - % & T
Support
1-st line resolution - %
Process, procedures up to date & agreed %
Applications
Defined - % Compliant - %
Scaled - # & % Monitored - %
Wintel
Defined - % Compliant - %
Scaled - # & % Monitored - %
SAP
Defined - % Compliant - %
Scaled - # & % Monitored - %
Infrastructure
Defined - % Compliant - %
Scaled - # & % Monitored - %
Infrastructure
Fail over - % & T
DRP - % & T
Scale / Patch / upgrade compliance - % & T
Call take
Duration call take %
Abandoned call %
Overdue
Type/Sev %
Policies
Defined %
Agreed %
Up to date %
Follow up
Type/Sev %
Back log
To assign #
Type / sev %
Per status in life cycle #

23. MONEY????

24. From Lean IT
with permission from Steve Bell
From Lean Enterprise
with permission from Jez Humble
Catch-ball

25. Charter of Intent Service Management Charter
Event Scope Leadership / Coordination (Normal) Schedule
Product or Service Compute, Storage, Service Desk, etc.
Executive
Sponsor
VP or C-level
Days of
Product/Service
Specific Conditions
What circumstances you're including and
excluding? (type of customer, time of year, etc.)
I&O Manager
If needed - often Director or
Manager level
Start/End
Times
Customer Demand
How many times is this done per wk, qtr, mo, or
yr.? (if helpful to know)
Team Leader
Skilled person leading the
activity
Location(s) of
service
Location of documentation SharePoint location
First Step Task on first process block
OBP Manager Not always needed
Product -
Service Date
exceptions
Always a nice touch; keeps
the team from wandering.Last Step Task on last process block
Boundaries & Limitations
What is the team NOT authorized to change, do,
etc.?
OP Manager
The person arranging logistics
(reserving the room, ordering
food, sending meeting notices,
etc.)
Interim
Briefing(s)
Aid in consensus building
and organizational learning.
Reporting cycle
Daily, weekly, quarterly
(add in roles if helpful)
Event Drivers Significant Contacts
1 What dependencies in services Providers Name Contact Name Contact Information
2 What dependencies on other partners (significant 1
3 Assumptions 2
4 3
5 4
Goals & Measureable Objectives 5
1 Aim for objectives w/ measurable targets (from X to X). 6
2 7
3 8
4 9
5 10
Planned Deliverables by Output-based Partners On-Call Support
1 Application mgt Function Name Contact Information
2 Compute storage 1
SMEs that may not be
needed full time
3 Service Desk 2
4 3
5 4
Planned Deliverables by Operating Partners On-Call Support
1 Application mgt Function Name Contact Information
2 compute storage 1
SMEs that may not be
needed full time
3 Service Desk 2
4 3
5 4
Potential Obstacles Approvals
1 Application mgt Executive Sponsor Output-based Partner Operating Partner
2 Compute storage
3 Service Desk
4 Signature: Signature: Signature:
5 Date: Date: Date:
Version Control
1 Version no. Reason for change
2 Creation Date
3 Revised Date
4 Author

26. MTBAK
My first KPI

27. DO
Review KPIs regularly against a target to
identify trends and define corrective
actions
Measure leading KPIs on the shortest
interval they an be controlled
Cascade targets and align KPIs between
all levels of the organization
Bring information to life by visualizing KPIs
using graphs, charts, and white-boards
DON’T
Measure too many things or measure just
because you can
Use complex calculations, percentages, or
ratios when a simple number is available
Try to roll-up or aggregate every KPI
between each level of the organization
Use KPIs to punish people – instead, use
them as a tool to improve performance
Tips

28. ITSMF UK
Premier Gate, Easthampstead Road, Bracknell,
RG12 1JS, United Kingdom
Tel: +44 (0) 118 918 6500 | Web: www.itsmf.co.uk
Conference KPI: fill in your evaluation please!!