Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Daniel Breston - DevOps metrics that matter

298 views

Published on

During Daniels presentation he showed you not only how to create meaningful metrics, but also how to develop a maturity matrix that makes sense to guide your journey.

Published in: Government & Nonprofit
  • Be the first to comment

  • Be the first to like this

Daniel Breston - DevOps metrics that matter

  1. 1. DevOps Metrics that matter
  2. 2. BritAn Too many years in IT Principal Consultant for Virtual Clarity ITSM-DevOps-Lean-AgileITSM DevOps Institute trainer Gamingworks GromIT associate Active BCS & itSMFUK Conference speaker Blogs & webcasts Daniel Breston @DanielBreston
  3. 3. Just curious ? Is ‘Gaming metrics’ something that happens where you work? No, our reports accurately reflect facts and actions Yes we “game the numbers since we manage from a distance Not sure! ? Does your business lifecycle match your IT lifecycle from their point of view? Yes, most of the time Yes, some of the time No not really
  4. 4. Business & Technology Don’t measure me on cost or traditional IT metrics, but on the metrics of the business. Jim Fowler, CIO, GE Capital
  5. 5. Employee view of a KPI “Tell me how you measure me, and I will tell you how I will behave” Eliyahu M. Goldratt
  6. 6. Measure, Metric, KPI • Measure: a standard unit used to express the size, amount, or degree of something • Metric: A system or standard of measurement
  7. 7. KPI: Indicator of Performance that is KEY An agreed measure that allows us to: • Assess our performance against a baseline or expectation • Immediately spot issues, problems or waste • Enables a behaviour • Let’s us continuously improve • Is a Metric that Matters • Links to a Value statement Steve Bell” of LeanIT Strategies: “A useful metric that makes the invisible visible”
  8. 8. Measurement systems fail because REASONS FOR FAILURE Measure what is easy, not what is important Not created with people involved Information is wrong or misleading Reported too late to be actionable Baseball bat GOOD MEASUREMENTS Minimize quantity Maximize accuracy Minimize effort Minimize ambiguity Maximise behaviours Maximise being SMART
  9. 9.  9  KPI: Does this make sense? ∏= = n i i E EMSizeAPM 1 xx ∑= += 5 1 x01.0 j jSFBE where Effort kSLOC kiloSourceLinesOfCo de Effort Multiplier: linear effect on effort Scale Factor: exponential effect on effort Constants that can be used to calibrate the formula KPI: so what do you think?
  10. 10. Better, faster, safer value as often & early as possible ITSM AgileLean DevOps DevOps
  11. 11. DevOps: values & principles • Focus on people • Embrace change and experimentation • Focus on people • Embrace change and experimentation CultureCulture • “Continuous Delivery” • “Infrastructure as Code” • “Continuous Delivery” • “Infrastructure as Code” • Focus on producing value for the end user • Small batch sizes • Focus on producing value for the end user • Small batch sizes • Measure everything • Show the improvement • Measure everything • Show the improvement • Open information sharing • Collaboration and communication • Open information sharing • Collaboration and communication AutomationAutomation LeanLean MetricsMetrics SharingSharing
  12. 12. DevOps Toll Gates @VirtualClarity
  13. 13. KPI Design An owner Definition Value Goal Equation Unit of measure Frequency Data location Dependency Most importantly – the required response when checked
  14. 14. ABCs of KPIs Aligned Provides clear line of sight between strategic goals and individual targets and activities (cascaded targets) Actionable The person using the indicator should be capable of influencing and improving it (accountability aligned with autonomy) Balanced Evaluates performance from multiple perspectives (leading and lagging; customer and business; efficiency, effectiveness, quality, time, cost, etc.) Controllable Small number and owned Consistent Definitions, targets, data collection and analysis methods remain constant from one period to the next (KPIs matrix and definitions) Comparable Measured against a historical baseline, a current plan, and a future target (base, plan, and target) KPI ABCs
  15. 15. 1. What is the vision? 3. Where do we want to be? 2. Where are we now? 4. How do we get there? 5. Did we get there? 6. How do we keep the momentum going? Business Vision, mission, goals and objectives Baseline Assessments Measurable targets Service & process improvement Measurements and metrics Audits and reviews Image : © Crown copyright 2007. Reproduced with permission of OGC. Agile ITSM
  16. 16. Sailing to an island The journey normally takes 4 days Day-1 Day-2 Day-3 Day-4 Analogy (or the way we really work??)
  17. 17. The captain checks his position after 4 days Necessary and time consuming course correction is needed Day-1 Day-2 Day-3 Day-4Day-5&6 (lost) Oops to late!
  18. 18. The captain checks his position on a daily basis Daily correction of the position Partly day-5 (lost) Day-1 Day-2 Day-3 Day-4 DevOps: Frequency is KEY
  19. 19. The captain checks his position every half hour Continuous and immediate correction with minimal time loss On time Day-1 Day-2 Day-3 Day-4 An important difference between Lagging and Leading KPIs! DevOps: better, faster, safer
  20. 20. KPI Tree Quality People Productivity Cost Safety Strategy
  21. 21. DevOps metrics that matter Customer Cost of delay Get it back
  22. 22. KPI tree example Operations Outage vital businessfunctions - % & € (cost impact) Lost opportunities -% & € (cost impact) Release Traced - % Categorized - % (risk based) Decided - % Executed - % Stabilized– T Process, procedures up to date & agreed % Run Availability- % Reliability– T Maintainability– T Resilience– T Process, procedures up to date & agreed % Asset Process, procedures up to date & agreed % Major & Minor Traceable % Planned % Documented % Tested % Profiled % Approved % Emergency Documented % Profiled % Tested % Approved % Infrastructure Traceable % Planned % Documented % Tested % Profiled % Approved % IMAC In time % 1-st time right % Online Vitalbusiness Applicationsmonitored % Failures in execution/performance% Workorder In time % 1-st time right % Batch Jobs under control % Failures in execution % Exceed time window % Wintel Fail over - % & T DRP - % & T Scale / Patch / upgrade compliance - % &T SAP Fail over - % & T DRP - % & T Scale / Patch / upgrade compliance - % & T Support 1-st line resolution - % Process, procedures up to date & agreed % Applications Defined - % Compliant - % Scaled - # & % Monitored - % Wintel Defined - % Compliant - % Scaled - # & % Monitored - % SAP Defined - % Compliant - % Scaled - # & % Monitored - % Infrastructure Defined - % Compliant - % Scaled - # & % Monitored - % Infrastructure Fail over - % & T DRP - % & T Scale / Patch / upgrade compliance - % & T Call take Duration call take % Abandoned call % Overdue Type/Sev % Policies Defined % Agreed % Up to date % Follow up Type/Sev % Back log To assign # Type / sev % Per status in life cycle #
  23. 23. MONEY????
  24. 24. From Lean IT with permission from Steve Bell From Lean Enterprise with permission from Jez Humble Catch-ball
  25. 25. Charter of Intent Service Management Charter Event Scope Leadership / Coordination (Normal) Schedule Product or Service Compute, Storage, Service Desk, etc. Executive Sponsor VP or C-level Days of Product/Service Specific Conditions What circumstances you're including and excluding? (type of customer, time of year, etc.) I&O Manager If needed - often Director or Manager level Start/End Times Customer Demand How many times is this done per wk, qtr, mo, or yr.? (if helpful to know) Team Leader Skilled person leading the activity Location(s) of service Location of documentation SharePoint location First Step Task on first process block OBP Manager Not always needed Product - Service Date exceptions Always a nice touch; keeps the team from wandering.Last Step Task on last process block Boundaries & Limitations What is the team NOT authorized to change, do, etc.? OP Manager The person arranging logistics (reserving the room, ordering food, sending meeting notices, etc.) Interim Briefing(s) Aid in consensus building and organizational learning. Reporting cycle Daily, weekly, quarterly (add in roles if helpful) Event Drivers Significant Contacts 1 What dependencies in services Providers Name Contact Name Contact Information 2 What dependencies on other partners (significant 1 3 Assumptions 2 4 3 5 4 Goals & Measureable Objectives 5 1 Aim for objectives w/ measurable targets (from X to X). 6 2 7 3 8 4 9 5 10 Planned Deliverables by Output-based Partners On-Call Support 1 Application mgt Function Name Contact Information 2 Compute storage 1 SMEs that may not be needed full time 3 Service Desk 2 4 3 5 4 Planned Deliverables by Operating Partners On-Call Support 1 Application mgt Function Name Contact Information 2 compute storage 1 SMEs that may not be needed full time 3 Service Desk 2 4 3 5 4 Potential Obstacles Approvals 1 Application mgt Executive Sponsor Output-based Partner Operating Partner 2 Compute storage 3 Service Desk 4 Signature: Signature: Signature: 5 Date: Date: Date: Version Control 1 Version no. Reason for change 2 Creation Date 3 Revised Date 4 Author
  26. 26. MTBAK My first KPI
  27. 27. DO Review KPIs regularly against a target to identify trends and define corrective actions Measure leading KPIs on the shortest interval they an be controlled Cascade targets and align KPIs between all levels of the organization Bring information to life by visualizing KPIs using graphs, charts, and white-boards DON’T Measure too many things or measure just because you can Use complex calculations, percentages, or ratios when a simple number is available Try to roll-up or aggregate every KPI between each level of the organization Use KPIs to punish people – instead, use them as a tool to improve performance Tips
  28. 28. ITSMF UK Premier Gate, Easthampstead Road, Bracknell, RG12 1JS, United Kingdom Tel: +44 (0) 118 918 6500 | Web: www.itsmf.co.uk Conference KPI: fill in your evaluation please!!

×