Enterprise Services Planning: Defining Key Performance Indicators

1,629 views

Published on

Defining KPIs for use in Enterprise Services Planning and with Kanban systems. Understanding the difference between KPIs, Improvement Guides, and General Health Indicators. Understanding how KPIs drive behavior such as establishing multiple classes of service. Relating KPIs to evolutionary change. KPIs are Fitness Criteria Metrics with defined threashold values

Published in: Leadership & Management
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,629
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
70
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide
  • Fitness criteria are metrics that measure things customer or other external stakeholders value such as delivery time, quality, predictability, conformance to regulatory requirements or metrics that value actual outcomes such as customer satisfaction or employee satisfaction
  • Fitness criteria are metrics that measure things customer or other external stakeholders value such as delivery time, quality, predictability, conformance to regulatory requirements or metrics that value actual outcomes such as customer satisfaction or employee satisfaction
  • Fitness criteria are metrics that measure things customer or other external stakeholders value such as delivery time, quality, predictability, conformance to regulatory requirements or metrics that value actual outcomes such as customer satisfaction or employee satisfaction
  • Fitness criteria are metrics that measure things customer or other external stakeholders value such as delivery time, quality, predictability, conformance to regulatory requirements or metrics that value actual outcomes such as customer satisfaction or employee satisfaction
  • Traditional change is an A to B process. A is where you are now. B is a destination. B is either defined (from a methodology definition) or designed (by tailoring a framework).
    To get from A to B, a change agency* will guide a transition initiative to install destination B into the organization.
    *either an internal SEPG or external consultants
  • Median is always less than the mean and lies between the mode and the mean. Median is less sensitive to the tail on distribution and hence less variable in the presence of assignable/special cause variation causing a long tail. However, it is the mean that is used in Little’s Law and therefore we do care about risks that affect the tail in the distribution when using Little’s Law to forecast.
  • Enterprise Services Planning: Defining Key Performance Indicators

    1. 1. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Learn to care about what the customer cares about KPIs should shape improvements to service delivery Enterprise Services Planning Defining Key Performance Indicators Presenter David J. Anderson Swift Kanban Webinar 29 April 2015
    2. 2. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Lean Kanban North America 2015 conference • “Back to our roots” • Implementing Kanban • Looking to the future… • Enterprise Services Planning
    3. 3. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. • 2 Days of Learning Sessions • Choose Your ½ Day Workshops • Risk Profiling – David J Anderson • Cost of Delay – Don Reinertsen • Project Management with Kanban • Cynefin 101 for Portfolio Kanban • Simple Probabilistic Forecasting …and more! • Becoming Data-Driven • Objective Retrospectives • Forecasting • Enterprise Kanban & Lean Startup • Scrumban • Kanban Coaching • Blockers for Improvement • RBS Project Sizing • Kanban Academic Research …
    4. 4. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Kanban experience reports Ultimate Software • BazaarVoice • web recommendations app and mobile development Including…
    5. 5. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
    6. 6. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
    7. 7. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Fitness for Purpose
    8. 8. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Which system is fitter? We don’t know! System B is faster but without understanding customer expectations, both may be fit enough 0 2 4 6 8 10 12 14 Lead Time (Days) System A Frequency 0 5 10 15 20 25 30 5 10 15 20 25 30 More Lead Time in Days System B Frequency Mean 17 days Mean 12 days
    9. 9. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Measuring delivery against expectation 0 2 4 6 8 10 12 14 Lead Time (Days) System A Frequency 0 2 4 6 8 10 12 Lead Time Expectation Spread (Days) System A Frequency 0 5 10 15 20 25 30 5 10 15 20 25 30 More Lead Time in Days System B Frequency 0 5 10 15 20 25 30 35 40 45 -15 -10 -5 0 5 10 15 20 More Lead Time Expectation Spread (Days) System B Frequency Mean 17 days Mean 12 days System B is clearly fitter! System B delivers 5/7 within expectations System A only delivers 3/7 within expectations
    10. 10. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. What makes a pizza delivery service “fit for purpose” ? Fitness criteria are metrics that measure things customers value when selecting a service again & again  Delivery time  Quality  Predictability  Safety (or conformance to regulatory requirements)
    11. 11. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Meet Neeta - a project manager Neeta’s team are working late (again) Neeta needs to feed them with pizza What attributes do her team care about in a pizza delivery service? • Delivery time = approximately 1 hour • Non-functional quality = tasty & hot • Functional quality (order accuracy) = doesn’t matter if small mistakes are made, geeks will eat any flavor of pizza • Predictability = +/- 30 minutes is acceptable • Safety = so long as health & safety in food preparation is good, it’s fine
    12. 12. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Neeta is also a working mom! Neeta gets home late. Her kids are really hungry and even though she shouldn’t she decides to order pizza for them What makes a pizza delivery service acceptable to her kids age 4, 6, 9 & 11 years? • Delivery time = 20 minutes • Non-functional quality = doesn’t matter too much, it’s pizza!!! • Functional quality (order accuracy) = it must be cheese pizza! No other flavor is acceptable! (even if you take the pepperoni off) • Predictability = +/- 5 minutes maximum!!! • Safety = only mommy worries about that stuff!
    13. 13. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. To be “fit for purpose” there is a product component & a service delivery component We need to offer a selection of different recipes which are tasty & popular. However, we must also deliver with speed & predictability Lesson
    14. 14. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Modern creative & knowledge worker businesses often obsess with product definition & strategy Operational excellence and service delivery excellence are often overlooked or treated as inferior management skills Lesson
    15. 15. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Neeta has 2 identities – Mother and Project Manager Each of Neeta’s identities represents a different market segment for the pizza delivery service Traditional demographic & income group segmentation does not accurately capture the context to understand “fit for purpose” Nor, for that matter, do personas. As Neeta represents two segments not just one persona Lesson
    16. 16. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Exercise – Understanding Fitness for Purpose Pick a service with which you are familiar Consider what makes it “fit for purpose?” Which attributes make you select the service, again and again? What are your expectations for each attribute?  Why?  What (business) risks drive your expectations?
    17. 17. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Fitness Criteria Drive Evolutionary Change
    18. 18. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Fitness criteria are metrics that measure observable external outcomes Fitness criteria are metrics that measure things customers or other external stakeholders value  Delivery time  Quality  Predictability  Safety (conformance to regulatory requirements) or metrics that qualitatively assess actual outcomes such as  customer satisfaction  employee satisfaction
    19. 19. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Evolutionary change has no defined end point Evolving Process Roll forward Roll back Initial Process Future process is emergent Evaluate Fitness Evaluate Fitness Evaluate Fitness Evaluate Fitness Evalua Fitnes We don’t know the end-point but we do know our emergent process is fitter!
    20. 20. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Business Risks, Fitness Criteria & Classes of Service should all align Business risks are things which are uncertain that affect the performance of our business such as nature of demand, consistency of supply, delivery predictability, seasonal windows of opportunity, time value of money Classes of service offered should align with business risks Metrics used to evaluate service delivery capability should be fitness criteria that are derived from specific business risks For example, opportunity cost of delay requires us to measure lead time and understand sensitivity to schedule uncertainty
    21. 21. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Select Key Performance Indicators Carefully! KPIs should be fitness criteria metrics with threshold values that represent “good enough” – the level where the service delivery is “fit for purpose” KPIs should assess service delivery capability and indicate fitness for purpose. In doing so, a KPI indicates your likelihood of success – of surviving and thriving - by adequately satisfying your customers? KPIs should be recognizable by your customers as something meaningful! If your customer doesn’t recognize the metric it isn’t a “key” “performance” indicator, it is some other kind of metric
    22. 22. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Other Useful Metrics Some other metrics are useful  Those which guide improvements  Those which indicate general health Is your metric evaluating and guiding a specific change to improve fitness of your business such as an initiative to improve vendor response times? Or, is it a general business health indicator such as liquidity? If neither of these, then it is a metric that you almost certainly don’t need! It should be removed!
    23. 23. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Defining Fitness Criteria
    24. 24. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Market Adoption Lifecycle Segmentation Enthusiasts Early Adopters Early Majority Late Majority Laggards Rate Of Market Adoption time Moore’s Chasm Little Chasm Hip Cool Buggy Community development Niche Market Features Good func quality Adequate non-func quality Permission Giving Early adopter Exceptional func and non- func quality Cost Effective Broad Features Exceptional func and non- func quality Low Cost Easy Access Forced adoption Viewed as taxation Fit for purpose Fit for purposeChanges over time
    25. 25. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Customer Storytelling & Clustering Tell stories about real customers, their motivation, what they buy and why. Cluster similar stories Give each cluster a “nickname” e.g. • “All ins” • “Aspirationals” • “Bet hedgers” • “Boy scouts”
    26. 26. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. You can’t just ask! Neeta, how fast would you like your pizza delivered? How predictable do you need us to be with our delivery estimate? Customers will tend to tell you they need better service and more features than they really need! Would you pay more for the things you say you need and want? No, probably not! Believe what customers actually do, do not believe what they say they’ll do! Actually behavior will vary from declared intent!
    27. 27. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Who knows your customers? Front line staff  Those who take and those who deliver orders  Those who provide “customer care”  Often the lowest paid staff in a business  Often the highest turnover, shortest tenured positions  And yet, they have the vital information that enables the business to survive, thrive and compete
    28. 28. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Pizza boy knows Neeta’s Story! Staff who meet customers can be trained to learn what matters to them and why Create ways to capture customer stories or directly involve customer facing staff when defining customer segments fitness criteria
    29. 29. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. GT car manufacturer story A well known manufacturer of GT cars determined customers were prepared to wait 21 months to take delivery They learned this by letting delivery time slip to 27 months and receiving cancellations and customers switching to a rival manufacturer Determining fitness criteria thresholds by reducing service levels until customer complaints rise to dangerous levels isn’t a “safe to fail” approach! Damaging your brand, your reputation and your profitability is a strange way to discover how to be… “fit for purpose!”
    30. 30. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Retarding customer service until customers complain vehemently or take their business elsewhere could be damaging • Undermines brand • Damages reputation • Loss of market share • Loss of revenue Probing for threshold values by reducing service quality isn’t “safe to fail” Is it “safe to fail”? We need general guidance that allows us to probe for fitness criteria threshold values that is “safe to fail” If we can’t ask, and we can’t allow service to decline until complaints make the threshold evident, what can we do?
    31. 31. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Probing for Threshold Levels
    32. 32. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Look for clusters or patterns of demand, or patterns of similar expectations, or new sources of demand that may represent an emerging segment Probe with classes of service Create a class of service to respond to the believed new segment • Set service levels at or close to anticipated threshold levels
    33. 33. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Probe with classes of service Observe take up of class of service • Is it over-used? (or abused?) If so tighten qualification criteria • Is it under-used? Consider removing it • Is it used but you fail to deliver to expectations? Do people complain? If no then consider removing it. You are over promising Fixed delivery date class of service emerged this way. Initially abused by marketing, eligibility criteria were tightened up.
    34. 34. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Telecom Equipment Example A platform maintenance department at a telecom equipment manufacturer receives demand only from internal application departments… … Each request is tagged with the originating telco operator for whom the request is being implemented. Each operator is given a lane on the kanban board
    35. 35. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Telecom Equipment Example Imagine 3 American telco operators with different strategic positions… • Verizon value quality most • Sprint value time-to-market • Voicestream/T-Mobile USA values low cost Now design and offer 3 classes of service… • High quality, tight “done” criteria for each step • Short lead time – pull priority, looser “done” criteria • Low cost – junior staff, lowest priority compared to other work
    36. 36. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Different lanes, different risks Done F E I Engin- eering Ready Deploy- ment Ready G D GY PB MN 10 ∞ P1 AB Ongoing Development Testing Done Verification Acceptance 10 10 Verizon 10 10 Sprint T-Mobile 10 DE DA Each lane represents a different source of demand but also different fitness criteria and threshold values Different classes of service and different pull criteria policies are defined for each lane providing service levels tuned to the “fitness for purpose” expectations of each customer
    37. 37. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. To have confidence you are offering a service that is “fit for purpose”, you must offer different classes of service To serve more than one market segment adequately, you must offer a selection of classes of service Lesson
    38. 38. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Classes of service should align to market segments and fitness criteria (or stakeholders needs) Lesson KPIs cannot be general! They need to be tied to customer expectations. Different segments have different expectations. Hence, different threshold levels of the KPI
    39. 39. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Classifying Metrics
    40. 40. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. 85% at 10 days Mean 5 days 98% at 25 days ChangeRequests ProductionDefects 85% at 60 days Mean 50 days 98% at 150 days Median 45 days Lead Time Distribution is a KPI
    41. 41. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Blocker Cluster Data Guides Improvements Harvested blocker tickets over a 1 month period Cluster blockers based on the stories behind the delay Each cluster represents a risk Identify Likelihood, Total Impact & Average Impact Identify whether occurs in the tail of the lead time distribution Define risk reduction & mitigation actions http://www.klausleopold.com/2013/09/blocker-clusters-problems-are-not.html
    42. 42. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Avg. Lead Time Avg. Delivery RateWIP Pool of Ideas Ready To Deliver Cumulative Flow is a General Health Indicator
    43. 43. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Test Ready Flow Efficiency is a General Health Indicator F E I G D GY PB DE MN P1 AB Customer Lead Time Waiting Waiting WaitingWorking Ideas Dev Ready 5 Ongoing Development Testing Done 3 35 UAT Release Ready ∞ ∞ Flow efficiency % = Work Time x 100% Lead Time Working WaitingWorking
    44. 44. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Liquidity is a General Health Indicator The volume of pull transactions in a kanban system defines its liquidity
    45. 45. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Volatility is a General Health Indicator The derivative of liquidity shows us kanban system volatility
    46. 46. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Wrap Up
    47. 47. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. ESP Training Modules Module 1 Portfolio Management (Day 1 & 2)  Strategy, fitness for purpose, KPIs, Cost of Delay, Scheduling, Sequencing, Portfolio risk, risk hedging, risk profiling, aligning strategy & capability, strategy review Module 2 Enterprise Services (Day 3)  Understanding kanban systems, real options, upstream Kanban, commitment & replenishment, lead time, chance vs assignable cause variation Module 3 Project & Demand Management (Day 4)  Demand analysis, demand shaping, capacity planning, project forecasting, risk review, labor pool liquidity, workflow liquidity Module 4 Real Options, Portfolios, Programs, Dependencies & Scaling(Day 5)  Scaling Kanban, dependencies, visualizing dependencies, portfolio Kanban, stand ups, service delivery reviews, ops reviews
    48. 48. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. ESP Training ESP training is delivered on-premises with clients around the world To order an ESP training class, contact Wes Harris, Commercial Director of David J. Anderson & Associates, wes@djaa.com
    49. 49. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. Thank you!
    50. 50. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc. About David Anderson is an innovator in management for 21st Century professional services businesses. He leads a training, consulting, publishing and event planning company dedicated to developing, promoting and implementing new management thinking & methods… David has 30+ years experience in the high technology industry starting with computer games in the early 1980’s. He has led software organizations delivering superior productivity and quality using innovative methods at large companies such as Sprint and Motorola. David defined Enterprise Services Planning and originated the Kanban Method an adaptive approach to improved service delivery. His latest book, published in June 2012, is, Lessons in Agile Management – On the Road to Kanban. David is also Chairman of Lean Kanban Inc., a business operating globally, dedicated to providing quality training & events that bring Kanban and Enterprise Services Planning to a broad audience of professionals around the world.
    51. 51. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.

    ×