Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Agile KPIs vs. Traditional KPIs – A mind shift


Published on

Managers encounter continued pressure to deliver more software in less time and they tend to introduce many different KPIs to measure success. But why do they introduce KPIs in the first place? Which are the good KPIs? Which ones are not useful? And which ones are the harmful ones? This white paper presents some of the most common KPIs and the expected outcomes that you could find using them. Agile Vs. Waterfall

Published in: Technology
  • In Section 4, you have the "chicken" and "pig" roles reversed. The Agile team are the pigs. Their bacon is on the line. They are the people actually doing the work.
    Are you sure you want to  Yes  No
    Your message goes here

Agile KPIs vs. Traditional KPIs – A mind shift

  1. 1. Agile KPIs vs. Traditional KPIs – A mind shift 1. Executive summary Managers encounter continued pressure to deliver more software in less time and they tend to introduce many different KPIs to measure success. But why do they introduce KPIs in the first place? Which are the good KPIs? Which ones are not useful? And which ones are the harmful ones? This white paper presents some of the most common KPIs and the expected outcomes that you could find using them. 2. Historically why did we measure success? Success is difficult to measure because it has many different variables and each stakeholder will have a different view of the definition of success: the business owner usually considers success a short “time to market” and within budget, while a technical architect might consider stability or scalability as the success criteria. Historically KPI measure has been used for the following reasons: 1. Analyse the current process to identify and understand gaps; 2. Promote behaviours that will lead to better results; 3. Understand how successful we have been and produce a lessons learnt report; 4. Punish or reward teams, individuals or departments The way the KPIs will be used in the teams can lead to great success or tremendous failure. A KPI is like any other tool that you could find in a toolbox. A hammer can be use to hang a painting or it can be used to break a window. It is up to the relevant stakeholders to understand how should they apply the information provided by the KPI. The strength of Agile is to fail fast and cheaply, learning from the mistakes occurred. Whilst setting up the Agile environment, one should always ask the following two questions: A) Is the KPI allowing me to improve the process? B) Is the process going against the Agile principles? Using the punishment approach in an Agile environment will discourage the people to be free to speak up their mind. That can lead to the situation when the people would hide their mistakes and therefore no useful and relevant lessons learnt report can be applied. Ultimately, the teams will increase the technical and non-technical debt. Eventually, there will be a price to pay for that debt and the longer is unidentified and consequently fixed, the more exponentially costly it will be to repair the damages produced. Furthermore, one of the strengths of Agile comes from an environment where the team members can feel free to search for the best value for the product owners and for the final users. Set up KPIs that will not affect the stability, the processes or the team. The positive reward approach is not necessarily a good approach either. For instance, do not believe that working hard for many hours is a good KPI. The University of Nebraska proved that working extra 10 hours a week can reduce the overall productivity by up to 40% (Raynar 1997). Also, it is important to note that positive reinforcement for one team member can be seen as a punishment for another one. As a consequence, avoiding punishment or reward is the better solution.
  2. 2. 2.1. Traditional vs. Agile KPIs The KPI to understand success is an inheritance of the Waterfall mentality as historically the organisations did not have any earlier warning tools. Managers tend to use this mechanism because the “lessons learnt” stage is at the end of the project. However, the KPIs to improve the process are the foundation of Agile: Tips: 1. Setting up the right KPIs that will improve the processes and will lead to more quality software in less time. 2. Setting up managerial KPIs to analyse the quantity or quality of your software will harm your Agile processes. 3. Work smart, not hard. 2.2. The right KPIs in Agile Agile is a methodology that promotes change and adaptability. Managers have to understand that the KPIs should reflect the reality, should be careful not to restrict freedom, and should avoid rigidity and static behaviours. The classic example of this is to standardise a value per point in terms of time or cost. The ultimate outcome of such action is that the teams won’t have a sense of achievement and success during the sprints. Finding the best KPIs for your environment requires a high level of understanding of the situation, empathy and patience. The golden rule should be: find your teams’ weakness and setup KPIs that will promote Agile principles. Also analyse: a) Does this KPI promote individuals and interactions or processes and tools? b) Does this KPI promote more working software or more comprehensive documentation? c) Does this KPI promote customer collaboration or more contract negotiation? d) Does this KPI promote flexibility to change or rigidity to follow a plan? the So that so we We want to do software We make more software in the next sprint Processes Setup a KPI to improve so we so that We want to do software How much software we will have developed We can understand project success Setup a KPI to understand
  3. 3. Examples of KPI (let’s play a bit of Game Theory): • “We are going to compare team’s efficiency to promote the next managers”. If we apply some Game Theory principles to this KPI we will come to the following conclusion: team members are competing among themselves for the best results. What initially seems like a good idea, in reality, it contradicts principle “A” and, for the long term, the output of this is KPI will be that team members will no longer collaborate. Thus, the general combined efficiency of the teams will be reduced. • “We are going to measure the velocity of the teams”. Velocity is a great internal tool that each team uses to understand their strengths and weaknesses, and bring better estimates in future, therefore reducing the risk. If a manager tries to compare velocities, then the logical reaction is that there will be an inflation of the point value. That will dilute the information and the estimations will become unstable and increase the risk. This happened because it was against the principle “A”. • “We are going to measure client satisfaction”. Satisfaction of the clients can only be achieved by increasing customer collaboration (principle “C”). That KPI will positively influence your teams to improve your process. • “We are going to measure the number of Agile best practices used during the sprints”. This will promote almost all principles, improving processes that will thus lead to more working software. To note that there are a few KPIs that can be used in any environment: • Continuous Improvement Index; create a culture of improving and of sharing ideas; • Agile best practices; • Engagement levels with the users; • Teams velocity. 3. Agile Maturity Assessment Many people have theoretical knowledge of Agile but not everyone is aware of the cultural and organizational challenges that the transformation towards Agile requires. Generally, creating the right environments requires a dual approach: a) Top-down approach: the organizational needs to understand how mature they are and its next steps. This will require buy in and support from senior stakeholders; b) Bottom-up approach: how are the technical processes, architecture and tools running on the day to day. Once the assessment has been completed, the right KPIs can be suggested to promote Agile behaviours in your environment. Important tip: Do not try to setup KPIs before conducting an Agile Maturity Assessment by an Agile Coach with extensive practical experience. He should understand deeply the strengths and weakness of the individuals, the teams, the organization, the dynamics of the business processes and the appetite for change.
  4. 4. 4. KPI analysis The following table shows some of the possible KPIs and what do they promote. It is important to note that some KPIs are useful to improve the processes for the Agile teams (i.e., chickens) but some other stakeholders (i.e., pigs) could misuse them. Managerial KPI Agile KPI Punish or reward Understand success Prom ote behaviours Understand gaps KPI The KPI promotes Customer satisfaction X X X Do not use any KPI as punishment or reward X X Client oriented focus Best Agile practices used eg. XP, Lean, SAFE, Kanban Scrum. X X X Increase productivity Velocity (points per sprint) X X X Increase efficiency Team stability X X Will generate better teamwork Attrition X X Team collaboration Time to market X X X X X Flexibility of the teams to adapt Agile adoption X X X Promotes best practices Business involvement X X X X Close the gap between the IT and the business Working software X It can lead to an endless loop understanding productivity (quantity vs. quality) Number of releases X X X It can promote releasing more or less than what is actually needed On time delivery X X X This could lead to under-commitment from the teams Value delivered X X Good KPI but problematic as sometimes value is out of the control of the teams Compare teams velocity X Inflation of points Individual velocity Poor KPI as it will lower morale and increase micromanagement Number of code lines Poor KPI as developers write inefficiently Percentange of sprint commitment X X X Attention required as this KPI can reduce quantity committed and reduce moral Defects density X X X Good KPI but attention not to lower amount of software needed Value of points Poor KPI - Inflate points Responsiveness to change X X X X Improves collaboration Time spent in improvements X X X X Improves processes Unfinished stories X X X X X Understand waste and value Use and Agile Maturity assessment (Fuller 2014) X X X X Understand Agile journey and create the right KPI Convert or measure points into days X X Poor KPI: motionless mentality. The team does not feel that they are succeeding Compare teams/ people X Terrible KPI: no collaboration People performances X Terrible KPI: Burnout effect/ low moral. Low collaboration These KPIs can be used in any environment as they promote good Agile practices.
  5. 5. Some other KPIs that you could consider could be: While choosing your KPI you can also take a look to some of the following metrics: lead time, defect count (at various phases; i.e., what’s a bug?), work in progress, code coverage, responsiveness to change, team stability, velocity (story points or story count per sprint), return on investment, innovations per sprint, continuous improvement time, failure load (fire-fighting time), iteration burn-down, unfinished stories, innovation index, customer satisfaction, un- deployed stories, number of blocks, budget/schedule compliance flow efficiency (lead time/touch time), release burn-up, projected capacity of next sprint (in person days), projected capability (in story points range) of next sprint, planned capability of the sprint (in story points), accepted velocity - story points done and accepted by product owner in the sprint, discovered work added to the backlog (in story points and stories), scope increase/decrease during the sprint/release, technical debt incurred this sprint (in story points), cost of sprint, sprint predictability, actual stories completed vs. committed stories, retrospective process improvement, communication, understanding of sprint scope and goal, user stories delivered versus user stories accepted, defect-removal efficiency (DRE), value delivered, on time delivery (Leigh 2015). 5. Summary and take away tips a) Setting up KPIs that will promote the improvement of the processes and will lead your teams to more quality software in less time. b) Setting up managerial KPIs to analyse the quality and quantity of the software will harm your processes. c) “Increments of improvements” is the best KPI to make more software with better quality. Do not use KPIs as means to punish or reward your people in Agile. d) Every few sprints conduct an Agile Maturity Assessment to understand your progress and the next steps. About the Author Javier Espinosa de los Monteros Foret is an independent Agile Coach that after obtaining two first class degrees, Management and IT, worked for more than 10 years in international projects across Europe leading Agile and Waterfall software projects. He obtained his MBA from IE Business School and he is also a Scrum Master Certified, Change Manager certified APMG, Project Manager (Prince2 Practitioner) and Programme Manager (MSP Practitioner) certified. Acknowledgements Special thanks to Simson Leigh, my good friend and excellent Agile coach, for his constant advice and support.
  6. 6. Bibliography Fuller, Dan. Measuring Agile Adoption Maturity. 21 May 2014. maturity (accessed Septembre 10, 2014). Raynar, Randolph Thomas and Karl A. “SCHEDULED OVERTIME AND LABOR PRODUCTIVITY: QUANTITATIVE ANALYSIS.”Journal of Construction Engineering and Management 123 (1997): 2.