Your SlideShare is downloading. ×
0
So – are we gettingany better?Amdocs case studyShirly Paster-BenorAgile Israel 2012
Contents    ● Background    ● The need for measurements and the conflict    ● Key Performance Indicators (KPIs) guidelines...
“…we provide state-of-the-artcustomer experience systemproducts and services, allowingService Providers to achieve theirbu...
annual revenues in excess of$3 billionglobal employees20,000customers in over50 countriesSome of our customers
PBG Division – Product Business Group        ~4000 employees                                     8 lines of business      ...
From Team Agility to Enterprise Agility                   © 2010 – Proprietary of Amdocs
Wait!            Information Security Level 2 – Sensitive7           © 2012 – Proprietary and Confidential Information of ...
Are we getting any better?                Information Security Level 2 – Sensitive8               © 2012 – Proprietary and...
Are my projects progressing as planned?                  Are we ready to deliver the release on time?     What is our cost...
Measurements? OK     .….but we are working Agile…                  Information Security Level 2 – Sensitive10             ...
The conflict     ● Self managed teams     ● Subjective estimation units     ● Dynamic backlog     ● Fixed sprints     ● Co...
We need KPIs that will         improve business     performance through Agile          behavior driving                 In...
4 guidelines          Information Security Level 2 – Sensitive13        © 2012 – Proprietary and Confidential Information ...
Aligned with organization goals                  Information Security Level 2 – Sensitive14                © 2012 – Propri...
Drives Agile & Lean behavior                 Information Security Level 2 – Sensitive15               © 2012 – Proprietary...
Same KPIs for top management and for teams                        Information Security Level 2 – Sensitive16              ...
Transparent - reflects reality                 Information Security Level 2 – Sensitive17               © 2012 – Proprieta...
Implementation process     Design KPIs with      stakeholders                           Expose to     Training & Buy In   ...
Our selection     KPIs supporting biz target     ● Productivity     ● Readiness     ● Predictability     ● Cycle time     ...
Productivity                    Release 8.1                                                                 Release 8.2   ...
21                                                                               100                                      ...
Predictability                      Information Security Level 2 – Sensitive22                    © 2012 – Proprietary and...
EVM – a tool for project management                                         Scope                                         ...
Impediments RCA                    10 Stories impeded                                                         52 days wast...
KPIs tips     ● Used by management as well as by teams for continuous       improvement     ● Provides fuel for meaningful...
Thank You!                  Information Security Level 2 – Sensitive26                © 2012 – Proprietary and Confidentia...
Average cycle time                          Information Security Level 2 – Sensitive27                        © 2012 – Pro...
Upcoming SlideShare
Loading in...5
×

"So – are we getting better?”

831

Published on

"So – are we getting better?”
By Shirly Paster-Benor @ Agile Israel 2012

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
831
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Amdocs is considered to be one of the successful companies thanks to a unique business strategy
  • Galileo-started with Scrum for development teams, evolved into Lean and Kanban in the customer teams and our vision is to reach the enterpriseIn order to tackle the issues at hand we need to move up the phases of enterprise agilityPhase 1 – Development AgilityPhase 2 – Business Agility – bringing agility to the business leaders and product managersPhase 3 – Enterprise Agility – scaling agility to the entire enterprise
  • Our goal is not being more Agile, it is to get better. We want to see measurements, we want to see numbers. All Agile scans are to measure whether we are more Agile.
  • + Do we have bottlenecks? Where are they? Where can we improve? How can I identify areas to improve?
  • The KPIs must be aligned to Agile and Lean values like: Continuous and self improvement, inspect and adopt, transparency, trust, self commitment, collaboration, etc.We must choose KPIs that will assist us in the journey of Agile&Lean implementation, they will support and advance the implementation.Why is it improtant? Because we believe that Agile and Lean will advance our efforts to become better and improve the biz goals that we chosen to focus on.
  • We must use same language throughout the organisation.We need to ensure that the measures are starting from the field, so the field can inspect & adopt and self improve, we also deliver them to management so they can see how organization in general is improving, but the message is that we creates the measures for the focus Teams and the Opportunity Teams.There is no point to have measure that management look at and the field don’t know what to do with the data and how to improve.We are looking for more tangible measures that will guide us to concrete actions in order to improve.
  • The KPIs should reflect reality, be based on real objective data that is easy to continuously collect.
  • I am going to present part of our KPIs and reports, I’ll explain for each KPI which biz target it promotes and how it drives better Agile behavior.You need to understand that defining the KPIs is the easy part of the story. Buy in and implementation in such a large organization with about 150 Scrum teams and dozens of managers is the real challenge, and I’ll discuss the implementation process shortly, after the KPIs presentation.
  • I am going to present part of our KPIs and reports, I’ll explain for each KPI which biz target it promotes and how it drives better Agile behavior.You need to understand that defining the KPIs is the easy part of the story. Buy in and implementation in such a large organization with about 150 Scrum teams and dozens of managers is the real challenge, and I’ll discuss the implementation process shortly, after the KPIs presentation.
  • Productivity is giving an answer to: how much SW are we able to deliver?This KPI came to support biz goal of: more features for less $Let’s analyze this chart: it is showing productivity for all PBG products and teamsWe can see in the chart results in 2 aspects: velocity = # of stories done, productivity = # of stories per man hourWhy # of stories instead of story points? # of stories can be added up across teams + there are surveys showing that along time story sizing is constant so that # of stories is stable. In order to understand the results, we correlated it with other KPIs/reports like: team stability, sprint/iteration readiness, and other information regarding releases and sprints timelines.Analysis: 1- [add the red boxes and blue line] From 2009 to 2010 we improved as it was time when Agile implementation started to mature. 2009 avg = 0.14, 2010 avg= 0.18, improvement of ~30%. From 2010 to 2011 no improvement. Why? Our experience shows that productivity is improved at first stages of implementation, till teams are reaching the ‘Norming’ phase, then, it is hard to see improvement in this area, there are improvements in other areas.2- [Add arrows] the picks are usually towards end of release , and the bottoms are usually at beginning of release due to low readiness3- Velocity vs. productivity – in Amdocs teams dynamic is high, stability is not high,… thus making velocity less effective as a KPI. In places where team stability is high and attrition is low, Velocity may be the more ‘classic’ Agile KPI.Remarks: Managers do not track numbers, only trends. Improvement along the timeline.For teams level we use story points as basis. It raises question like: can’t a FT cheat and blow the estimations? The answer is: no, there are balances in the process, PdO is there to challenge the team.How this KPI drives Agile/Lean behavior?Teams are trying to commit for more and improve. It is self continuous improvement.Also- it strengthen the team commitment as a team because we are not measuring personals.This KPI results led us to the Readiness KPI
  • Readiness came to support improvement in performance (productivity) and indirectly supports biz goal of: more features for less $This KPI is giving an answer to: Are we ready for the next sprint/iteration?It presents # of ready stories (prioritized and with DoD) per FT, vs. the avg # of done stories in last 5 sprints.This is a time-dependent KPI, meaning: we do not expect 100% readiness long before the sprint start, it should be rising gradually towards 1 week before sprint start.In this chart we see results for a group of FTs (product) . [Add the line+box]. We consider everything above 200% as waste, you see in first 2 sprints 500% ready stories and on the end of second sprint which was end of release big amount of stories were descoped = waste= sunk cost. Then a new release started and backlog grew but not for the same levels.How this KPI drives Agile/Lean behavior?Very much ! It drives low WIP, where we say we do not want to see more stories then needed because they might come irrelevant and then all the invested work is waste, and also do not want to see ready stories long before the sprint start.This KPI served us in our coaching efforts because it well reflects our point.In the old world of Amdocs, a ‘Utilization’ measurement was practiced, and here we show exactly why keeping the people busy not necessarilty leads to better results. (even the opposite)
  • Predictability came to support biz goal of: being able to commitThis KPI is going through all levels in PBG, and giving an answer to: can I commit the next release?Predictability is measured by delivered vs. committed stories in %.**for impeded stories there is a small bonus, for extra stories also small bonusHere we see an example of a certain FT standing on ~80% predictability.The low predictability in first 2 sprints is due to low readiness. (we made the RCA)100% is not a good result as it implies that team does not challenge themselves……This is a very important KPI, as our customers wants us to commit. It does not stand alone – it must be analyzed together with productivity readiness and more.However- it raised a lot of conflicts: when we produced this cross teams, it drived a wrong behavior from middle managers side- they set a number as a goal for all teams, instead of giving the teams data as a basis for self improvement.Over all PBG predictability is 70%, and this is a number that can give head of PBG the confidence to commit.We decided to use this number in the teams level as a basis for sprint/iteration retrospective, for self improvement. (they set their own target)How this KPI drives Agile/Lean behavior?It is emphasizing the value of self commitment.
  • RCA for a specific project – enables the OT to analyse where are the main bottlenecks which wastes the most time and holds the most stories based on real objective data. Then the team can RCA the process or the contributors to it and remove them.In this project we see:Design – 2 stories impeded, 15 days holdExternal dependencies – 10 stories impeded, 52 days holdInfra - 1 stories impeded, 4 days holdOther- 3 stories impeded, 15 days hold
  • Average cycle time supports biz goal of: reduce time to marketIt presents the average duration in weeks for all epics (a bunch of stories with biz value) ended in a month.The ‘clock’ is starting once Epic is written by the PdM and then go through design phase (HLD), dev. and integrated testing. Then the clock stops.This KPI is used by projects OTs and above (products managers, BU mgrs). It is one KPI which does not use by the teams themselves. How this KPI drives Agile/Lean behavior?1- It is difficult to say that our releases are much shorter now, but in average, each requirement is taking less, hence enabling low WIP and better performance.2- We are able to release SPs or short releases more frequently.
  • Transcript of ""So – are we getting better?”"

    1. 1. So – are we gettingany better?Amdocs case studyShirly Paster-BenorAgile Israel 2012
    2. 2. Contents ● Background ● The need for measurements and the conflict ● Key Performance Indicators (KPIs) guidelines ● Amdocs KPIs and views ● Questions? Information Security Level 2 – Sensitive2 © 2012 – Proprietary and Confidential Information of Amdocs
    3. 3. “…we provide state-of-the-artcustomer experience systemproducts and services, allowingService Providers to achieve theirbusiness goals and gain acompetitive edge”
    4. 4. annual revenues in excess of$3 billionglobal employees20,000customers in over50 countriesSome of our customers
    5. 5. PBG Division – Product Business Group ~4000 employees 8 lines of business 35 product More then 100 components 6 locations 1 portfolio Information Security Level 2 – Sensitive5 © 2012 – Proprietary and Confidential Information of Amdocs
    6. 6. From Team Agility to Enterprise Agility © 2010 – Proprietary of Amdocs
    7. 7. Wait! Information Security Level 2 – Sensitive7 © 2012 – Proprietary and Confidential Information of Amdocs
    8. 8. Are we getting any better? Information Security Level 2 – Sensitive8 © 2012 – Proprietary and Confidential Information of Amdocs
    9. 9. Are my projects progressing as planned? Are we ready to deliver the release on time? What is our cost? Can we reduce it? Can we deliver more?How efficient are we? Can I commit to my customers?Do we have enough work for next iteration? Information Security Level 2 – Sensitive9 © 2012 – Proprietary and Confidential Information of Amdocs
    10. 10. Measurements? OK .….but we are working Agile… Information Security Level 2 – Sensitive10 © 2012 – Proprietary and Confidential Information of Amdocs
    11. 11. The conflict ● Self managed teams ● Subjective estimation units ● Dynamic backlog ● Fixed sprints ● Commitment based on past results ● Scope change is embraced Information Security Level 2 – Sensitive11 © 2012 – Proprietary and Confidential Information of Amdocs
    12. 12. We need KPIs that will improve business performance through Agile behavior driving Information Security Level 2 – Sensitive12 © 2012 – Proprietary and Confidential Information of Amdocs
    13. 13. 4 guidelines Information Security Level 2 – Sensitive13 © 2012 – Proprietary and Confidential Information of Amdocs
    14. 14. Aligned with organization goals Information Security Level 2 – Sensitive14 © 2012 – Proprietary and Confidential Information of Amdocs
    15. 15. Drives Agile & Lean behavior Information Security Level 2 – Sensitive15 © 2012 – Proprietary and Confidential Information of Amdocs
    16. 16. Same KPIs for top management and for teams Information Security Level 2 – Sensitive16 © 2012 – Proprietary and Confidential Information of Amdocs
    17. 17. Transparent - reflects reality Information Security Level 2 – Sensitive17 © 2012 – Proprietary and Confidential Information of Amdocs
    18. 18. Implementation process Design KPIs with stakeholders Expose to Training & Buy In Management Stable Measure Continuous Start Measure (+believe) Improvement Information Security Level 2 – Sensitive18 © 2012 – Proprietary and Confidential Information of Amdocs
    19. 19. Our selection KPIs supporting biz target ● Productivity ● Readiness ● Predictability ● Cycle time Performance improvement views/reports ● EVM ● Impediments RCA Information Security Level 2 – Sensitive19 © 2012 – Proprietary and Confidential Information of Amdocs
    20. 20. Productivity Release 8.1 Release 8.2 Information Security Level 2 – Sensitive20 © 2012 – Proprietary and Confidential Information of Amdocs
    21. 21. 21 100 200 300 400 500 600 700 800 900 0 2011.06.13 2011.06.17 2011.06.21 2011.06.25 2011.06.29 2011.07.03 2011.07.07 Readiness 2011.07.11 2011.07.15 2011.07.19 2011.07.23 2011.07.27 2011.07.31 2011.08.04 2011.08.08 2011.08.12 2011.08.16 2011.08.20 2011.08.24 Release end 2011.08.28 2011.09.01 2011.09.05 2011.09.09 2011.09.13 2011.09.17 2011.09.21 2011.09.25 2011.09.29 2011.10.03Information Security Level 2 – Sensitive 2011.10.07 2011.10.11 2011.10.15 2011.10.19© 2012 – Proprietary and Confidential Information of Amdocs 2011.10.23 Sprint end % ready stories
    22. 22. Predictability Information Security Level 2 – Sensitive22 © 2012 – Proprietary and Confidential Information of Amdocs
    23. 23. EVM – a tool for project management Scope changes Reference % done scope % consumed budget Information Security Level 2 – Sensitive23 © 2012 – Proprietary and Confidential Information of Amdocs
    24. 24. Impediments RCA 10 Stories impeded 52 days wasteNumber of impediments Cumulative impediments duration Information Security Level 2 – Sensitive 24 © 2012 – Proprietary and Confidential Information of Amdocs
    25. 25. KPIs tips ● Used by management as well as by teams for continuous improvement ● Provides fuel for meaningful conversation. ● Crystalize 2-3 business goals and derive KPIs from them ● Choose a small set of metrics and diagnostics. ● Follow trends, not numbers. ● Make sure it is easy to collect. ● Provides feedback on a frequent and regular basis. ● Do not compare teams results, measure each unit’s improvement Information Security Level 2 – Sensitive25 © 2012 – Proprietary and Confidential Information of Amdocs
    26. 26. Thank You! Information Security Level 2 – Sensitive26 © 2012 – Proprietary and Confidential Information of Amdocs
    27. 27. Average cycle time Information Security Level 2 – Sensitive27 © 2012 – Proprietary and Confidential Information of Amdocs
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×