![Summary
Manage Risk With The End in Mind
So4ware Quality
• To manage risk in our so4ware projects, we must
consider the quality (or lack of [risk]) from all
viewpoints and blindspots
• Take advantage of all the collaboraVon, checks
and balances within the agile process to build
miVgaVon within your agile process at each step
– Review the course before the race
– Learn at each step and review with risk mindset
• Keep your eye on the ball – So4ware Quality
@philiplew @xboso, 78 © 2017 XBOSo4, Inc.- All Rights Reserved.](https://image.slidesharecdn.com/managingagilesoftwareprojectsunderuncertainty-aqaa-170414033031/85/Managing-Agile-Software-Projects-With-Risk-and-Uncertainty-78-320.jpg)
Uploaded byXBOSoft
Managing Agile Software Projects With Risk and Uncertainty
The document discusses managing software projects under uncertainty using agile methodologies to understand and mitigate risks. It outlines the importance of risk management, highlights common pitfalls in agile processes, and presents techniques to identify and address risks effectively. The content also emphasizes that software quality is intrinsically linked to risk management and that agile frameworks can help improve this relationship.
More Related Content
![[Agile Portugal 2014] - Agile Decision Support System for Upper Management - ...](https://cdn.slidesharecdn.com/ss_thumbnails/agileportugal2014v10final-140626034616-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Managing Agile Software Projects With Risk and Uncertainty
Managing Agile Software Projects With Risk and Uncertainty
- 1. Managing Agile So,ware Projects Under Uncertainty Rocket Science? Applying the rightthinking and techniques within an agile framework to understand and manage risk. @philiplew @xboso, 1 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 2. Meet Your Instructor • Phil Lew – Telecommunications consultant and network designer – Team Lead, Data warehousing product development – Software product manager, BI product – COO, large IT services company – CEO, XBOSoft, software qa and testing services • Relevant specialties/Research – Software quality process improvement – Software usability evaluation – Software quality in use / UX design @philiplew @xboso, 2 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 3.
- 4.
- 5. SeRng ExpectaVons • InteracVve-ask quesVons • Group exercises • I won’t read the slides… • Slides for you as a take-away • Exercises – You get what you put in, Be All In – Not the “pitcher and glass” method • This is just an appeVzer, part of a full day workshop 5 @philiplew @xboso,
- 6. What Got Us Here • Smaller teams • Faster iteraVons • Listening to the user – ConVnuous beta – Data collecVon& analyVcs © 2017 XBOSo4, Inc.- All Rights Reserved. 6 • CommunicaVon • Working smarter • Analysis, adapVon and improvement 1. Changes in technology (mobile, cloud) 2. Changes in business models 3. Many failures… @philiplew @xboso,
- 7. Agile Problems Resistance to Change DIstrust Requirements Churn Frozen Requirements Agile Doing – Not Being Process Inconsistency Lack Test AutomaVon RetrospecVves Not Valuable AgileFall Lack Customer- User Understanding Agile Failures – Why? © 2017 XBOSo4, Inc.- All Rights Reserved. 7 Let’sVote – You get to pick the top 3 @philiplew @xboso,
- 8. Agile Success ExecuVve Support User Involvement Scoped Value Skilled Players Agile Process Proficiency Clear Business ObjecVves High Use of AutomaVon Consistent Dev, IntegraVon and Release PracVces CollaboraVve Behaviors Full Focus Agile Success – Why? © 2017 XBOSo4, Inc.- All Rights Reserved. 8 Let’sVote – You get to pick the top 3 @philiplew @xboso,
- 9.
- 10. So4ware Quality and Risk • All Companies are becoming So,ware Companies • So4ware is a part of almost all value chains. • Technology and business condiVons make consistently providing value difficult to accomplish without miVgaVng large amounts of risk. @philiplew @xboso, 10 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 11. Is the So4ware Ready for Release? • Am I done tesVng? • What if ____? • What did we forget? • _________ • What are some ‘what ifs or quesVons that your boss asks? @philiplew @xboso, 11 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 12. Agenda for Today 1. Risk, Uncertainty and So4ware Quality 2. Risk and Where it Comes From 3. Agile and Risk 4. Risk MiVgaVon and Frameworks @philiplew @xboso, 12 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 13. Process Versus Quality • The process for building so4ware and the resulVng so4ware product are intertwined. • We thought (CMMI) that reliable, repeatable processes for building so4ware led to equally good quality so4ware product. • This is not enVrely true. – “Efficiency and repeatability do not equal quality.” Why? – And neither does velocity! – What happens when you drive too fast? @philiplew @xboso, 13 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 14. Risk, Defects or Lack Of, and Quality • So4ware quality is o4en reduced to so4ware tesVng and finding defects… • Finding defects is seen as a primary and someVmes sole risk miVgaVon pracVce. • Finding defects doesn’t necessarily drive quality, but links quality to the absence of defects. • Absence of defects is not always = quality so4ware delivered. • Quality needs a broader definiVon as does Risk Management @philiplew @xboso, 14 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 15.
- 16.
- 17. The Realm of So4ware Project Risks Personnel Shortfalls Shortfalls in external components& services Real-time performance shortfalls Straining Computer Science capabilities User can’t use it Gold Plating Developing the wrong features-wrong thing Unrealistic schedules Unrealistic budget Stream of changing requirements Software Risk Management, B. Boehm 1989 People ProductProcess 17 Security holes @philiplew @xboso,
- 18.
- 19. Technology Risks • Plarorm can’t adapt to changing needs • Too hard to implement, too complicated with this stack • Doesn’t integrate well with other 3rd party tools • Can’t find people who have the right skills • What else in terms of technology risks? @philiplew @xboso, 19 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 20. Product Risks • Delivered too late, 1st mover or even 2nd mover advantage dissipated • Doesn’t do what the customer/user wants • Too slow • We missed the mark, it does what we wanted but… • Harder to do than we thought • ______ @philiplew @xboso, 20 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 21. Contextual Influencers on Risk • Project size • Team make up • Complexity • Timing • CompeVVon • Market demand • User ExpectaVons (quality expectaVons) • Stakeholders • Company history and culture Can these be controlled or not? @philiplew @xboso, 21 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 22. System Complexity Factors Nature of ExternalInterface Database Size CPU Execution Time Constraints Failure Handling Main Storage Constraints Development Environment Factors Analyst Capability Application Experience Language Experience Programmer Capability Development Schedule Experience with Subcontractors Consequences of Not Meeting The Quality Requirements Loss of Life Loss of Property Performance Degradation Loss of Data Interruption of Service Inconvenience System Complexity Factors Development Environment Factors Risk Contextual Influencers @philiplew @xboso, 22 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 23. Process Risks • Too slow • Not repeatable • Delicate and FrAgile – Easily broken • Not adaptable • Not consistent • Too dependent on a superhero @philiplew @xboso, 23 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 24. Delivery Risk • Risks that add costs or stop business revenue due to delayed launch or even cancellaVon. – How many of you have had project delays? • Not enough people to do what we promised, so we deliver late • Got a slow start in understanding what is wanted • Had to make some big unexpected changes in the middle • Towards the end, we found out what we delivered was not exactly what was wanted • Harder than we thought, took a while, couldn’t figure out some things. @philiplew @xboso, 24 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 25. Delivery Delay Risk • Project management and development processes and the right funcVonality mean zero if the applicaVon works unpredictably, is slow, or breaks down o4en. • In addiVon to on-Vme, on-budget and on- scope delivery, business value is generated by the funcVonality working like it should. @philiplew @xboso, 25 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 26. Business Risks • Risks that make the applicaVon hard to maintain and adapt to changing business requirements • Lack of agility thus damages future business revenue. – Underlying structure and architecture • O4en forgouen or overlooked, why? • Too late, didn’t get market share • Didn’t develop right features for what the market wanted @philiplew @xboso, 26 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 27. Customer SaVsfacVon Risks • Customer/end user doesn’t like/use what we made • Customer is dissaVsfied with what we delivered • Customer miscommunicated what they wanted • Customer didn’t understand what they wanted @philiplew @xboso, 27 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 28.
- 29. Uncertainty Versus Risks • What is an uncertainty? • Outcome that has a probability > 0%; – but < 100% • UncertainVes become risks if they have a probability of a negaVve consequence. P(n) • As P(n) increases --> higher the risk. • If P(n) = 0, then it is merely an uncertainty. • What uncertainVes in your project do you have that are not necessarily risks? Any? @philiplew @xboso, 29
- 30. Uncertainty with Possible NegaVve Outcomes = Risk • How can we know the probability of a negaVve consequence? • Can we be certain P(negaVve outcome) = 0 ? • What I'll eat tonight is uncertain. Is that a risk? • What are risks in your projects? • What are the negaVve consequences if any (P(n)=0)? @philiplew @xboso, 30
- 31. Understanding NegaVve Consequences • SomeVmes you don’t know or understand what you are afraid of, or trying to avoid • They may be bigger than you think – And could even be project threatening • They may be smaller than you think, or even = 0 based on an assumpVon you overlooked • Do you have any examples when you overesVmated or underesVmated negaVve consequences? @philiplew @xboso, 31
- 32. Let’s Talk About Probability • Most of us subconsciously assume a normal distribuVon when it comes to uncertainty • That is what we are used to when we were given grades in school, remember? • But what are the characterisVcs of a normal distribuVon? @philiplew @xboso, 32
- 33.
- 34.
- 35. Fat and Narrow Tails Make a Difference • What risksare fat and narrow tailed? • With a fat tailed risk you can be catastrophically wrong. • What would be “Ruin” when it comes to software development? @philiplew @xboso, 35
- 36. Why It’s Important to Understand Normal and Fat • TradiVonal risk management strategies rely on and assume a normal bell curve but in reality, projects don't behave this way. Let’s look at this closer. • What things are normal and what types of occurrences don’t follow a normal distribuVon? – Height of people in the room @philiplew @xboso, 36 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 37. Normal Exercise – EnVre Class • Normal distribuVons vary in the neighborhood of its average, but have few data points beyond 3 standard deviaVons from that average. • Do a survey of the class and calculate the average height of people in the room for men, and for women. • Average height of men is 5’10” with standard deviaVon of 4”. Chances of a 6’10” man are very small. @philiplew @xboso, 37 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 38. Fat Tail Exercise – Small Groups • hup://www.boxofficemojo.com/genres/chart/? id=documentary.htm • Calculate the average and standard deviaVon for these documentary movies revenue. • What are the differences you observe between documentary movies and people’s heights? • How can you apply this to your work in determining what is a risk and not, what risks to pay auenVon to and NOT. @philiplew @xboso, 38 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 39. In the End, We Want to • Find problems early • Find big problems • Avoid problems altogether – **Problems by nature have negaVve consequences – Our job is to decrease this probability • Agile helps us with this, but we need to focus on those acVviVes with the right viewpoint to gain this insight @philiplew @xboso, 39
- 40. Why is Probability Important? • It changes the whole conversaVon • “When can you get this done?” • “Depends on how much risk and certainty you want” • What other quesVons does your boss ask that should be answered with a probability distribuVon? 40 @philiplew @xboso,
- 41. Risk Is a Serious Problem ProbabiliVes Are Not in Our Favor • So4ware is increasingly componenVzed, complex and distributed. • Development frameworks ease the complexity of creaVng these applicaVons, but are limited in helping test the applicaVons. • OrganizaVons want to reduce maintenance costs while shortening project life cycles via agile pracVces. • Risk is not decreasing, nor is the risk of catastrophe. @philiplew @xboso, 41 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 42. Why Do We Ignore or Forget Risks 1. Focused on velocity 2. Focused on funcVonality 3. Risk management is too heavy 4. Known or idenVfied risks but no acVon Why do you forget or ignore risks? @philiplew @xboso, 42 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 43. Overconfidence is Natural • We are naturally overconfident (cogniVve bias) • OpVmisVc bias leads us to take risks and underesVmate the odds we face • 80% of small businesses fail in the first 18 months according to Forbes – Surveys indicate that entrepreneurs give themselves a 60% chance of success – To what extent will the outcome depend on what you do? – 80% @philiplew @xboso, 43 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 44. Tendency to Only See Readily ‘Available’ Evidence • Social and economic pressures favor overconfidence • We tend to have an illusion of control – When we esVmate a quanVty, we rely on informaVon that comes to mind – We need to allow for the informaVon that does not come to mind! @philiplew @xboso, 44 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 45. Premortem • To overcome this overconfident opVmisVc cogniVve bias, we can use a premortem. • Enables those who had any concern about the project can freely menVon the concerns. • Great safeguard against groupthink. • Give team members ‘permission’ to be a pessimist. @philiplew @xboso, 45 1. Imagine that a year has passed. 2. Your project has been shown to be an utter disaster 3. Briefly explain (write) why it was a disaster. © 2017 XBOSo4, Inc.- All Rights Reserved.
- 46. Exercise : Pre-Mortem (Small Groups) • ObjecVve – Use collecVve experience and learning in the front-end rather than back-end of a project • Number of players – Divide up into groups of ____ • DuraVon – 5 minutes – “What will go wrong?” – “How will this end in disaster?” • Visualize that you are at the end of the project, and it was a complete failure. IdenVfy why it failed. @philiplew @xboso, 46
- 47.
- 48. The Cone of Uncertainty • Barry Boehm: esVmates may be off by 4x • Decisions decrease uncertainty, but not necessarily… • What can we learn from this? Source: http://www.construx.com/Thought_Leadership/Books/ The_Cone_of_Uncertainty/@philiplew @xboso, 48
- 49. Cone of Learning • To purposefully decrease uncertainty and understand risks, we need to purposefully learn and uncover informaVon. • Based on Edgar Dale’s Cone of Learning • Fastest path to learning, is through CollaboraVon and “Doing the Real Thing”. @philiplew @xboso, 49
- 50. Agile and Risk • With agile methods, teams produce incrementally using iteraVons, called sprints • Each build is planned out and prioriVzed, executed, assessed (issues and improvement, done/not), with review and conVnuous prioriVzaVon as new ‘things’ come up. • At each step/phase/acQvity in the agile process you can focus on what you learned and specific risk viewpoints and miQgaQon acQons. @philiplew @xboso, 50 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 51.
- 52. Risks During Agile Exercise or Not: List out risks that you may encounter; what could go wrong during each phase? 1. During the beginning-planning & product management 2. During the sprint 3. During the retrospecVve @philiplew @xboso, 52 © 2017 XBOSo4, Inc.- All Rights Reserved. Break up in small groups and list out the risks
- 53. Product Management (1) Risk MiQgaQon Delivery delay • Drive towards esVmaVng accurately. • Align expectaVons and what can and cannot be delivered in planning meeVng. • Don’t over promise Even though the applicaVon works, it doesn’t work as it was intended to do. Review backlog user stories to ensure you understand them fully. Scope creep Evaluate exisVng backlog, new items, add/delete and trade out. Personnel loss Ensure full stack team with cross disciplinary skills Forgouen or overlooked risk When developing the backlog explicitly ask to idenVfy the potenVal problems. @philiplew @xboso, 53 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 54. Product Management (2) Risk MiQgaQon ApplicaVon hard to maintain and adapt to changing business requirements- poor underlying structure and architecture In the sprint planning meeVng cover more than what is immediate (to do). • How will this feature be implemented within our exisVng architecture? • Are our esVmates the ‘quick and dirty’ method (increasing debt) or can we do it a beuer way? Forgouen or overlooked • Interview those that are not part of the planning exercise. • IdenVfy the knowable risks when generaVng the iniVal backlog. • Build miVgaVon for risks into the definiVon of done. • Generate stories for less common risks and add them to the backlog. • Review risks when grooming stories • Allocate Vme during planning to idenVfy emerging risks. @philiplew @xboso, 54 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 55. Risk MiQgaQon Planning takes too much Vme and therefore doesn’t get done, or done well/completely. • Ensure a clear prioriVzaVon process • Ensure that planning and grooming sessions have clear output EsVmaVon process is not clear, not standard and not oriented toward improvement • Ensure clear acceptance criteria within the user story and the definiVon of done. Stakeholders say, “I didn’t know that” or don’t officially ‘approve’. • Develop guidelines for documentaVon, tracking and reporVng to allow visibility and transparency of all project aspects Delays due to who decides as the ulVmate authority on a feature-or other decision • Setup guidelines for agile movement, for decision making if someone is hit by the bus Product Management (3) @philiplew @xboso, 55 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 56. Sprint (1) Risk MiQgaQon • TesVng incomplete, or rushed at the end • Implement test driven development • Include tesVng in the definiVon of done • DemonstraVons at the end of the sprint not conclusive (done or not?) • Insufficient integraVon, or integraVon too late • Review stories to ensure acceptance criteria are clear • DefiniVon of done includes tesVng • DefiniVon of done includes integraVon • Make user stories’ done/acceptance criteria clear • API automated tesVng • We deliver product and find out we have some security and performance issues which require architectural changes • Define measurements for many viewpoints on improvement, not just velocity. Maintain velocity and focus on form. • Periodically test non funcVonal requirements as part of definiVon of done. • Include performance criteria in definiVon of done • Performance geRng worse and worse, works but very slow • Implement performance tesVng criteria and scenario tesVng for most common user scenarios @philiplew @xboso, 56 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 57. Sprint (2) Risk MiQgaQon Risks in the quality of a delivered applicaVon. Even though the applicaVon works, it doesn’t work as well as it should. • Maintainability • Usability Therefore, cannot fulfill what it was intended to do. DefiniVon of done includes some level of predefined non-funcVonal tesVng, where comparisons are done including: • Code reviews • HeurisVc and observaVonal usability tesVng before its ALL done, along the way @philiplew @xboso, 57 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 58. Risk MiQgaQon-Items to Examine Risks forgouen or unknown • Gather risk data though surveys when the program stakeholders are geographically diverse. • Interview customers or potenVal customers. Feature turned out too hard to implement or took too much Vme Ask these quesVons: • How was the feature implemented? Did we make it harder than we needed to? • Did we make any architecture decisions from before that impeded this feature • Is the underlying structure and architecture able to meet future needs Scope creep Examine unplanned work, check esVmates Personnel loss Review issues created due to lack of skills or people RetrospecVve (1) @philiplew @xboso, 58 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 59. Risk MiQgaQon Delivered what we could, but did a work around • Develop a user story to do it to match with overall architecture • Examine exisVng architecture Too difficult to implement a funcVon, took much longer than we thought • Walkthrough design consideraVons on periodic basis • Ensure team skills are up to par • Ensure clear definiVon of done • For tough features, increase collaboraVon regarding detailed design • Examine esVmaVng errors IntegraVon too late, product didn’t work together, just pieces • Define and implement conVnuous integraVon process • Integrate daily RetrospecVve (2) @philiplew @xboso, 59 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 60.
- 61. Risk MiQgaQon RetrospecVve not held or was not producVve • Led by an independent facilitator, could be another team or execuVve • Held in a safe place –quiet, not disturbed • Not a witch hunt –Focus on process rather than people • Plan the event –send out notes on potenVal problems to get people thinking • Have the right people involved • Record results in open area, allocate and monitor acVons RetrospecVve (4) @philiplew @xboso, 61 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 62. Retro (5) Risk MiQgaQon Confusion of roles and responsibiliVes, something is forgouen or falls off the plate that was important • InsVll Primary ownership, even with 2 team-mates • Analyze items that fell off previously to determine esVmaVon errors • Examine all unplanned work-Necessary? Time crunch at the end, not enough Vme to get done • InsVll in daily standups - address delays Can’t deliver what we thought, too complex or we didn’t understand well enough • Daily standups address esVmated points for a story, re-assess and split up • For increased story points, product owner feedback required @philiplew @xboso, 62 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 63.
- 64.
- 65. Classical Risk Management • IdenVfy • Evaluate • PrioriVze and plan • MiVgate There are even standards on on risk management. Risk Management Planning Risk Assessment Risk MiQgaQon Monitor & Review Risks @philiplew @xboso, 65 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 66.
- 67.
- 68. Issues in Managing Risk 1. What can you control and what can you not? 2. What can you prepare for and what not? 3. What should you prepare for and what not? 4. What else? @philiplew @xboso, 68 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 69.
- 70.
- 71.
- 72.
- 73. Risk Census Example Risk Probability of Risk Size of Loss (Days) Risk exposure Feature as implemented doesn’t saVsfy the customer/ user, must do over again or revise 20% 15 3 Historical data access and quality of data 30% 20 6 Chosen development plarorm, lack of internal skills and hard to find people 50% 20 10 Workarounds used but sacrifice long term viability and flexibility 60% 60 36 Crunched at the end. Only got done half of the tesVng 90% 15 4.5 Performance criteria not met at delivery 20% 30 6 Poor communicaVons with off-shore team (Vme difference) 40% 5 2 Total exposure 67.5 @philiplew @xboso, 73 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 74.
- 75.
- 76.
- 77.
- 78. Summary Manage Risk With The End in Mind So4ware Quality • To manage risk in our so4ware projects, we must consider the quality (or lack of [risk]) from all viewpoints and blindspots • Take advantage of all the collaboraVon, checks and balances within the agile process to build miVgaVon within your agile process at each step – Review the course before the race – Learn at each step and review with risk mindset • Keep your eye on the ball – So4ware Quality @philiplew @xboso, 78 © 2017 XBOSo4, Inc.- All Rights Reserved.
- 79. Thanks QuesVons and Answers Philip Lew philip.lew@xbosoft.com Some resources:http://www.xbosoft.com/knowledge_center/ www.xbosoft.com @philiplew @xboso, Are You Standing on Solid Ground? 79 © 2017 XBOSo4, Inc.- All Rights Reserved. Slides will be posted on slideshare. Follow XBOSoft on LinkedIn and be notified.