Agile Project Risk - An Organizational View - Webinar Slides
•
1 like•265 views
Join Philip Lew from PNSQC and Yingki Kwong from PPLF (Portfolio and Project Leaders Forum) as they discuss Agile Risk and Uncertainty. Agile is designed to handle uncertainty in requirements as new features are requested and their priorities shift in real time. Agile sprints produce frequent software releases based on direct input from the business. This tight coupling with the business enables, in theory, early detection of defects in requirements and designs; as high level user stories / scenarios are elaborated to produce detail requirements that support design, development, and implementation. However, in chasing agility, projects often ignore or can only poorly understand the uncertainties and associated project risks introduced by important Agile processes, such as the backlog. With relentless sprints, it is easy to view completed sprints as a proxy for progress. The risk trap is poor understanding of the probability and impact of the actual project risks associated with implementing certain user stories incorrectly (scope / quality risks) and actual velocity falling short of the expected (schedule / budget risks). A key question is: how do we mitigate the uncertainties that are introduced by using Agile? Phil Lew suggests that an important problem is that we sometimes carry assumptions which either cause us to spend too much effort on things we can’t control or give us unfounded comfort and reassurance. If we can’t understand the uncertainties and risks, how can we have confidence in our software as systems become more complex? Phil supplements classic risk management techniques with a lifecycle approach on risk and uncertainty to identify and address the uncertainties that matter—and those that don’t. Then Phil outlines methods that you can use to address these risks while maintaining rhythm in your agile software processes. Come and learn about risks you never thought of and see how you can manage or avoid them.
Recommended
Recommended
More Related Content
Similar to Agile Project Risk - An Organizational View - Webinar Slides
Similar to Agile Project Risk - An Organizational View - Webinar Slides (20)
Recently uploaded
Recently uploaded (20)
Agile Project Risk - An Organizational View - Webinar Slides
- 2. • Senior leaders of project-based organizations • Discuss common problems and solutions in project portfolio management and project office governance • From a wide spectrum of commercial industries, government agencies, and not-for-profit organizations • Contact Jeff Oltmann, jeff@spspro.com @philiplew @pnsqc 2
- 3. PNSQC Pacific Northwest Software Quality Conference • The oldest software quality conference in North America now in its 37th year. • Annual conference held in Portland, Oregon • Non-profit organization run by volunteers. • PNSQC is a hybrid professional conference that focuses on the software quality practitioner. • Conference speakers and participants are people sharing their own experiences in both success and failure (not by consultants nor vendors). • A range of topics and speakers - everything from management methods and distributed teams to agile, automation, Devops and security. 3
- 4. Our Speaker Today Philip Lew PNSQC Board Member, Program Chair, and Speaker • CEO, XBOSoft • Relevant specialties and passions o Software quality process, evaluation, measurement and improvement o Software quality in use / UX design o Mobile User Experience and usability o Cycling and travel o He never learned how to read 4
- 5. Our Moderator Today @philiplew @pnsqc 5 Ying Ki Kwong PNSQC Volunteer and Speaker PPLF Member • Statewide QA Program Manager, Office of the State CIO, State of Oregon • Relevant specialties and passions o Quality & risk management o Enterprise IT project management o Complex systems and complexity o Volunteering for local nonprofits o Travel
- 6. Background for Today § At PNSQC 2018, Phil & Ying Ki presented a paper on quality & risk management, from the perspective of enterprises acquiring systems. § https://www.pnsqc.org/quality-risk-management-challenges-when- acquiring-enterprise-software-systems/ § Various subjects were covered, with some attention on Agile. § Feedback received about the community’s need for improved general understanding around where Agile meets enterprise quality & risk management. 6 So, today’s talk by Phil co-sponsored by PPLG and PNSQC.
- 7. House Rules § Participants other than the speakers are muted § Questions via the Zoom control panel or through Twitter @PNSQC § Questions may be asked throughout the webinar - we’ll try to answer them at the end § You will receive info on recording and slides after the webinar 7 Webinar Hashtag: #AgileRisk Now the featured speaker...
- 9. Agenda (1) Agile Risks Reasons for failure Agile process Risk types Uncertainty and probabilities 9
- 11. What Got Us Here • Smaller teams • Faster iterations • Listening to the user • Continuous beta • Data collection & analytics 11 • Communication/Collaboration • Full stack teams • Retrospectives • Analysis, adaption and improvement 1. Changes in technology (mobile, cloud) 2. Changes in business models 3. Many failures… @philiplew @pnsqc
- 12. Agile and Risk • Chasing agility and velocity à ignore risks inherently introduced by Agile • Mistake completed sprints for progress • Velocity attained but... Still don’t deliver what is wanted and needed (MVP). • Backlog mismanagement • User stories (especially for complex features) not detailed enough for management to ascertain that features-functions will meet business needs. • Hard to implement, yet critical or complex features not implemented (stay delayed in backlog) or incorrectly implemented. @philiplew @pnsqc 12
- 19. Technology Risks • Platform can’t adapt to changing needs • Too hard to implement, too complicated with current technology stack • Doesn’t integrate well with other 3rd party tools • Can’t find people who have the right skills • What else in terms of technology risks? @philiplew @pnsqc 19
- 20. Product Risks • Delivered too late, 1st mover or even 2nd mover advantage dissipated • Doesn’t do what the customer/user wants • We missed the mark, it does what we wanted but… • Too slow • Harder to do than we thought • Can’t deliver sought after/differentiator features/ functions • “Move it to the backlog for now…” • ______ @philiplew @pnsqc 20
- 21. Process Risks • Too slow • Not repeatable • Delicate and FrAgile – Easily broken • Not adaptable • Not consistent • Too dependent on a superhero @philiplew @pnsqc 21
- 25. Fat and Narrow Tails Make a Difference • What risks are fat and narrow tailed? • With a fat tailed risk you can be catastrophically wrong. • What would be “Ruin” when it comes to software development? @philiplew @pnsqc
- 26. Why It’s Important to Understand Normal and Fat • Traditional risk management strategies rely on and assume a normal bell curve but in reality, projects don't behave this way. • What things are normal and what types of occurrences are not normal (don’t conform to normal distribution)? • Height of people in the room @philiplew @pnsqc 26
- 27. Normal Exercise – Entire Class • Normal distributions vary in the neighborhood of its average, but have very few data points beyond 3 standard deviations from that average. • Do a survey of the class and calculate the average height of people in the room for men, and for women. • Average height of men is 5’10” with standard deviation of 4”. • 3 standard deviations - Chances of a 6’10” man are very small. @philiplew @pnsqc 27
- 28. Fat Tail Exercise – Small Groups • http://www.boxofficemojo.com/genres/chart/? id=documentary.htm • Calculate the average and standard deviation for these documentary movies revenue. • What are the differences you observe between documentary movies and people’s heights? • How can you apply this to your work in determining what is a risk and not, what risks to pay attention to and NOT. @philiplew @pnsqc 28 average $11,393,365.45 standard deviation $17,319,262.05
- 29. In the End, We Want to • Find problems early • Find big problems • Avoid problems altogether • **Problems by nature have negative consequences • Our job is to decrease this probability • Agile helps us with this, but we need to focus on those activities with the right viewpoint to gain this insight @philiplew @pnsqc 29
- 31. • Software is increasingly componentized, complex and distributed à hidden risks. • Development frameworks ease complexity in creating software, but are limited in helping test the applications. • Organizations want to reduce maintenance costs while shortening project life cycles via agile practices. Faster is not safer… • Risk is not decreasing, nor is the risk of catastrophe. @philiplew @pnsqc 31 Risks Are Not In Our Favor
- 32. Why Do We Ignore or Forget Risks 1. Focused on velocity 2. Focused on functionality • Do the easy stuff first 3. Risk management is too heavy 4. Known or identified risks but no action Why do you forget or ignore risks? @philiplew @pnsqc 32
- 33. Overconfidence is Natural • We are naturally overconfident (cognitive bias) • Optimistic bias leads us to take risks and underestimate the odds we face • 80% of small businesses fail in the first 18 months according to Forbes • Surveys indicate that entrepreneurs give themselves a 60% chance of success • To what extent will the outcome depend on what you do? – 80% @philiplew @pnsqc 33
- 34. Tendency to Only See Readily ‘Available’ or Latest Evidence • Social and economic pressures favor overconfidence • We tend to have an illusion of control • When we estimate a quantity, we rely on information that comes to mind • We need to allow for the information that does not come to mind! • We weight evidence of most recent experiences more than we should. • We want to believe and make ourselves right. @philiplew @pnsqc 34
- 35. Premortem • To overcome this overconfident optimistic cognitive bias, we can use a premortem. • Enables those who had any concern about the project can freely mention the concerns. • Great safeguard against groupthink. • Give team members ‘permission’ to be a pessimist. @philiplew @pnsqc 35 1. Imagine that a year has passed. 2. Your project has been shown to be an utter disaster 3. Briefly explain (write) why it was a disaster.
- 37. Risks During Agile Exercise (or Not): List out risks that you may encounter; what could go wrong during each phase? 1. During the beginning-planning & product management 2. During the sprint 3. During the retrospective @philiplew @pnsqc 37 Break up in small groups and list out the risks
- 38. Product Management (1) Risk Mitigation Delivery delay • Drive towards estimating accurately. • Align expectations and what can and cannot be delivered in planning meeting. • Reduce/manage expectations. Even though the application works, it doesn’t work as it was intended to do. Review backlog user stories to ensure you understand them fully. Scope creep Evaluate existing backlog, new items, add/delete and trade out. Personnel loss Ensure full stack team with cross disciplinary skills Forgotten or overlooked risk When developing the backlog explicitly ask to identify the potential problems. @philiplew @pnsqc 38
- 39. Product Management (2) Risk Mitigation Application hard to maintain and adapt to changing business requirements- poor underlying structure and architecture In the sprint planning meeting cover more than what is immediate (to do). • How will this feature be implemented within our existing architecture? • Are our estimates the ‘quick and dirty’ method (increasing debt) or can we do it a better way? Forgotten or overlooked • Interview those that are not part of the planning exercise. • Identify the knowable risks when generating the initial backlog. • Build mitigation for risks into the definition of done. • Generate stories for risks and add them to the backlog. • Specifically allocate time during planning to identify emerging risks. @philiplew @pnsqc 39
- 40. Risk Mitigation Planning takes too much time and therefore doesn’t get done, or done well/ completely. • Ensure a clear prioritization process • Ensure that planning and grooming sessions have clear output Estimation process is not clear, not standard and not oriented toward improvement • Ensure clear acceptance criteria within the user story and the definition of done. Stakeholders say, “I didn’t know that” or don’t officially ‘approve’. • Develop guidelines for documentation, tracking and reporting to allow visibility and transparency of all project aspects Delays due to who decides as the ultimate authority on a feature-or other decision • Setup guidelines for agile decision making. Product Management (3) @philiplew @pnsqc 40
- 41. Sprint (1) Risk Mitigation • Testing incomplete, or rushed at the end • Implement test driven development • Include testing in the definition of done • Demonstrations at the end of the sprint not conclusive (done or not?) • Insufficient integration, or integration too late • Review stories to ensure acceptance criteria are clear • Definition of done includes testing and integration • API automated testing • We deliver product and find out we have some security and performance issues which require architectural changes. • Worsening performance • Define measurements for many viewpoints on improvement, not just velocity. • Periodically test non functional requirements as part of definition of done. • Include performance criteria in definition of done • Implement performance testing criteria and scenario testing for most common user scenarios @philiplew @pnsqc 41
- 42. Sprint (2) Risk Mitigation • Poor quality of a delivered application. Even though the application works, it doesn’t work as well as it should (i.e. usability, security, performance). • Cannot fulfill what it was intended to do. Definition of done includes some level of predefined non-functional testing, where comparisons are done including: • Code reviews • Heuristic and observational usability testing before its “Done” and too late. @philiplew @pnsqc 42
- 43. Risk Mitigation-Items to Examine Risks forgotten or unknown • Gather risk data though surveys when the program stakeholders are geographically diverse. • Interview customers or potential customers. Feature turned out too hard to implement or took too much time Ask these questions: • How was the feature implemented? Did we make it harder than we needed to? • Did we make any architecture decisions from before that impeded this feature • Is the underlying structure and architecture able to meet future needs Scope creep Examine unplanned work, check estimates Personnel loss Review issues created due to lack of skills or people Retrospective (1) @philiplew @pnsqc 43
- 44. Risk Mitigation Delivered what we could, but did a work around • Examine existing architecture Too difficult to implement a function, took much longer than we thought • Walkthrough design considerations on periodic basis • Assess team skills • Ensure clear definition of done • For tough features, increase collaboration regarding detailed design • Examine estimating errors Integration too late, product didn’t work together, just pieces • Define and implement continuous integration process • Integrate daily Retrospective (2) @philiplew @pnsqc 44
- 45. Risk Mitigation Lack of performance, load and security testing until the end Build in non-functional testing as a user story for periodic execution, i.e. once/ month • Didn’t get ‘Done’ • Customer not satisfied with the feature although we did what he said/in the user story • Review user stories for ambiguity and completeness • Ambiguous assumptions and responsibilities • User stories’ done criteria insufficient • User stories too big Too much stuff in the sprint, didn’t finish, many things added in during the sprint Define method and logic to add/subtract from the sprint during the sprint Retrospective (3) @philiplew @pnsqc 45
- 46. Risk Mitigation Retrospective not productive • Led by an independent facilitator, could be another team or executive • Held in a safe place –quiet, not disturbed • Not a witch hunt –Focus on process rather than people • Plan the event –send out notes on potential problems to get people thinking • Have the right people involved • Record results in open area, allocate and monitor actions Retrospective (4) @philiplew @pnsqc 46
- 47. Retrospective (5) Risk Mitigation Confusion of roles and responsibilities, something is forgotten or falls off the plate that was important • Instill Primary ownership, even with 2 team-mates • Analyze items that fell off previously to determine estimation errors • Examine all unplanned work- Necessary? Time crunch at the end, not enough time to get done • Instill in daily standups - address delays Can’t deliver what we thought, too complex or we didn’t understand well enough • Daily standups address estimated points for a story, re-assess and split up • For increased story points, product owner feedback required @philiplew @pnsqc 47