The pressure to deliver faster to the market has never been more insistent and pervasive than today’s business environment. The Agile world of iterative and incremental delivery has enabled great advances in terms of delivery speed; however, the lack of an integrated risk framework is creating challenges in terms of matching speed with quality. On the one hand, the standards-setting organizations such as the Project Management Institute (PMI) have updated their book of knowledge (PMBOK v7) to move away from highly prescriptive processes to lean thinking. On the other hand, Agile standards themselves have started to emerge, recognizing the need for some prescriptive guidelines on coming up with release and iteration goals. Struggling in between this continuum are the innovative technology projects that wonder how “creativity can be timeboxed” to deliver value! While the impact of leadership to form the team and the organizational culture to embrace continuous learning are unquestionable, it is important to realize that the areas of strategy, leadership, and culture are not substitutes for the lack of risk-based project thinking. When delivering IT applications that are contain inherent conceptual, technical, and compliance risks, a more systematic approach is needed. In this presentation, you will hear about the emerging space of IT initiatives that are impacted by such risks and the need to adopt risk-based frameworks in application lifecycle management. You will also see practical examples of how risk-based lifecycle management can be done in real-time.

1. ®
Rethinking Risk-Based Project
Management in the Emerging
IT initiatives
Presented by
Adam Sandman
Northeastern University Annual Leadership &
Project Management Symposium, 2022

2. ®
®
About Me
 Adam Sandman was a programmer
from the age of 10 and has been
working in the IT industry for over
25 years.
 Currently Adam is the Founder and CEO
of Inflectra Corporation, where he is
interested in technology, business and
innovation.
 Adam lives in Washington, DC, USA
twitter: @adammarksandman
linkedin: https://www.linkedin.com/in/adamsandman/

3. ®
3 | 9/25/2022 © Copyright 2006-2021 Inflectra Corporation
Agenda
Background
 Agile Overview
 Risk Management
Risks in Software Projects
Risks in Software Testing
How Does ML/AI Change Things?
Questions?

4. ®
Background

5. ®
Agile Overview

6. ®
®
How Agile Changed Project Management
Requirements
Gathering
Business
Design
Unit
Testing
Traditional Waterfall Methodology
Release Complete
Technical
Design
Systems Development
Integration
Testing
Acceptance
Testing
(UAT)
Limited
Upfront
Planning
Release #1 Release #2
Agile Methodology
It 1 It 2 It 3 It 4 It 5 It 6
Release #3
It 7 It 8 It 9
Release #4
It 10 It 11 It 12
Iteration N
Iteration
Planning
Release Complete
Development, Continuous Integration, Unit Testing,
Functional & Acceptance Testing
The agile manifesto challenged the orthodoxy around software development, freeing it
from the tyranny of process and rigidity, and embracing user feedback and change.

7. ®
®
Agile – Led to Continuous Delivery
Iteration
Iteration
Scope
Tested,
Integrated
Working
System
Iteration
Plan
Release Plan
Organizations quickly realized how much more efficient they could be having integrated
teams that collaboratively define the requirements, plan the releases and sprints, test the
product during development and deploy the latest update in a seamless way

8. ®
What is Risk Management?

9. ®
9 | 9/25/2022 © Copyright 2006-2021 Inflectra Corporation
®

10. ®
10 | 9/25/2022 © Copyright 2006-2021 Inflectra Corporation
®

11. ®
11 | 9/25/2022 © Copyright 2006-2021 Inflectra Corporation
®

12. ®
®
What are Risks?
 Risk is a concept that describes a possible future event that
might happen at some point in the future.
 Compare with an issue which is something that is occurring now
 Risks can be positive – an uncertain event that benefits you
in the future, e.g. winning the lottery
 Generally Risks are considered negative – an uncertain
event that could be detrimental to the success of the
project, e.g. the system will not handle the possible future
user load.

13. ®
®
Identifying Risks using a Breakdown
 Use a hierarchical Risk Breakdown Structure to identify
“all” risks that may be on a project:

14. ®
How “Bad” Are The Risks?

15. ®
®
One Measure: Probability
 How likely it will be that the risk will happen

16. ®
®
Another Measure: Impact
 How serious it will be if the risk happens.

17. ®
®
Putting Them Together: Exposure
 This is a measure of how exposed you are to a specific risk
Exposure = Probability X Impact:

18. ®
Reducing Risks: Mitigations

19. ®
®
Introducing Mitigations
 Actions we can take to reduce either probability or impact:
after mitigation

20. ®
®
Integrating Risks into Project Management
 Too often risk management is its own ‘discipline’ isolated
away from the project management office and the teams.
 Risks are in reality integral to the project:
 Risks mitigations result in tasks that need to be in the plan
 Risks affect requirements and need to be accounted for
 Tests can be done to understand the severity of the risks

21. ®
Risks in Software Projects

22. ®
®
Types of Risks on Software Projects
 Conceptual
 Are we building the right thing?
 Technical
 Will it technically work?
 Schedule
 Will it be delivered ‘on-time’
 Agile Can Help!
Conceptual
Technical
Schedule

23. ®
®
Reduce Risks with Agile
 Conceptual Risk – What? Why?
 Deliver small increments & get feedback
 Stakeholders can “see” something tangible, grasp the concept
 Technical Risk – How?
 Use ‘spike’ solutions to provide out the technical risks early
 Change course before too much has been invested
 Schedule Risk – When?
 Release in small increments, adjust scope vs. dates
 Often teams fixate on these risks vs. the other ones
In Agile projects, it is a team job to manage these risks!

24. ®
Agile Itself is Not a Panacea!

25. ®
®
Schedule Risks
 Agile teams unfortunately often focus solely on mitigating
these schedule risks.
 Timeboxing iterations/sprints to 2-weeks reduces the
chance that a risk will materialize in that time period.
Limited
Upfront
Planning
Release #1 Release #2
Agile Methodology
It 1 It 2 It 3 It 4 It 5 It 6
Release #3
It 7 It 8 It 9
Release #4
It 10 It 11 It 12
Iteration N
Iteration
Planning
Release Complete
Development, Continuous Integration, Unit Testing,
Functional & Acceptance Testing

26. ®
®
Conceptual & Technical Risks
 Conceptual Risk – What? Why?
 You can only mitigate these risks if you gather real feedback.
 You need to let users see how the product actually looks/works
 They need to be able to validate the concept through practical
usage
 You need to carefully choose user stories/epics that illustrate
sufficient aspects of the concept.
 Technical Risk – How?
 You need to plan for real “spike” iterations/sprints
 Developers are often wedded to solutions before they are proved
 Need to spend time identifying possible technical risks and make
time in the plan for enough spike sprints.

27. ®
®
Real-Word Examples
 The Un-Tethered Architecture for the US Military
 Technical Risk
 The original launch of the iPhone
 Conceptual Risk
 Technical Risk
 The Lean Startup
 Conceptual Risk
 Windows 8
 Conceptual Risk

28. ®
What About Compliance Risks?

29. ®
®
Bridging Between Agile and Compliance
Support for Regulated Processes Modern, Agile Processes
• Baselining
• Risk Management
• Electronic Signatures
• Data Privacy
• Compliance
• Workflow
• Requirements Mgt
• On-Premise
• CI / CD
• Scrum
• Kanban
• Scaled Agile
• DevOps
• Distributed Teams
• Remote Working
• Cloud-Based
Genius of the “AND”

30. ®
Risks in Hardware Projects
Failure Mode & Effects Analysis (FMEA)

31. ®
®
How Does FMEA Change Risks
 When using an FMEA approach/methodology, there are
actually three key measures:
 Probability - how likely the risk is to happen (1-5)
 Impact - how severe the risk will be if that happens (1-5)
 Detectability - how easy it is to observe the risk (1-5)
Risk Priority Number (RPN) =
Impact X Probability X Detectability

32. ®
®
How Does FMEA Change Risks
 When using an FMEA approach/methodology, there are
actually three key measures:
 Probability - how likely the risk is to happen (1-5)
 Impact - how severe the risk will be if that happens (1-5)
 Detectability - how easy it is to observe the risk (1-5)

33. ®
Risks in Software Testing
Risk Based Testing

34. ®
Testing = Manage Technical Risk

35. ®
®
There Is Never Enough Time for Testing
How do we decide what testing should be done?
How do we decide what tests should be run?
What tests can we perform to reduce overall risk?

36. ®
Solution: Risk Based Testing

37. ®
®
Create Test Cases to Address Risks
 Starting with the highest exposure risks, create (or prioritize
existing) tests that explicitly verify and mitigate the most
exposed identified risks:
Test Name
1 PCI Security Testing
2 Sample integration test
3 Test DB Volume with 2 GB
4 Usability test book pages
5 A/B test pricing pages

38. ®
®
Prioritize Tests that Touch Multiple Risks
 Map Tests to Risk, if a test covers multiple risks, then it’s
the highest priority:
Risk Name Probability
Impact Exposure TC1 TC2 TC3 TC4 TC5 TC6 TC7 TC8 TC9 TC10
The v1.1 release may not be ready in time Likely Critical 12
We may not get enough authors to sign up Unlikely Catastrophic
8
The database may not support the volume Unlikely Critical 6
The book pages may not be easy enough to use Possible Marginal 6
The software licenses may be too expensive Unlikely Marginal 4
The client team may not be ready for UAT Rare Critical 3
Risk that the system will break Likely Marginal 8
The system will not comply with PCI-DSS 4.0 Possible Critical 9
Risk that the integration system will not work for our product Likely Critical 12
Tests

39. ®
Risks in Cyber Security
A Risk-Based Approach to Security

40. ®
40 | 9/25/2022 © Copyright 2006-2021 Inflectra Corporation
®
Real World Example – Inflectra®

41. ®
How Does ML/AI Change
Things?
Who Checks the Machines?

42. ®
®
How Does AI Affect Risks in Projects?
1. AI can avoid bias in traditional risk management
(hindsight bias, confirmation bias, etc.) and find ‘zero-day’
risks.
2. AI can assist in mining data to better analyze and rate
risks in terms of exposure and RPN metrics.
3. AI can introduce new risks since it uses non-deterministic
algorithms and the boundary conditions and edge cases
are often not well understood
4. How do we “test” or validate an AI model in the classic
sense of testing?

43. ®
®
Putting It All Together…

44. ®
44 | 9/25/2022 © Copyright 2006-2021 Inflectra Corporation
Wrap Up & Final Thoughts
 Regardless of your role (developer, business analyst, project
manager, tester), risk management is core discipline
 Understand how to integrate risk management practices
into your projects practically, take baby steps
 Risk Management can be integrated into software
development, testing, hardware projects & cybersecurity
 AI and ML have the ability to radically change the risk
profile of our projects, both for better and worse!

45. ®
45 | 9/25/2022 © Copyright 2006-2021 Inflectra Corporation
®
Acknowledgements
 I would like to thank Dr. Sriram
Rajagopalan, who is a project
management and leadership program
faculty member.
 Dr. Rajagopalan was important in
bringing the risk management rigor to
Inflectra and our Spira products. He also
works as an Enterprise Agile Evangelist
for Inflectra.

46. ®
46 | 9/25/2022 © Copyright 2006-2021 Inflectra Corporation
Questions?