Software projects are known to have challenges with estimation, uncertainty, risk, and commitment—and the most valuable projects often carry the most risk. Other industries also encounter risk and generate value by understanding and managing that risk effectively. Todd Little explores techniques used in a number of risky businesses—product development, oil and gas exploration, investment banking, medicine, weather forecasting, and gambling—and shares what those industries have done to manage uncertainty. With studies of software development estimations and uncertainties, Todd discusses how software practitioners can learn from a better understanding of uncertainty and its dynamics. In addition, he introduces techniques and approaches to estimation and risk management including utilizing real options and one of its key elements—understanding commitment. Take away a better understanding of the challenges of estimation and what software practitioners can do to better manage estimation, risks, and their commitments.

1.
MD
Half‐day Tutorial
6/3/2013 8:30 AM
"Dealing with Estimation,
Uncertainty, Risk, and Commitment"
Presented by:
Todd Little
Landmark Graphics Corporation
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888‐268‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Todd Little
Landmark Graphics Corporation
Todd Little is a senior development manager for Landmark Graphics. Todd has been involved
in most aspects of software development with a focus on commercial software applications for
oil and gas exploration and production. He is a coauthor of the Declaration of Interdependence
and a founding member and past president of the Agile Leadership Network. Todd has served
on the board of directors of both the Agile Alliance and the Agile Leadership Network, is a
coauthor of Stand Back and Deliver: Accelerating Business Agility, has written several articles
for IEEE Software, and posts all his publications and presentations on his
websitetoddlittleweb.com.

3. Dealing with Estimation, Uncertainty,
Risk, and Commitment :
An Outside-In Look at Agility and Risk Management
Todd Little
Sr. Development Manager
Landmark Software & Services

4. Agenda
•
•
•
•
•
•
Introduction
Exercise
Estimation and Uncertainty
Exercise
Break
Risk Management, Real Options and
Commitment
• Exercise
• Summary

5. A Game?

6. Form into teams.
Pick a name...
Shout out the name.

7. On a piece of paper, write
1. Your Name
2. Your Team Name
3. Team to complete task A
first
4. Team to complete task B
first

8. Create a six sided random number generator.
Something like a dice like thingy.
Every member of the team needs to roll a double six.
http://www.flickr.com/photos/palooza/3309242555/
task “A”...

9. Create a four sided random number generator.
Something like a dice like thingy.
Every member of the team needs to roll a double six.
http://www.flickr.com/photos/palooza/3309242555/
task “B”...

10. Hurricane Rita

11. Risk and Uncertainty
Risk
Uncertainty
Frank Knight
Immeasurable
Quantifiable
PMI
Risk can be positive or
negative
???
English
A situation involving
exposure to danger
The state of being uncertain
Not known or established;
questionable

12. Delivery Challenges/Failures
Succesful
35%
Failed
19%
Challenged
46%
Standish Group 2006, reported by CEO Jim Johnson, CIO.com, ‘How to Spot a Failing Project’

13. Managing the Coming Storm
Inside the Tornado
Project Kickoff
When will we get the requirements?
All in good time, my little pretty, all in good time
But I guess it doesn't matter anyway
Just give me your estimates by this afternoon
Team Unity
Not so fast! Not so fast! ... I'll have to give the matter a little
thought. Go away and come back tomorrow
No, we need something today!
Ok then, it will take 2 years.
No, we need it sooner.
Doesn't anybody believe me?
I already promised the customer it will be out in 6 months
You're a very bad man!

14. We’re not in Kansas Anymore
Developer Hero
I may not come out alive, but I'm goin' in there!
Reorg
The Great and Powerful Oz has got matters well in hand.
My! People come and go so quickly here!
Testing
"Hee hee hee ha ha! Going so soon? I wouldn't hear
of it! Why, my little party's just beginning!

15. Why is Software Late?
Genuchten 1991 IEEE
General
Manager
Project
Manager
1
10
Insufficient front end planning
2
3
Unrealistic project plan
3
8
Project scope underestimated
4
1
Customer/management changes
5
14
Insufficient contingency planning
6
13
Inability to track progress
7
5
Inability to track problems early
8
9
Insufficient Number of checkpoints
9
4
Staffing problems
10
2
Technical complexity
11
6
Priority Shifts
12
11
No commitment by personnel to plan
13
12
Uncooperative support groups
14
7
Sinking team spirit
15
15
Unqualified project personnel
Item

16. The Context of Feedback

17. Why is Software Late?
Genuchten 1991 IEEE
General
Manager
Project
Manager
4
1
Customer/management changes
10
2
Technical complexity
2
3
Unrealistic project plan
9
4
Staffing problems
7
5
Inability to track problems early
11
6
Priority Shifts
14
7
Sinking team spirit
3
8
Project scope underestimated
8
9
Insufficient Number of checkpoints
1
10
Insufficient front end planning
12
11
No commitment by personnel to plan
13
12
Uncooperative support groups
6
13
Inability to track progress
5
14
Insufficient contingency planning
15
15
Unqualified project personnel
Item

18. Why is Software Late?
Genuchten 1991 IEEE
General
Manager
H
L
H
M
H
Project
Manager
H
H
H
H
L
Item
Customer/management changes
Overall complexity
Unrealistic project plan
Staffing problems
Insufficient front end planning

19. The Goal
Time
To Spec
Within Budget
On

20. Da Plan, Boss – Da Plan

21. IEEE Software, May/June 2006

22. Accuracy of Initial Estimate
Actual
Initial Estimate vs. Actual Duration
Ideal
LGC Data
DeMarco
Initial Estimate

23. Probability Distribution Curve
Distribution Curve of Actual/Estimated (DeMarco data vs. LGC)
(Demarco data is Effort/Effort; LGC data is Duration/Duration)
2.5
2
Frequency
1.5
DeMarco
LGC
1
0.5
0
0
1
2
3
4
(Actual/Estimated)
5
6
7
8

24. Log Normal Distribution
• Estimation Accuracy follows a Log Normal
distribution
Median
Mean
0.1
1
10
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5

25. Uncertainty Bounds
Cumulative Distribution Function of Actual/Estimate Ratio
100%
90%
80%
CDF Probability
70%
DeMarco Data
DeMarco Log-Normal
Landmark Data
Landmark Log-Normal
60%
50%
40%
30%
20%
10%
0%
0.1
1
Ratio of Actual/Estimate
10

26. How does Estimation Accuracy
Improve Over Time?
Cone of Uncertainty from Boehm
Relative Cost Range
4.0
2.0
1.5
1.25
1.0
0.8
0.67
0.5
0.25
Feasibility
Concept of
Operation
Requirements Product Design Detail Design
Spec
Spec
Spec
Accepted
Software

27. Landmark Cone of Uncertainty
Estimation Error over Time
Acutal Total Duration / Estimated Total Duration
10
1
0.1
0.00
0.10
0.20
0.30
0.40
0.50
0.60
Percent of Actual Duration
0.70
0.80
0.90
1.00

28. But is Uncertainty Really Reduced?
“Take away an ordinary person’s illusions and
you take away happiness at the same time.”
Henrik Ibsen--Villanden

29. Remaining Uncertainty

30. Remaining Uncertainty

31. Sources of Error
• Bias
• Uncertainty Range
• Scope Creep

32. Sources of Error
• Bias
Median
Mean
Estimate
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5

33. Negotiation Bias
• "It is difficult to get a man to
understand something when his
salary depends upon his not
understanding it.“
» Upton Sinclair:

34. Politics and Schedule Estimation
• Developers tend to be temperamentally opposed
to the use of negotiating tricks. Such tricks
offend their sense of technical accuracy and fair
play. Developers don't want to offer lopsidedly
high initial estimates even when they know that
customers, marketers, or bosses will start with
lopsidedly low bargaining positions.
– Steve McConnell
http://www.stevemcconnell.com/ieeesoftware/bp03.htm

35. Space Shuttle Challenger
Engineers
Probability of loss of life
Management
1 in 100
1 in 100,000

36. Overconfidence in Ranges
• Most people are significantly overconfident about
their estimates, especially educated professionals
90% Confidence
Interval
Group
Harvard MBAs
Chemical Co. Employees
Chemical Co. Employees
Computer Co. Managers
Computer Co. Managers
AIE Seminar (before training)
AIE Seminar (after training)
Subject
General Trivia
General Industry
Company-Specific
General Business
Company-Specific
General Trivia & IT
General Trivia & IT
% Correct (target 90%)
40%
50%
48%
17%
36%
35%-50%
~90%
34

37. Test 1 (Jørgensen IEEE
Software 2008)
Group
Guidance
A
800
B
40
C
4
D
None
Result
160

38. Test 1
Group
Guidance
Result
A
800
300
B
40
100
C
4
60
D
None
160

39. Test 2
Group
Guidance
Result
A
Minor
Extension
New
Functionality
Extension
50
B
C

40. Test 2
Group
Guidance
Result
A
Minor
Extension
New
Functionality
Extension
40
B
C
80
50

41. Test 3
Group
A
B
Guidance
Future work at
stake, efficiency
will be measured
Control
Result
100

42. Test 3
Group
A
B
Guidance
Future work at
stake, efficiency
will be measured
Control
Result
40
100

43. Understand Bias
• "What gets us into trouble is not what we
don't know. It's what we know for sure that
just ain't so.“
» Mark Twain

44. Create a six sided random number generator.
Something like a dice like thingy.
Every member of the team needs to roll a double six.
http://www.flickr.com/photos/palooza/3309242555/
task “A”...

45. Relative Sizing

46. Did you know? (Jørgensen IEEE
Software March 2013)
• The US has 155 million more inhabitants
than Mexico, but Mexico has 100 million
fewer inhabitants than the US.
• Poland has 10 million more inhabitants
than Romania, but Romania has about the
same number of inhabitants as Poland.
• Austria’s population is 70% of Hungary’s,
while Hungary’s population is 80% of
Austria’s.

47. Velocity Helps Remove Bias
•
𝑆𝑡𝑜𝑟𝑦 𝑃𝑜𝑖𝑛𝑡𝑠
𝑆𝑡𝑜𝑟𝑦 𝑃𝑜𝑖𝑛𝑡𝑠
𝐼𝑡𝑒𝑟𝑎𝑡𝑖𝑜𝑛
= 𝐼𝑡𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑠
Projected Ship Date
11/18/2010
9/29/2010
8/10/2010
6/21/2010
5/2/2010
3/13/2010
1/22/2010
12/3/2009
10/14/2009
8/25/2009
0
1
2
3
4
Iteration
5
6
7
8

48. Story
Estimate
But Velocity is not a Silver Bullet

49. Sources of Error
• Range of Uncertainty

50. Estimation Errors
• Lan Cao - Estimating Agile Software
Project Effort: An Empirical Study

51. Lan Cao - Estimating Agile Software Project
Effort: An Empirical Study

52. Sources of Error
• Scope Creep

53. Scope Creep
• Capers Jones
–2% per month
–27% per year

54. We want this

55. Uncertainty
Know that
we know
Know that
we don’t know
Don’t know that
we know
Knowable
Don’t know that
we don’t know
Unknowable

56. Uncertainty
Know that
Wishful Thinking
we know
Know that
Discoverable Risks
we don’t know
p10
p50
Don’t know that
we know
Knowable
Don’t know that
Uncertainty
we don’t know
Management
Unknowable
p90

57. Estimation Exercise

58. Risky Business

59. War
They couldn't hit an elephant at this dist…
General John B. Sedgwick, Union Army Civil
War officer's last words, uttered during the Battle
of Spotsylvania, 1864

60. Gordon the
Guided Missile

61. Movies

62. Books

63. Poker

64. Texas Hold’em:
Which is the best hole hand?
A
B
C

65. Texas Hold’em:
Which is the best hole hand?
A
B
C
33.5%
29.6%
36.5%

66. Poker Metric:
Percent of Hands Won

67. Oil & Gas Exploration

68. Oil & Gas Exploration

70. Which Risks Are Important

71. Real Options
• The right — but not the obligation — to
undertake certain actions prior to an expiry
date

72. Airline Tickets

74. Real Options
• The right, but not the obligation to take
some action prior to an expiry date

75. Real Options
Value of Information
Value of Flexibility

76. Value of Information

77. Value of Information

78. Increasing Value & Cost of Info.
Aim for this
range
EVPI
Value or Cost
$$$
EVI
• EVPI – Expected Value of
Perfect Information
• ECI – Expected Cost of
Information
• EVI – Expected Value of
Information
ECI
$0
Perfect
Information
Low certainty
High certainty
76

79. Value of Flexibility
• The right — but not the
obligation — to undertake
certain actions prior to an
expiry date

80. Making An Agile Conference Agile
1000
Submissions for 2011 compared to 2010
900
800
700
600
Y2010
5 Extra weeks to
review sessions
500
Y2011-IT1
400
Y2011-IT2
300
200
100
0
0
10
20
30
40
Days Before Deadline
50
60
70

81. Estimated Agile 2010 Attendance
1800
1600
1400
1200
1000
800
Most Likely
600
Pessimistic
400
Optimistic
200
0
16
15
14
13
12
11
10
9
8
Weeks Out
7
6
5
4
3
2
1
FINAL

82. Do you have a choice?
Option
Commitment
Decision
80

84. Agile projects need risk management too

85. We’ll
rock at
Scrum
negligible
Early
Finish
Evaluatio
n
Sickness
Task
Switching
showstopper
impact
opportunity
Analyze risks
Scope
creep
Alien
Invasion
improbable
probability
near certainty

86. Risk Burndown
30
25
Risk
20
Risk 5
15
Risk 4
Risk 3
Risk 2
Risk 1
10
5
0
1
2
3
4
Iteration
5
6
7

87. Risk
Technical Uncertainty
Uncertain
Benefits
Lack of
Standardization
Technical Uncertainty
Manual Process
Complexity
Rules Engine
Complexity
Uncertain
Benefits
Lack of
Standardization
Manual Process
Complexity
Rules Engine
Complexity

88. Categories of risk
Business Case
Failure
Delivery Failure
Quality and Market
Acceptance
Political Risk
Collateral Damage

89. Two Risk Perspectives
Business
Perspective
Individual
Perspective

90. Collateral Damage

91. Delivery Failure results in Collateral
Damage

92. Titanic
• Time pressure
• Feedback blocks
• Unsinkable

93. I’m beginning to
think it wasn’t such
a good idea to turn
off those unit tests

94. Collateral Damage
Business
Perspective
Individual
Perspective
Could be catastrophic
Some individuals may be willing to
take on more risk than desired

95. Collateral Damage Management
Inspect
Honor feedback
Incremental Delivery
An effective roll-back strategy

96. Business Case Failure

97. Business Case Failure

98. Features and Functions
Never or
Rarely Used:
64%
Rarely
19%
Sometimes
16%
Always or
Often Used:
20%
Often
13%
Never
45%
Always
7%
Standish Group Study, reported by CEO Jim Johnson, XP2002

99. Powerful Questions are:
What are we building?
What Business are we in?
What
Building are
we in?

100. The Purpose Alignment Model
98

101. In Practice
CAN WE
CREATE A
DIFFERENTIATED
PARTNERSHIP?
MINIMIZE /
ELIMINATE
99
INNOVATE,
CREATE
ACHIEVE AND
MAINTAIN
PARITY,
MIMIC,
SIMPLIFY

102. Applicable at all Levels
• Corporate
Strategy
• Product Strategy

103. 101
A View of Strategy - Apple
ATT
PERIPHERALS
NEW PRODUCT
DESIGN
USER EXPERIENCE
CONTENT
DISTRIBUTION
MS OFFICE
INTEL HARDWARE
OTHER
SOFTWARE

104. Business Case
Business
Perspective
Individual
Perspective
Sustainable competitive advantage
How can I sell this so that I can get
more budget?

105. Delivery Failure
Over Budget
Late

106. Uncertainty Bounds
Cumulative Distribution Function of Actual/Estimate Ratio
100%
90%
80%
CDF Probability
70%
DeMarco Data
DeMarco Log-Normal
Landmark Data
Landmark Log-Normal
60%
50%
40%
30%
20%
10%
0%
0.1
1
Ratio of Actual/Estimate
10

107. The A/B/C List sets proper
expectations
A
MUST be completed in order to ship the product and the
schedule will be slipped if necessary to make this
commitment.
B
Is TARGETED to be completed in order to ship the product,
but may be dropped without consequence.
C
Is NOT TARGETED to be completed prior to shipping, but
might make it if time allows.
Only “A” features may be committed to customers.
If more than 50% of the planned effort is allocated to “A”
items the project is at risk.

108. A/B/C List
50%
25%
25%
B
C D
A
Typical Delivery
1.2
1
0.8
0.6
A
B
Backlog Plan
C
0.4
0.2
50%
Target
Delivery Date
r
D
ec
em
be
r
N
ov
em
be
ct
O
m
be
te
Se
p
100%
ob
er
r
t
us
Au
g
ly
Ju
Ju
ne
ay
M
il
Ap
r
ar
ch
M
ry
ua
Fe
br
Ja
nu
ar
y
0

109. A/B/C List
50%
25%
25%
A B
Uncertainty Risk
C D
1.2
1
0.8
0.6
A
B
Backlog Plan
C
0.4
0.2
50%
Target
Delivery Date
r
D
ec
em
be
r
N
ov
em
be
ct
O
m
be
te
Se
p
100%
ob
er
r
t
us
Au
g
ly
Ju
Ju
ne
ay
M
il
Ap
r
ar
ch
M
ry
ua
Fe
br
Ja
nu
ar
y
0

110. Estimating Story Difficulty
Estimation
Value
XL
L
M
S
S
M
L
Cost
XL

111. Product Innovation Flow
Hot Items
Product
Backlog
Release
Backlog
A
Items
Items
Idea Filter
Sales
Adaptive Activities
Services
RTM
B&C
Project Sanction
Iteration
Backlog
Newly Discovered
Flexible Scope
Backlog
CORE Activities
Most Items for consideration in next release
Backlog
Burnup
B/C/D
A
Customer
Support

112. Risk Management = Expectation Management
No
Surprises!

113. Iteration Review Questions
• What promises have been made, to whom were
they made, and who made them?
• What key decisions or commitments might we
have to make within the next 2 iterations?
• Questions to ask team anonymously
–
–
–
–
When will we be ready to ship?
Will we be able to keep our promises?
Is the team healthy and operating effectively?
Is the team on the right path?

114. Ask the Team
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
2/6/2011
2/26/2011
3/18/2011
4/7/2011
4/27/2011
5/17/2011
6/6/2011
6/26/2011
7/16/2011

115. Cost of Delay

116. Target
Best possible scenario if
everything went perfectly.
Uncertainty
Value
Plan
Planned scope for the
release at the optimal time
that it can be released.
Contract
Minimum scope for the
release at the latest date
that it can be released.
Tim e

117. Uncertainty
Value
C
B
A
Tim e

118. Tools for Delivery Risk
Stop Dreaming
1
0.9
Cost of Delay
0.8
0.7
0.6
0.5
0.4
0.3
0.2
Ask the Team
0.1
0
2/6/2011/26/2011
2
3/18/2011/7/2011/27/2011
4
4
5/17/2011/6/2011/26/2011
6
6
7/16/2011

119. Delivery Risk
Business
Perspective
Individual
Perspective
Cost of Delay
Business Credibility
Personal Credibility

120. Market Acceptance
Features and Quality

121. “I skate to
where the
puck is
going to
be, not
where it
has been.”

122. Allow Mid Course Corrections
Zone of success
Increasing Knowledge
Planned
Completion
Planned Path
Start
Actual Path
As Knowledge
increases Leaders use
iterations to guide
project towards
enhanced goal
Actual Completion

123. The Cone of Uncertainty
We expect uncertainty and manage for it through iterations,
anticipation, and adaptation.

125. The Purpose Alignment Model
123

126. Quality and Market Acceptance
Strategies
Seek and honor feedback
Incremental Delivery

128. Wrong Priorities

129. Compute This
"I recently asked a colleague [CIO] whether he
would prefer to deliver a project somewhat late
and over-budget but rich with business benefits or
one that is on-time and underbudget but of scant
value to the business. He thought it was a tough
call, and then went for the on-time scenario.
Delivering on-time and within budget is part of his
IT department's performance metrics. Chasing
after the elusive business value, over which he
thought he had little control anyway, is not."
Cutter Sr. Consultant Helen Pukszta

130. Software Tornado Example
Item
-15M
Product Acceptance
Uncertainty
Schedule Uncertainty
0M
15M
30M
45M
██ ███ ███ ███ ███ ███ ███
█ ███ ███ ██
General Market Uncertainty
██ ██
Cost Uncertainty
██ █

131. Software Tornado Example
Item
-15M
0M
15M
30M
45M
Product Acceptance
Uncertainty
███ ███ ███ ███ ███ ███
Schedule Uncertainty
███ ███ ██
General Market Uncertainty
██ ██
Cost Uncertainty
██ █

132. Software Tornado Example
Item
-15M
0M
15M
Product Acceptance
Uncertainty
███ ███
Schedule Uncertainty
███ ██
General Market Uncertainty
██ ██
Cost Uncertainty
██ █
30M
45M

133. The Measurement Inversion
In a business case, the economic value of measuring a variable is usually
inversely proportional to the measurement attention it typically gets.
Lowest
Information Value
• Initial cost
Most Measured
• Long-term costs
• Cost saving benefit other
than labor productivity
• Labor productivity
• Revenue enhancement
• Technology adoption rate
Highest
Information Value
• Project completion
Least Measured
132

134. Market Acceptance
Business
Perspective
Individual
Perspective
Impact to revenue
Impact to credibility
Cost of rework
Priorities are often guided by other
perceived constraints

135. Political Risk

136. Change
or
Maintain

137. Political Challenges
Boss and above
You
Direct Reports
Boss’s Peers
Peers
Peers’ Reports
Boss’s Indirect
Peers
Indirect Peers
Indirect

138. Network
What’s in it for them for everyone to succeed?
What do they worry about losing if you are successful?

139. Long Ago and Far, Far Away…

141. Collegial
Compliance
Creative
Tension
More useful
Collaboration
NonCollaborator Collaborator
Collaborating with Non-Collaborators
Combative
Common Practice
Agree
Agreement
Disagree

142. Agile Leadership

143. Politics
Business
Perspective
Politics do not add to business value
other than by luck
Individual
Perspective
Perception trumps Reality
Politics forms Perception

144. Final Summary
Risk
Business Perspective
Business Case Strategic alignment with
market need
Individual Perspective
Demonstrate Confidence
Collateral
Damage
Could be catastrophic
Make or Break career.
• Build quality in
Individuals may be willing to • Feedback loops
take more risk than the
organization
Delivery of
Market/
Quality Need
Impact to revenue
Impact to credibility
Cost of rework
Depends on the individual - • Customer Feedback
Priorities are often guided
by other perceived
constraints
Personal credibility
• Cost of Delay
• Tornado Charts
Delivery on
Cost of Delay
Time or within Reduction of ROI
Budget
Business Credibility
Politics
Markets only care if politics Perception trumps Reality
influence external
Politics forms Perception
perception.
Tools
• Purpose Alignment
Model
• Networking &
Transparency
• Collaborating with
Non-Collaborators

145. Simulation Exercise
ID: 1
Value: Sum of all dice
Major feature
Test: Roll 3 dice - 2 or more dice are the
same
Commit Points: 2

146. Stories and Scoring
We will run 3 iterations,
with 10 rolls of the dice per
iteration
ID: 1
Value: Sum of all dice
Major feature
WIP limit of one story in
progress
Must commit to 15 pts for
the release, and 5 pt per
iteration
Test: Roll 3 dice - 2 or more dice are the
same
Commit Points: 2
• Points (Value pts and Commit pts) are scored for accepted stories.
• Total points are Value + 10*Commit Points Made.
• Value is just an estimate…the market will decide

147. Special Actions
• Information
– At the cost of one roll, you may pre-roll your next turn
prior to selecting the story to work on.
• Flexibility
– At the cost of one roll, you may designate that the
story that you will be working on can be refactored at
a future date, i.e. you first accept the story, but if a
future roll gives an improved result, you may use that
roll for the new value
• It is ok abandon a story
• It is ok to redo a story

148. Estimating Story Difficulty
Estimation
Value
XL
L
M
S
S
M
L
Cost
XL

149. The Progress Board
Backlog
Committed In Work
Accepted
Committed for
this release (A)
Committed for
this iteration
Passes test
criteria
Cutline
Potential stories
In Development

150. Risk Management is not Risk Aversion

151. Contact
Todd Little
todd@toddlittleweb.com
www.toddlittleweb.com
www.accelinnova.com

152. Extra Slides

153. Why is
Software
Late?

154. From the home office in Duncan, Oklahoma
Dubai, UAE
Top Ten reasons why software is
late

155. Top Ten reasons why software
is late
10: Requirements, what Requirements?
What you want, baby I
got it
R-E-Q-U-I-R-E
Find out what it
means to me

156. Top Ten reasons why software
is late
9: Dependencies on other groups that were late

157. Top Ten reasons why software
is late
8: Over-optimistic Schedule Estimation
Always look on the bright side of code
.......
Always look on the bright side of code
.......
The code’s a piece of $#!^,
when we look at it
We can always overlook a minor kink . . . .
It probably compiles, it might even link . .
Surely that must mean it doesn’t stink

158. Top Ten reasons why software
is late
7: Those weren’t MY estimates
Scheduling Ritual
How low can you go!

159. Top Ten reasons why software
is late
6: Not enough testers or documentation
resources.
Who needs them anyway? We
put those bugs--I mean features-in there on purpose. Besides, it
was difficult to program, it should
be difficult to use.

160. Top Ten reasons why software is
late
5: Offshore and Outsourcing issues
My source code lies over the ocean,
My source code lies over the sea .
My source code lies over the ocean,
Oh bring back my source code to me
.....
Bring Back, Bring Back,
oh bring back my source code to me, to me
Bring Back, Bring Back,
oh bring back my source code to me

161. Top Ten reasons why software is
late
4: One word, Ch-ch-ch-changes

162. Top Ten reasons why software
is late
3: I can’t get no, System Admin
– I can’t get no, CM action
– ‘cause I try,
– ..and I try,
– ….and I try,
– ……and I try….

163. Top Ten reasons why software
is late
2: You didn’t give me the headcount that
you promised

164. Top Ten reasons why software
is late
1: Weren’t you doing the backups!?

165. Successful Projects?

166. Risk and Context
• One Size Doesn’t Fit All

167. Context Leadership Model
Bulls
Sheep Dogs
Cows
Uncertainty
Colts
Project Complexity

168. Context Leadership Model
High
Bulls
Simple, young projects.
Need agility
Tight Teams
Uncertainty
Colts
Agility to handle uncertainty
Process definition to cope
with complexity
Sheep Dogs
laissez faire
Cows
Complex, mature market
Need defined interfaces
Low
Low
High
Project Complexity

169. Bull Product Release

170. Reduce Uncertainty or Complexity
Uncertainty
Complexity
Attribute
Score
Attribute
Score
Market
███
Team Size
█████████
Technical
███
Mission Critical
█████████
# Customers
█████████
Team Location
█████████
Duration
█████████
Team Maturity
███
Change
███
Domain Gaps
███
Dependencies
█████████
Opportunities to Reduce Uncertainty:
Opportunities to Reduce Complexity:
• Use proven technologies
• Reduce project duration
• Collocate the team
• Break project into sub-projects

171. Partitioning
Colt
Project
Bull
Program
Dog
Project
Cow
Project
Remember: Loose Coupling and Strong Cohesion

172. Project Leadership Guide
Create
Change
High
Deploy
Market
Differentiating
Invent
Ad Hoc
Eliminate
Change
Offload
Low
Embrace
Change
Outsource
Low
Agile
Control
Change
Manage
Structured
Mission Critical
High

173. Portfolio Management
RAPID Quadrant Assessment
12.0
Uncertainty
Uncertainty
10.0
8.0
6.0
4.0
2.0
0.0
0.0
5.0
10.0
15.0
20.0
Project Complexity
Project Complexity
25.0
30.0

174. Not all dogs are the same

175. Financial Markets
A severe depression like that of 1920-21 is
outside the range of probability.
Harvard Economic Society, Weekly Letter,
November 16, 1929.

177. Getting Better

178. New Product Development
I think there is a world market for about five
computers.
Thomas J. Watson, chairman of IBM, 1943.

179. Getting Better
Estimation Quality Factor (EQF)
9
8
7
6
5
4
3
2
1
0
1
2
3
Years
4

180. How do “Risky Businesses” work
“It’s tough to make predictions,
especially about the future.”
Yogi Berra, Niels Bohr

181. Exercise
Low
Distance from Las Vegas to
Houston, Texas
Height of the Empire State
Building
Population of Sweden
U.S. Oil Consumption/day
Water in a 100 gallon vat filled
with sand
Med High

182. Exercise
Low Med
Distance from Las Vegas to
Houston, Texas
Height of the Empire State
Building
Population of Sweden
1222
U.S. Oil Consumption/day
20MM
Water in a 100 gallon vat filled
with sand
35
1453
9MM
High

183. Uncertainty
Value
Maximal Schedule
Success Flexible
Scope
Flexible
Minimal
Success
Failure
Tim e

184. Estimation accuracy improves
(Eveleens and Verhoef)

185. Estimation accuracy constant
(Eveleens and Verhoef)

186. Estimation accuracy decreases
(Eveleens and Verhoef)