Increase Business Value with Effective Decision Management
1. Increasing Business Value
with Decision Management
David G. Ullman
President, Robust Decisions Inc.
Jeffrey Barnes
President, Compelevent Solutions
Pr
Copyright, Robust Decisions, 2002 1
www.robustdecisions.com
2. Goals
• Define Decision Management
• Describe major Decision Management objectives
• Managing uncertainty
• Basing decisions on satisfaction and risk
• Deciding what to do next
• Demonstrate achieving the objects using an RFP
example
• Discuss how Decision Management can help you
produce more successful decisions
Copyright, Robust Decisions, 2002 2
www.robustdecisions.com
3. The value of information
DECISION
Judgment
KNOWLEDGE
Increasing
Behavior value
MODELS
Relationships
DATA
Copyright, Robust Decisions, 2002 3
www.robustdecisions.com
4. What makes critical decisions
so difficult?
• There are many stakeholders: customers,
influencers, decision makers and approvers
• Their knowledge is uncertain: qualitative,
evolving and incomplete
• Their perspectives about what is important varies
• They must base decisions on expected satisfaction
and assessment of risk
• Not clear what to do next "The only real problem in
life is what to do next."
Arthur C. Clark
Copyright, Robust Decisions, 2002 4
www.robustdecisions.com
5. What is Decision Management?
A process that ensures
Given stakeholder perspectives and
situation
their uncertain knowledge
are effectively utilized in determining
what to do next to make the
Goal
best possible choices with
known expected satisfaction and risk
Copyright, Robust Decisions, 2002 5
www.robustdecisions.com
6. Symptoms of Poor Decision
Management
• Decisions that were declared final are later revisited
• Team members don’t buy into decisions
• Team decisions are made by edict or by loudest voice
• Not all the expertise of the team members is utilized.
• There is little confidence in decisions based on
uncertain information and unknown risk
• Decisions are not justified, documented, or reused
Copyright, Robust Decisions, 2002 6
www.robustdecisions.com
7. Cost of poor decisions
Product development:
• A poor decision - 20 people spend one
week reworking a critical problem: $75,000
wasted on burdened salaries alone
Product procurement:
• A poor decision - Wastes product
acquisition/implementation costs and
creates excessive support costs. Expected
added business value delayed, diminished
or not realized entirely.
Copyright, Robust Decisions, 2002 7
www.robustdecisions.com
9. Simple example: RFP evaluation
• RFP contains
• Detailed list of requirements that define the
ideal system - developed by functional
representatives (stakeholders) in the
organization
• The responses
• 4 proposals submitted
• Your organization has dealt with some of the
vendors before
Copyright, Robust Decisions, 2002 9
www.robustdecisions.com
10. Evaluation criteria given in the RFP
• Cost Criteria, < $100K 30%
What if one vendor
• Technical Criteria proposes 90%?50%
• System response time, <3 sec
• Training time, > 95% proficiency in 72 hrs
How do you manage Is risk a useful
• Ease of use, > 75% satisfaction
qualitative criteria? criterion?
(they • Risk ofthe meeting criteria, low
are often not
discriminators)
• ……….. Whose
• Management Criteria weightings? 20%
• Strong management team
• Experienced with similar problems
• ……….. Total 100%
Copyright, Robust Decisions, 2002 10
www.robustdecisions.com
12. A more sophisticated evaluation method
Measures of
satisfaction
(The Decision Matrix)
Performance Management
Response Training Ease of Strong Proj. Team Total
Cost time time use mgt team experience sum/wt
wt .3 .5 .2 1.0
Vendor 1 4 3 2 1 3 3 16/2.8
Vendor 2 4 4 4 3 5 4 24/3.9
Vendor 3 4 3 4 4 4 4 23/3.8
Vendor 4 4 5 5 4 2 2 22/3.9
“The Mechanical Design Process” 3rd
Confidence vendor
meets criterion
edition, David G. Ullman, McGraw Hill, 2003
5 = very high
1 = very low Which will you
choose?
Copyright, Robust Decisions, 2002 12
www.robustdecisions.com
13. Decision Management
improving the decision matrix
• Minimal additional work
• Inclusion of stakeholders perspectives First topic
• Management of uncertain knowledge and
differing perspectives about what is important
• Real time analysis of satisfaction and risk
• Guidance about what to do next to find the
vendor most likely to be the best choice
• Capture and document the deliberation for
justification and reuse
Copyright, Robust Decisions, 2002 13
www.robustdecisions.com
14. Uncertain evaluation
For Vendor 1
Response Training Ease of Strong Proj. Team
Cost time time use mgt team experience
Evaluator 1 4 3 2 4 4 3
Evaluator 2 4 4 3 2 3 3
Evaluator 3 4 3 4 2 1 3
Evaluator 4 4 4 5 1 4 2
How do you manage the
Confidence that
vendor meets criterion
differences?
5 = very high
1 = very low
Copyright, Robust Decisions, 2002 14
www.robustdecisions.com
15. How do you manage the differences?
Average? – Averages are always brown.
Discussion? – Discussions can be
dominated by loudest voices.
– How can you account for
knowledge differences?
Voting? – Move to Florida!
What about uncertainty?
Copyright, Robust Decisions, 2002 15
www.robustdecisions.com
16. The effect of knowledge in
qualitative evaluations
For Vendor 1 Knowledge scale
Expert
Strong Mgt Knowledge Experienced
Team Informed
Evaluator 1 4 Experienced Amateur
Weak
Evaluator 2 3 Informed Unknowledgeable
Evaluator 3 1 Weak
Evaluator 4 4 Experienced
Average = 3 Team score including
Confidence that
vendor meets criterion
knowledge = 4.1
5 = very high
Copyright, Robust Decisions, 2002 16
1 = very low
www.robustdecisions.com
17. The effects of uncertainty during
quantitative evaluations
• RFP - Training time > 95% proficiency in 72 hours
• In reality you will be delighted with 95%, but not sure it is Target
achievable. Disgusted if < 85% uncertainty
• Vendor bids - 95%
• Vendor’s reality – sure she can get 91% + 5% Evaluation
• Your estimate of vendor’s reality is 87% + 8% uncertainty
• If you could run an evaluation experiment you
might find 90% + 8% Changes
• After the system is in use the reality is 94% with time
Copyright, Robust Decisions, 2002 17
www.robustdecisions.com
18. Decision Management
improving the decision matrix
• Minimal additional work
• Inclusion of stakeholders perspectives
• Management of uncertain knowledge and
differing perspectives about what is important Next topic
• Real time analysis of satisfaction and risk
• Guidance about what to do next to find the
vendor most likely to be the best choice
• Capture and document the deliberation for
justification and reuse
Copyright, Robust Decisions, 2002 18
www.robustdecisions.com
19. Risk Analysis
• Risk is not a criterion!!
• Risk is a measure of uncertainty.
• Risk is one measure of how well each
alternative meets each criterion.
• There is risk that the system response time will be >3 sec
• There is risk that the training time will be < 95% proficiency in
72 hrs
• There is risk that the ease of use will be < 75% satisfaction
• There is risk that the management team is not strong
• There is risk that they are not experienced with similar
problems
Copyright, Robust Decisions, 2002 19
www.robustdecisions.com
20. There are two kinds of risk
• Satisfaction Risk – The possibility that an
alternative will not meet a criterion ideal target
• Risk in choosing an alternative that can achieve
only 93% proficiency
• Not really a risk measure, but lack of satisfaction
• Downside risk – The possibility that the actual
satisfaction will be worse than expected
• We think that it will be 93% but there is a
chance it will only be 90%
• Also upside risk
Copyright, Robust Decisions, 2002 20
www.robustdecisions.com
21. Vendor 1 16/2.8 Variation captured by the
Which one will
Vendor 2 24/3.9 evaluations of different
you choose?
stakeholders
Vendor 3 23/3.8
Vendor 4 22/3.9
Ideal satisfaction - 4.0?
Satisfaction risk V4
V2
3.9
Weighted V3 3.8
Satisfaction
Downside risk
It is as important to understand the uncertainty
as it is to believe the satisfaction!
Copyright, Robust Decisions, 2002 21
www.robustdecisions.com
22. Decision Management
improving the decision matrix
• Minimal additional work
• Inclusion of stakeholders perspectives
• Management of uncertain knowledge and differing
perspectives about what is important
• Real time analysis of satisfaction and risk
• Guidance about what to do next to find the vendor
most likely to be the best choice Final topic
• Capture and document the deliberation for
justification and reuse
Copyright, Robust Decisions, 2002 22
www.robustdecisions.com
23. "The only real problem in life is what
to do next." Arthur C. Clark
Vendor 1 16/2.8
Vendor 2 24/3.9
Vendor 3 23/3.8
Vendor 4 22/3.9
Traditionally
• Choose vendor with highest score (satisfaction)
• Loose sleep If you choose not to decide, you
• Re-bid still have made a choice. Neil Peart,
drummer and lyricist for Rush.
Copyright, Robust Decisions, 2002 23
www.robustdecisions.com
24. Decision Management basis for
“what to do next” analysis
Traditionally all that is available
• Satisfaction
• Value of information analysis
The needed
(a sensitivity analysis) richness
supplied by
• Level of consensus decision
• Level of knowledge or management
tools
uncertainty
Copyright, Robust Decisions, 2002 24
www.robustdecisions.com
25. Decision Management suggests
1. Choose an alternative (The team has reached
consensus, from all perspectives on a specific
alternative)
2. Work toward consensus (Increase team consensus
on identified critical evaluations)
3. Increase knowledge (Decrease uncertainty on
identified critical evaluations)
4. Refine criteria (Identified critical criteria need to be
refined)
Copyright, Robust Decisions, 2002 25
www.robustdecisions.com
26. A “What to do next report”
Build consensus on Strong management team for Vendor 1
Build consensus on System response time for Vendor 2
Increase team knowledge about Ease of use for Vendor 1
Increase team knowledge about Strong management team
for Vendor 2
Refine team experience criterion
Copyright, Robust Decisions, 2002 26
www.robustdecisions.com
27. The deliberation
Initial what to next report
• Build consensus New what to next report
Strong management
team for Vendor 1 • Increase knowledge
about Ease of use for
• Build consensus Vendor 3
System response
time for Vendor 2
Based on results team
recommends Vendor 3
to decision maker
Copyright, Robust Decisions, 2002 27
www.robustdecisions.com
28. What is Decision Management?
A process that ensures
stakeholder perspectives and
their uncertain knowledge
are effectively utilized in determining
what to do next to make the
best possible choices with
known expected satisfaction and risk
Copyright, Robust Decisions, 2002 28
www.robustdecisions.com
29. On the next project
• Manage uncertainties
• Integrate stakeholder knowledge and perspectives
• Calculate satisfaction and risk
• Analyze what to do next
You will
• Have more successful decisions
• Get more buy-in from stakeholders
• Keep out of trouble
• Document how decisions were reached for
justification and reuse
• Deliver on promises made in the business case for
the RFP
Copyright, Robust Decisions, 2002 29
www.robustdecisions.com
31. You now get to make the decision…
“What to do next”
Copyright, Robust Decisions, 2002 31
www.robustdecisions.com
32. For RFP problem.
Copyright, Robust Decisions, 2002 32
www.robustdecisions.com
33. Robust Decisions Inc.
Delivering decision management
consulting, training and tools.
Developers of Accord™
the real time, easy to use
decision management
application for distributed
or co-located teams
Copyright, Robust Decisions, 2002 33
www.robustdecisions.com
