1. Validating AHP and ANP in
Exercises with Known Answers
By
Rozann Saaty
Creative Decisions Foundation
4922 Ellsworth Avenue
Pittsburgh, PA 15213
Email: rozann@creativedecisions.net
Fax: 412-681-4510
1
2. Validation Examples
• Fundamental Scale of the AHP
• Single Pairwise Comparison Matrices
• Hierarchical Validation Examples
• Networks with Feedback and Dependence
• Saaty Compatibility Index
• Complex Model Validation with Benefits,
Opportunities, Costs and Risks
2
3. Fundamental Scale of Absolute Numbers
1 Equal importance
3 Moderate importance of one over another
5 Strong or essential importance
7 Very strong or demonstrated importance
9 Extreme importance
2,4,6,8 For Intermediate values
1.1,1.2,…,1.9 Decimals for very close comparisons
Use Reciprocals for Inverse Comparisons
3
4. Three Examples of Estimating Relative
Values in a single pairwise matrix
Relative areas of shapes
Relative weights of objects
Relative distances to cities
4
6. One Individual’s Judgment Matrix
for Estimating Relative Areas
Circle Triangle Square Diamond Rec- Eigenvector Actual
tangle (priority Relative
vector of Sizes from
relative Measuring
sizes) Figures
Circle 1 9 2.5 3 6 0.488 0.467
Triangle 1/9 1 1/5 1/3.5 1/1.5 0.049 0.046
Square 1/2.5 5 1 1.7 3 0.233 0.241
Diamond 1/3 3.5 1/1.7 1 1.5 0.148 0.149
Rectangle 1/6 1.5 1/3 1/1.5 1 0.082 0.097
*Only the judgments in bold must be made, the others are
automatically determined
6
7. A Relative Priority Scale Applies to a Particular Group of
Objects – if the Objects Change, the Priorities Change
Relative Relative
Priority Priority
Scale Scale
with Five with
Figures Square
removed
Circle .467 .615
Triangle .046 .061
Square .241 xxxxxxxxxxxxx
xxxxxxxxxxxxx
Diamond .149 .196
Rectangle .097 .128
7
8. Relative Weights of Objects
Weight Radio Typewriter Large Projector Small Eigenvector Actual
Attache Attache
Results Relative
Case
Weights
Radio 1 1/5 1/3 1/4 4 0.09 0.10
Typewriter 5 1 2 2 8 0.40 0.39
Large 3 1/2 1 1/2 4 0.18 0.20
Attache
Case
Projector 4 1/2 2 1 7 0.29 0.27
Small 1/4 1/8 1/4 1/7 1 0.04 0.04
Attache
Case
8
9. Distances of Cities from Philadelphia
Comparison Cairo Tokyo Chicago San London Montreal Eigen- Distance to Relative
of Distances Francisco vector Philadelph Distance
from ia in miles
Philadelphia
Cairo 1 1/2 8 3 3 7 0.263 5,729 0.278
Tokyo 3 1 9 3 3 9 0.397 7,449 0.361
Chicago 1/8 1/9 1 1/6 1/5 2 0.033 660 0.032
San 1/3 1/3 6 1 1/3 6 0.116 2,732 0.132
Francisco
London 1/3 1/3 5 3 1 6 0.164 3,658 0.177
Montreal 1/7 1/9 1/2 1/6 1/6 1 0.027 400 0.019
9
10. Validation with Hierarchies
• An Investment Model
• Barzilai’s Wrong Hierarchical Structures –
we show how to do them correctly
10
11. An Investment Example
using Arithmetic
Calculating investment returns using arithmetic
AHP needs to match these results if it is valid
C1 C2 Total Normalized
Interest Capital Gains Return Total
Return Return Return
(relative returns
of the two
investments)
Investment A $3 $10 $13 13/34 = 0.382
Investment B $6 $15 $21 21/34 = 0.618
Totals $9 $25 $34
11
12. The Same Investment Example
using AHP in the Wrong Way
Obtain priorities by normalizing dollars in each column, add and
re-normalize to get final results
C1 C2 Sum of Normalized
Interest Capital Normalized Sum
Return Gains Return
(norm.) Return
(norm.)
Investment 3/9 10/25 11/15 11/30=.367
A ≠.382
Investment 6/9 15/25 19/15 19/30=.633
B ≠.618
Totals 1 1 2
12
13. Using AHP the Right Way
C1 C2 Total Weight and
9/34 25/34 Dollars Add to get the
Correct Relative
Priorities
A 3/9×9/34=3/34 10/25×25/34=10/34 13 13/34 =0 .382
B 6/9×9/34=6/34 15/25×25/34=15/34 21 21/34 = 0.618
Dol- 9 25 34
lars
13
14. OBJECTIONS
• I don’t want to go through all this nonsense to
figure out correct weights for the criteria.
• Ans: We are only doing this to show AHP can
produce correct results with actual scales. In
practice one would combine numbers from the
same scale under a single criterion first using
whatever formulas are appropriate.
• Oh, very well, but what if you have intangibles?
• Ans: We establish the importance of the criteria
through pairwise comparisons.
14
15. Conclusion
• In any validation exercise it is necessary to
establish the weights of the criteria correctly
so they will give the results you are
assuming. You cannot assume criteria
weights at the top of the hierarchy and
results at the bottom that are in conflict.
15
17. Barzilai’s Hierarchical Validation
Example
The president and three vice-presidents of a company are analyzing
their marketing options. The company produces and sells a single
product for a fixed price through five stores in the city. Stores 1 and 2
are in the city’s West Side, store 3 is at City Centre, and stores 4 and 5
are in the East Side. They all agree to define the company’s value
function as its total annual revenue where represents annual sales in
millions of dollars in store i, i = 1,…,5. The company needs to choose
between marketing strategies A and B. These strategies will result in
annual revenue of P = (3, 3, 1, 1, 1) if strategy A is chosen and Q = (1,
1, 1, 3, 3) in millions of dollars if strategy B is implemented.
Barzilai concludes that v(x) = x1+ x2 + x3 + x4 + x5 and since
v(P) = v(Q) = 9 (millions of dollars) the two alternatives are equally preferred
17
18. Barzilai’s hypothesis:
grouping the stores in different
ways should give the same result
if AHP is valid.
What he finds: grouping the stores in
different ways gives different results.
Why? He does not understand that criteria
weights are not arbitrary; they depend on
the structure and the assumed results.
18
19. Three Structures
(we show two here)
Barzilai creates three different structures
by grouping the five stores into
territories in three ways, making the
casual and unjustifiable assumption that
the criteria and subcriteria weights
have default equal values, regardless of
the structure.
19
23. How Do We Validate the ANP?
• Re-do the Investment Example as an ANP model
and show the results again validate expectations.
• Estimate market share using a network (Walmart,
Kmart and Target example)
• Do a complex BOCR model (ANWR voting
results)
23
24. Investment Example as ANP Model
Table 4 Data for the Investment Problem
Criteria Alternatives
Criteria
Cluster A1
Interest Return Capital Gains C2 First A2
Return C1 Capi- Inve Second
Inter- tal stme Invest
est Gains nt ment
1Cri- C1 -
Alternatives teria Interest 0 0 3 6
Cluster
C2 -
Alternative A Alternative B
Capital
Gains 0 0 10 15
2Alt-
ernat A - First
ives inv. 3 10 0 0
B-
Second
inv. 6 15 0 0
24
26. ANP RESULTS
Final Priorities(same as
Limit Matrix
Hierarchical Results)
(normalize by cluster to get final results)
Investmt 1 .3824
Investmt 2 .6176
Interest .2647
Capital .7352
Gains
26
28. Comparison with Actual Data
Mass Merch- Market Share ANP Model
andizers Actual (1998) Results
Walmart .548 .573
(58Billion)
Kmart .259 .221
(27.5 Billion) 20.3
Target .192 .206
(20.3 Billion)
Closeness of Results:
Saaty Compatibility Index: 1.011
28
29. Saaty Compatibility Index
• Compute the Saaty Compatibility Index by
forming two pairwise comparison matrices: A
from the results and B from the actual relative
values.
• Transpose the A matrix and perform
Hadamard matrix multiplication (cell-wise): AT
x B. Sum each row and sum those results.
Divide by n2. The result is the Saaty
Compatibility Index. A value of 1.0 means the
vectors exactly coincide.
• The closer to 1.0, the better the result.
29
30. The Next Step in Validation
• Validating multilevel ANP models.
• The ANWR model for determining
whether or not the US should allow
drilling for oil in Alaska.
• Results:
30
31. • In a recent poll, native Alaskans supported
opening ANWR to oil and gas exploration.
The question asked was “Do you believe oil
and gas exploration should or should not be
allowed within the ANWR Coastal Plain?”
Poll Results Model
Results
77.7%
22.3%
31
