This chapter discusses the methodology for policy analysis. The analyst's basic procedure is to: 1) help the decision-maker determine their objectives, 2) search for possible alternatives to meet the objectives, 3) determine the consequences of each alternative, and 4) rank the alternatives according to the decision-maker's criteria. While this framework provides guidance, applying analysis to policy problems remains an art. Successful analysis requires continuous refinement of the problem formulation, alternatives considered, and models used to predict outcomes.

1. Analysis for Public Decisions
Ch. 4: A Basic Framework for Policy Analysis
- E.S. Quade
Presented by Erin Mihalik

2. Quick Summary
This chapter discusses the methodology as if the
analysis were being carried out for a single
decision-maker who commissioned it.

3. • The successful application of analysis to policy
problems is an art.
• There are principles and procedures which will
offer guidance, but these cannot be followed
blindly.

4. • For policy analysis to be successful, it must:
– Discover a course of action that (if not clearly the
best available option) is at least satisfactory
– Have its findings accepted and incorporated into a
policy or decision that can be implemented
WITHOUT being so modified that it no longer
brings about the desired result.

5. The analyst’s basic procedure:
1. To help the decision-maker determine what he
wants.
2. To search out the possible ways of getting what
he wants
3. To work out the consequences that would
follow a decision to adopt each of the
alternatives, and
4. To rank the alternatives according to a criterion
specified by the decision-maker (or to present
them to him for ranking along with the
necessary information)

6. That sounds easy…
TOO EASY!

7. The Problem…
Public decisions don’t always fit into the “basic
procedure”


8. BUT, even so, the “basic procedure” is a good
starting point and a reasonable strategy for
discovering good solutions.


9. The Elements of Analysis
1. The Objective
2. The Alternatives
3. The Impacts
4. The Criteria
5. The Model (or Models)

10. 1. The Objectives
The objectives are what a decision-maker seeks
to accomplish or attain by means of his decision.
One of the hardest tasks an analyst has is
to discover if the stated objectives are
REALLY the objectives desired!

11. 2. The Alternatives
• The alternatives are the options or means
available to the decision-maker by which, it is
hoped, the objectives can be attained.

12. Juvenile Delinquency, for example…
Alternatives don’t have to be obvious!
Education… Recreation…
So what are some Family Police
alternatives for subsidy…. surveillance…
combatting juvenile
delinquency? Low-income
housing…
And whatever other
options we might
discover later!

13. 3. The Impacts
The designation of a particular alternative as the
means of accomplishing the objective implies a
certain set of consequences. Some of these
consequences are benefits and others are costs.
Huh??? In English It just means that no matter which
please, Mr. E.S. Quade! alternative we choose, there will be
positive and negative side effects!

14. The Impacts
There may be other impacts associated with an
alternative that, while they have little effect on
the attainment of the desired objective, they
should still be considered. These are called
spillovers or externalities.

15. By broadening the objectives, the externalities
can be internalized or made part of the study.

16. • In the narrow sense, costs are the resources
required to implement an alternative that is
no longer available for other uses once it is
implemented.
• In the broadest sense, costs are the
“opportunities foregone” – all the things we
cannot have or do once we have a particular
alternative.

17. Many, but not all, costs can be expressed in
dollars or other quantitative terms.
One example would be
traffic fatalities.

18. Traffic Fatalities Example:
This example includes things that can and
cannot be expressed quantitatively.
Delay caused to motorists by lowering the speed
limit can be partially expressed quantitatively…
But things like irritation to the drivers and
speeding elsewhere might lead to MORE
accidents! This is very hard to quantify.

19. 4. The Criteria
A criterion is a rule or standard by which to rank
the alternatives in order of desirability.
This provides a way to relate
objectives, alternatives, and impacts.

20. 5. The Model
Any decision analysis needs a process that can predict or
indicate the consequences that flow from an alternative.
This is provided by the model.
Abstractly, a model is just a set of generalizations or
assumptions about the world. It’s a simplification of the
real world.
Policy Models are usually elaborate mathematical
structures programmed for a computer.
Other models are simple mental models and used
throughout the analytic process.

21. The Process of Analysis
When searching for the best alternative, we go
through 5 stages that must be linked together
and repeated several times.

22. The Five Stages
• Formulation – clarifying and constraining the
problem and determining objectives.
• Search – identifying, designing, and screening the
alternatives.
• Forecasting – predicting the future environment
or operational context.
• Modeling – building and using models to
determine the impacts.
• Evaluation – comparing and ranking the
alternatives.

23. The Importance of Being Repetitive
Sure! But it’s very possible that you
don’t fully understand your
problem, so I’m going to ask you a
I have a problem I want lot of repetitive questions so we can
you to help me analyze. solve the REAL issue!

24. Successful analysis depends upon a continuous
cycle of:
• Formulating the problem
• Selecting objectives
• Designing alternatives
• Building better models

25. Repetition
It’s important to cycle and recycle through the
questions so the answers become more focused.
Are you sure?
Really sure?
???

26. The Iterative Nature of Analysis
Determining
Clarifying the
objectives and
problem
criteria
Searching out
Opening new and designing
alternatives alternatives
Questioning ITERATION! Collecting data
assumptions and information
Interpreting Building and
results testing models
Evaluating costs Examining
and alternatives for
effectiveness feasibility

27. Formulation
Formulation encompasses an attempt to isolate
the questions or issues involved to fix the
context within which these issues are to be
resolved, to clarify the objectives, to discover
the major factors that are operative, and to get
some feel for the relationships among them.
This stage is SUPER important! All the time we spend redefining the
problem helps to make the problem much more clear. A clear problem
helps us find a better solution!

28. An Example…
Eh-hmmmmm!!!
Tell me about your
There’s way too much turnover in my training
problem…
program! Everyone seems to quit or get fired!
Help me reduce the turnover!!

29. The same “problem” can have different perspectives
from different levels within the company!
My company doesn’t HAVE a
turnover problem. We train people
to do dangerous jobs – if the
turnover’s high, it’s in the safety Well, that
interest of those we terminate and certainly changes
the other workers! I don’t want things!
anyone getting hurt!

30. To solve one problem, you may need
to solve other problems, first!
• For example, urban air pollution.
First you need to consider other aspects of city
life, like local transportation and controls… and
these might affect housing and jobs!
A systematic investigation of these issues might
alter your perception of the issue.

31. Formulation is highly subjective
We consider what facts will be meaningful to the decision-maker.
In the transportation example, we should consider the practical
things, but we can leave the outlandish hypotheticals at home!

32. Search
The search phase is concerned with finding the
alternatives and the data and relationships on which
the analysis is to be based. It is usually more
productive to look for additional alternatives than it is
to look for more precise schemes for comparison.
It’s important to
consider a wide range
of alternatives!

33. Search is the activity that depends most
on talents that the analyst may not have.
Legal! Scientific!
Economic!
Sociology!
Engineering!
Artistic!
Medical!
Cost
Analysis!
Language!
An analyst’s success may often depend on the
technical competence of his associates!

34. Every system belongs to a hierarchy.
There are subsystems for every system and
there are wider systems that the system you’re
studying forms a subsystem. Clear thinking will be
necessary to find the
links between all these
systems so I can
formulate all the costs
and benefits!
Your
System

35. Forcasting
• The impacts or consequences that follow from
the decision to accept and implement a
particular alternative depend upon both:
– The properties of the alternative, AND
– On the situation or environment that exists when
and during the time the alternative is to be
effective.

36. The Transportation Example…
• The population distribution in the region may
affect the choice of routes and carriers.
– If the population is evenly distributed, carriers
who can make many stops will be preferred.

37. Scenario writing
• In policy studies, predictions of the future environment is
most often done by scenario writing.
• A scenario is a description of the essential features of the
future context in which the alternatives are to be implanted.
• Scenario writing is the preparation of a logical sequence of
hypothetical (but credible events) that could lead from the
present to the future. Exactly! Personal
commuter dragons aren’t
Does that mean no exactly … credible.
personal commuter
dragons?

38. Modeling
• Building a model of the system and its
environment is the standard approach to
compare the costs associated with different
ways of operating a future system.
• A model could be a simple table or graph, or it
may be written in a mathematical language.
• If you are lucky, the same model can be used
to predict outcomes from a competing system
– if not, another model must be built.

39. Systems Engineering Models
The end objective is to optimize the performance of a system, so the
model building must be subservient to this objective. A systems
engineering team must:
1. Ensure model building is carried out with a sense of purpose
(designed as efficiently as possible)
2. Tie together the various specializations that may be needed for
building models of the subsystems.
3. Ensure that work is concentrated where it is most needed. As a
general rule, models should be kept as simple as possible.
4. Decide when the model is adequate for the purposes for which it is
needed
5. If the model is used for planning, see that an effective dialogue is
conducted between the systems team and the managers who will use
the model. This dialogue must start when the model is BEING built.

40. Evaluation
Systems analysis has used two principal conceptual approaches
to rank the alternatives:
1. Fix the task or the level of effectiveness and then seek to
determine the alternative which is likely to achieve this level
of effectiveness or accomplish the task at the lowest cost.
2. Fixed budget. For a specified cost level or budget to be used
in attaining the objective the analysis attempts to determine
which alternative will produce the highest effectiveness.
The ideal is to do more than to prepare a comparison of
the alternatives ; we also want to rank them according
to criteria, so the decision-maker’s choice is easier!

41. Cost-Effective Approach
• Cost-effectiveness approaches are useful
when the relative merit of numerous
proposals is under investigation.
• It is not as useful when the question is one of
absolute merit (such as in deciding between
funding adult education or highway
construction).

42. Fixed Budget Approach
• This can be useful for questions of absolute merit,
even if we can’t measure accomplishment in a
clear way.
• We can “normalize” the alternatives and develop
a table of comparative effectiveness measures
(this can be both qualitative and quantitative).
Then decision-makers may be able to decide on
the basis of their own subjective criteria which
type of program they prefer.

43. • When decision-makers are judging whether something
is worth the cost, the most common approach is to
express the benefits and costs associated with each
alternative in dollars as a function of time, discount the
future benefits and costs at some appropriate rate, and
compare the alternatives on the basis of the present
value of net benefits.
• Alternatives can also be compared on the basis of the
internal rate of return (discount rate so NPV=0). This is
the “classical” cost-benefit approach – it’s hard to use
well when analyzing complex policy issues.

44. • It may not be possible for the analysts to prepare
an unambiguous ranking of the alternatives.
• When this occurs, the best scheme may be to list
the characteristics and impacts of the
alternatives and let the decision-makers
determine the ranking using their own
judgment.

45. No method of comparison is likely to answer all the questions.
What if my method of
comparison isn’t able to
answer all the questions Don’t worry!! No method of
the decision-makers want comparison is going to answer
to consider? ALL the questions. Just do your
best to anticipate and answer
as much as you can!

46. Conclusive Summary
• Even if the model and its inputs are excellent, the
conclusions proposed may be unacceptable due
to reasons based on morale, tradition, politics,
and organizational behavior.
• It’s important for the analysis USER to distinguish
what the study actually shows from any
recommendations made by the analyst based on
what the analyst FEELS should be done.
– Some say an analyst should not even make his
conclusion known.

47. • Whether or not the decision-maker made a
right decision based on analysis can never be
verified. Even long after the decision has been
made, we may still have no way of telling
whether the best action was chosen.

48. The Process Illustrated
• There are as many ways to approach a
problem as there are analysts.
• We’ll take a look at one example…

49. Short-Haul Transportation
In this example, we turn to short-haul transportation systems
about 10-15 years in the future.
Item Cost Beneft
Jet aircraft Serious noise and air polution Reduced travel time
Automobiles Pollution, personal injuries, Increased individual
energy shortages, urban sprawl mobility
High-speed railways Noise pollution; land required
So this study must consider the following:
Obvious Considerations Less Obvious
Time saved Environmental effects
Monetary Cost Societal effects
Additionally, how the costs/benefits are distributed among
social groups and localities must be considered.

50. Potential procedure for carrying out
the analysis:
Choice of a
criterion:
selection of
impacts
Design of
alternative
systems
Building
Design of
models to Prediction of Comparison
cases for
predict impacts of cases
comparison
impacts
Prediction of
operational
context

51. The impacts must be worked out on a
case-by-case basis
In the transportation hypothetical, we have the
following categories:
Impact Impacted
Transportation service impacts Occur to users of the system.
Financial impacts Occur to the operators and society
Economic impacts Involve changes in income and employment
Community impact Changes in the activity patterns, tax base,
and environment
Distributional impacts This considers how the various aggregate
impacts are distributed among different
social groups and locations

52. Developing the model…
Because in this example, there are so many
impacts which are diverse and difficult to value
monetarily, the desirable approach would be to
present the impacts in their natural units to be
ranked by the decision-makers.

53. Example of a model illustrating impact values…
Impacts Base VTOL TACV Impacts Base VTOL TACV
CTOL case case CTOL case case
case case
Transportation service Community impacts
impacts
Passengers (millions yearly) 7 4 9 Noise (thousand households 10 1 20
Door-to-door trip time (avg 2 1.5 2.5 Air pollution (% all emissions) 3 9 1
hr)
Door-to-door trip cost (avg. 17 28 20 Petroleum savings (%) 0 -20 +30
in $)
Airport congestion (% 0 5 10 Households displaced 0 20 500
reduction
Financial Impacts Land taken (acres) 0 25 8000
Investment costs ($ 150 200 2000 Taxes lost ($ millions) 0 0.2 2.0
millions)
Net annual subsidy ($ 0 0 90 Landmarks destroyed None None Ft. X
millions)
Economic impacts (peak yr Distribution impacts
Added jobs (thousands) 20 25 100 % low-income trips taken 7 1 20
Added sales ($ millions) 50 88 500 % of noise-impacted low- 2 16 40
income households

54. • Having developed the required models, measures
of the various impacts are obtained and the set
presented to the decision-makers for their
inspection in the form of a “scorecard”.
• This alone can’t be expected to complete the
analytic work, for the decision-makers will have
questions and will want additional comparisons
made and further systems and mixes of
alternatives investigated.
• But after these questions are answered, the
decision maker will be in a better position to
make a decision than before the analysis.

55. Analysis for Public Decisions
Ch. 4: A Basic Framework for Policy Analysis
- E.S. Quade
Presented by Erin Mihalik
Questions?