choice experiments

1. Priyanka Upreti
10731
Agricultural Economics
CHOICE EXPERIMENTS
FOR SOCIO-ECONOMIC
RESEARCH

2. CONTENTS
Introduction
Comparison with other stated preference techniques
Concept and approach
How to design a choice experiment
Various issues and considerations
Strengths and weaknesses
Uses

3. INTRODUCTION
• To investigate the impact of various policies of government related to social sector or
before introducing any product or program or in social planning, it is prerequisite to
elicit individuals’ preferences.
• Generally, preferences can be recorded via two ways: revealed preference approach
and stated preference approach.
 Revealed preferences: observing choices that people have made in the real world
(what individuals actually did i.e. revealed by their behaviour)
 Stated preferences: here we ask, “If you faced this particular situation, what would
you do?” (hypothetical choices). It is statement of their preferences via their choices.

4. The comparison between revealed preference and stated
preference
Revealed preferences Stated preferences
Performance
information
• The result of the actual behaviour
• Consistent with the behaviour in real
market
• We can get ‘choice’ results
• Expression under the hypothetical situation
• Possibility of inconsistent with the behaviour in
the real market
• We can get ‘ranking’, ‘rating’, ‘choice’ etc.
Alternatives • Only existing alternatives • Existing and non-existing alternatives
Attributes • Measurement error
• Limited range of attributes’ levels
• Possibility of collinearity among
attributes
• No measurement error
• Extensibility of the range of attributes’ levels
• Controllability of the collinearity among attributes
Choice set • Non-clear • Clear
Number of
response(s)
• One response per respondent • One or more response(s) per respondent
Source: Nobuhiro Sanko, 2001

5. Stated
preference
method
decision tree
Source: Anna Merino - Castello, 2003

6. Characteri-
stics of
stated
preference
approaches
Source: Anna Merino - Castello, 2003

7. Choice experiments
• Initially developed by Louviere and Hensher (1982) and Louviere and Woodworth (1983) in the
marketing economics and transportation literature.
• It allows researchers to uncover how individuals value selected attributes of a program, product or
service by asking them to state their choice over different hypothetical alternatives.
• Each alternative is described by several characteristics, known as attributes.
• Can be used for products and services not traded on a market, such as for a new product under
development and not yet commercially available.
• A monetary value is included as one of the attributes, along with other attributes of importance.
• When individuals make their choice, they implicitly make trade-offs between the levels of the
attributes in the different alternatives presented in a choice set (Alpizar et al., 2001)
• Choice experiments were inspired by the Lancasterian microeconomic approach (Lancaster, 1966), in
which individuals derive utility from the characteristics of the goods rather than directly from the
goods themselves.

8. • It has its theoretical foundation in random utility theory and relies on the assumptions of
economic rationality and utility maximization.
• In stating a preference the individual is assumed to choose the alternative that yields his/her
highest individual benefit, known as utility. The utility yielded by an alternative is assumed to
depend on the utilities associated with its composing attributes and attribute levels.
• Yiq is the utility of individual q for the ith alternative and is assumed to be a function of its
attributes, Xi is a vector of attributes for the ith alternative accompanied by a set of weights, bi,
that establish the relative contribution of each attribute to the utility associated with the ith
alternative.
• Used to determine the significance of the attributes that describe the good or service and the
extent to which individuals are willing to trade one attribute for another.
Yiq = Xibi +uiq

9. Choice experiment: concept
and approach
• CEs are samples of choice sets or choice
scenarios drawn from the universe of all
possible choice sets.
• CE comprises of the following elements-
1) A set of fixed choice options that have explicit
names.
2) A set of attributes that describe potential
differences in the choice options.
3) A set of levels or values assigned to each
attribute of each choice options to represent a
range of variation in that attribute appropriate
to the research objectives of a particular study.
4) A sample of subjects evaluates all or a subset
of the choice sets in the total experiment and
chooses one of the possible options available to
be chosen in each set.
Attributes
Levels

10. How to design a choice experiment?
Identifying the good or service to
be valued
Designing on what attributes and levels
fully describe the good or service
Constructing an experimental design
Constructing the survey
Administering survey to respondents
Analysis of data
• Research question
• Needs of the client
• Attributes: The independent variable whose effect are
being tested,
• Level: the options or increments of an attribute

11. 2. Deciding on what attributes and levels fully describe the good or service:
• It requires:
Good understanding of target population’s perspective and experience (different
cultural and language setting)
Policy concerns so requires involvement of local institutions and policy-makers during
the preparatory stage
Published and grey literature, such as policy documents and government reports.
augment secondary sources with primary data to ensure that the DCE is tailored to the
study setting (semi-structured interviews, focus group discussions)
Local researchers: encourage respondents to be more open and allow them to express
themselves in their local languages.
Content analysis

12. • No restrictions on the number of attributes that could be included in a DCE, though
in practice most DCEs have contained fewer than 10 to ensure that respondents are
able to consider all attributes listed when making their choice (DeShazo and Fermo
2002).
• It is important to avoid conceptual overlap between two or more of the attributes,
known as inter-attribute correlation - prevents the accurate estimation of the main
effect of a single attribute.
• Typically the levels chosen should reflect the range of situations that respondents
might expect to experience
• There should be a base level that reflect the prevailing working conditions.
Additional levels are then established that represent a reasonable improvement from
the base.

13. Example
Estimation of economic values
of changes in ecological, social
And economic conditions of the
wetland.
The number of wetland management
scenarios that can be generated from
5 attributes, 2 with 4 levels and the
remaining 3 with 2 levels, is
42⁎23=128.
Source: Birol E. et al, 2006

14. 3. Constructing an experimental design:
• DCE elicitations rely on an effective experimental design -- the combination of attributes and
attribute levels presented to respondents (choice sets).
• Full factorial design can be generated which consists of all possible combinations of the levels
of the attributes, and permits estimation of main effects and interactions.
• A main effect refers to the direct independent effect on the choice variable of the difference in
attribute levels. An interaction effect is the effect on the choice variable obtained by varying
two or more attribute levels together (Effect of one factor depends on the level of another).
• Generally, the number of possible profiles is an where a is the number of levels and n is the
number of possible attributes.
• If the number of levels varies across attributes then the number of possible hypothetical
profiles is an x bm where a and b are the different attributes levels and n and m are the different
attributes.

15. Example of full factorial design
• Respondents’ preferences towards a public transport service
• Total combinations: 23 = 8
Characteristics:
• It is orthogonal:
- rows are perfectly uncorrelated,
- Each pair of levels occurs equally often.
• It is balanced:
- Each level appears an equal no. of time.
• It allows to estimate main effects and
two‐way or higher interactions.
• It also happens to be an orthogonal array.
- All possible interactions are estimable.
Attributes Fare Travel time Service frequency
Levels High (1)
Low (-1)
Slow (-1)
Fast (1)
Infrequent (-1)
Frequent (1)
Attributes
Fare
(A)
Travel
time (B)
Frequen
cy (C)
A*B B*C A*C A*B*C
Scenari
os
1
2
3
4
5
6
7
8
1
1
1
1
-1
-1
-1
-1
1
1
-1
-1
1
1
-1
-1
-1
1
-1
1
-1
1
-1
1
1
1
-1
-1
-1
-1
1
1
-1
1
1
-1
-1
1
1
-1
-1
1
-1
1
1
-1
1
-1
-1
1
1
-1
1
-1
-1
1

16. Fractional factorial design
• Full factorial designs are practical only for small problems involving either small numbers of
attributes or levels or both.
• Too costly and tedious to rate all possible combinations in a full factorial design.
• Fractional factorial designs consists of carefully chosen subset of a full factorial design. The
subset is chosen so as to exploit the principles of:
1. Hierarchical ordering principle: lower order effects are more likely to important than higher
order effects, so when resources are scarce priority should be given to estimation of lower order
effects.
2. Effect sparsity principle: the number of relatively important effects in a factorial experiment
are small.
3. Effect heredity principle: in order of an interaction to be significant, at least one of its parent
factor should be significant.

17. • Fractional factorial design is supported by the reason that usually only some
interactions are significant or researcher’s interest. Louviere (1988) analyzed how
much variability in behavioural response main effects and interactions explain:
(a) Main effects explain the largest amount of variance in response data, often 80% or
more;
(b) Two-way interactions account for the next largest proportion of variance, although
this rarely exceeds 3% - 6%;
(c) Three-way interactions account for even smaller proportions of variance, rarely
more than 2% - 3% (usually 0.5% - 1%) and;
(d) Higher order terms account for minuscule proportions of variance.

18. • These are expressed by Ik-p, where I = no. of levels, k = no. of factors and p = size of fraction of
full factorial used. p = no. of generators, assignments as to which effects or interactions are
confounded, i.e. cannot be estimated independently with each other.
• It will be 1/(Ip) fraction of the full factorial design.
• E.g. 25-2 design = ¼ of full factorial design, only 8 runs rather than 32 runs.
• These designs can be obtained from:
SPSS’s ORTHOPLAN command
available in some literatures
OPTEX procedure in the SAS statistical software
web- based catalogue approach (e.g. www.york.ac.uk/depts/maths/tables/orthogonal.htm
http://www.itl.nist.gov/div898/handbook/pri/section3/pri3347.htm )
FrF2 package in R (works along with DoE.base or DoE.wrapper package)

19. Choice set creation
A choice set (treatment) is
one scenario provided for
evaluation by respondents. A
series of differing choice
sets are provided.
Source: Birol E. et al, 2006

20. 1. Simultaneous choice set creation – LMN method
Used when one wants a design wherein choice sets each contain N alternatives of M
attributes of L levels each.
2. Sequential choice set criterion:
(1) Shifting: produce one alternative with
factorial design and another by shifts
of original alternative.
(2) Foldover: produce one alternative from
factorial design. Shifts the first two columns with 3rd as such. Shuffle each of the two
separately and choose one from each; these become choice set 1.
3. Randomized choice sets creation: reflects the fact that respondents are
randomly selected to receive different versions of the choice sets.

21. Problems of factorial designs:
• Too many scenarios and games.
• Trivial questions: dominant scenarios and transitivity
• Contextual constraints: some alternatives are not possible in real
market situation.

22. The other methods:
Design Main work Purpose Assumptions At the expense of Supporting reason
Fractional
factorial design
Selecting specific
scenarios or games from
full factorial design
Reducing number of
games
Some or all of
interactions are
not significant
Some or all of
interactions
Many parts are explained
only by main effects
Removing trivial
games
Removing trivial games Reducing number of
games, removing
valueless questions
Dominance
(Preference),
Transitivity
Orthogonality Trivial games bring less
information and make
respondents stop thinking
seriously.
Contextual
constraints
Removing scenarios
which are
technologically
impossible or
unreasonable
Reducing number of
games and achieving
realistic situation
The criteria of
technological
impossibility and
unreasonableness
Orthogonality Analysis, using scenarios
which are technologically
impossible or unreasonable,
is suspicious
Block design Division of the games
into more than one
part, each of which
must be fractional
factorial design
Reducing number of
games per
respondent
Homogeneity Individual
estimation
Individual estimation is less
important compared to
universal estimation
Random
selection
Choosing randomly
from candidates of
games
Reducing number of
games per
respondent
Homogeneity Individual
estimation
Individual estimation is less
important compared to
universal estimation

23. Considerations for choice designs
• Orthogonality
• Balanced
• Minimum overlap: minimize the number of times each level appears in a
choice set.
• Utility balance: Ensure that no choice set contains either a dominant
alternative that every rational person would want or a terrible alternative
that no one would want.
• D-efficiency: measure for most efficient design. D-optimal designs are
constructed to minimize the generalized variance of the estimated
regression coefficients.

24. Generating and pre-testing the questionnaire
• Created choice sets form the basis for questionnaire.
• Typically DCEs ask respondents to consider up to 18 choice sets, representing a practical limit of
how many comparisons can be completed before boredom sets in (Hanson et al. 2005;
Christofides et al. 2006).
• The questionnaire should be clearly presented and contain a standard introduction to DCE with
choice set examples.
• To minimize any bias caused by the order in which the choice sets occur or the attributes are
described, several versions of the questionnaire are produced in which choice sets and
attributes are presented in different orders (Kjaer et al. 2006).
• Pictures, diagrams and symbols may aid comprehension, and are particularly relevant for
conducting a DCE in low-income countries where literacy cannot be assumed.
• Collect data on socio-economic indicators to allow analysis of the impact of individual
characteristics on the choices made.

25. • Administering the questionnaire: self or trained fieldworkers.
• Piloting the questionnaire: when working across cultures and several languages. Helps to
review element of design process, selection and definition of attributes, their levels,
respondent’s understanding of the task, their ease of comprehension and whether the
number of choice sets can be managed by the target population (Hall et al. 2004).
• Once the choice sets and questionnaire design are finalized, the DCE questionnaire can be
administered to collect data via face-to-face interview, telephone interview, mailed
questionnaire etc.
• Some issues:
Attribute non-attendance: respondents often do not consider all attributes presented in
the tasks but make choices on only a sub-set of attributes, can cause biased parameter
estimates.
Response certainty: consistency between hypothetical and real choices
Rationality of choices

26. Study on estimating participation
in biodiversity conservation
by Grenier R. et al, 2014
Stated approach to estimate
attribute non-attendance
Supplementary question for
accounting for the risk related
to response certainty

27. Tests for rationality behind respondents’ answers
• In a study on ‘Modelling Recreation Demand using Choice Experiments:
Climbing in Scotland’ by Hanley N. et al, 2000 suggested to:
• Include for a sub-set of respondents a choice pair where one alternative
strictly dominates the other. By making these alternatives identical except
for price. Respondents would be expected to reject the more expensive
option.
• Include, for a different subset of individuals, identical choice pairs as their
first and fourth choice occasion. The answer which respondents gave in the
first instance was expected to be the same as the answer they gave when the
pair was repeated.

28. Analysis of DCE data
• Involves regression models that have a dichotomous or
polychotomous categorical dependent variable, such as a probit, logit,
or conditional logit model i.e. multinominal logit specification.
• MNL model converts observed choice frequencies (being estimated
probabilities) into utility estimates via the logistic function.
• The utility associated with every attribute level can be estimated and
we can construct total utility.
• Softwares: LIMDEP, NLOGIT, SPSS, SAS, STATA etc.

29. Conditional logit model
• Assumes IIA
(independence of
irrelevant alternatives)
property: the relative
probabilities of two
options being chosen
are unaffected by
introduction or removal
of other alternatives.
• If violated, results will
be biased.
• To test whether the CL
model is appropriate,
the Hausman and
McFadden (1984) test
for the IIA property is
employed.
Source: Birol E. et al, 2006

30. Random parameter logit model
• CL model assumes homogenous preferences across respondents.
• RPL model accounts for unobserved, unconditional heterogeneity in order to
account for preference heterogeneity in pure public goods.
• Superior to the CL model in terms of overall fit and welfare estimates.
• Explains sources of heterogeneity by including interactions of respondent-
specific social, economic and attitudinal characteristics with choice specific
attributes in the utility function.
• Higher overall fit.
• Other models: nested logit, latent class model, mixed logit
Source: Birol E. et al, 2006
Random parameter logit model with interactions:

31. Estimation of willingness to pay
• After estimating parameters, welfare measures, in the form of marginal willingness to
pay (WTP), can be determined by estimating the marginal rate of substitution between
the change in the wetland management attribute in question and the marginal utility of
income represented by the coefficient of the payment attribute.
• Marginal WTP values, for each of the wetland management attributes estimated using
the Wald procedure (Delta method) in LIMDEP 8.0 NLOGIT 3.0.
Source: Birol E. et al, 2006

32. Strengths
• Forces respondents to consider trade-offs between attributes.
• CE designs can avoid multicollinearity problems which often troubles revealed preference
analysts.
• CE can be used to study preferences for attribute levels beyond the existing, e.g. changes in
access prices beyond the range currently observed.
• Relatively lower cost compared with other experimental approaches
• Able to generate estimates of the values of many different alternatives from the one application.
• Makes the frame of reference explicit to respondents via the inclusion of an array of attributes
and product alternatives.
• Enables implicit prices to be estimated for attributes.
• Enables welfare impacts to be estimated for multiple scenarios.
• Can be used to estimate the level of consumer demand for alternative service product in non-
monetary terms.

33. Weaknesses
• Welfare estimates obtained with CE are sensitive to study design. For example, the
choice of attributes, the levels chosen to represent them, and the way in which
choices are relayed to respondents may all impact on the values of estimates of
consumers‘ surplus and marginal utilities.
• Fractional factorial designs used in practice deliberately confound two-way and
higher order interactions with lower order estimates in order to make the design
small.
• One weakness of discrete dependent variable model is that mean and variances on
the latent scale are perfectly confounded means they can’t be separated. It creates
problem in interpretation of output of a regression model.

34. Uses
• In agricultural and food economics
• Environmental and resource economics- environment impact assessment,
modelling recreation demand, biodiversity conservation.
• Health economics.
• Estimating the implied willingness to pay (WTP) for goods and services.
• Predicting uptake and refining new product development.
• Variations of product attributes.
• Demand estimates and optimum pricing.
• Estimating the effects of product characteristics on consumer choice.

35. Conclusion
• In the past few decades, the CE method has become popular among the public and
private sectors as a guide for efficient and effective decision-making.
• In the public sector, to capture the marginal economic values—costs and benefits of
improvements in, or provision of, public goods and services. Used to conduct cost–
benefit analysis.
• In the private sector, used as a market-research tool that helps in understanding the
demand structures for private goods and services not yet on the market.
• As compared to the other stated preference techniques CE presents multiple choices
to respondents and thus providing a greater degree of public participation and
avoids many of problems associated with other methods.
• The success of a CE depends on the design of the experiment which is a dynamic
process involving definition of attributes, attribute levels and customisation, context
of the experiment, experimental design and questionnaire development.