- The document compares choice experiments and prediction markets for collecting preference data in conjoint analysis. Choice experiments involve presenting customers with subsets of product concepts and asking them to choose one, while prediction markets gamify the process by having customers trade securities predicting market shares. - Both approaches were used on a sample conjoint analysis problem involving attributes of a mahjong parlor. The results showed a high correlation between the estimated market shares and partial utilities derived from each approach. - This indicates prediction markets can substitute for choice experiments by effectively collecting preference data for conjoint analysis. However, more case studies are needed to generalize this conclusion.

1. © Hajime Mizuyama
A comparison between choice experiments
and prediction markets for collecting
preference data in conjoint analysis
Hajime Mizuyama
Dept. of Industrial & Systems Engineering,
Aoyama Gakuin University
mizuyama [at] ise.aoyama.ac.jp
ISOMS 2013 @ Osaka 2/June/2013

2. © Hajime Mizuyama
• In today's changing market, developing a new product is vital for a company
to survive and sustain.
• The product should be made different from competing ones and attractive to
target customers by introducing new attributes.
• Candidate attributes should be devised and properly evaluated, and the both
tasks can be supported by crowdsourced knowledge and intelligence.
• This talk focuses on the conjoint analysis, which has been widely used for
supporting the evaluation task.
• It evaluates the partial utility of each candidate attribute based on preference
data collected from potential customers through virtual choice experiments.
Research background

3. © Hajime Mizuyama
• Conducting choice experiments on crowds is tedious and costly, and it is not
easy to properly motivate them to express their true preference.
• A potentially effective solution to this issue is to gamify the process of
collecting preference data, and it can be realized through prediction markets.
• To present how to utilize prediction markets for collecting preference data
and how to derive the partial utilities of candidate attributes from the data.
• To actually conduct conjoint analysis on a simple example problem using
choice experiment approach and prediction market approach respectively.
• To confirm that the two approaches lead to similar conclusions and hence
prediction markets can be substituted for choice experiments.
Research objective

4. © Hajime Mizuyama
• Research background and objective
• Choice experiment approach
• Prediction market approach
• Comparison between the approaches: A simple case
• Conclusions
Agenda

5. © Hajime Mizuyama
• There are N candidate attributes, B1, B2, …, BN, which can be incorporated into
the new product under development.
• The concept of a new product can be expressed as a bundle of attributes.
Thus, any possible subset of the candidate attributes represents a new
product concept.
• The empty set corresponds to the product concept which does not include
any newly proposed attributes and its attractiveness is treated as the baseline.
• Each possible product concept can be denoted by x = (x1, x2, …, xN)T,
where xn = 1 represents that the attribute Bn is included in the concept
and xn = 0 indicates that it is not.
Product concepts and attributes

6. © Hajime Mizuyama
• Various small subsets of possible product concepts x1, x2, … to be compared
are created through the design of experiments technique*.
• For each of the subset, virtual choice experiments are conducted on some
potential customers.
• That is, the customers are asked to choose a product to buy from the subset,
assuming a shopping occasion where only the products in the subset are
available.
• The choice data collected in this manner are then used to statistically
estimate the attribute utilities according to the logit choice model.
Choice experiment approach: How to derive partial utilities
* E.g., Design and Analysis of Choice Experiments Using R: A Brief Introduction
H. Aizaki and K. Nishimura
Agricaltural Information Research, vol.17, no.2, pp. 86-94 (2008)

7. © Hajime Mizuyama
Overview of choice experiment approach
Attribute B1
Attribute B2
Attribute B3
Subset 1
Subset 2
Subset 3
Combine
Attribute
utilities
a1, a2, …
Derive
Collecting
preferences
on concepts
through
choice
experiments
Choice
data
Evaluate

8. © Hajime Mizuyama
• The more attractive the product concept x is, the higher the probability a
customer chooses it from among alternatives.
• The logit choice model defines the probability SA(x) as:
where A(x) is the attractiveness of the product concept x, defined as:
The attractiveness of the baseline concept is taken as a unit, an represents the
partial utility added by including the attribute Bn, and a = (a1, a2, …, aN)T.
Logit choice model

9. © Hajime Mizuyama
• Research background and objective
• Choice experiment approach
• Prediction market approach
• Comparison between the approaches: A simple case
• Conclusions
Agenda

10. © Hajime Mizuyama
• Some subsets of possible product concepts x1, x2, … to be compared are
created through the design of experiments technique.
• For each of the subset, likely values of their relative market shares p1, p2, …
are evaluated as a set through a prediction market*.
• That is, some potential customers etc. are asked to trade prediction securities
corresponding to the products in the subset, whose payoffs are proportional
to the relative market share values.
• The relative market share values p1, p2, … are then used to statistically
estimate the attribute utilities according to the logit market share model.
Prediction market approach: How to derive partial utilities
*A Knowledge-Driven Approach Using Prediction Markets for Planning and Marketing of a New Product
H.Mizuyama
Proc. of the 4th World Conference on Production and Operations Management, July (2012)

11. © Hajime Mizuyama
Overview of prediction market approach
Attribute B1
Attribute B2
Attribute B3
Attribute
utilities
a1, a2, …
Derive
Subset 1
Subset 2
Subset 3
Combine
Collecting
preferences
on concepts
through
prediction
markets
Relative
market shares
p1, p2, …
Evaluate

12. © Hajime Mizuyama
How to use a prediction market
A market for EMSPS
controlled by a CMM
Bid & ask offers
Market prices
= Estimated
shares
Relative
market share
prediction
security
(RMSPS)
The designer, some other employees,
loyal customers, etc. of the company.
Concept
x1
Concept
x2
Concept
x3
Payoffs proportional
to the shares
estimated by
the whole results

13. © Hajime Mizuyama
• The more attractive the product concept x is, the higher the market share it
will achieve if actually launched.
• The logit market share model defines the market share SA(x) as:
where A(x) is the attractiveness of the product concept x, defined as:
The attractiveness of the baseline concept is taken as a unit, an represents the
partial utility added by including the attribute Bn, and a = (a1, a2, …, aN)T.
• The model can be linearlized as:
Logit market share model

14. © Hajime Mizuyama
• Research background and objective
• Choice experiment approach
• Prediction market approach
• Comparison between the approaches: A simple case
• Conclusions
Agenda
Do these approaches lead
to similar conclusions?

15. © Hajime Mizuyama
Baseline design:
• Service facility (a Mahjong parlor targeting college students),
which is more than 10 minutes away from a station, located on the ground
floor of a building, playing pop music for BGM, about 10 people capacity, and
prohibiting smoking.
Candidate attributes:
B1: Within 5min. B2: Within 10 min.
B3: Underground floor B4: Upper floor
B5: Classical music B6: Jazz
B7: 30 people cap. B8: 50 people cap.
B9: Smoking separated B10: Smoking allowed
Case description

16. © Hajime Mizuyama
How comparison is made
Choice experiments:
• 6 students at Aoyama Gakuin
University
• 27 choice experiments:
A choice is made from among 3
candidate shop profiles.
• Partial utilities are derived from the
preference data (choice data)
based on the logit choice model.
• Relative market shares are
estimated using the model.
Prediction markets:
• 6 students at Aoyama Gakuin
University
• 9 market sessions (LMSR):
6 candidate shop profiles are
compared.
• Relative market shares are
evaluated according to the market
prices.
• Partial utilities are derived from the
relative market shares based on
the logit market share model.

17. © Hajime Mizuyama
0.10 0.15 0.20 0.25
0.00.10.20.30.4
Shares estimated by choice experiments
Sharesestimatedbypredictionmarkets
Estimated market shares
Correlation
coefficient
= 0.69
Relative shares estimated by choice experiments
Relativesharesestimatedbypredictionmarkets

18. © Hajime Mizuyama
B1 B2 B3 B4 B5 B6 B7 B8 B9 B10
-0.50.00.51.01.5
B1 B2 B3 B4 B5 B6 B7 B8 B9 B10
-0.20.20.40.6
Estimated partial utilities
Partial utilities estimated by choice experiments
Partial utilities estimated by prediction markets
Correlation
coefficient
= 0.90

19. © Hajime Mizuyama
• Research background and objective
• Choice experiment approach
• Prediction market approach
• Comparison between the approaches: A simple case
• Conclusions
Agenda

20. © Hajime Mizuyama
• A prediction market approach to collecting preference data from the crowd
for conjoint analysis is proposed.
• It is confirmed with an example conjoint analysis problem that the proposed
approach and a conventional approach using choice experiments lead to
similar conclusions.
• Thus, it seem that the proposed prediction market approach can be
substituted for tedious choice experiments.
• However, this is a tentative conclusion based only on a single case. Further
research is needed by piling up case studies.
Conclusions

21. Thank you for your kind attention!
Questions and comments are welcome.
mizuyama [at] ise.aoyama.ac.jp