Share your thoughts online:

#SKIMwebinar

Mobile Choice-based Conjoint
A valid alternative

Gerard Loosschilder
CMethO

J...
Mobile surveys: it’s time to act
Increasingly people take surveys on a
mobile platform. Filtering them out
may skew the sa...
But we should do a better job tailoring to their needs.
The respondent may have something better to do
Survey is
enjoyable...
Today’s presentation is about the
validity of mobile choice-based conjoint

Introducing 3*3 mCBC

4

SKIM Webinar “Mobile ...
Adapting to the shorter attention span of the mobile user
By reducing the number of choice tasks and number of choices per...
At the same valid results as a traditional CBC exercise
The good news

The bad news

The MAE of share predictions is
<4% a...
3*3 mCBC requires a larger sample size
As a rule of thumb, sample size is
tied to number of parameters to be
estimated, de...
3*3 mCBC requires a well-balanced research design
In a 3*3 mCBC research design,
the number of data points
collected is mu...
Empirical evidence

How do we know if mCBC
using a 3*3 design is valid?

9

SKIM Webinar “Mobile Choice-based Conjoint: A ...
Because we have validated it …

We determined the relation between
sample size and convergent validity
It shows that at in...
… in a study into pain killers and batteries
Attributes:
 Brand
 Admin method
 Dosage
 Package
 Price
3-5 levels each...
Attribute importance values show great similarities
at all sample sizes for mobile CBC
The values for
batteries are
equall...
However, at larger sample sizes, preference shares get
closer to benchmark leg, demonstrating greater validity
10.0%

Mean...
Conditions to make 3*3 mCBC work – research design
Every display of product profiles in a choice task is valuable. It
shou...
Conditions to make 3*3 mCBC work – real estate
There is no space for large
amounts of information
Max. 5 attributes

Text ...
Powered by

Interaction design
http://tinyurl.com/skimmcbc

16

SKIM Webinar “Mobile Choice-based Conjoint: A valid altern...
3*3 mCBC in Price Sensitivity Analyses – batteries

http://tinyurl.com/skimmcbc
17

SKIM Webinar “Mobile Choice-based Conj...
3*3 mCBC in Price Sensitivity Analyses – pain killers

http://tinyurl.com/skimmcbc
18

SKIM Webinar “Mobile Choice-based C...
3*3 mCBC in Telecom – design of a postpaid portfolio

http://tinyurl.com/skimmcbc
19

SKIM Webinar “Mobile Choice-based Co...
Mobile in MaxDiff applied to Semantics
and other tests of claims and benefit statements

http://tinyurl.com/skimmcbc
20

S...
To conclude

3*3 mCBC is a valid addition

21

SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
To conclude – mCBC is a valid addition
Do

Don’t

• Take market knowledge into
account

• Use it for large CBCs

• Use a l...
Next steps
• SKIM researchers will continue to validate mCBC …
• For allocation studies
• For large research designs using...
Share your thoughts online:

#SKIMwebinar

Go to www.skimgroup.com/webinars
for today’s presentation slides and more!
Gera...
Upcoming SlideShare
Loading in …5
×

Webinar Mobile Conjoint Analysis: Are the results valid?

946 views

Published on

Mobile devices such as tablets and smartphones are the survey platforms of the future. At SKIM, we’re preparing for this future by adapting survey interaction design to the specifics of mobile; shorter and more engaging choice exercises, with fewer choice tasks and fewer options per task. However, when doing this, we must make sure that the results of a mobile Choice-Based Conjoint (CBC) study are valid; as in, it delivers the same results as the same CBC study would on a traditional platform.

John and Gerard shared the results in empirical comparative studies between mobile and traditional CBC, to determine if and when the results of a mobile CBC are valid. Based on these results, they provided guidelines to determine when it is right to utilize mobile CBC.

For more information about SKIM's webinars, visit www.skimgroup.com/webinars.

Published in: Business, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
946
On SlideShare
0
From Embeds
0
Number of Embeds
272
Actions
Shares
0
Downloads
12
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Webinar Mobile Conjoint Analysis: Are the results valid?

  1. 1. Share your thoughts online: #SKIMwebinar Mobile Choice-based Conjoint A valid alternative Gerard Loosschilder CMethO John Ashraf Conjoint expert Abigail Joffre Today’s webinar host
  2. 2. Mobile surveys: it’s time to act Increasingly people take surveys on a mobile platform. Filtering them out may skew the sample. In Standard survey design 7% 4% % of people taking the test survey by platform Smartphone 89% Tablet PC Mobile survey design Screen Source: Burke, July 2013 Out Biased sample 2 2013 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  3. 3. But we should do a better job tailoring to their needs. The respondent may have something better to do Survey is enjoyable PC 24% Tablet Mobile ST Mobile+ 9% PC This is bad news Mobile ST Mobile+ PC 79% Tablet 14% 10% Would take another survey on the same device Responding was very easy 53% 38% 31% This is bad news Tablet 66% 48% Mobile ST 35% Mobile+ This is bad news 32% Source: Burke, July 2013. Mobile ST is a standard survey on a smartphone; Mobile+ is a dedicated mobile survey design 3 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  4. 4. Today’s presentation is about the validity of mobile choice-based conjoint Introducing 3*3 mCBC 4 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  5. 5. Adapting to the shorter attention span of the mobile user By reducing the number of choice tasks and number of choices per task Three choice tasks 5 Three options each SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative” Completed in
  6. 6. At the same valid results as a traditional CBC exercise The good news The bad news The MAE of share predictions is <4% at the correct sample size So far, it is impossible to reduce the number of choice tasks and options per task to fewer than the 3*3 platform Overall results, conclusions and recommendations are the same 6 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  7. 7. 3*3 mCBC requires a larger sample size As a rule of thumb, sample size is tied to number of parameters to be estimated, determined by the number of attributes and levels Traditionally, we recommend a minimal sample size of n= 200 for any conjoint study. Now, the sample size goes up with the number of parameters to be estimated. 7 If # of parameters is Sample size n= 12 (3 atts*4 levels) 400 16 (4 atts*4 levels) 800 20 (5 atts*4 levels) 1,200 25 (5 atts*5 levels) 1,600 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  8. 8. 3*3 mCBC requires a well-balanced research design In a 3*3 mCBC research design, the number of data points collected is much smaller than the number of parameters to be estimated. So, the key success factor is a high-quality research design 8 Frequency balanced Every combination of two attributes and their levels is represented well by all research designs Market knowledge included We take current market shares and knowledge about consumer preferences into account in design generation SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  9. 9. Empirical evidence How do we know if mCBC using a 3*3 design is valid? 9 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  10. 10. Because we have validated it … We determined the relation between sample size and convergent validity It shows that at increasing sample sizes, mobile CBC is a good approximation of common CBC on traditional platforms 10 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  11. 11. … in a study into pain killers and batteries Attributes:  Brand  Admin method  Dosage  Package  Price 3-5 levels each; 16 parameters Attributes:  Brand  Price  Size 3-5 levels each; 12 parameters What is the validity of mobile CBC at various sample sizes if compared with the same conjoint study in a traditional way at n= 200? 11 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  12. 12. Attribute importance values show great similarities at all sample sizes for mobile CBC The values for batteries are equally similar Attribute importance Pain killers 40% 30% Brand 20% Admin method 10% Dosage 0% Package Price Variations in attribute importance values are insignificant Platform and sample size 12 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  13. 13. However, at larger sample sizes, preference shares get closer to benchmark leg, demonstrating greater validity 10.0% Mean Absolute Error values (MAE, in %) MAE value in % 7.5% The break point is at n= 400; twice the sample size of the benchmark 5.0% Batteries Pain killers 2.5% Especially at pain killers at 16 parameters (instead of 12 for batteries) 0.0% 200 13 300 400 500 600 700 800 Sample size in the mobile leg Mean Absolute Error (MAE) is a metric expressing how far apart the preferences shares are as measured by mCBC at increasing sample sizes compared with the benchmark CBC at n= 200 900 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  14. 14. Conditions to make 3*3 mCBC work – research design Every display of product profiles in a choice task is valuable. It should not be wasted on prohibitions or alternative specific designs. • We can not tolerate any reduction in D-efficiency values So for now, we can apply mCBC in some standard situations. We do not apply 3*3 mCBC in situations in which there are prohibitions, utility balanced designs and alternative-specific designs 14 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  15. 15. Conditions to make 3*3 mCBC work – real estate There is no space for large amounts of information Max. 5 attributes Text is succinct, keywords only No complex descriptions Pictures are icons or schematic Be careful with package or ad testing 15 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  16. 16. Powered by Interaction design http://tinyurl.com/skimmcbc 16 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  17. 17. 3*3 mCBC in Price Sensitivity Analyses – batteries http://tinyurl.com/skimmcbc 17 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  18. 18. 3*3 mCBC in Price Sensitivity Analyses – pain killers http://tinyurl.com/skimmcbc 18 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  19. 19. 3*3 mCBC in Telecom – design of a postpaid portfolio http://tinyurl.com/skimmcbc 19 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  20. 20. Mobile in MaxDiff applied to Semantics and other tests of claims and benefit statements http://tinyurl.com/skimmcbc 20 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  21. 21. To conclude 3*3 mCBC is a valid addition 21 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  22. 22. To conclude – mCBC is a valid addition Do Don’t • Take market knowledge into account • Use it for large CBCs • Use a larger sample • Include research design restrictions • Use succinct text, iconized visuals • Use it for package, ad or concept work • Have five attributes at five levels max 22 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  23. 23. Next steps • SKIM researchers will continue to validate mCBC … • For allocation studies • For large research designs using consideration sets • For more complex visuals • Keep improving UI for Mobile CBCs • Across dominant platforms • Translate learnings back to any CBC 23 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  24. 24. Share your thoughts online: #SKIMwebinar Go to www.skimgroup.com/webinars for today’s presentation slides and more! Gerard Loosschilder John Ashraf Abigail Joffre Chief Methodology Officer g.loosschilder@skimgroup.com @gloosschilder Conjoint expert j.ashraf@skimgroup.com a.joffre@skimgroup.com +1 201 963 8430 @SKIMgroup

×