Gerard Loosschilder
CIO & Partner
Robin de Rooij
Senior Manager
Abigail Joffre
Today’s webinar host
#SKIMwebinar
Share you...
Mobile surveys: it’s time to act
Increasingly people take surveys on a
mobile platform. Filtering them out
may skew the sa...
So we better format right…
3 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
Introducing 3*3 mCBC
Today’s presentation is about the
validity of mobile choice-based conjoint
4 SKIM Webinar “Mobile Cho...
Adapting to the shorter attention span of the mobile user
By reducing the number of choice tasks and number of choices per...
At the same valid results as a traditional CBC exercise
The good news
The MAE of share predictions is
<4% at the correct s...
3*3 mCBC requires a larger sample size
As a rule of thumb, sample size is
tied to number of parameters to be
estimated, de...
3*3 mCBC requires a well-balanced research design
In a 3*3 mCBC research design,
the number of data points
collected is mu...
How do we know if mCBC
using a 3*3 design is valid?
Empirical evidence
9 SKIM Webinar “Mobile Choice-based Conjoint: A val...
Because we have validated it …
We determined the relation between
sample size and convergent validity
It shows that at inc...
… in a study into pain killers and batteries
11
What is the validity of mobile CBC at various sample sizes
if compared wit...
Attribute importance values show great similarities
at all sample sizes for mobile CBC
12
The values for
batteries are
equ...
However, at larger sample sizes, preference shares get
closer to benchmark leg, demonstrating greater validity
13
Mean Abs...
Conditions to make 3*3 mCBC work – research design
Every display of product profiles in a choice task is valuable. It
shou...
Conditions to make 3*3 mCBC work – real estate
15
There is no space for large
amounts of information
Max. 5 attributes
Tex...
16
Interaction design
http://tinyurl.com/skimmcbc
Powered by
SKIM Webinar “Mobile Choice-based Conjoint: A valid alternati...
3*3 mCBC in Price Sensitivity Analyses – batteries
17
http://tinyurl.com/skimmcbc
SKIM Webinar “Mobile Choice-based Conjoi...
3*3 mCBC in Price Sensitivity Analyses – pain killers
18
http://tinyurl.com/skimmcbc
SKIM Webinar “Mobile Choice-based Con...
3*3 mCBC in Telecom – design of a postpaid portfolio
19
http://tinyurl.com/skimmcbc
SKIM Webinar “Mobile Choice-based Conj...
Mobile in MaxDiff applied to Semantics
and other tests of claims and benefit statements
20
http://tinyurl.com/skimmcbc
SKI...
We capture actual consumer behavior
in a mobile shopping exercise taken on mobile devices
Packaging can be
seen in detail ...
The exercise allows us to optimize features
of the mobile retail environment and its products
22
Product rankings in
searc...
Replicating online retail environments transcends
to services offered by telecom networks
23
A build-your-own post-paid su...
… and by publishing houses…
24
The original website of the publishing house
to have consumers sign up for a subscription …...
3*3 mCBC is a valid addition
To conclude
25 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
To conclude – mCBC is a valid addition
Do
• Take market knowledge into
account
• Use a larger sample
• Use succinct text, ...
Next steps
• SKIM researchers will continue to validate mCBC …
• For allocation studies
• For large research designs using...
Go to www.skimgroup.com/webinars
for today’s presentation slides and more!
Robin de Rooij
Senior Manager
r.derooij@skimgro...
Upcoming SlideShare
Loading in...5
×

Webinar Mobile Research: Are the results valid?”

280

Published on

Mobile devices such as tablets and smartphones are the survey platforms of the future. At SKIM, we’re preparing for this future by adapting survey interaction design to the specifics of mobile; shorter and more engaging choice exercises, with fewer choice tasks and fewer options per task. However, when doing this, we must make sure that the results of a mobile Choice-based Conjoint (CBC) study are valid; as in, it delivers the same results as the same CBC study would on a traditional platform.

Gerard and Robin shared the results in empirical comparative studies between mobile and traditional CBC, to determine if and when the results of a mobile CBC are valid. Based on these results, they provided guidelines to help you determine when it is right to utilize mobile CBC.

For more information about SKIM's webinars, visit http://www.skimgroup.com/webinars.

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
280
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Webinar Mobile Research: Are the results valid?”

  1. 1. Gerard Loosschilder CIO & Partner Robin de Rooij Senior Manager Abigail Joffre Today’s webinar host #SKIMwebinar Share your thoughts online: Mobile Choice-based Conjoint A valid alternative
  2. 2. Mobile surveys: it’s time to act Increasingly people take surveys on a mobile platform. Filtering them out may skew the sample. 2 Source: Burke, July 2013 89% 4% 7% 2013 Smartphone Tablet PC Screen In Out Biased sample Standard survey design Mobile survey design % of people taking the test survey by platform SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  3. 3. So we better format right… 3 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  4. 4. Introducing 3*3 mCBC Today’s presentation is about the validity of mobile choice-based conjoint 4 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  5. 5. Adapting to the shorter attention span of the mobile user By reducing the number of choice tasks and number of choices per task 5 Three choice tasks Completed inThree options each SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  6. 6. At the same valid results as a traditional CBC exercise The good news The MAE of share predictions is <4% at the correct sample size Overall results, conclusions and recommendations are the same The bad news So far, it is impossible to reduce the number of choice tasks and options per task to fewer than the 3*3 platform 6 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  7. 7. 3*3 mCBC requires a larger sample size As a rule of thumb, sample size is tied to number of parameters to be estimated, determined by the number of attributes and levels Traditionally, we recommend a minimal sample size of n= 200 for any conjoint study. Now, the sample size goes up with the number of parameters to be estimated. If # of parameters is Sample size n= 12 (3 atts*4 levels) 400 16 (4 atts*4 levels) 800 20 (5 atts*4 levels) 1,200 25 (5 atts*5 levels) 1,600 7 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  8. 8. 3*3 mCBC requires a well-balanced research design In a 3*3 mCBC research design, the number of data points collected is much smaller than the number of parameters to be estimated. So, the key success factor is a high-quality research design Frequency balanced Every combination of two attributes and their levels is represented well by all research designs Market knowledge included We take current market shares and knowledge about consumer preferences into account in design generation 8 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  9. 9. How do we know if mCBC using a 3*3 design is valid? Empirical evidence 9 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  10. 10. Because we have validated it … We determined the relation between sample size and convergent validity It shows that at increasing sample sizes, mobile CBC is a good approximation of common CBC on traditional platforms 10 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  11. 11. … in a study into pain killers and batteries 11 What is the validity of mobile CBC at various sample sizes if compared with the same conjoint study in a traditional way at n= 200? Attributes:  Brand  Price  Size 3-5 levels each; 12 parameters Attributes:  Brand  Admin method  Dosage  Package  Price 3-5 levels each; 16 parameters SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  12. 12. Attribute importance values show great similarities at all sample sizes for mobile CBC 12 The values for batteries are equally similar Variations in attribute importance values are insignificant 0% 10% 20% 30% 40% Attributeimportance Platform and sample size Pain killers Brand Admin method Dosage Package Price SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  13. 13. However, at larger sample sizes, preference shares get closer to benchmark leg, demonstrating greater validity 13 Mean Absolute Error (MAE) is a metric expressing how far apart the preferences shares are as measured by mCBC at increasing sample sizes compared with the benchmark CBC at n= 2000.0% 2.5% 5.0% 7.5% 10.0% 200 300 400 500 600 700 800 900 MAEvaluein% Sample size in the mobile leg Mean Absolute Error values (MAE, in %) Batteries Pain killers The break point is at n= 400; twice the sample size of the benchmark Especially at pain killers at 16 parameters (instead of 12 for batteries) SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  14. 14. Conditions to make 3*3 mCBC work – research design Every display of product profiles in a choice task is valuable. It should not be wasted on prohibitions or alternative specific designs. • We can not tolerate any reduction in D-efficiency values So for now, we can apply mCBC in some standard situations. We do not apply 3*3 mCBC in situations in which there are prohibitions, utility balanced designs and alternative-specific designs 14 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  15. 15. Conditions to make 3*3 mCBC work – real estate 15 There is no space for large amounts of information Max. 5 attributes Text is succinct, keywords only No complex descriptions Pictures are icons or schematic Be careful with package or ad testing SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  16. 16. 16 Interaction design http://tinyurl.com/skimmcbc Powered by SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  17. 17. 3*3 mCBC in Price Sensitivity Analyses – batteries 17 http://tinyurl.com/skimmcbc SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  18. 18. 3*3 mCBC in Price Sensitivity Analyses – pain killers 18 http://tinyurl.com/skimmcbc SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  19. 19. 3*3 mCBC in Telecom – design of a postpaid portfolio 19 http://tinyurl.com/skimmcbc SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  20. 20. Mobile in MaxDiff applied to Semantics and other tests of claims and benefit statements 20 http://tinyurl.com/skimmcbc SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  21. 21. We capture actual consumer behavior in a mobile shopping exercise taken on mobile devices Packaging can be seen in detail through a zoom-in feature The shopping basket selections can be reviewed SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”21
  22. 22. The exercise allows us to optimize features of the mobile retail environment and its products 22 Product rankings in search results Ratings & Reviews Price & Promotions Images and iconography Product descriptions and sizes 24 hour protection Product callouts Shipping options SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  23. 23. Replicating online retail environments transcends to services offered by telecom networks 23 A build-your-own post-paid subscription environment inspired by Hi, a subsidiary brand of KPN in NL A build-your-own post-paid subscription environment inspired by Tele2, an independent price fighter brand in NL SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  24. 24. … and by publishing houses… 24 The original website of the publishing house to have consumers sign up for a subscription … … converted into a conjoint choice exercise to mimic the online environment SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  25. 25. 3*3 mCBC is a valid addition To conclude 25 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  26. 26. To conclude – mCBC is a valid addition Do • Take market knowledge into account • Use a larger sample • Use succinct text, iconized visuals • Have five attributes at five levels max Don’t • Use it for large CBCs • Include research design restrictions • Use it for package, ad or concept work 26 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  27. 27. Next steps • SKIM researchers will continue to validate mCBC … • For allocation studies • For large research designs using consideration sets • For more complex visuals • Keep improving UI for Mobile CBCs • Across dominant platforms • Translate learnings back to any CBC 27 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”
  28. 28. Go to www.skimgroup.com/webinars for today’s presentation slides and more! Robin de Rooij Senior Manager r.derooij@skimgroup.com @Robin_de_Rooij Abigail Joffre a.joffre@skimgroup.com +1 201 963 8430 @SKIMgroup Gerard Loosschilder Chief Innovation Officer g.loosschilder@skimgroup.com @gloosschilder #SKIMwebinar Share your thoughts online:
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×