Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

"Big simulators with dozens of products: Which conjoint method is most suitable?" at ART Forum 2017

240 views

Published on

Download this presentation https://goo.gl/c77VU7

The key question in this presentation was to check what extent the discrepancy between the number of concepts in a task (3, or 4) versus the number of products in the conjoint simulation model (20, or 50) may influence the results of the study, by comparing CBC, ACBC and PBC (preference based conjoint).

Published in: Business
  • Be the first to comment

  • Be the first to like this

"Big simulators with dozens of products: Which conjoint method is most suitable?" at ART Forum 2017

  1. 1. Big simulators with dozens of products: Which conjoint method is most suitable? Jeroen Hardon | Marco Hoogerbrugge
  2. 2. 2 Required prior knowledge: CBC vs ACBC Choice-Based Conjoint: (nearly) balanced level frequencies for every respondent Adaptive Choice-Based Conjoint: level frequencies are skewed and different for every respondent depending on his/her earlier answers in BYO task 0 2 4 6 8 10 12 14 16 Frequency 30 GB was the respondent’s preferred level in the earlier BYO task 0 2 4 6 8 10 12 14 16 Frequency
  3. 3. 3 Introduction
  4. 4. 4 What we test What we simulate What is our problem?
  5. 5. 5 Holdout tasks with 50-100 products are impossible(?) Hit rate in CBC 20.5% > Much better than random (5%) > But very poor in absolute sense > And it will get worse in a real simulator MAE 1.81% > Nearly random (1.85%) We have a wrong prediction for 80% of our respondents In this test study we used a holdout task with 20 concepts.
  6. 6. 6 Performance of standard ACBC* is very similar Hit rate 19.8% > Although slightly lower, the hit-rates are very comparable to CBC > And again, it will get worse in a real simulator MAE 1.83% > Nearly random (1.85%) * With 3 concepts per screen, no screening section, BYO tasks included in estimation
  7. 7. 7 Preference-Based Conjoint (PBC) o level 1 (fixed) o level 2 (fixed) o level 3 (fixed) o level 4 (fixed) o level 5 (fixed) o level 6 (fixed) o respondent's choice in BYO task (flexible) o respondent's choice in BYO task (flexible)
  8. 8. 8 Preference-Based Conjoint (PBC) o level 1 (fixed) o level 2 (fixed) o level 3 (fixed) o level 4 (fixed) o level 5 (fixed) o level 6 (fixed) o respondent's choice in BYO task (flexible) o respondent's choice in BYO task (flexible) o respondent's choice in BYO + 1 level (flexible) o respondent's choice in BYO - 1 level (flexible)
  9. 9. 9 Preference-Based Conjoint (PBC) as a midway between CBC and ACBC 0 2 4 6 8 10 12 14 16 Frequency 0 2 4 6 8 10 12 14 16 Frequency 0 2 4 6 8 10 12 14 16 Frequency ACBC PBC (for example) CBC
  10. 10. 10 Test study
  11. 11. Different “legs”, each 250 respondents 11 1. CBC 2. ACBC* 3. PBC* 4. PBC2* 5. PBC2, frequency of all attributes dependent on initial preference 6. PBC*, half with 4 concepts/task, half with partial profile 10 concepts/task * Frequencies of three attributes dependent on initial preference (e.g. minutes and data, but not brand)
  12. 12. 12 Note on research leg with 10 concepts/screen Task 1-6 Task 7-9 Task 10-12 Holdout task
  13. 13. 13 Results
  14. 14. 0.01 0.015 0.02 0.15 0.2 0.25 0.3 14 Scatterplot of hit rate and MAE Good direction Wrong direction Hit rate (FC-based) MAE (SoP-based)
  15. 15. 15 PBC clearly performs better than currently available methods 0.01 0.015 0.02 0.15 0.2 0.25 0.3 Hit rate (FC-based) MAE (SoP-based) PBC ACBC CBC
  16. 16. 16 PBC2 worse than PBC 0.01 0.015 0.02 0.15 0.2 0.25 0.3 Hit rate (FC-based) MAE (SoP-based) PBC ACBC CBC PBC2
  17. 17. 17 PBC2 worse than PBC 0.01 0.015 0.02 0.15 0.2 0.25 0.3 Hit rate (FC-based) MAE (SoP-based) PBC ACBC CBC PBC2 PBC (all atts) PBC2 (all atts)
  18. 18. 18 PBC2 worse than PBC 0.01 0.015 0.02 0.15 0.2 0.25 0.3 Hit rate (FC-based) MAE (SoP-based) PBC ACBC CBC PBC2 PBC2 (all atts) PBC (partial concepts)
  19. 19. Adding BYO data improves hit rates even further 0.8% 1.2% 1.6% 2.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% Hit rate (FC-based) PBC PBC2 (all atts) ACBC CBC PBC2 PBC (partial concepts) Note: for ACBC this is the default setting 19 MAE (SoP-based)
  20. 20. Adding brand covariate improves hit rates further 0.8% 1.2% 1.6% 2.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% Hit rate (FC-based) MAE (SoP-based) PBC PBC2 (all atts) ACBC CBC PBC2 PBC (partial concepts) 20
  21. 21. 21 By combining multiple approaches we successfully improved the hit-rate from ~20% to ~40% > New method – PBC > Adding tasks with 10 partial profile concepts, sorted by brand > Adding BYO data to estimation > Adding current brand as a “smart” covariate
  22. 22. 22 Further research needed > Applying PBC to all attributes seems promising > Although we doubled the hit-rate from 20% to 40%, there are opportunities to improve further > Can we have a holdout tasks with 50 products? > More test studies needed to validate findings
  23. 23. SKIMgroup.com Thank you Jeroen Hardon Director Methodology & Innovation j.hardon@skimgroup.com Marco Hoogerbrugge Research Director m.hoogerbrugge@skimgroup.com

×