Conjoint Analysis Alternatives in Questionnaire Design


Published on

Conjoint analysis in survey research is outmoded: static, closed, attribute-based in a real-time, turn-on-a-dime, conversational world. Heuristic methods offer a cheaper, faster, more actionable framework for both qualitative and quantitative work. This deck briefly outlines the quantitative framework.

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Conjoint Analysis Alternatives in Questionnaire Design

  1. 1. Trading up from tradeoffs: heuristic questionnaire design
  2. 2. Closed models have drawbacks “We got him!”
  3. 3. Why should static attribute models still drive research? • Developed before automated data collection • Designs based on team illusions, qualitative research, “limited” room • Require introspection and abstraction outside normal life, disclaiming the “X factor” • Create falsely dichotomous assumptions that inadequately inform decisions • Assume an overall “trade-off” between quality and price, when stakeholders may not
  4. 4. When is a decision difficult to model? • Specifiers, purchasers and users may differ • Manufacturers cannot easily change attributes and levels, even with high need • High stakes: decisions can disable or kill • Benefits and risks cannot be reliably generalized or projected • Commitment time permits re-evaluation, but decision burden for self (and often others) is long-term: cars, colleges, real estate
  5. 5. When is a decision a tradeoff? • Attributes are intrinsic to the product, and can only exist at one level at one time • Reasonable people agree on a product’s actual attribute levels, and are aware of those salient to them • Stakes and commitment times are significant; only infrequent re-evaluation is possible • Product data is empirical, not proxy • Many decisions will not meet these criteria
  6. 6. “Messaging for market segments” isn’t real life • Product profiles often present an artificial “full information” context – What stakeholders don’t care about, they are less likely to actually know • Specifiers seldom have a complete competitive set to consider • People make decisions, not segments, strata or audience groups – Context meets content – Mfr label says one thing, we do another
  7. 7. Human brains make bite-sized decisions • We use “heuristics” –decision shortcuts – because time is short and our brains are too small to consider everything at once • Heuristics can be simple: “Never pay extra for national brand peas,” or as complex as choosing a life partner • We always break our own rules – Heuristics are subject to mediating factors, e.g. anchoring and adjustment, priming; as well as situational constraints
  8. 8. Conjoint methods are product-centric • Experimental designs sometimes select profiles based on initial preferences, but attributes/levels are still pre-fabricated • Conjoint designs also assume: – Attributes represent a single construct, apart from interactions used in the experimental design – The distance between levels represents a finite, measurable value, that exists irrespective of any respondent’s reference point
  9. 9. The choice task subject is only human • She’s focusing on a few attributes, and making assumptions about those not shown – To complete the task in a reasonable time within her context • But analysts assume that she considered all and only the stimuli, in a zero-sum game • Table stakes may assume false importance, because excluded factors are the real drivers, and/or because the levels offered were not salient or even believable
  10. 10. Heuristic designs identify and leverage decision drivers • Respondents’ domains, measures and thresholds populate and limit the stimuli (not just profiles) presented – No two respondents may see the same questions or profiles – Base sizes for simulations will differ, since those whose benchmarks are unmet will not “contribute” to projected interest or share – Range of threshold values is defined by respondents, not a priori • Studies are cheap, fast, transparent and thus easily integrated
  11. 11. The “voice of the customer” is an N of 1 • Traditional respondent-level conjoint outputs: – Profile 1 preference share = XX% and so on – Imputed importance utilities and interpolated preference shares for the scenarios not presented • Heuristic studies: – Domain/measure (“attribute”) 1 = Z, with threshold of X, attribute 2 = C, with threshold of Y, and so on – Preference share given respondent’s thresholds (+/- X %) = XX% – Multiple scenarios can be presented, all salient to the respondent’s benchmarks
  12. 12. Heuristic designs help optimize decision support • Eliciting barriers to information-seeking, consideration, selection and purchase, including communication gaps • Developing support to facilitate use • Validating domains of unmet need and benchmark(s), often contrasting user, retailer, distributor, funder perspectives – In one case, arguing for a subsequently successful launch, albeit with an inferior delivery system
  13. 13. Thank you for listening! Laurie Gelb