1. An Introduction to MaxDiffWebinar: Tuesday August 24th, 2010

2. Agenda What is MaxDiff? Why all the fuss? Problems with Ratings Scales, and why MaxDiff is better Fielding and Analyzing MaxDiff questions Getting started 2

3. What is MaxDiff? Maximum Difference Scaling (MaxDiff) is a way of evaluating the importance (or preference) of a number of alternatives It is a discrete choice technique: respondents are asked to make simple best/worst choices Maxdiff has the advantage that it is very simple for the respondent, but gives extremely rich information to the researcher 3

4. But Wait! I can Do That Without MaxDiff! Traditionally, Marketing Research has used ratings scales to determine importance or preference, like this: There seems to be so much more information here. So why not just use ratings scales? 4

5. The Dark Secrets of Importance Ratings Scales Despite their popularity, ratings scales have several significant flaws that researchers are often unaware of, including: Scale-use bias Poor discrimination and comparison Poor predictive capability Cultural variance Let’s look at a some of these issues 5 For a good overview, see “Testing Alternatives to Importance Ratings”, Chrzanand Golovashkina, 2007

6. Scale Use Bias Are these respondents really different in their preferences? 6

7. Ratings Scales and the Lack of Constraints A big problem with traditional ratings scales is that they do not force the respondent to make a choice. Often this can mean that our data is meaningless. Consider the following question: Believe it or not, we once saw almost exactly this question fielded in a study. Please don’t do this! 7

8. The Advantages of MaxDiff Methods OK! I am convinced that traditional ratings scales have many problems. But what about MaxDiff? A well-designed Maxdiff exercise has the following properties: It is easy for respondents It is culturally invariant (as “most” and “least” are easily translated and have little room for interpretation) It forces a trade-off – respondents cannot just say everything is ‘very important’ There is no scale bias The data can be very powerful for predictive and clustering purposes It is robust for testing a ‘laundry list’ of unalike things 8

9. MaxDiff Questions In its simplest form a MaxDiff question is just a list of alternatives, with the respondent being asked to identify the Most/Least (or Best/Worst) pair: i.e. the ones with the ‘Maximum Difference’ For a small number of alternatives, asking a single question can suffice 9

10. MaxDiff with Multiple Questions When there are more alternatives we will want to ask each respondent multiple MaxDiff questions, with different combinations of alternatives in each ‘task’ The number of tasks shown depends on the total number of alternatives, and the number shown per task. Specialist software is typically used to design the tasks e.g. SPSS ORTHOPLAN, Sawtooth 10

11. Analyzing the Results There are three major options for analyzing the results of a MaxDiff exercise, each with its own strengths and weaknesses: Count Analysis: Simply tallying the number of times each alternative is chosen as ‘Most’ or ‘Least’ important by the population Pros: Very simple, provides population-level preferences Cons: Limited usefulness for analysis beyond the basics Logit Modeling: Using a logit model to calculate ‘utilities’ for each alternative Pros: Fast, scaled utilities can be compared and ‘share of utility’ can be calculated for population and for segments Cons: More complex than simple counts. Aggregate-level only results Hierarchical Bayes (HB) or Latent Class: More advanced mathematical techniques Pros: Robust techniques that produce respondent-level utilities. Results can be used in simulators or for segmentation Cons: Require advanced software and practitioner expertise 11

12. Interpreting the Numbers Counts: The simple counts can be expressed as percentages, for the number of times an item was shown as ‘best’ or ‘worst’. These can be ordered and reported. Some analysts like to show the differences (i.e. the number of times something was identified as ‘best’ minus the number of times it was identified as ‘worst’ Utilities: Logit and HB methods give each attribute tested a ‘utility’. Broadly speaking this is a measure of how strongly the attribute contributes to a decision Share of Utility: The utilities can be rescaled and converted into probabilities (using a logit transform). These can then be compared to give the relative weight of each attribute Simulation and Segmentation: Respondent-level utilities can be further analyzed either through segmentation or simulation 12

13. Example: A Custom MaxDiff Simulator 13

14. Some Tips for Effective MaxDiff Start simple: Don’t try to cram too much into your MaxDiff Keep it Short: Even though MaxDiff is a very easy task for respondents, having too many items in a task, or too many tasks, will cause respondents to answer with less care Make the scenario specific: A good scenario leaves the respondent with no doubt how to answer, and makes your analysis more effective 14

15. Getting Started MaxDiff is an exciting technique that is getting more and more popular – however, researchers need to continue moving away from traditional importance ratings and to these new techniques While the more advanced uses of MaxDiff are incredibly powerful, don’t be intimidated by their complexity or cost – even a simple MaxDiff with Counting Analysis can be better at extracting true user importance than the use of ratings scales Single Question MaxDiff with Logit Utilities Analysis is available now from Survey Analytics – see the following demonstration 15

16. Parametric Marketing LLC Parametric® provides research and analytics services to business professionals and policy makers. We specialize in analyzing consumer behavior and advising companies on the financial implications of product, pricing, promotional and brand decisions. With our expertise in advanced research, data mining and business & financial modeling, we are uniquely equipped to help our clients address their most demanding problems. Our innovative, interactive tools bring advanced techniques – such as conjoint, discrete choice modeling and customer valuation – within the reach of real business people. We also license our technology and provide analytics consulting to marketing research industry partners. Parametric was founded in 2003 by Chris Robson PRC, and Scott Laing PRC 400 E Evergreen Boulevard, Suite 303 Vancouver WA 98660 Phone: +1 360.696.2929 info@paramktg.com www.paramktg.com 16