Successfully reported this slideshow.

×

# Bad Boy Matrix Question - Whatcha gonna do when they come for you?

## More Related Content

### Bad Boy Matrix Question - Whatcha gonna do when they come for you?

1. 1. BAD BOY MATRIX QUESTION Whatcha gonna do when they come for you? Florian Tress ODC Services GmbH, Germany GENERAL ONLINE RESEARCH 2012 5 – 7 March 2012 at the DHBW Mannheim
2. 2. 2 THE VENN DIAGRAM OF FIELDWORK Fun & entertainment Solve problems Incentives Collect (a lot of) valid data Disport Gain a lot of knowledge Curiosity Implement special (and possibly boring) Express an opinion methods PEOPLE RESEARCHERS
3. 3. 3 BAD BOY MATRIX QUESTION RESEARCHERS  Standardized data format, comparability of responses (with common orientation)  Maximize amount of data, minimize length of interview  Facilitate statistical procedures, e.g. factorial analysis, indices, etc.  Availability of validated test instruments , e.g. Big Five PEOPLE  Monotonous, boring  Overwhelming variety of options: “decision paralysis”  Nondifferentiation, satisficing behavior  Another question format would be more appropriate: Inferior data
4. 4. 4 STANDARD ALTERNATIVES MULTIPLE CHOICE  Matrix with a two point scale  Additional option “None of these” (mandatory question) RANKING  Bring statements in an order  Perfectly differentiated data
5. 5. 5 SPECIAL ALTERNATIVE: THE CAROUSEL  Only one statement presented at once  New statements slide in from the left  Response options remain in the same place
6. 6. 6 SPECIAL ALTERNATIVE: DRAG ME  All statements presented at once  Center stands for respondent  Measures distance from center  Arrangement of statements unimportant
7. 7. 7 COMPARISON OF THESE ALTERNATIVES Monadic Questionnaire : Random assignment to one of these alternatives n = 1080; 216 interviews per alternative; good spread over age and education Indicators: Comparability, Trustworthiness, Data Quality, Satisfaction, Technical Requirements
8. 8. 8 COMPARABILITY: BRAND LIKEABILITY Multiple Choice Carousel Amazon 73 Amazon 4,1 3 Factors (Cum.%: 56) | Cronbach„s α: 0,77. 3 Factors (Cum.%: 58) | Cronbach„s α: 0,78. Nivea 66 Nivea 4,1 Google 62 Google 3,9 IKEA 54 Volkswagen 3,7 Volkswagen 53 Ikea 3,5 Matrix Nutella 52 Siemens 3,5 Coca Cola 47 Amazon 4,0 BMW 3,5 BMW 43 Nivea 3,9 Coca Cola 3,4 Siemens 39 Google 3,9 Nutella 3,4 McDonalds 39 Volkswagen 3,7 McDonalds 3,2 Nutella 3,6 Ranking Coca Cola 3,5 Drag Me Siemens 3,5 Amazon 6,6 BMW 3,5 Amazon 68 3 Factors (Cum.%: 57) | Cronbach„s α: 0,73. 3 Factors (Cum.%: 49) | Cronbach„s α: n.a. Nivea 5,8 IKEA 3,5 Google 66 Google 5,0 McDonalds 3,1 Nivea 61 Volkswagen 4,6 3 Factors (Cum.%: 60) | Cronbach„s α: 0,80. Coca Cola 57 BMW 4,2 Ikea 52 Coca Cola 4,2 Siemens 52 Nutella 3,9 Volkswagen 51 IKEA 3,8 Nutella 49 Siemens 3,7 BMW 48 McDonalds 3,2 McDonalds 44
9. 9. 9 COMPARABILITY: ATTITUDES Multiple Choice Carousel A 62 A 3,9 3 Factors (Cum.%: 50) | Cronbach„s α: 0,56. 3 Factors (Cum.%: 61) | Cronbach„s α: 0,80. C 44 C 3,7 E 38 B 3,5 F 30 D 3,5 D 28 E 3,4 Matrix B 28 F 3,4 G 24 A 3,8 H 3,1 J 18 B 3,6 G 3,1 H 14 C 3,5 J 3,0 I 11 D 3,5 I 2,9 E 3,2 Ranking F 3,2 Drag Me G 3,1 C 6,4 H 3,0 A 65 3 Factors (Cum.%: 63) | Cronbach„s α: 0,82. 3 Factors (Cum.%: 51) | Cronbach„s α: n.a. A 6,4 I 2,8 C 63 D 4,9 J 2,8 E 55 E 4,9 3 Factors (Cum.%: 63) | Cronbach„s α: 0,82. F 52 F 4,6 G 50 B 4,0 D 49 J 3,8 B 44 I 3,6 J 41 H 3,4 I 39 G 3,0 H 32 (A) Wenn ich gute Erfahrungen mit einer Marke mache, empfehle ich sie aktiv weiter. (B) Werbung sollte mich stärker über Marken informieren, die ich noch nicht kenne. (C) Ich habe feste Marken, die ich bevorzugt einkaufe. (D) Neuartige und innovative Produkte passen gut zu meinem Lebensstil. (E) Ich probiere häufig neue Marken aus, die ich noch nicht kenne. (F) Ich bevorzuge Marken, die auf eine lange Tradition zurückblicken. (G) Ich bin bereit, für Markenprodukte mehr Geld auszugeben. (H) Werbung sollte mich stärker über Marken informieren, die ich bereits gut kenne. (I) Je bekannter eine Marke ist, desto leichter kann man ihr vertrauen. (J) Die Bekanntheit einer Marke sagt etwas ihre Qualität aus.
10. 10. 10 RESULT: COMPARABILITY Results correspond (roughly) for the most / least likeable brands (agreeable statements) But: Drag Me seems to measure something different / to be biased by third variables WINNER LOSER
11. 11. 11 TRUSTWORTHINESS: BRAND LIKEABILITY Follow-Up-Exploration: Why do you think, this is the most / least likeable brand? MULTIPLE CHOICE MATRIX CAROUSEL Ø Words pos.: 5,5 Ø Words pos.: 6,8 Ø Words pos.: 7,7 Ø Words neg.: 5,4 Ø Words neg.: 7,3 Ø Words neg.: 7,0 Non-Response: 3% Non-Response: 2% Non-Response: 3% RANKING DRAG ME Ø Words pos.: 6,3 Ø Words pos.: 7,2 Ø Words neg.: 6,7 Ø Words neg.: 7,2 Non-Response: 3% Non-Response: 2%
12. 12. 12 TRUSTWORTHINESS: ATTITUDES Follow-Up-Exploration: Why did you agree / disagree with this statement? MULTIPLE CHOICE MATRIX CAROUSEL Ø Words pos.: 7,4 Ø Words pos.: 7,9 Ø Words pos.: 9,3 Ø Words neg.: 8,6 Ø Words neg.: 9,6 Ø Words neg.: 10,0 Non-Response: 11% Non-Response: 9% Non-Response: 8% RANKING DRAG ME Ø Words pos.: 8,3 Ø Words pos.: 8,4 Ø Words neg.: 8,1 Ø Words neg.: 8,6 Non-Response: 7% Non-Response: 7%
13. 13. 13 RESULT: TRUSTWORTHINESS WINNER LOSER nonspecific / general specific / rich in detail „because I like it“ „good quality, fair prices“
14. 14. 14 DATA QUALITY: NONDIFFERENTIATION 13% 8% 8% 6% 5% 4% 0% 0% 0% 0% Multiple Choice Ranking Matrix Carousel Drag Me Brand Likeability Attitudes Low sample size!
15. 15. 15 DATA QUALITY: NONDIFFERENTIATION younger  40  older 7% Age 3% 5% 2% 1% 6% 2% 5% 2% 3% 3% 2% Brand Likeability Attitudes Brand Likeability Attitudes Brand Likeability Attitudes lower A-L. higher 7% Education 3% 4% 3% 2% 6% 2% 5% 3% 2% 2% 2% Brand Likeability Attitudes Brand Likeability Attitudes Brand Likeability Attitudes MULTIPLE CHOICE MATRIX CAROUSEL Low sample size!
16. 16. 16 RESULT: DATA QUALITY The data is perfectly differentiated with the Ranking and Drag Me Question. Among the other three alternatives, the Carousel performs best. WINNER LOSER
17. 17. 17 SATISFACTION Layout Usability Comprehensibility Carousel 71 20 5 Carousel 75 19 3 Carousel 74 15 2 Ranking 67 22 11 Ranking 73 20 6 Ranking 71 15 4 DragMe 66 23 10 DragMe 71 18 8 Matrix 70 16 4 Matrix 65 25 9 Matrix 70 23 6 DragMe 70 16 5 Multiple Choice 63 31 4 Multiple Choice 66 28 3 Multiple Choice 67 17 4 Topic Length Fun Carousel 64 15 8 Carousel 69 15 13 Carousel 62 14 10 Ranking 57 14 11 Ranking 66 18 11 Ranking 57 13 10 DragMe 55 16 11 DragMe 64 19 11 Matrix 54 13 12 Multiple Choice 54 15 11 Multiple Choice 61 21 15 DragMe 54 16 11 Matrix 54 12 14 Matrix 60 22 15 Multiple Choice 53 14 12
18. 18. 18 SATISFACTION Overall Carousel 59 24 14 Ranking 58 25 16 DragMe 57 25 15 Matrix 46 34 15 Multiple Choice 45 35 17 WINNER LOSER
19. 19. 19 SUMMARY MULTIPLE RANKING MATRIX CAROUSEL DRAG ME CHOICE Comparability Trustworthiness Data Quality Satisfaction Technical none JScript none JScript, Flash JScript, Flash Requirements
20. 20. 20 RECOMMENDATIONS CHECK, IF YOUR STUDY PERMITS THE USAGE OF JSCRIPT AND FLASH! (in most cases, it will) SELECT THE QUESTIONTYPES CAREFULLY! (there might be better alternatives) LAYOUT AND USABILITY MATTER! (the longer the interview, the more they matter) IF YOU HAVE DOUBTS ASK YOUR FIELDWORK PROVIDER! (they should have enough experience)
21. 21. FLORIAN TRESS THANK YOU VERY MUCH! www.odc-services.com f.tress@odc-services.com @FTress