SlideShare a Scribd company logo
BAD BOY MATRIX QUESTION
Whatcha gonna do when they come for you?




Florian Tress
ODC Services GmbH, Germany

GENERAL ONLINE RESEARCH 2012
5 – 7 March 2012 at the DHBW Mannheim
2
THE VENN DIAGRAM OF FIELDWORK


     Fun & entertainment     Solve problems


Incentives
                             Collect (a lot of) valid data

               Disport
                                Gain a lot of knowledge
 Curiosity

                             Implement special
                             (and possibly boring)
        Express an opinion   methods




             PEOPLE          RESEARCHERS
3
                        BAD BOY MATRIX QUESTION

  RESEARCHERS

 Standardized data format, comparability of responses (with common orientation)
 Maximize amount of data, minimize length of interview
 Facilitate statistical procedures, e.g. factorial analysis, indices, etc.
 Availability of validated test instruments , e.g. Big Five


          PEOPLE

 Monotonous, boring
 Overwhelming variety of options: “decision paralysis”
 Nondifferentiation, satisficing behavior
 Another question format would be more appropriate: Inferior data
4
          STANDARD ALTERNATIVES



                  MULTIPLE CHOICE
                   Matrix with a two point scale
                   Additional option “None of these” (mandatory question)




RANKING
 Bring statements in an order
 Perfectly differentiated data
5
SPECIAL ALTERNATIVE: THE CAROUSEL


                   Only one statement presented at once
                   New statements slide in from the left
                   Response options remain in the same place
6
SPECIAL ALTERNATIVE: DRAG ME




                     All statements presented at once
                     Center stands for respondent
                     Measures distance from center
                     Arrangement of statements unimportant
7
         COMPARISON OF THESE ALTERNATIVES

          Monadic Questionnaire : Random assignment to one of these alternatives


        n = 1080; 216 interviews per alternative; good spread over age and education


Indicators: Comparability, Trustworthiness, Data Quality, Satisfaction, Technical Requirements
8
                                                                 COMPARABILITY: BRAND LIKEABILITY
                                                            Multiple Choice                                                                                   Carousel

                                                  Amazon                            73                                                              Amazon                           4,1
3 Factors (Cum.%: 56) | Cronbach„s α: 0,77.




                                                                                                                                                                                           3 Factors (Cum.%: 58) | Cronbach„s α: 0,78.
                                                    Nivea                         66                                                                  Nivea                          4,1
                                                   Google                        62                                                                  Google                        3,9
                                                     IKEA                   54                                                                  Volkswagen                        3,7
                                              Volkswagen                   53                                                                          Ikea                      3,5
                                                                                                               Matrix
                                                  Nutella                  52                                                                      Siemens                       3,5
                                               Coca Cola                 47                  Amazon                                       4,0          BMW                      3,5
                                                     BMW                43                     Nivea                                      3,9    Coca Cola                      3,4
                                                 Siemens               39                     Google                                      3,9       Nutella                     3,4
                                               McDonalds               39                Volkswagen                                     3,7      McDonalds                     3,2
                                                                                             Nutella                                    3,6
                                                               Ranking                    Coca Cola                                    3,5                    Drag Me
                                                                                            Siemens                                   3,5
                                                  Amazon                       6,6              BMW                                   3,5           Amazon                       68




                                                                                                                                                                                           3 Factors (Cum.%: 57) | Cronbach„s α: 0,73.
3 Factors (Cum.%: 49) | Cronbach„s α: n.a.




                                                    Nivea                    5,8                IKEA                                  3,5            Google                     66
                                                   Google                  5,0            McDonalds                                 3,1               Nivea                    61
                                              Volkswagen                 4,6                   3 Factors (Cum.%: 60) | Cronbach„s α: 0,80.       Coca Cola                   57
                                                     BMW                4,2                                                                            Ikea                 52
                                               Coca Cola                4,2                                                                        Siemens                 52
                                                  Nutella              3,9                                                                      Volkswagen                 51
                                                     IKEA              3,8                                                                          Nutella               49
                                                 Siemens              3,7                                                                              BMW                48
                                               McDonalds             3,2                                                                         McDonalds               44
9
                                                                                          COMPARABILITY: ATTITUDES
                                                               Multiple Choice                                                                                                                               Carousel

                                              A                                        62                                                                                           A                                                 3,9
3 Factors (Cum.%: 50) | Cronbach„s α: 0,56.




                                                                                                                                                                                                                                               3 Factors (Cum.%: 61) | Cronbach„s α: 0,80.
                                              C                              44                                                                                                     C                                              3,7
                                              E                         38                                                                                                          B                                             3,5
                                              F                    30                                                                                                               D                                             3,5
                                              D                   28                                                                                                                E                                           3,4
                                                                                                                                       Matrix
                                              B                   28                                                                                                                F                                           3,4
                                              G                  24                                         A                                                        3,8            H                                         3,1
                                              J             18                                              B                                                       3,6             G                                         3,1
                                              H           14                                                C                                                     3,5               J                                        3,0
                                               I         11                                                 D                                                     3,5                I                                      2,9
                                                                                                            E                                                   3,2
                                                                     Ranking                                F                                                   3,2                                          Drag Me
                                                                                                            G                                                 3,1
                                              C                                         6,4                 H                                                3,0                    A                                          65




                                                                                                                                                                                                                                               3 Factors (Cum.%: 63) | Cronbach„s α: 0,82.
3 Factors (Cum.%: 51) | Cronbach„s α: n.a.




                                              A                                         6,4                  I                                             2,8                      C                                          63
                                              D                              4,9                            J                                              2,8                      E                                    55
                                              E                              4,9                                      3 Factors (Cum.%: 63) | Cronbach„s α: 0,82.                   F                                  52
                                              F                            4,6                                                                                                      G                                  50
                                              B                          4,0                                                                                                        D                                 49
                                              J                         3,8                                                                                                         B                               44
                                               I                       3,6                                                                                                          J                              41
                                              H                       3,4                                                                                                            I                            39
                                              G                     3,0                                                                                                             H                        32

                                                   (A) Wenn ich gute Erfahrungen mit einer Marke mache, empfehle ich sie aktiv weiter. (B) Werbung sollte mich stärker über Marken informieren, die ich noch nicht kenne. (C) Ich
                                                   habe feste Marken, die ich bevorzugt einkaufe. (D) Neuartige und innovative Produkte passen gut zu meinem Lebensstil. (E) Ich probiere häufig neue Marken aus, die ich noch
                                                   nicht kenne. (F) Ich bevorzuge Marken, die auf eine lange Tradition zurückblicken. (G) Ich bin bereit, für Markenprodukte mehr Geld auszugeben. (H) Werbung sollte mich stärker
                                                   über Marken informieren, die ich bereits gut kenne. (I) Je bekannter eine Marke ist, desto leichter kann man ihr vertrauen. (J) Die Bekanntheit einer Marke sagt etwas ihre Qualität aus.
10
                    RESULT: COMPARABILITY

Results correspond (roughly) for the most / least likeable brands (agreeable statements)

 But: Drag Me seems to measure something different / to be biased by third variables




                  WINNER                                 LOSER
11
          TRUSTWORTHINESS: BRAND LIKEABILITY
          Follow-Up-Exploration: Why do you think, this is the most / least likeable brand?




MULTIPLE CHOICE                               MATRIX                                   CAROUSEL

Ø Words pos.: 5,5                        Ø Words pos.: 6,8                          Ø Words pos.: 7,7
Ø Words neg.: 5,4                        Ø Words neg.: 7,3                          Ø Words neg.: 7,0
Non-Response: 3%                         Non-Response: 2%                           Non-Response: 3%


                        RANKING                                    DRAG ME

                    Ø Words pos.: 6,3                          Ø Words pos.: 7,2
                    Ø Words neg.: 6,7                          Ø Words neg.: 7,2
                    Non-Response: 3%                           Non-Response: 2%
12
                    TRUSTWORTHINESS: ATTITUDES
             Follow-Up-Exploration: Why did you agree / disagree with this statement?




MULTIPLE CHOICE                              MATRIX                                     CAROUSEL

Ø Words pos.: 7,4                       Ø Words pos.: 7,9                        Ø Words pos.: 9,3
Ø Words neg.: 8,6                       Ø Words neg.: 9,6                        Ø Words neg.: 10,0
Non-Response: 11%                       Non-Response: 9%                         Non-Response: 8%


                        RANKING                                  DRAG ME

                    Ø Words pos.: 8,3                        Ø Words pos.: 8,4
                    Ø Words neg.: 8,1                        Ø Words neg.: 8,6
                    Non-Response: 7%                         Non-Response: 7%
13
                        RESULT: TRUSTWORTHINESS




                                                         WINNER




    LOSER
nonspecific / general                           specific / rich in detail
„because I like it“                          „good quality, fair prices“
14
         DATA QUALITY: NONDIFFERENTIATION
   13%




                                           8%
         8%

                                     6%
                                                             5%
                                                        4%




                  0%   0%                                         0%   0%

Multiple Choice   Ranking             Matrix           Carousel   Drag Me
                            Brand Likeability   Attitudes


                                    Low sample size!
15
                                             DATA QUALITY: NONDIFFERENTIATION
          younger  40  older




                                        7%
Age




                                                                                          3%
                                                        5%              2%                                                1%
                                        6%                                                                2%
                                                                                          5%
                                                        2%              3%                                                3%
                                                                                                          2%

                                 Brand Likeability   Attitudes   Brand Likeability     Attitudes   Brand Likeability   Attitudes
lower A-L. higher




                                        7%
      Education




                                                                                          3%
                                                        4%
                                                                        3%                                                2%
                                        6%                                                                2%
                                                                                          5%
                                                        3%              2%                                2%              2%

                                 Brand Likeability   Attitudes   Brand Likeability     Attitudes   Brand Likeability   Attitudes



                                        MULTIPLE CHOICE                       MATRIX                           CAROUSEL

                                                                         Low sample size!
16
                  RESULT: DATA QUALITY
   The data is perfectly differentiated with the Ranking and Drag Me Question.

        Among the other three alternatives, the Carousel performs best.




                                                                           WINNER
LOSER
17
                                       SATISFACTION

             Layout                             Usability                  Comprehensibility
      Carousel        71    20 5           Carousel     75     19 3         Carousel           74     15 2

       Ranking        67   22 11            Ranking     73     20 6          Ranking           71     15 4

       DragMe         66   23 10            DragMe      71    18 8            Matrix           70     16 4

        Matrix        65   25 9              Matrix    70     23 6           DragMe            70     16 5

Multiple Choice   63        31   4   Multiple Choice   66     28 3    Multiple Choice          67     17 4




              Topic                               Length                                Fun
      Carousel        64    15 8           Carousel    69    15 13          Carousel          62     14 10

       Ranking        57   14 11            Ranking    66    18 11           Ranking          57    13 10

       DragMe         55   16 11            DragMe     64    19 11            Matrix          54    13 12

Multiple Choice       54   15 11     Multiple Choice   61    21 15           DragMe           54    16 11

        Matrix        54   12 14             Matrix    60    22 15    Multiple Choice         53    14 12
18
                  SATISFACTION
                     Overall
      Carousel            59        24   14

       Ranking            58        25    16

       DragMe             57        25   15

        Matrix       46        34        15

Multiple Choice      45        35        17




        WINNER                 LOSER
19
                             SUMMARY



                  MULTIPLE
                             RANKING   MATRIX    CAROUSEL          DRAG ME
                   CHOICE


Comparability


Trustworthiness


Data Quality


Satisfaction

Technical
                   none      JScript   none     JScript, Flash   JScript, Flash
Requirements
20
            RECOMMENDATIONS
CHECK, IF YOUR STUDY PERMITS THE USAGE OF JSCRIPT AND FLASH!
                    (in most cases, it will)


           SELECT THE QUESTIONTYPES CAREFULLY!
             (there might be better alternatives)


                LAYOUT AND USABILITY MATTER!
        (the longer the interview, the more they matter)


     IF YOU HAVE DOUBTS ASK YOUR FIELDWORK PROVIDER!
             (they should have enough experience)
FLORIAN TRESS




THANK YOU
VERY MUCH!
             www.odc-services.com
             f.tress@odc-services.com
             @FTress

More Related Content

Viewers also liked

Ch 7 design of rcc footing
Ch 7 design of rcc footingCh 7 design of rcc footing
Ch 7 design of rcc footing
MD.MAHBUB UL ALAM
 
Stress Among teenes (questionare)
Stress Among teenes (questionare)Stress Among teenes (questionare)
Stress Among teenes (questionare)
VaishnaviSSSikarwar
 
Línea del tiempo del computador
Línea del tiempo del computadorLínea del tiempo del computador
Línea del tiempo del computador
lucasdn
 
Mass Media and Society, Chapter 1: Media and Culture
Mass Media and Society, Chapter 1: Media and CultureMass Media and Society, Chapter 1: Media and Culture
Mass Media and Society, Chapter 1: Media and Culture
czavisca
 
Holderness Coastline - Management
Holderness Coastline - ManagementHolderness Coastline - Management
Holderness Coastline - Management
Steven Heath
 
Natürlich Online
Natürlich OnlineNatürlich Online
Natürlich Online
Florian Tress
 
Energia mar
Energia marEnergia mar
Energia mar
mjtecno
 
Build Features, Not Apps
Build Features, Not AppsBuild Features, Not Apps
Build Features, Not Apps
Natasha Murashev
 

Viewers also liked (9)

Ch 7 design of rcc footing
Ch 7 design of rcc footingCh 7 design of rcc footing
Ch 7 design of rcc footing
 
Stress Among teenes (questionare)
Stress Among teenes (questionare)Stress Among teenes (questionare)
Stress Among teenes (questionare)
 
eBook-Begynd med CRM!
eBook-Begynd med CRM!eBook-Begynd med CRM!
eBook-Begynd med CRM!
 
Línea del tiempo del computador
Línea del tiempo del computadorLínea del tiempo del computador
Línea del tiempo del computador
 
Mass Media and Society, Chapter 1: Media and Culture
Mass Media and Society, Chapter 1: Media and CultureMass Media and Society, Chapter 1: Media and Culture
Mass Media and Society, Chapter 1: Media and Culture
 
Holderness Coastline - Management
Holderness Coastline - ManagementHolderness Coastline - Management
Holderness Coastline - Management
 
Natürlich Online
Natürlich OnlineNatürlich Online
Natürlich Online
 
Energia mar
Energia marEnergia mar
Energia mar
 
Build Features, Not Apps
Build Features, Not AppsBuild Features, Not Apps
Build Features, Not Apps
 

More from Florian Tress

Personality Matters - Qualitative Online Research
Personality Matters - Qualitative Online ResearchPersonality Matters - Qualitative Online Research
Personality Matters - Qualitative Online Research
Florian Tress
 
The Monetary Value of Good Questionnaire Design
The Monetary Value of Good Questionnaire DesignThe Monetary Value of Good Questionnaire Design
The Monetary Value of Good Questionnaire Design
Florian Tress
 
Surveytainment 2.0
Surveytainment 2.0Surveytainment 2.0
Surveytainment 2.0
Florian Tress
 
Let me surveytain you! Designing appealing questionnaires for better results
Let me surveytain you! Designing appealing questionnaires for better resultsLet me surveytain you! Designing appealing questionnaires for better results
Let me surveytain you! Designing appealing questionnaires for better results
Florian Tress
 
Challenges on the Way to a Mobile Panel
Challenges on the Way to a Mobile PanelChallenges on the Way to a Mobile Panel
Challenges on the Way to a Mobile Panel
Florian Tress
 
Rich Profiles
Rich ProfilesRich Profiles
Rich Profiles
Florian Tress
 
Are well-selected panelists better respondents?
Are well-selected panelists better respondents?Are well-selected panelists better respondents?
Are well-selected panelists better respondents?
Florian Tress
 

More from Florian Tress (7)

Personality Matters - Qualitative Online Research
Personality Matters - Qualitative Online ResearchPersonality Matters - Qualitative Online Research
Personality Matters - Qualitative Online Research
 
The Monetary Value of Good Questionnaire Design
The Monetary Value of Good Questionnaire DesignThe Monetary Value of Good Questionnaire Design
The Monetary Value of Good Questionnaire Design
 
Surveytainment 2.0
Surveytainment 2.0Surveytainment 2.0
Surveytainment 2.0
 
Let me surveytain you! Designing appealing questionnaires for better results
Let me surveytain you! Designing appealing questionnaires for better resultsLet me surveytain you! Designing appealing questionnaires for better results
Let me surveytain you! Designing appealing questionnaires for better results
 
Challenges on the Way to a Mobile Panel
Challenges on the Way to a Mobile PanelChallenges on the Way to a Mobile Panel
Challenges on the Way to a Mobile Panel
 
Rich Profiles
Rich ProfilesRich Profiles
Rich Profiles
 
Are well-selected panelists better respondents?
Are well-selected panelists better respondents?Are well-selected panelists better respondents?
Are well-selected panelists better respondents?
 

Recently uploaded

foodgasm restaurant and Bar pune road.docx
foodgasm restaurant and Bar pune road.docxfoodgasm restaurant and Bar pune road.docx
foodgasm restaurant and Bar pune road.docx
PraghyaBhandari
 
A Complete Guide of Dubai Freelance Visa and Permit in 2024
A Complete Guide of Dubai Freelance Visa and Permit in 2024A Complete Guide of Dubai Freelance Visa and Permit in 2024
A Complete Guide of Dubai Freelance Visa and Permit in 2024
Dubiz
 
Patrick Dwyer Merrill Lynch - Founder of the Dwyer Family Foundation
Patrick Dwyer Merrill Lynch - Founder of the Dwyer Family FoundationPatrick Dwyer Merrill Lynch - Founder of the Dwyer Family Foundation
Patrick Dwyer Merrill Lynch - Founder of the Dwyer Family Foundation
Patrick Dwyer Merrill Lynch
 
Green Minimalist Aesthetic Project Proposal Presentation.pdf
Green Minimalist Aesthetic Project Proposal Presentation.pdfGreen Minimalist Aesthetic Project Proposal Presentation.pdf
Green Minimalist Aesthetic Project Proposal Presentation.pdf
shivamkush646
 
BBA Final SML 501 INTERNATIONAL BUSINESS .pdf
BBA Final SML 501 INTERNATIONAL BUSINESS .pdfBBA Final SML 501 INTERNATIONAL BUSINESS .pdf
BBA Final SML 501 INTERNATIONAL BUSINESS .pdf
mcdopex6
 
Corporate Governance for South African Mining Companies
Corporate Governance for South African Mining CompaniesCorporate Governance for South African Mining Companies
Corporate Governance for South African Mining Companies
James AH Campbell
 
TEST BANK For Auditing & Assurance Services A Systematic Approach, 12th Editi...
TEST BANK For Auditing & Assurance Services A Systematic Approach, 12th Editi...TEST BANK For Auditing & Assurance Services A Systematic Approach, 12th Editi...
TEST BANK For Auditing & Assurance Services A Systematic Approach, 12th Editi...
kevinkariuki227
 
Andrew Wilmot- Ecentric Payment Systems: Macro Trends Shaping Payments in Afr...
Andrew Wilmot- Ecentric Payment Systems: Macro Trends Shaping Payments in Afr...Andrew Wilmot- Ecentric Payment Systems: Macro Trends Shaping Payments in Afr...
Andrew Wilmot- Ecentric Payment Systems: Macro Trends Shaping Payments in Afr...
itnewsafrica
 
Innovation Hub_ Spotlight on Toms River's Role as a Beacon for Entrepreneuria...
Innovation Hub_ Spotlight on Toms River's Role as a Beacon for Entrepreneuria...Innovation Hub_ Spotlight on Toms River's Role as a Beacon for Entrepreneuria...
Innovation Hub_ Spotlight on Toms River's Role as a Beacon for Entrepreneuria...
Philip M Caputo
 
Cracking the Customer Experience Code.pptx
Cracking the Customer Experience Code.pptxCracking the Customer Experience Code.pptx
Cracking the Customer Experience Code.pptx
Workforce Group
 
WAM Corporate Presentation July 2024.pdf
WAM Corporate Presentation July 2024.pdfWAM Corporate Presentation July 2024.pdf
WAM Corporate Presentation July 2024.pdf
Western Alaska Minerals Corp.
 
Zodiac Signs and Fashion: Dressing to Suit Your Astrological Style
Zodiac Signs and Fashion: Dressing to Suit Your Astrological StyleZodiac Signs and Fashion: Dressing to Suit Your Astrological Style
Zodiac Signs and Fashion: Dressing to Suit Your Astrological Style
my Pandit
 
Navigating Change Strategies for Effective Transition and Operational Plannin...
Navigating Change Strategies for Effective Transition and Operational Plannin...Navigating Change Strategies for Effective Transition and Operational Plannin...
Navigating Change Strategies for Effective Transition and Operational Plannin...
Brian Frerichs
 
The Most Innovative Blockchain Company to Follow in 2024 (5).pdf
The Most Innovative Blockchain Company to Follow in 2024 (5).pdfThe Most Innovative Blockchain Company to Follow in 2024 (5).pdf
The Most Innovative Blockchain Company to Follow in 2024 (5).pdf
analyticsinsightmaga
 
AI at Work​ The demystification of AI and real-world stories on how to apply ...
AI at Work​ The demystification of AI and real-world stories on how to apply ...AI at Work​ The demystification of AI and real-world stories on how to apply ...
AI at Work​ The demystification of AI and real-world stories on how to apply ...
Auxis Consulting & Outsourcing
 
Connected Small Boat Protection Solution | July 2024
Connected Small Boat Protection Solution | July  2024Connected Small Boat Protection Solution | July  2024
Connected Small Boat Protection Solution | July 2024
Hector Del Castillo, CPM, CPMM
 
Best Practices in Negotiation for Business Management Students
Best Practices in Negotiation for Business Management StudentsBest Practices in Negotiation for Business Management Students
Best Practices in Negotiation for Business Management Students
zoyaws1
 
PAWFESSIONAL ELVA MAX.pdfAAAAAAAAAAAAAAAAAAA
PAWFESSIONAL ELVA MAX.pdfAAAAAAAAAAAAAAAAAAAPAWFESSIONAL ELVA MAX.pdfAAAAAAAAAAAAAAAAAAA
PAWFESSIONAL ELVA MAX.pdfAAAAAAAAAAAAAAAAAAA
lawrenceads01
 
Retail Store Scavenger Hunt powerpoint slides
Retail Store Scavenger Hunt powerpoint slidesRetail Store Scavenger Hunt powerpoint slides
Retail Store Scavenger Hunt powerpoint slides
JairSemexant
 
شركات إبراهيم العرجاني: لدعم الاقتصاد المصري
شركات إبراهيم العرجاني: لدعم الاقتصاد المصريشركات إبراهيم العرجاني: لدعم الاقتصاد المصري
شركات إبراهيم العرجاني: لدعم الاقتصاد المصري
إبراهيم العرجاني
 

Recently uploaded (20)

foodgasm restaurant and Bar pune road.docx
foodgasm restaurant and Bar pune road.docxfoodgasm restaurant and Bar pune road.docx
foodgasm restaurant and Bar pune road.docx
 
A Complete Guide of Dubai Freelance Visa and Permit in 2024
A Complete Guide of Dubai Freelance Visa and Permit in 2024A Complete Guide of Dubai Freelance Visa and Permit in 2024
A Complete Guide of Dubai Freelance Visa and Permit in 2024
 
Patrick Dwyer Merrill Lynch - Founder of the Dwyer Family Foundation
Patrick Dwyer Merrill Lynch - Founder of the Dwyer Family FoundationPatrick Dwyer Merrill Lynch - Founder of the Dwyer Family Foundation
Patrick Dwyer Merrill Lynch - Founder of the Dwyer Family Foundation
 
Green Minimalist Aesthetic Project Proposal Presentation.pdf
Green Minimalist Aesthetic Project Proposal Presentation.pdfGreen Minimalist Aesthetic Project Proposal Presentation.pdf
Green Minimalist Aesthetic Project Proposal Presentation.pdf
 
BBA Final SML 501 INTERNATIONAL BUSINESS .pdf
BBA Final SML 501 INTERNATIONAL BUSINESS .pdfBBA Final SML 501 INTERNATIONAL BUSINESS .pdf
BBA Final SML 501 INTERNATIONAL BUSINESS .pdf
 
Corporate Governance for South African Mining Companies
Corporate Governance for South African Mining CompaniesCorporate Governance for South African Mining Companies
Corporate Governance for South African Mining Companies
 
TEST BANK For Auditing & Assurance Services A Systematic Approach, 12th Editi...
TEST BANK For Auditing & Assurance Services A Systematic Approach, 12th Editi...TEST BANK For Auditing & Assurance Services A Systematic Approach, 12th Editi...
TEST BANK For Auditing & Assurance Services A Systematic Approach, 12th Editi...
 
Andrew Wilmot- Ecentric Payment Systems: Macro Trends Shaping Payments in Afr...
Andrew Wilmot- Ecentric Payment Systems: Macro Trends Shaping Payments in Afr...Andrew Wilmot- Ecentric Payment Systems: Macro Trends Shaping Payments in Afr...
Andrew Wilmot- Ecentric Payment Systems: Macro Trends Shaping Payments in Afr...
 
Innovation Hub_ Spotlight on Toms River's Role as a Beacon for Entrepreneuria...
Innovation Hub_ Spotlight on Toms River's Role as a Beacon for Entrepreneuria...Innovation Hub_ Spotlight on Toms River's Role as a Beacon for Entrepreneuria...
Innovation Hub_ Spotlight on Toms River's Role as a Beacon for Entrepreneuria...
 
Cracking the Customer Experience Code.pptx
Cracking the Customer Experience Code.pptxCracking the Customer Experience Code.pptx
Cracking the Customer Experience Code.pptx
 
WAM Corporate Presentation July 2024.pdf
WAM Corporate Presentation July 2024.pdfWAM Corporate Presentation July 2024.pdf
WAM Corporate Presentation July 2024.pdf
 
Zodiac Signs and Fashion: Dressing to Suit Your Astrological Style
Zodiac Signs and Fashion: Dressing to Suit Your Astrological StyleZodiac Signs and Fashion: Dressing to Suit Your Astrological Style
Zodiac Signs and Fashion: Dressing to Suit Your Astrological Style
 
Navigating Change Strategies for Effective Transition and Operational Plannin...
Navigating Change Strategies for Effective Transition and Operational Plannin...Navigating Change Strategies for Effective Transition and Operational Plannin...
Navigating Change Strategies for Effective Transition and Operational Plannin...
 
The Most Innovative Blockchain Company to Follow in 2024 (5).pdf
The Most Innovative Blockchain Company to Follow in 2024 (5).pdfThe Most Innovative Blockchain Company to Follow in 2024 (5).pdf
The Most Innovative Blockchain Company to Follow in 2024 (5).pdf
 
AI at Work​ The demystification of AI and real-world stories on how to apply ...
AI at Work​ The demystification of AI and real-world stories on how to apply ...AI at Work​ The demystification of AI and real-world stories on how to apply ...
AI at Work​ The demystification of AI and real-world stories on how to apply ...
 
Connected Small Boat Protection Solution | July 2024
Connected Small Boat Protection Solution | July  2024Connected Small Boat Protection Solution | July  2024
Connected Small Boat Protection Solution | July 2024
 
Best Practices in Negotiation for Business Management Students
Best Practices in Negotiation for Business Management StudentsBest Practices in Negotiation for Business Management Students
Best Practices in Negotiation for Business Management Students
 
PAWFESSIONAL ELVA MAX.pdfAAAAAAAAAAAAAAAAAAA
PAWFESSIONAL ELVA MAX.pdfAAAAAAAAAAAAAAAAAAAPAWFESSIONAL ELVA MAX.pdfAAAAAAAAAAAAAAAAAAA
PAWFESSIONAL ELVA MAX.pdfAAAAAAAAAAAAAAAAAAA
 
Retail Store Scavenger Hunt powerpoint slides
Retail Store Scavenger Hunt powerpoint slidesRetail Store Scavenger Hunt powerpoint slides
Retail Store Scavenger Hunt powerpoint slides
 
شركات إبراهيم العرجاني: لدعم الاقتصاد المصري
شركات إبراهيم العرجاني: لدعم الاقتصاد المصريشركات إبراهيم العرجاني: لدعم الاقتصاد المصري
شركات إبراهيم العرجاني: لدعم الاقتصاد المصري
 

Bad Boy Matrix Question - Whatcha gonna do when they come for you?

  • 1. BAD BOY MATRIX QUESTION Whatcha gonna do when they come for you? Florian Tress ODC Services GmbH, Germany GENERAL ONLINE RESEARCH 2012 5 – 7 March 2012 at the DHBW Mannheim
  • 2. 2 THE VENN DIAGRAM OF FIELDWORK Fun & entertainment Solve problems Incentives Collect (a lot of) valid data Disport Gain a lot of knowledge Curiosity Implement special (and possibly boring) Express an opinion methods PEOPLE RESEARCHERS
  • 3. 3 BAD BOY MATRIX QUESTION RESEARCHERS  Standardized data format, comparability of responses (with common orientation)  Maximize amount of data, minimize length of interview  Facilitate statistical procedures, e.g. factorial analysis, indices, etc.  Availability of validated test instruments , e.g. Big Five PEOPLE  Monotonous, boring  Overwhelming variety of options: “decision paralysis”  Nondifferentiation, satisficing behavior  Another question format would be more appropriate: Inferior data
  • 4. 4 STANDARD ALTERNATIVES MULTIPLE CHOICE  Matrix with a two point scale  Additional option “None of these” (mandatory question) RANKING  Bring statements in an order  Perfectly differentiated data
  • 5. 5 SPECIAL ALTERNATIVE: THE CAROUSEL  Only one statement presented at once  New statements slide in from the left  Response options remain in the same place
  • 6. 6 SPECIAL ALTERNATIVE: DRAG ME  All statements presented at once  Center stands for respondent  Measures distance from center  Arrangement of statements unimportant
  • 7. 7 COMPARISON OF THESE ALTERNATIVES Monadic Questionnaire : Random assignment to one of these alternatives n = 1080; 216 interviews per alternative; good spread over age and education Indicators: Comparability, Trustworthiness, Data Quality, Satisfaction, Technical Requirements
  • 8. 8 COMPARABILITY: BRAND LIKEABILITY Multiple Choice Carousel Amazon 73 Amazon 4,1 3 Factors (Cum.%: 56) | Cronbach„s α: 0,77. 3 Factors (Cum.%: 58) | Cronbach„s α: 0,78. Nivea 66 Nivea 4,1 Google 62 Google 3,9 IKEA 54 Volkswagen 3,7 Volkswagen 53 Ikea 3,5 Matrix Nutella 52 Siemens 3,5 Coca Cola 47 Amazon 4,0 BMW 3,5 BMW 43 Nivea 3,9 Coca Cola 3,4 Siemens 39 Google 3,9 Nutella 3,4 McDonalds 39 Volkswagen 3,7 McDonalds 3,2 Nutella 3,6 Ranking Coca Cola 3,5 Drag Me Siemens 3,5 Amazon 6,6 BMW 3,5 Amazon 68 3 Factors (Cum.%: 57) | Cronbach„s α: 0,73. 3 Factors (Cum.%: 49) | Cronbach„s α: n.a. Nivea 5,8 IKEA 3,5 Google 66 Google 5,0 McDonalds 3,1 Nivea 61 Volkswagen 4,6 3 Factors (Cum.%: 60) | Cronbach„s α: 0,80. Coca Cola 57 BMW 4,2 Ikea 52 Coca Cola 4,2 Siemens 52 Nutella 3,9 Volkswagen 51 IKEA 3,8 Nutella 49 Siemens 3,7 BMW 48 McDonalds 3,2 McDonalds 44
  • 9. 9 COMPARABILITY: ATTITUDES Multiple Choice Carousel A 62 A 3,9 3 Factors (Cum.%: 50) | Cronbach„s α: 0,56. 3 Factors (Cum.%: 61) | Cronbach„s α: 0,80. C 44 C 3,7 E 38 B 3,5 F 30 D 3,5 D 28 E 3,4 Matrix B 28 F 3,4 G 24 A 3,8 H 3,1 J 18 B 3,6 G 3,1 H 14 C 3,5 J 3,0 I 11 D 3,5 I 2,9 E 3,2 Ranking F 3,2 Drag Me G 3,1 C 6,4 H 3,0 A 65 3 Factors (Cum.%: 63) | Cronbach„s α: 0,82. 3 Factors (Cum.%: 51) | Cronbach„s α: n.a. A 6,4 I 2,8 C 63 D 4,9 J 2,8 E 55 E 4,9 3 Factors (Cum.%: 63) | Cronbach„s α: 0,82. F 52 F 4,6 G 50 B 4,0 D 49 J 3,8 B 44 I 3,6 J 41 H 3,4 I 39 G 3,0 H 32 (A) Wenn ich gute Erfahrungen mit einer Marke mache, empfehle ich sie aktiv weiter. (B) Werbung sollte mich stärker über Marken informieren, die ich noch nicht kenne. (C) Ich habe feste Marken, die ich bevorzugt einkaufe. (D) Neuartige und innovative Produkte passen gut zu meinem Lebensstil. (E) Ich probiere häufig neue Marken aus, die ich noch nicht kenne. (F) Ich bevorzuge Marken, die auf eine lange Tradition zurückblicken. (G) Ich bin bereit, für Markenprodukte mehr Geld auszugeben. (H) Werbung sollte mich stärker über Marken informieren, die ich bereits gut kenne. (I) Je bekannter eine Marke ist, desto leichter kann man ihr vertrauen. (J) Die Bekanntheit einer Marke sagt etwas ihre Qualität aus.
  • 10. 10 RESULT: COMPARABILITY Results correspond (roughly) for the most / least likeable brands (agreeable statements) But: Drag Me seems to measure something different / to be biased by third variables WINNER LOSER
  • 11. 11 TRUSTWORTHINESS: BRAND LIKEABILITY Follow-Up-Exploration: Why do you think, this is the most / least likeable brand? MULTIPLE CHOICE MATRIX CAROUSEL Ø Words pos.: 5,5 Ø Words pos.: 6,8 Ø Words pos.: 7,7 Ø Words neg.: 5,4 Ø Words neg.: 7,3 Ø Words neg.: 7,0 Non-Response: 3% Non-Response: 2% Non-Response: 3% RANKING DRAG ME Ø Words pos.: 6,3 Ø Words pos.: 7,2 Ø Words neg.: 6,7 Ø Words neg.: 7,2 Non-Response: 3% Non-Response: 2%
  • 12. 12 TRUSTWORTHINESS: ATTITUDES Follow-Up-Exploration: Why did you agree / disagree with this statement? MULTIPLE CHOICE MATRIX CAROUSEL Ø Words pos.: 7,4 Ø Words pos.: 7,9 Ø Words pos.: 9,3 Ø Words neg.: 8,6 Ø Words neg.: 9,6 Ø Words neg.: 10,0 Non-Response: 11% Non-Response: 9% Non-Response: 8% RANKING DRAG ME Ø Words pos.: 8,3 Ø Words pos.: 8,4 Ø Words neg.: 8,1 Ø Words neg.: 8,6 Non-Response: 7% Non-Response: 7%
  • 13. 13 RESULT: TRUSTWORTHINESS WINNER LOSER nonspecific / general specific / rich in detail „because I like it“ „good quality, fair prices“
  • 14. 14 DATA QUALITY: NONDIFFERENTIATION 13% 8% 8% 6% 5% 4% 0% 0% 0% 0% Multiple Choice Ranking Matrix Carousel Drag Me Brand Likeability Attitudes Low sample size!
  • 15. 15 DATA QUALITY: NONDIFFERENTIATION younger  40  older 7% Age 3% 5% 2% 1% 6% 2% 5% 2% 3% 3% 2% Brand Likeability Attitudes Brand Likeability Attitudes Brand Likeability Attitudes lower A-L. higher 7% Education 3% 4% 3% 2% 6% 2% 5% 3% 2% 2% 2% Brand Likeability Attitudes Brand Likeability Attitudes Brand Likeability Attitudes MULTIPLE CHOICE MATRIX CAROUSEL Low sample size!
  • 16. 16 RESULT: DATA QUALITY The data is perfectly differentiated with the Ranking and Drag Me Question. Among the other three alternatives, the Carousel performs best. WINNER LOSER
  • 17. 17 SATISFACTION Layout Usability Comprehensibility Carousel 71 20 5 Carousel 75 19 3 Carousel 74 15 2 Ranking 67 22 11 Ranking 73 20 6 Ranking 71 15 4 DragMe 66 23 10 DragMe 71 18 8 Matrix 70 16 4 Matrix 65 25 9 Matrix 70 23 6 DragMe 70 16 5 Multiple Choice 63 31 4 Multiple Choice 66 28 3 Multiple Choice 67 17 4 Topic Length Fun Carousel 64 15 8 Carousel 69 15 13 Carousel 62 14 10 Ranking 57 14 11 Ranking 66 18 11 Ranking 57 13 10 DragMe 55 16 11 DragMe 64 19 11 Matrix 54 13 12 Multiple Choice 54 15 11 Multiple Choice 61 21 15 DragMe 54 16 11 Matrix 54 12 14 Matrix 60 22 15 Multiple Choice 53 14 12
  • 18. 18 SATISFACTION Overall Carousel 59 24 14 Ranking 58 25 16 DragMe 57 25 15 Matrix 46 34 15 Multiple Choice 45 35 17 WINNER LOSER
  • 19. 19 SUMMARY MULTIPLE RANKING MATRIX CAROUSEL DRAG ME CHOICE Comparability Trustworthiness Data Quality Satisfaction Technical none JScript none JScript, Flash JScript, Flash Requirements
  • 20. 20 RECOMMENDATIONS CHECK, IF YOUR STUDY PERMITS THE USAGE OF JSCRIPT AND FLASH! (in most cases, it will) SELECT THE QUESTIONTYPES CAREFULLY! (there might be better alternatives) LAYOUT AND USABILITY MATTER! (the longer the interview, the more they matter) IF YOU HAVE DOUBTS ASK YOUR FIELDWORK PROVIDER! (they should have enough experience)
  • 21. FLORIAN TRESS THANK YOU VERY MUCH! www.odc-services.com f.tress@odc-services.com @FTress