Survey & Questionnaire Design in Applied Marketing Research

15,064 views
14,470 views

Published on

This lecture introduces the considerations for survey research. It also outlines some of the important elements for effective survey design.

Published in: Business
3 Comments
25 Likes
Statistics
Notes
No Downloads
Views
Total views
15,064
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
3
Comments
3
Likes
25
Embeds 0
No embeds

No notes for slide
  • Personal Surveys In-home or place of work Advantages control/flexibility quality and level of response Disadvantages cost Used in b2b & other situations Appointments Structured/unstructured Intercept Surveys Public places Consumers/shoppers Low cost (high volume, minimal travel) Shopping centre approaches flexibility compromised by time pressures low co-operation rates sample bias Telephone Surveys Consumer and b2b research Sampling, random or Random Digit Dialling (RDD) Generally CATI (Computer Assisted Telephone Interviews) inflexible/invisible sample bias participation rates non-availability/ gatekeepers non participation legal issues
  • Postal Surveys Cheap and quick advance warning - questionnaire - follow-up letter - telephone call short, closed questions only inflexible lack of control non-response bias literacy rates interest levels Email Surveys Advantages and uses are as for telephone interviews Will be self-completed Response rates may be low Target population must have access to e-mail Researchers must have access to e-mail addresses of target population
  • Survey & Questionnaire Design in Applied Marketing Research

    1. 1. Survey Design Week 5 (3) Dr. Kelly Page Cardiff Business School E: pagekl@cardiff.ac.uk T: @drkellypage T: @caseinsights FB: kelly@caseinsights.com
    2. 2. Survey Research <ul><li>Use of a questionnaire to gather facts, opinions and attitudes </li></ul><ul><li>Cross-sectional and/or longitudinal </li></ul><ul><li>Individuals NOT groups (e.g., stakeholders) </li></ul><ul><li>Objectives </li></ul><ul><ul><li>General (e.g., describe situation ; facts ; find opinions ; explore underlying reasons ) </li></ul></ul><ul><ul><li>Identify sample characteristics </li></ul></ul><ul><ul><li>Measure many differing characteristics and/or constructs </li></ul></ul><ul><li>Advantages: </li></ul><ul><ul><li>Quick , cheap & efficient </li></ul></ul><ul><ul><li>but… a poor survey is worse than no survey at all </li></ul></ul>
    3. 3. What do Surveys Measure? <ul><li>Characteristics of People </li></ul><ul><ul><li>Behaviours: actions that are observable by others (e.g., product usage) </li></ul></ul><ul><ul><li>Demographics: Characteristics that are directly observable and directly verifiable (e.g., gender) </li></ul></ul><ul><ul><li>Sociographics: Social characteristics (e.g., income) </li></ul></ul><ul><ul><li>Geographics: Geographic characteristics (e.g., place of birth, residence) </li></ul></ul><ul><li>Mindsets of People </li></ul><ul><ul><li>Beliefs, emotions, behavioural intent (i.e, attitudes) </li></ul></ul><ul><ul><li>Perceptions (e.g., ease of use) </li></ul></ul><ul><li>Preferences of People </li></ul><ul><ul><li>Wants and/or needs (i.e., motivations) </li></ul></ul><ul><li>Specific Constructs: </li></ul><ul><ul><li>Satisfaction, confidence, knowledge etc </li></ul></ul>
    4. 4. Survey Methods (1) <ul><li>Personal </li></ul><ul><ul><li>Door-to-door interviews </li></ul></ul><ul><ul><li>Executive Interviews </li></ul></ul><ul><li>Intercept Surveys </li></ul><ul><ul><li>Mall-intercept Interviews </li></ul></ul><ul><li>Telephone Interviews </li></ul><ul><ul><li>Central location telephone interviews </li></ul></ul><ul><ul><li>Computer-assisted telephone interviews (CATI) </li></ul></ul><ul><ul><li>Completely automated telephone surveys (CATS) using IVR technology </li></ul></ul>
    5. 5. Surveys Methods (2) <ul><li>Self-administered </li></ul><ul><ul><li>Completed by participants with no assistance from an interviewer (e.g., in-class) </li></ul></ul><ul><li>Mail (postal) surveys </li></ul><ul><ul><li>Ad-hoc or one-shot mail surveys sent without prior contact from researcher </li></ul></ul><ul><ul><li>Mail panels: pre-contacted and pre-screened participants who are periodically sent questionnaires </li></ul></ul><ul><li>Internet Surveys </li></ul><ul><ul><li>Email questionnaires </li></ul></ul><ul><ul><li>Web survey </li></ul></ul><ul><ul><li>Bulletin Boards </li></ul></ul><ul><ul><li>Downloadable </li></ul></ul>
    6. 6. Internet Surveys <ul><li>ADVANTAGES </li></ul><ul><li>Fast to deploy & report </li></ul><ul><li>Inexpensive (reduced costs) </li></ul><ul><li>Easier to manage </li></ul><ul><li>A utomated error reduction </li></ul><ul><li>I nterface error reduction </li></ul><ul><li>I nterviewer influence reduced </li></ul><ul><li>Personalisation </li></ul><ul><li>High response rates </li></ul><ul><li>Hard-to-reach respondents </li></ul><ul><li>DISADVANTAGES </li></ul><ul><li>Lack of probability sample : no list of users exists! (e.g., email addresses) </li></ul><ul><li>C are needed in interpreting the results </li></ul><ul><li>S elf-selection bias </li></ul><ul><li>Respondent authenticity </li></ul><ul><li>Unrestricted </li></ul><ul><li>D u plicate responses </li></ul><ul><li>Security </li></ul>
    7. 7. Determining Survey Method (1) <ul><li>Sampling precision </li></ul><ul><ul><li>Required level of sampling precision is important </li></ul></ul><ul><ul><li>E.g., low = convenience sample = self-admin in-class </li></ul></ul><ul><ul><li>E.g., high = random sample = central located telephone </li></ul></ul><ul><li>Budget </li></ul><ul><ul><li>For example: Budget = £10,000; sample size required = 1000 </li></ul></ul><ul><ul><li>Cost of mall-intercept = £27.50 per interview = £27,500 </li></ul></ul><ul><ul><li>Cost of internet survey = £0.50 per survey = £500 </li></ul></ul><ul><li>Requirements for participant reactions (e.g., taste test). </li></ul><ul><li>Quality of data (e.g., validity & reliability) </li></ul>
    8. 8. Determining Survey Method (2) <ul><li>Length of Questionnaire </li></ul><ul><ul><li>Affects response rates (non-response error) and response error </li></ul></ul><ul><li>Incidence rate </li></ul><ul><ul><li>% of people in general population that fit the characteristics to be sampled </li></ul></ul><ul><ul><li>Need to consider search costs to locate the people required </li></ul></ul><ul><ul><li>e.g., door-to-door interviewing very expensive for low-incidence </li></ul></ul><ul><li>Structure of Questionnaire </li></ul><ul><ul><li>Structured = closed questions, sequence = mail, self-administered, telephone </li></ul></ul><ul><ul><li>Unstructured = open questions = face-to-face interviewing </li></ul></ul><ul><li>Scheduled time available </li></ul>
    9. 9. Errors in Survey Research <ul><li>Random Error (i.e., random sampling error) </li></ul><ul><ul><li>Error that results from chance variation </li></ul></ul><ul><ul><li>Chance variation: difference between sample value & true value of population mean </li></ul></ul><ul><ul><li>Can be reduce by increasing the sample size </li></ul></ul><ul><li>Systematic Error (i.e., non-sampling error) </li></ul><ul><ul><li>Results from mistakes or problems in execution of the research design </li></ul></ul><ul><ul><li>Sample design & measurement error </li></ul></ul>
    10. 10. METHOD COMPARATOR Personal Intercept Telephone Postal Internet Highest flexibility in collecting data Best Best Ok Worst Ok Maximum diversity of questions Best Best Worst Worst Ok Best use of physical stimuli Best Best Worst Worst Good Effective sample control Best Best Good Worst Worst Controlling data collection environment Best Best Ok Worst Worst Restricting field interviewer effects Worst Worst Ok Best Best Maximising quantity of data Best Ok Worst Worst Ok Highest response rate Best Best Ok Worst Best Protecting respondent anonymity Worst Worst Ok Best Best Avoiding socially desirable answers Worst Worst Best Best Ok Keeping costs low Worst Ok Good Best Best Probing questions Best Best Worst Worst Ok Fast results Ok Ok Best Worst Best Building rapport with respondents Best Ok Ok Worst Worst
    11. 11. Discussion Questions <ul><li>Which survey method would you select for the following and why? </li></ul><ul><ul><ul><li>Highly personal & controversial questions </li></ul></ul></ul><ul><ul><ul><li>A survey that needs to completed quickly </li></ul></ul></ul><ul><ul><ul><li>Boring questions </li></ul></ul></ul><ul><ul><ul><li>It would be difficult to find the respondent at home </li></ul></ul></ul><ul><ul><ul><li>A survey of a particular organisation </li></ul></ul></ul><ul><ul><ul><li>Finances are very limited </li></ul></ul></ul><ul><ul><ul><li>Questionnaire is difficult to follow </li></ul></ul></ul><ul><ul><ul><li>Questionnaire is relatively long </li></ul></ul></ul><ul><ul><ul><li>A high response rate is important </li></ul></ul></ul>
    12. 12. Questionnaire Design Week 5 (3) Dr. Kelly Page Cardiff Business School E: pagekl@cardiff.ac.uk T: @drkellypage T: @caseinsights FB: kelly@caseinsights.com
    13. 13. Questionnaires <ul><li>Questionnaire </li></ul><ul><ul><li>A set of questions/items designed to generate the data necessary to accomplish the objectives of the research project (i.e., survey instrument) </li></ul></ul><ul><ul><li>Formalised schedule for for collecting information from participants </li></ul></ul><ul><ul><li>Standardises the wording and sequencing of questions </li></ul></ul><ul><ul><li>Imposes uniformity on data-gathering process (e.g., reliability, reduces error) </li></ul></ul><ul><ul><li>Facilitate data processing </li></ul></ul><ul><li>Improper design = </li></ul><ul><ul><li>Inaccurate data & information </li></ul></ul><ul><ul><li>Higher costs </li></ul></ul><ul><ul><li>Poor decision making </li></ul></ul>GOOD QUESTIONNAIRE DESIGN IS IMPERATIVE FOR GOOD DATA
    14. 14. Questionnaire Design Process See ‘survey data collection methods’ in previous lecture notes
    15. 15. 3) Question / Item Design <ul><li>Measurement </li></ul><ul><ul><li>The process of assigning numbers or labels to person’s, objects, or events in accordance with specific rules for representing quantities and qualities of attributes e.g., Biological Sex: 0 = Males, 1 = Female </li></ul></ul><ul><li>Levels of Measurement </li></ul><ul><ul><li>Nominal (a) </li></ul></ul><ul><ul><li>Ordinal (b) </li></ul></ul><ul><ul><li>Interval </li></ul></ul><ul><ul><li>Ratio </li></ul></ul>b) How frequently do you use the Internet?  Once a month 1  Twice a month 2  Once a week 3  Twice a week 4 a) Please indicate your gender?  Male 0  Female 1
    16. 16. Question Response Format (1) <ul><li>Open-ended questions </li></ul><ul><ul><li>participant responds in own words </li></ul></ul><ul><ul><li>open response option (a) </li></ul></ul><ul><li>Close-ended questions </li></ul><ul><ul><li>participants required to choose from a pre-determined list of answers </li></ul></ul><ul><ul><li>Dichotomous (b) </li></ul></ul><ul><ul><li>Multichotomous c) </li></ul></ul><ul><li>Partially open-ended (d) </li></ul>a) Why is brand X better? b) Have you used the Internet today?  Yes  No c) Which best represents your age? a) Under 17 d) 36-50 b) 18-25 e) 51-65 c) 26-35 f) 66+ d) What is your preferred brand of car?  BMW  Mercedes  Ford  Other (please specify) ________________
    17. 17. Question Response Format (2) <ul><li>Scale-response questions </li></ul><ul><ul><li>Close-ended questions in which the response choices are designed to capture intensity of participants feelings </li></ul></ul><ul><ul><li>Measure subjective and abstract concepts </li></ul></ul><ul><ul><li>e.g., brand loyalty, satisfaction </li></ul></ul><ul><li>Unidimensional = measure only one attribute of a concept </li></ul><ul><ul><li>e.g., 10-items = perceived ease of web use </li></ul></ul><ul><li>Multidimensional = measure several dimensions of a concept </li></ul><ul><ul><li>e.g., 10-items = perceived ease of web use for a) finding information; b) communicating; c) conducting transactions etc. </li></ul></ul>
    18. 18. Types Measurement Scales (1) <ul><li>Graphic Rating Scales </li></ul><ul><ul><li>Graphic continuum anchored by extremes </li></ul></ul><ul><ul><li>Participants asked to mark on continuum </li></ul></ul><ul><ul><li>Continuous interval level of measurement </li></ul></ul><ul><li>Itemized Rating Scales </li></ul><ul><ul><li>Selection from limited number of ordered categories </li></ul></ul><ul><ul><li>Ordinal scaled </li></ul></ul><ul><li>Rank order scale </li></ul><ul><ul><li>Comparative scales as compare two or more items and rank each item </li></ul></ul><ul><ul><li>e.g., please rank the brands listed where 1 = most preferred and 5 = least preferred (BMW, Mercedes, Ford etc) </li></ul></ul><ul><ul><li>Ordinal scaled </li></ul></ul>
    19. 19. Types Measurement Scales (2) <ul><li>Paired Comparisons </li></ul><ul><ul><li>Pick one of two objects from a set, based on some stated criteria </li></ul></ul><ul><ul><li>Easier to select, than rank (as in rank-order scales) </li></ul></ul><ul><ul><li>Order bias is overcome (e.g., order of brands/criteria) </li></ul></ul><ul><ul><li>No. of criteria should remain small </li></ul></ul>a) Please tell me which characteristics in each pair is ‘more important’ to you when selecting a sun care product. a) Tans evenly b) Tans without burning a) Prevents burning b) Protects against burning & tanning a) Good value for money b) Goes on evenly a) Not Greasy b) Does not stain clothing
    20. 20. Types Measurement Scales (3) <ul><li>Constant Sum Scales </li></ul><ul><ul><li>Ask participants to divide a given number of points (e.g., 100) among two or more attributes, based on importance to him/her </li></ul></ul><ul><ul><li>No. of points = ranking assigned = Ordinal scaled </li></ul></ul><ul><ul><li>Advantages: If two items perceived equal importance, can be assigned equal values </li></ul></ul>a) Please allocate 100 points among the characteristics of sports clothing below such that allocation = importance. The more points allocated = the more important. All points should add up to 100. Is comfortable to wear _______ Is durable _______ Is made by a well-known brand _______ Is made in UK _______ 100 pts
    21. 21. Types Measurement Scales (4) <ul><li>Semantic Differential Scales </li></ul><ul><ul><li>Examines strengths & weaknesses of a product/company image by having participant rank it between dichotomous pairs of words/phrases (1-7) </li></ul></ul><ul><ul><li>Means of responses are then plotted as a profile </li></ul></ul><ul><ul><li>Interpretation of rating of 4 = neutral, or unable to relate to anchor </li></ul></ul><ul><ul><li>Ordinal scaled </li></ul></ul>Modern 1 2 3 4 5 6 7 Old-fashioned Aggressive 1 2 3 4 5 6 7 Defensive Friendly 1 2 3 4 5 6 7 Unfriendly Reliable 1 2 3 4 5 6 7 Unreliable Helpful 1 2 3 4 5 6 7 Unhelpful Successful 1 2 3 4 5 6 7 Unsuccessful
    22. 22. Types Measurement Scales (5) <ul><li>Likert Scales </li></ul><ul><ul><li>Series of statements expressing either a favourable or unfavourable attitudes toward the concept under investigation </li></ul></ul><ul><ul><li>Participants indicate level of agreement </li></ul></ul><ul><ul><li>Scores are summated to equal respondents attitudes on item, dimension and/or scale </li></ul></ul><ul><ul><li>Interval scaled </li></ul></ul> SD Neutral SA The registration was simple 1 2 3 4 5 6 7 Registration did not take long to complete 1 2 3 4 5 6 7 The registration informed me about the site 1 2 3 4 5 6 7 Registration was non-threatening 1 2 3 4 5 6 7
    23. 23. 4. Question Wording (1) <ul><li>Objectionable questions </li></ul><ul><ul><ul><li>How old are you? ________________ (WRONG) </li></ul></ul></ul><ul><ul><ul><li>Please indicate your age: (RIGHT) </li></ul></ul></ul><ul><li>Bias in more than one direction </li></ul><ul><ul><ul><li>Are you against same sex marriage and in favour of legislation to ban it? (WRONG) </li></ul></ul></ul><ul><ul><ul><li>What is your view of same sex marriage? (RIGHT) </li></ul></ul></ul><ul><li>Emotionally charged words </li></ul><ul><ul><ul><li>Was the government negligent by ignoring the views of the people whom were opposed to the war in Iraq? (WRONG) </li></ul></ul></ul><ul><ul><ul><li>Given the views expressed against the ware in Iraq, what should have the government done? (RIGHT) </li></ul></ul></ul>
    24. 24. 4. Question Wording (2) <ul><li>Threats to self-esteem </li></ul><ul><ul><li>How often have you driven home drunk (WRONG) </li></ul></ul><ul><ul><li>Have you ever driven a vehicles after you have had too much to drink (RIGHT) </li></ul></ul><ul><li>Personalisation </li></ul><ul><ul><li>Dear Kelly Page … How would you rate ….. (WRONG) </li></ul></ul><ul><ul><li>Dear valued customer …. How would you rate ….. (RIGHT) </li></ul></ul><ul><li>Make sure the wording is clear </li></ul><ul><li>Consider respondents willingness & ability to answer </li></ul>
    25. 25. 5. Questionnaire Flow & Layout <ul><li>Screening questions: Use to identify appropriate respondents </li></ul><ul><li>Questions to gain interest (Warm-ups): Easy to answer questions </li></ul><ul><li>General Questions: e.g., behavioural question about purchase/usage </li></ul><ul><li>Difficult & Complicated questions: e.g, attitudes, rankings etc </li></ul><ul><li>Position sensitive, threatening, and demographic questions at the end e.g., age, income etc </li></ul><ul><li>Distinguish instructions e.g., capitals, italics </li></ul>Insert prompters at strategic points throughout
    26. 27. 6-10. Design Process <ul><li>Evaluate questionnaire </li></ul><ul><ul><li>Are all questions necessary </li></ul></ul><ul><ul><li>Is questionnaire too long </li></ul></ul><ul><ul><li>Will the questions provide insight required </li></ul></ul><ul><ul><li>Is it well-layout & easy to follow </li></ul></ul><ul><ul><li>Are the instructions clear </li></ul></ul><ul><ul><li>Consider data processing & coding </li></ul></ul><ul><li>Obtain Approvals </li></ul><ul><li>Pre-test & Revise </li></ul><ul><li>Prepare final copy </li></ul><ul><li>Implement survey </li></ul><ul><ul><li>Field instructions & call/contact sheets for management </li></ul></ul>
    27. 28. Other t h ings to avoid … <ul><li>Question bias </li></ul><ul><li>Complex questions </li></ul><ul><li>Unspecific questions </li></ul><ul><li>Jargon, shorthand </li></ul><ul><li>Sophisticated/uncommon words </li></ul><ul><li>Negative questions </li></ul><ul><li>Hypothetical questions </li></ul><ul><li>Potential ambiguity </li></ul><ul><li>Over-tight banding </li></ul><ul><li>Overlapping responses </li></ul><ul><li>Not allowing for ‘other’ or ‘don’t know’ responses </li></ul>
    28. 29. The content of this work is of shared interest between the author, Kelly Page and other parties who have contributed and/or provided support for the generation of the content detailed within. This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 2.0 UK: England & Wales. http://creativecommons.org/ Kelly Page (cc)

    ×