Vovici Vision 2011: New Design Better Surveys and Boost Response Rates b


Published on

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • http://blog.vovici.com/blog/bid/20582/The-Seven-Steps-of-Highly-Successful-Surveys
  • http://blog.vovici.com/blog/bid/18140/Reminder-Invitations-Double-Survey-Response-Ratehttp://blog.vovici.com/blog/bid/24732/Optimize-rather-than-Maximize-Response-Rates
  • http://blog.vovici.com/blog/bid/26604/Response-Rates-Driven-by-16-Major-Factors
  • http://blog.vovici.com/blog/bid/26604/Response-Rates-Driven-by-16-Major-Factors
  • http://blog.vovici.com/blog/bid/53107/The-Science-of-Email-Marketing-Applied-to-Surveys
  • http://blog.vovici.com/blog/bid/22595/E-mail-List-Rental-Guidelineshttp://blog.vovici.com/blog/bid/25131/Panel-Rental-Guidelineshttp://blog.vovici.com/blog/bid/18232/Representative-Web-Surveys-Require-Good-Email-Lists-of-Customers
  • http://blog.vovici.com/blog/bid/24309/Unsubscription-Survey-Results-Best-Practices
  • http://blog.vovici.com/Blog/bid/18184/Ensuring-Your-Survey-Invitation-Isn-t-Flagged-as-Spam
  • http://blog.vovici.com/Blog/bid/18205/Getting-Your-Survey-Invitation-Opened
  • http://blog.vovici.com/blog/bid/18192/Compelling-Survey-Invitations
  • http://blog.vovici.com/blog/bid/18258/Hurdles-in-Race-to-Turn-Recipients-into-Respondents
  • http://blog.vovici.com/Blog/bid/18098/Writing-Survey-Invitations-Six-Points-to-Cover
  • http://blog.vovici.com/blog/bid/54538/You-re-the-Top-You-re-the-Coliseum-Top-Box-Scores-vs-Means
  • http://blog.vovici.com/blog/bid/56330/Real-Time-Reports-Not-Always-Ready-for-Prime-Time
  • It's the desire to better understand the factors which can drive our confidence in the data - which has driven 2 large scale R&D initiatives in 2009.The initial study - "Sexy Questions, Dangerous Results" was presented at the Jan 09 NetGain 3 conference in Toronto. I am extremely grateful to the ResearchNow team, led by John Visser and Rasheeda Qureshi for their partnership on that study. We learned that by moving from a radio button, to a slider or drop down format - we would see significant differences in how often people say they brush their teeth ... how important issues are to Canadians ... and when they last bought products.
  • what level of inference would we be OK with on the "are you ..." question?
  • In our Quali/Quant design of 50 interview per interface - respondents rated both experiences similarly across enjoyment, ease of use, and desire to repeat.There are questions which need to be answered - like what will happen over time as survey takers experience the evolution of panels, and overall web experiences.
  • Generation Bof (Generation Whatever)Generation Niks (Generation Nothing)Baby BoomersUS – between 1946 and 1963Japan – between 1947 and 1949Russia – called Sputnik Generation because they grew up under communism, reachied middle age during the Gorbachev Revolution, and have lived through the transition to a Russian-style market economy.When working in an international environment, you should try and understand the research best practices of the regions you’re working with, when the local agencies usually don’t report generations neither should you probably. And take some history lessons: Understanding a coungry’s past and development phases will help you interpret respondent’s reactions better. Finally, nothing beats talking with locals: speak, whenever possible, to people from the countries you research to understand the nuances of their culture, tradition, and politics.
  • AttributesA competent translator has the following qualities:familiarity with the subject matter of the text being translated;a very good knowledge of the language, written and spoken, from which he is translating (the source language);an excellent command of the language into which he is translating (the target language);a profound understanding of the etymological and idiomatic correlates between the two languages; anda finely tuned sense of when to metaphrase ("translate literally") and when to paraphrase, so as to assure true rather than spurious equivalents between the source- and target-language texts.[30][edit] MisconceptionsA common misconception is that anyone who can speak a second language will make a good translator. In the translation community, it is generally accepted that the best translations are produced by persons who are translating into their own native languages,[31] as it is rare for someone who has learned a second language to have total fluency in that language. A good translator understands the source language well, has specific experience in the subject matter of the text, and is a good writer in the target language. Moreover, he is not only bilingual but bicultural.
  • Identify words you don’t want translatedBest to use native speakers in country since usage changesBack translation is especially important in medical or psychometric researchValidation = pilot test of the surveyMay need local Subject Matter Experts to reviewGallup does backtranslations of all their surveys: Gallup does this for all its surveys, having a different person, using a native English speaker to do itMore on backtranslation:
  • Understand the time differences that may separate various language populations
  • Skewing toward positive and negative - This is especially critical to plan for in Product Concept Testing and Brand Perception Surveys
  • Vovici Vision 2011: New Design Better Surveys and Boost Response Rates b

    1. 1. Design Better Surveys and Boost Response Rates
    2. 2. When It Comes to Surveys, Much to Complain About Too time consumingAsk too many personal questions Come at inconvenient times Are too long Cant be trusted 0% 5% 10% 15% 20% 25% Source: Vovici, n = 100 2
    3. 3. What Makes a Successful Survey? Meets its original goal Produces accurate data that can be used to make decisions from Improves the satisfaction level of its respondents Used to improve the sponsoring organization 3
    4. 4. The Seven Steps of Highly Successful Surveyors 1. Focus on a Goal 2. Survey the Right Group of People 3. Craft Your Invitation Carefully 4. Order Questions Logically 5. Write Objective Questions 6. Shorten the Survey 7. Close the Feedback Loop 4
    5. 5. Agenda Response Rate Targets Drivers of Response Rates Jumping the Hurdles to Response Craft Your Invitation Carefully Reporting Charts Q&A Multi-Language Surveys 5
    6. 6. Response Rate Targets
    7. 7. Oversimplified Look at Market Research “Rates” General Population Invitees Incidence Rate = Invitees ÷ General Population Invitation Readers Participants Participation Rate = Participants ÷ Invitees Click-Thru Rate = Participants ÷ Invitation Readers Completes Response Rate 1 ~= Completes ÷ Invitees Completion Rate = Completes ÷ Participants 7
    8. 8. Response Rate Goals Compliance Representative at Higher Confidence Representative by Cell Representative Overall 8
    9. 9. High Response Rates Not NeededHigher response rates do not produce much more accurate resultsFear is that low response rates indicate a potential non-response biasPew Research Center found a 25% response rate did not reduce quality ofsurvey estimates compared to a 50% response rate Source: “Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey”Holbrook, Krosnick and Pfent found: Little variance by response rate But aim for response rates of at least 20% (“Our findings might not generalize ... to declines in response rates below 20 percent.”) Source: “The Causes and Consequences of Response Rates in Surveys by the News Media and Government Contractor Survey Research Firms”
    10. 10. Drivers of Response Rates
    11. 11. Key Drivers to Response Rates Target Quantity of Audience Reminders History of Invitation Frequency Empanelment Invitation Salience Recency Exclusivity Incentives 11
    12. 12. Target Audience MattersResponse Rates Proportional to Strength of Relationship100% 80% 60% 40% 20% Low High 0% AverageSource: Vovici 12
    13. 13. Use Reminders to Improve Response Rate Cumulative Response Rate35%30%25%20%15% Optimize rather than maximize response rates10% 5% 0% Invitation 1st Reminder 2nd Reminder 3rd Reminder 13
    14. 14. History of Invitation Frequency For customers who aren’t members of a panel: The Boy Who Cried “Survey!” – too frequent requests for feedback lead to a decline in response rates Rare requests to participate in a survey produce a higher response Response rates suffer if Marketing sends out a high volume of email (“spam”) 14
    15. 15. Empanelment Respondents who have opted in to a panel expect to participate Engaged panelists will complain if they are not surveyed each month 15
    16. 16. Invitation Salience: Effect of Topic & Text on Response • Participate and earn a free gift card from Home Depot • Participate and win one of 3 cash prizes • Participate and $5 will be credited to the credit card of your choice • We’d like to know your thoughts about gasoline prices •This survey is super-short! (we promise) • Shorter than the typical survey! Takes only 10 minutes… • We only need to ask you about 10 questions • Your answers will remain completely confidential • We only need to ask you about 25 questions • We’d like to ask you questions about home improvement • We’d like to ask you questions about Space Tourism • Get a free copy of the results after you participate! • The results of the survey will be published in the New York Times! • Earn frequent flyer miles on the credit card of your choice Source: CMOR, “Creating a Respondent Cooperation Playbook” 16
    17. 17. Recency The longer the lag between the event and the survey the lower the response rate. The response rate for one transactional survey jumped from 15% to 63% by sending invite within 15 minutes of event instead of 24 hours later. 17
    18. 18. Exclusivity Emphasizing the exclusivity of the invitation will improve response rates. Example: "You are one of a small, select group of customers that we have invited to provide us feedback." Of course, dont say it if it isnt true! 18
    19. 19. Incentives & Response “It has been repeatedly shown that providing financial incentives as a persuasive motivator for reluctant respondents is a viable and effective technique for increasing response rates…” - “Demographics and Online Survey Response Rates”, Kurt Knapton and Steve Myers “....Goetz, Tyler and Cook…concluded that financial incentives increased response rates and showed no differences in the demographics of the incentive and non-incentive groups.” “....Mason, Lesser and Dillman (2003) found that non- response error may actually be reduced when financial incentives are used...” 19
    20. 20. Meta Analysis “Factors affecting response rates of the web survey: A systematic review” Computers in Human Behavior (March 2010) Weimiao Fan and Zheng Yan Synthesizes results of 300 studies into a model of the contributing factors of response rates Response rate = completes / invites 20
    21. 21. Factors Affecting Response Rates to Web Surveys Content of Target Invitation Technology web Survey delivery audience designs issues questionnaires Nature of the Type of sponsors of the Sampling Personalization population survey methods Survey software What the topic is Scarcity Contact delivery Demographics modes How long the survey takes to Access control complete Internet transmission Personality Contact quality capabilities Question writing Use of incentives Source: Fan and Yan, 2010 21
    22. 22. The Science of Email Marketing – Applied to Surveys Businesses are consumers. Try sending survey invitations on the weekends. Send invites very early in the morning. Use lots of links in your survey invites. Include reference information in your survey invitation, providing recipients other ways to provide feedback or seek assistance after the survey is closed. Subject lines should use the word survey and should not emphasize incentives. Emphasize in the invite that this is an exclusive opportunity to provide feedback (for those studies for which it is true, of course). Source: Dan Zarrella, “The Science of Email Marketing”, 2011 22
    23. 23. Jumping the Hurdles to Response
    24. 24. The Third Step: Craft Your Invitation Carefully On • For research to house lists, make sure Email List that lists are double opt-in Not on • Unsubscribe List For general market research, do not Not Flagged as buy lists Spam • Strongly consider using external panel Message Opened from a reputable provider Link Clicked 24
    25. 25. The Third Step: Craft Your Invitation Carefully On • Maintain a suppression list of Email List addresses that have opted-out Not on Unsubscribe • Provide different categories of List Not Flagged as Spam opt-out (e.g., marketing email vs. newsletters vs. surveys) Message Opened • Remove suppressed addresses Link prior to emailing Clicked 25
    26. 26. The Third Step: Craft Your Invitation Carefully On CAN-SPAM Compliance Email List • Truthful subject lines Not on Unsubscribe List • Valid “From” e-mail Not Flagged as Spam address • Message Physical address Opened • Valid opt-out link Link Clicked• Avoid SPAM filter trigger words (“Free, “$”, “act now”) 26
    27. 27. The Third Step: Craft Your Invitation Carefully On • Personal email address Email List rather than corporate Not on mail box Unsubscribe List Not Flagged as Spam Message Opened Link • Compelling subject line Clicked o “Help us improve our products” o “Share your opinion” 27
    28. 28. Clearing the Final Hurdle: Craft Your Invitation Carefully Less than 8 seconds to make a first impression On Tell recipients Email List what you want them to do as soon Not on as possible in the invitation Unsubscribe List Not Flagged as Know your audience and write the invitation with Spam this knowledge in mind Message Opened For transactional surveys, keep it timely Link Clicked For random samples, mentioning the exclusivity of the invite improves response rates Let them know how long the survey will be Consider using HTML e-mails vs. text 28
    29. 29. Hurdles in Race to Turn Recipients into Respondents On A well-crafted invitation is Email List essential to jumping response Not on hurdles. Unsubscribe List Not Flagged as Spam Message Opened Link Clicked 29
    30. 30. Craft Your Invitation Carefully
    31. 31. Sample Invitation 31
    32. 32. Clearing the Final Hurdle: Six Points to CoverIntro/Basic Appeal Your Youve been selected for a research OpinionCounts! study!Importance of Your participation in this survey Your participation in this survey willParticipation will help improve products and help make improvements to a services. government program.Survey Subject Wed like to know your thoughts Wed like to know your thoughtsMatter about gasoline prices. about dairy products.Flexibility/ We only need to ask you about This survey will only take a fewTime Burden 10 questions. moments of your time.Incentive Offered Participate and $5 will be Get a free copy of the results afterfor Participation credited to the credit card of you participate! Source: CMOR your choicePrivacy/Data As member of the Better Your privacy is important to us, yourConfidentiality Business Bureau we take your answers will be combined with privacy seriously and will others, and will never be linked with respect your confidentiality. you personally. 32
    33. 33. Common Elements of Email InvitationsIncentive amountExpected length Contact info Opt-out/DeclineSubject included Privacy Policy Deadline Panel reminder 0% 20% 40% 60% 80% 100%Source: Vovici review of 30 ESOMAR 26 documents 33
    34. 34. Invitations: To Disclose or Not to Disclose100% “The invitation text does not Exclude 90% include…the topic of the Include 80% survey, as this may affect the 70% panelist’s willingness to 60% participate and thus create 50% bias.” - Norstat 40% “Research shows that people 30% may be more likely to 20% respond when the invitation 10% references a local geography, 0% Normalized. a topic of interest, or an appealing reward. However, care must be taken not to Source: Vovici review of 30 ESOMAR 26 documents bias the responses.” - SSI
    35. 35. Effect of Topic & Text of Invitation on Response Rate • Participate and earn a free gift card from Home Depot • Participate and win one of 3 cash prizes • Participate and $5 will be credited to the credit card of your choice • We’d like to know your thoughts about gasoline prices •This survey is super-short! (we promise) • Shorter than the typical survey! Takes only 10 minutes… • We only need to ask you about 10 questions • Your answers will remain completely confidential • We only need to ask you about 25 questions • We’d like to ask you questions about home improvement • We’d like to ask you questions about Space Tourism • Get a free copy of the results after you participate! • The results of the survey will be published in the New York Times! • Earn frequent flyer miles on the credit card of your choice Source: CMOR, “Creating a Respondent Cooperation Playbook” 35
    36. 36. Different Constituencies Respond Differently American College of Cardiology Case Study Surveys sent between 10 am and 2 pm Tuesday through Thursday Discovered by analyzing response patterns to past surveys “Strike at the best time, even if it means waiting.” Timing of invitations What works well for consumers may work poorly for businesses “Is Sunday the new Tuesday?” for B2B surveys A staggered series of reminders makes optimizing delivery time less important A/B test Adapt the split tests used by marketing departments to pit invitations against one another Rinse and repeat to find what works best 36
    37. 37. Reporting
    38. 38. Good Reports Require Upfront Planning Sampling’s Impact on Reporting Creating questionnaires that facilitate results. Construct Reports Easily consumable reports that accurately detail trends and highlight conclusions. Present & Share Reports & Data Efficiently format and share your detailed reports and raw data 38
    39. 39. Qualitative/Quantitative Research• Random samples • Convenience samples• Closed questions Quantitative Research • Polls with comments• Limited response options • Online Communities• Can’t get the story behind • Focus Groups the story • Provide the Story behind• Provides hard data that Qualitative Research the story can be extrapolated to a • Can’t extrapolate to a larger audience larger audience 39
    40. 40. Ramifications for Reporting Sometimes lots of numbers are really qualitative research, because you can’t extrapolate from them Forget sophisticated reporting techniques if research is based on a convenience sample But… you don’t need a probability sample to learn that a single customer is unhappy 40
    41. 41. Structure the Questionnaire for Good DataPreload as much respondent demographic data as possibleUse conditional logic and piping to keep respondents from answering questions that aren’t applicable to themUse validation to ensure data is provided in the correct format 41
    42. 42. Structure Questions to Fit Analytic ModelsSet up parallel questions if you want gap analysis between importance and satisfactionInclude questions to support the quadrant analyses and segmentations you will want to reportWhen designing recurring surveys, develop multi- question indices to report on 42
    43. 43. Sample Gap Prioritization Matrix 43
    44. 44. Use Test Data Before Fielding Populate the survey with random test data Generate reports on the mock data to verify your questions will give you the information that you are seeking 44
    45. 45. Understand Your Audience Who, What, When Who will be looking at your data? What are their expectations? Format, delivery, depth. How often? 45
    46. 46. Building your Report Use a top-down approach Start by understanding the major themes, then drill down. Use filtering and cross-tabulation to find hidden patterns and trends. Preloaded historical and demographic data will provide essential context. Do others reach the same conclusions when they work with the data? Let the results shape the report Question order should not determine reporting order Abandon preconceived notions/self-fulfilled ‘realizations’/dramatic over- generalizations However, understand, explain and develop conclusions that seem contradictory or illogical 46
    47. 47. Logically Consolidate & Summarize• Keep it simple and assume nothing. • Be precise; remove platitudes; use examples and draw logical parallels • Include: an executive summary, your methodology, any special notes. Always cite all sources. • Call out significant findings. Use the “tell them, then tell them again” approach.• Avoid unintentional conclusions. Reconcile easily misunderstood or contradictory results.• Provide links to secondary research and articles that: • Provide perspective • Reinforce conclusions 47
    48. 48. Sharing Your Insight Build Reports that are Scalable & Repeatable Researcher’s quandary: People always want more. Provide a method for other business users to work with reports Real-time self service & Raw data (filtered or in its entirety) Choose The Right Format & Distribution Method Self-service vs. email vs. print Word, Excel, PowerPoint, PDF, etc. Consider the Future The most cost effective research is that which you have already done Integrate data into future research 48
    49. 49. Create Hierarchical Reports Hierarchical reports eliminate manual processes and offer personalized data per report userCase Study:Corporate Climate Survey, reviewed by 1000 individuals, with different views: Branch manager and assistant branch manager: branch results compared to district, region, nation District manager: each branch in district compared to district, region, nation Regional VP: each branch in district, each district in region, compared to nation Corporate executives: national results; drilldown to any level 49
    50. 50. Five-Part Essay vs. Inverted Pyramid Introduction Conclusion Essential Narration Information Important Information Affirmation Supplemental Information Negation Data Tables Conclusion 50
    51. 51. Reports & Presentations in OppositionReading a Report Attending a PresentationNot scheduled in advance ScheduledNo agenda Follows agendaPrivate SocialNo interaction with author Interaction with presenterCompeting demands on reader No competing demands on attendeeRecommends first Recommendations last Source: Lawrence Gibson, Eric Marder Associates 51
    52. 52. Charts
    53. 53. “My focusgroup likesthe green.” 53
    54. 54. Bar Charts are Easier to Interpret than Pie Charts Question 1 Question 2 Question 3 A B C D E A B C D E A B C D E
    55. 55. Bar Charts Often Inconsistent % Completely % Completely Likely to Satisfied Repurchase 30%Product B Product B 35% 43%Product D Product D 37% 0% 20% 40% 60% 0% 20% 40% 60% % Completely Likely to % Completely Likely to Recommend Expand Purchases Product B Product B Product D Product D 0% 10% 20% 30% 10% 12% 14% 16%
    56. 56. Summary Statistics Lose Information Mean = 3.05 5 54 4 43 3 32 2 21 1 10 0 0 Top Box = 20% 5 5 5 4 4 4 3 3 3 2 2 2 1 1 1 0 0 0
    57. 57. Common Mistakes with Trend Reports Showing one year of data Grouping data by short time periods Using trend reports at all Including recent responses in trend reports 57
    58. 58. Random Samples Converging As Sample Size Increases Series 1 Series 2 Series 310.0 9.0 8.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 1 10 109 208 19 28 37 46 55 64 73 82 91 100 127 136 145 154 163 172 181 190 199 217 226 235 244 253 262 271 280 289 298 118 58
    59. 59. Rich Media Implications
    60. 60. Use of Faces in Surveys A face of an interviewer or researcher will increase the accuracy of reporting Accuracy is maximized when the gender of the respondent and the displayed face match Male panelists are more likely to report accurately when the introduction is formal, rather than friendly Accuracy differed depending on the interaction of gender of the respondent, presence or absence of a face, gender of the face and style of introduction Source: NPD Group 60
    61. 61. Continuum of Rich Media Offerings More like More like face-to-face paper interviews surveys Source: Gina K. Walejko 61
    62. 62. Continuum of Rich Media Offerings Avatars with speech Faces with recordings Faces Surveys routed to human More like interviewers face-to- Automating traditional F2F face probes interviews Reintroduces social desirability bias 62
    63. 63. Rich Media Can Increase Completion Rates Source: Deborah Sleep & Jon Puleston, Quirks – December 2008 63
    64. 64. Element54 Research Data Consistency Respondent Usability 64
    65. 65. Balanced 5 Cell Design: 2000 Total Interviews Radio Button Anchor @ „1‟ Anchor @ „5‟Anchor @ ‟10‟ Experimental 65
    66. 66. Anchoring of Slider Position Matters Source: Element54 66
    67. 67. Inference vs. Instruction 42% did not read the question 33% did not read the "traditional" question Source: Element54 67
    68. 68. Equal Ratings for ‘Engagement’ Metrics 1.0 2.0 Survey Enjoyment 8.20 8.25 Ease of Use 8.86 8.85 Desire to Repeat 7.02 6.56 Source: Element54 68
    69. 69. Tips for Migrating to Rich Media Discretion is the better part of valor Split ballot test rich media questions against traditional questions to identify differences Visual choices introduce bias, especially for more abstract choices Use rich media in a considered manner 69
    70. 70. Q&A
    71. 71. Multi-Language Surveys
    72. 72. Benefits of Multi-Language Surveys Increase response rates Improve data quality Language-specific surveys Use country as a basis for reports and conditional branching Highly efficient means of conducting a single survey among a global customer base Allows you to consistently communicate with and gain feedback from global customers72 72
    73. 73. Translation throughout the Survey Research Process Study Survey Fielding Analysis Design Design73 73
    74. 74. Study Design What languages do you need to translate to? What internal and external resources will you use for translation? Once the results come in, how will you translate open-ended comments?74 74
    75. 75. Remember: Countries are Not Monoliths Most countries have multiple languages There is no Belgian language There is no Chinese language Even when two countries share a common language, you may need to budget for “localization” 75 7
    76. 76. Biggest Mistake in Multi-Language Studies: Unrealistic Schedules • Prepare a conservative schedule Study Design • Time-consuming translation phase after the master Survey Design questionnaire is complete • International holiday schedules can extend the time the Fielding survey is in the field • Translation of open-ended comments adds weeks to the Analysis analysis phase76 76
    77. 77. Translation throughout the Survey Research Process Study Survey Fielding Analysis Design Design77 77
    78. 78. Write Master Questionnaire with Translation in Mind Using your typical questionnaire would Increase translation costs Reduce representativeness of results Instead, structure for translation Avoid jargon, slang and technical terms Rewrite for readability Use as few open-ended questions as possible Use language-specific skip patterns78 78
    79. 79. Some Questions Need Alternates, Not Translations Think localization, not just translation Country-specific questions are often called-for No sense translating the choices of the question, “Which hotel did you choose?” Choice lists may need to be tailored for major brands in different markets Entire sections of the questionnaire may differ for structure of industry in that country Graphics and icons may be inappropriate in some countries or for some cultures79 79
    80. 80. Translation vs. Localization English United States Spanish French EnglishEnglish Spanish Master Canada Questionnaire French Japanese Mexico Spanish Japan Japanese 80
    81. 81. Areas of Special Interest Weights and measures Currency Time – 24 hour clock, GMT, Zulu, UTC, WET Use images without text to best use in many cultures May need to reduce questionnaire length to take into account overall budget for translations81 81
    82. 82. Generational Considerations Use Age Groupings rather than Generational Groupings for reporting across cultures Exception: Generational Groupings may be used for the US and Canada What is Generation X? Group born between 1964 and 1975 (US) Younger Westernized generation (India) Generation Golf (Germany) Génération Bof (France) Generation Niks (Netherlands)82 82
    83. 83. Scales Matter Some scales don’t translate well They become too wordy, too subjective Some scales exaggerate cultural issues Likert scales can be particularly difficult Safe choices ConstantSum MaxDiff83 83
    84. 84. Answers Patterns for Common Response Styles Completely Somewhat Neither agree Somewhat Completely Disagree Agree Response Style disagree disagree nor disagree agree agree Optimal Responding Extreme Response Style (ERS)Truncated Scales Response Range (RR) Mild Response Style Midpoint Response (MPR) AcquiescenceSocial/Anti-Social Styles Response Style (ARS) Disacquiescence Response Style (DARS) Socially Desirable Responding (SDR) Noncontingent Responding (NCR) 84
    85. 85. Response Styles by Country: Informed by Culture Source: Johnson, Kulesa, Cho, Shavitt, 2003; Vovici 85
    86. 86. Constant Sum Example “Please distribute 100 points amongst the Item Points following 5 items, based Optical zoom of 3X or 30 greater on their importance to you Resolution: 10 40 when buying digital Megapixels or greater Available in 5 cameras. An item that has multiplecolors no importance to you Battery Life 25 should be given no points. Weight 0 100 The total must equal 100 points.” 86
    87. 87. MaxDiff Example87 87
    88. 88. Important Safety Tips for Multi-Language Surveys Translation is the last part of the survey cycle Surveys must be finalized before having them translated to avoid confusion and re-work Do not trust machine translations!88 88
    89. 89. A Survey Translation is Not Just a Translation of the Survey Invitation and reminder emails Introductions Button captions Validation messages End pages Thank you messages Alerts89 89
    90. 90. Choose Your Translator Well The best translators have excellent understanding of the originating language, the target culture, the industry and survey research Too often, translation is “insourced” to employees who do not have the requisite experience Survey translations are more demanding than traditional translations Use only native speaking professional translators (preferably certified translators)90 90
    91. 91. Best Practices for Working with Translators Brief the translators on the intent of the survey Create a glossary of English terms and the target language for consistency Use native speakers to review the translation Kathleen Bostick and provide feedback VP Global Marketing Kathleen.bostick@lionbridge.com @Lion_MktgVEEP Pilot test of translation of survey may be http://blog.lionbridge.com/ required and will lengthen the process www.lionbridge.com A backtranslation may be recommended Source: Lionbridge Choose a tool that saves the question and Technologies translations.91 91
    92. 92. Translation throughout the Survey Research Process Study Survey Fielding Analysis Design Design92 92
    93. 93. Deadlines and Timing Timing can affect survey response rates Be knowledgeable of the holidays and cultural norms of your survey populations Avoid survey deadlines on holidays or weekends (in some cultures Friday/Saturday constitute the weekend, not Saturday/Sunday) Be mindful of the International Date Line and individual times zones within countries Be careful of time indicators – many cultures use a 24-hour clock93 93
    94. 94. Respondent Language Selection • Translated email invitations with links toappropriate translation • Or self-selection based on all available languages:94 94
    95. 95. Respondent Language Selection Auto-detect based on primary language set in browser Conditional logic, alerts and triggers based on language-specific surveys Respondents’ browser settings may not match their native language 95 9
    96. 96. Translation throughout the Survey Research Process Study Survey Fielding Analysis Design Design96 96
    97. 97. Analysis Considerations Some countries skew towards “positive” responses (Latin America, Japan) Others skew negative (Germany) Take special care with Product Concept Testing and Brand Perception Surveys Always analyze data by country, not just by region Even if you plan to aggregate by region for reporting purposes Within some countries, important regional differences are also worth investigating97 97
    98. 98. Optimal Data Storage for Survey Results Base Survey English 98
    99. 99. Optimal Data Storage for Survey Results French French Italian Italian German Base German Survey Spanish Spanish English Chinese Chinese Japanese Japanese 99
    100. 100. Optimal Data Storage for Survey Results French French Italian Italian German Base German Response Survey Database* Spanish Spanish English Chinese Chinese Japanese Japanese *Safe Harbor Certified 100
    101. 101. Optimal Data Storage for Survey Results French French Data Export Italian Italian German Base German Response Survey Database Spanish Spanish English Chinese Chinese Japanese Japanese 101
    102. 102. Optimal Data Storage for Survey Results French French Data Export Italian Italian German Base German Response Survey Database Spanish Spanish View English Chinese Report Chinese Japanese Japanese 102
    103. 103. Translation throughout the Survey Research Process Study Survey Fielding Analysis Design Design103 103
    104. 104. What is the Role of the Translator Seat? User role that provides access to translate specific surveys into specific languages Economically allows translators to provide value Provides security by limiting what the translator can see and do Option to export surveys as XLIFF file, an XML version of the survey which translation houses can rapidly use to translate a single survey into a variety of languages 104
    105. 105. Bad Translations: More Than Just Bad Data Significant schedule slips Due to re-fielding, Significant budget overruns new translation Non-blind surveys: Your brand loses credibility105 105
    106. 106. Thank You For Your TimeFeel free to email questions: Jeffrey Henning, PRC VP of Strategy & Founder Vovici jhenning@vovici.com http://blog.vovici.com 106