How Non-native English Speakers Respond to English Surveys
•
1 like•593 views
We like to think that everyone answering our surveys is perfectly fluent in English but let's be realistic. About 10% of Americans have difficulty reading/writing in English because it is not their native language. And when we apply standard techniques to identify which survey takers provide good data and which are simply giving random answers, we are often making the mistake of applying measurements that all require high level language skills.
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (16)
Similar to How Non-native English Speakers Respond to English Surveys
Similar to How Non-native English Speakers Respond to English Surveys (20)
More from Peanut Labs
More from Peanut Labs (20)
Recently uploaded
Recently uploaded (20)
How Non-native English Speakers Respond to English Surveys
- 1. Please tweet! @PeanutLabsMedia @LoveStats Please tweet! @LoveStats @PeanutLabsMediaPlease tweet! @LoveStats @PeanutLabsMedia I. Can’t. Understand. You: How non-native English speakers respond to English surveys Presented by Annie Pettit Chief Research Officer at Peanut Labs DIY Sample and Polling
- 2. Please tweet! @PeanutLabsMedia @LoveStats 20% of US residents don’t speak English at home 9% of US residents don’t speak English well
- 3. Please tweet! @PeanutLabsMedia @LoveStats Survey •12 minutes •Driving, shopping, electronics, eating out, working
- 4. Please tweet! @PeanutLabsMedia @LoveStats N=1212 N=228 N=204 Method
- 5. Please tweet! @PeanutLabsMedia @LoveStats DATA QUALITY
- 6. Please tweet! @PeanutLabsMedia @LoveStats Native Speakers Are Click Happy % of respondents choosing many answer options
- 7. Please tweet! @PeanutLabsMedia @LoveStats ESL Speakers Are Click Averse % of respondents choosing minimal answer options
- 8. Please tweet! @PeanutLabsMedia @LoveStats ESL Speakers Choose Red Herrings Sounds familiar? Looks familiar?
- 9. Please tweet! @PeanutLabsMedia @LoveStats ESL Speakers Don’t Follow Instructions Failure rate on “Choose 3 items”
- 10. Please tweet! @PeanutLabsMedia @LoveStats ESL Speakers Choose None of Above % choosing many None of Above answers Somewhat Good Very Good Extremely Good
- 11. Please tweet! @PeanutLabsMedia @LoveStats ESL Provide Shorter Verbatims % providing shortest responses Blank % in Q1 % in Q2 % in Q3 % in Q4 Answer Shortest Answers Longest Answers
- 12. Please tweet! @PeanutLabsMedia @LoveStats ESL are really fast or really slow % in Slowest to Fastest Deciles of Responses Slowest Decile Fastest Decile
- 13. Please tweet! @PeanutLabsMedia @LoveStats ESL Responders Make More Errors 0 1 2 3 4 5 6 7 8 9 10 11 Number of errors made
- 14. Please tweet! @PeanutLabsMedia @LoveStats RESULT OF THESE DATA QUALITY MEASURES
- 15. Please tweet! @PeanutLabsMedia @LoveStats Higher % of ESL People Cut Are their opinions less important? Is it fair to cut them? Somewhat good English Good English Excellent English
- 16. Please tweet! @PeanutLabsMedia @LoveStats Honest ESL People Get Cut Any final comments on this survey? •great and interesting survey •Have nothing for u guy, sorry im dumb lol lmao XD •i 'm very active •I enjoyed this survey •it was a good survey •it was very great to fill all ur surveys •very nice survey, not enough points •Well nothing to share now but their are something friendly way of living now. What 3 things do you do with your phone? •1..To call someone 2..To know what is going on around the world. 3..to text my friends •coupons, texts, facebook •i phone, awesome, can download games and I used iPhone and mobile •internet and check emails and talk and text •Looking information on the internet. Call the doctors. •Play games, Watch Youtube, and Watch read, Talk Text Listen to music •text. chatting with friends and talking on the phone
- 17. Please tweet! @PeanutLabsMedia @LoveStats Most Language Discriminatory Variables Beta Sig Under Click V1 -.283 .000 Contradict V1 -.150 .002 Can’t Rank -.137 .006 Lies -.151 .011 Under Click V2 -.132 .030 Eliminate based on language
- 18. Please tweet! @PeanutLabsMedia @LoveStats Eliminate based on quality Least Language Discriminatory Variables Beta Sig. Over Click V1 .004 .944 Over Click V2 -.005 .927 Straightlining V1 -.005 .926 Over Click V3 .005 .923 Don’t Follow Instructions -.008 .903 Verbatim V1 .011 .838 Invalid Phone Number .015 .756 Contradict V2 -.019 .704 quality quality
- 19. Please tweet! @PeanutLabsMedia @LoveStats CONCLUSIONS
- 20. Please tweet! @PeanutLabsMedia @LoveStats Recommendations •Choose less language discriminative data quality questions • Overclicking • Straightlining • Follow instructions • Verbatim quality • Contradictions • Red herrings • None of the above
- 21. Please tweet! @PeanutLabsMedia @LoveStats Recommendations • Avoid cultural references, slang, norms • Consider forcing minimum responses • Keep questions short • Create white space Have you seen any advertising for this brand? Please consider TV ads, billboards, internet ads, and other ads.
- 22. Please tweet! @PeanutLabsMedia @LoveStats Thank you! Annie Pettit Chief Research Officer annie@peanutlabs.com ca.linkedin.com/in/AnniePettit/ facebook.com/AnniePettit twitter.com/LoveStats Jonathan Cheriff Director of Sales & Marketing jonathan.cheriff@peanutlabs.com Find PeanutLabs on LinkedIn Facebook Twitter YouTube DIY SURVEY PROGRAMMING DIY POLLINGDIY SAMPLE