Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Survey Says! Measuring for Change

1,017 views

Published on

Here is my presentation from Big Design 2017.
Surveys are an essential yet often neglected research tool. Get the best possible data to inform your decisions by applying UX to survey design.

  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hey guys! Who wants to chat with me? More photos with me here 👉 http://www.bit.ly/katekoxx
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • @mselz I don't think so.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Mad that I missed the first two thirds. But nice to meet you. Did anyone shoot video?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Survey Says! Measuring for Change

  1. 1. @Curio_Research SURVEY SAYS! MEASURING WITH SURVEYS Lauren Isaacson Market and UX Research Consultant Curio Research 1
  2. 2. @Curio_Research THANK YOU FOR COMING!
  3. 3. @Curio_Research https://iwsmt-content-ok2nbdvvyp8jbrhdp.stackpathdns.com/9152012004322nXqEh.jpeg
  4. 4. @Curio_Research https://c1.staticflickr.com/4/3064/4566278908_c97b3a557a.jpg
  5. 5. @Curio_Research SURVEYS! WHAT ARE THEY GOOD FOR? Absolutely something… 5 https://upload.wikimedia.org/wikipedia/commons/thumb/3/3b/Love_Hate_Love_%2815139163553%29.jpg/1024px-Love_Hate_Love_%2815139163553%29.jpg
  6. 6. @Curio_Research WE LOVE TO SEND SURVEYS
  7. 7. “ @Curio_Research If you cannot measure it, you cannot improve it. -Lord Kelvin 7
  8. 8. @Curio_Research easy
  9. 9. @Curio_Research
  10. 10. @Curio_Research WE HATE TO GET SURVEYS
  11. 11. @Curio_Research https://upload.wikimedia.org/wikipedia/commons/thumb/f/f0/Busy_desk.svg/2000px-Busy_desk.svg.png
  12. 12. @Curio_Research https://cdn.pixabay.com/photo/2012/05/04/10/56/work-47200_960_720.png
  13. 13. @Curio_Research
  14. 14. @Curio_Research https://c1.staticflickr.com/6/5206/5279525928_199804e5c9_b.jpg
  15. 15. @Curio_Research WHAT MAKES A GOOD SURVEY? 15 http://www.thebluediamondgallery.com/pictures/survey.jpg
  16. 16. @Curio_Research KEEP IT SHORT
  17. 17. @Curio_Research THE SHORTER THE BETTER ➤ Have a singular focus - Keep the number of research objectives to a minimum ➤ Use quantitative questions - Open ended text response questions make the survey longer and more taxing on the respondent ➤ No longer than 10 minutes to take - Fewer Questions = More Responses + More Completes https://pixabay.com/p-161539/
  18. 18. @Curio_Research MOBILE FIRST 44% of SurveyMonkey surveys were taken on a mobile device last March* *SurveyMonkey European Survey Research Association 2017 Presentation
  19. 19. @Curio_Research DO YOU REALLY NEED TO ASK THAT? REALLY? ARE YOU SURE? If you knew this information, what would you do?
  20. 20. @Curio_Research MAKE ALL QUESTIONS TWEET LENGTH QUESTIONS Simply stated. Direct. Clear. Jargon free.
  21. 21. @Curio_Research BE FRIENDLY NOT FORMAL
  22. 22. @Curio_Research HELP RESPONDENTS GIVE YOU CORRECT ANSWERS
  23. 23. @Curio_Research HELP RESPONDENTS BE ACCURATE ➤ Make answer options exhaustive - Think about and do some research to give respondents every possible option ➤ Consider the ranges you offer carefully - Respondents will interpret the ranges you give as typical of the population - Below average, average, above average ➤ Don’t expect respondents to remember anything - Give specifics https://c1.staticflickr.com/8/7299/12033015254_7c20d3425c.jpghttp://maxpixel.freegreatpicture.com/static/photo/1x/Accuracy-Dartboard-Sport-Competition-Target-Darts-857092.jpg
  24. 24. @Curio_Research HELP RESPONDENTS BE ACCURATE ➤ Offer an opt-out - Other, None, Not Applicable ➤ Let people enter in their own answer - Other _________ • Only use this if you actually plan to use the data https://c1.staticflickr.com/8/7299/12033015254_7c20d3425c.jpg https://pixabay.com/p-1955257/
  25. 25. @Curio_Research DO NOT MAKE QUESTIONS REQUIRED
  26. 26. @Curio_Research https://s-media-cache-ak0.pinimg.com/736x/ba/1d/87/ba1d87c311ebb69b3704a1d74acb193c--orient-savior.jpg
  27. 27. @Curio_Research ➤ Do you have a non-human companion? Dog Cat
  28. 28. @Curio_Research ➤ What kind of pet do you have? Dog Cat
  29. 29. @Curio_Research ➤ What kind of pet do you have? Dog Cat Amphibian Reptile Rodent Fish Bird Other I don’t have a pet
  30. 30. @Curio_Research ➤ What kind of pet do you have? Dog Cat Amphibian Reptile Rodent Fish Bird Other I don’t have a pet
  31. 31. @Curio_Research ➤ What kind of pet do you have? RANDOMIZE ANCHOR Dog Cat Amphibian Reptile Rodent Fish Bird Other I don’t have a pet
  32. 32. @Curio_Research ➤ What kind of pet do you have? Short question Plain language Exhaustive answer options Allows multiple answers Randomized and Anchored It’s not mandatory Dog Cat Amphibian Reptile Rodent Fish Bird Other I don’t have a pet
  33. 33. @Curio_Research ➤ What kind of pet do you have? https://c1.staticflickr.com/8/7073/6878033654_cf98354007_b.jpg Dog Cat Amphibian Reptile Rodent Fish Bird Other I don’t have a pet
  34. 34. @Curio_Research SCALES
  35. 35. @Curio_Research GOOD SCALES ARE: ➤ Likert Scales - Odd numbers, 5 or 7 - Neutral is a perfectly valid answer ➤ Balanced - The same number of positive and negative options with a neutral option in the middle ➤ Vertical - Mobile first https://upload.wikimedia.org/wikipedia/commons/9/9e/Scales_of_Justice_%28PSF%29.png
  36. 36. @Curio_Research https://s-media-cache-ak0.pinimg.com/736x/ba/1d/87/ba1d87c311ebb69b3704a1d74acb193c--orient-savior.jpg
  37. 37. @Curio_Research ➤ Please rate your level of satisfaction with this conference: Excellent Good Decent Poor
  38. 38. @Curio_Research ➤ How is the conference going so far: Excellent Good Decent Poor
  39. 39. @Curio_Research ➤ How is the conference going so far: ExcellentGoodDecentPoor
  40. 40. @Curio_Research ➤ How is the conference going so far: ExcellentGoodDecentPoor Neutral
  41. 41. @Curio_Research ➤ How is the conference going so far: I’m having a great time! It’s pretty good It kind of bad It’s really bad Neutral
  42. 42. @Curio_Research ➤ How is the conference going so far: I’m having a great time! It’s pretty good It kind of bad It’s really bad Neutral
  43. 43. @Curio_Research ➤ How is the conference going so far: I’m having a great time! It’s pretty good It kind of bad It’s really bad Neutral Short question Plain language Neutral option Likert Scale Balanced scale Vertical
  44. 44. @Curio_Research ➤ How is the conference going so far: I’m having a great time! It’s pretty good It kind of bad It’s really bad Neutral https://www.flickr.com/photos/47823583@N03/8508102188/
  45. 45. @Curio_Research MINIMIZE GRID USAGE Relying on grids is cheating
  46. 46. @Curio_Research http://cdn2.content.compendiumblog.com/uploads/user/b8bbc9ab-67b1-4a8e-ac5c-5811daa967bd/1bfbf295-292e-4b39-a40d-a3fed11a5f1e/Image/31dd75016b423826eca4861f3d6f73f1/1grid.jpg
  47. 47. @Curio_Research BIAS IS REAL Agree or Disagree?
  48. 48. @Curio_Research ACQUIESCENCE & DESIRABILITY ➤ People naturally want to please and fit in with each other, even people they’ve never met ➤ Avoid: - Agree/Disagree - Yes/No - True/False ➤ Rely on actual usage or purchase data instead of doing a survey. https://www.flickr.com/photos/47652574@N06/35966892356/
  49. 49. @Curio_Research SAMPLING & RESEARCH BIAS ➤ Incorporate randomness into how you select respondents ➤ Dice, random number generators, etc. ➤ Be very careful about how your questions are worded ➤ Try to avoid revealing the sponsor of the survey if you can ➤ It’s easier than you think to communicate what you want people to answer. - Try deliberately using carefully crafted red herrings - Red herrings are also good for insuring data quality as well https://www.flickr.com/photos/9345620@N04/2651935525/
  50. 50. @Curio_Research https://s-media-cache-ak0.pinimg.com/736x/ba/1d/87/ba1d87c311ebb69b3704a1d74acb193c--orient-savior.jpg
  51. 51. @Curio_Research ➤ Do you use Gmail? Yes No
  52. 52. @Curio_Research ➤ Which of these email services do you use? Gmail Hotmail Yahoo! Mail Outlook I have my own email server Other ____________ I don’t use email
  53. 53. @Curio_Research ➤ Which of these email services do you use? Gmail Hotmail Yahoo! Mail Outlook I have my own email server Other ____________ I don’t use email
  54. 54. @Curio_Research ➤ Which of these email services do you use? Gmail Hotmail Yahoo! Mail Outlook I have my own email server Other ____________ I don’t use email FantasticMail
  55. 55. @Curio_Research ➤ Which of these email services do you use? Gmail Hotmail Yahoo! Mail Outlook I have my own email server Other ____________ I don’t use email FantasticMail RANDOMIZE ANCHOR
  56. 56. @Curio_Research ➤ Which of these email services do you use? Gmail Hotmail Yahoo! Mail Outlook I have my own email server Other ____________ I don’t use email FantasticMail Doesn’t lead Plain language Data quality red herring Allows multiple answers Randomized and Anchored It’s not mandatory
  57. 57. @Curio_Research ➤ Which of these email services do you use? Gmail Hotmail Yahoo! Mail Outlook I have my own email server Other ____________ I don’t use email FantasticMail https://www.flickr.com/photos/meme_tn/6878030444
  58. 58. @Curio_Research INCENTIVIZE
  59. 59. @Curio_Research INCENTIVES ARE NOT NECESSARILY DIFFICULT OR EXPENSIVE ➤ Methods: - Online gift cards - Account credit - Random prize drawing ➤ Incentives can greatly increase your response rate for better data analysis ➤ Offering an incentive demonstrates respect for your users. Their time is just as valuable as yours
  60. 60. @Curio_Research OPEN WITH GRATITUDE CLOSE WITH GRATITUDE And a final open end
  61. 61. @Curio_Research BE GRACIOUS ➤ Thank respondents for their participation at the opening and the closing ➤ Tell them how the data will be used - Will it improve a service they use? - Help a non-profit apply for a grant? ➤ Offer to share the results when the study is over ➤ Present a final open end so they can share something you might have missed http://maxpixel.freegreatpicture.com/static/photo/1x/Fig-Mourning-Angel-Sculpture-Gratitude-Consolation-1990440.jpg
  62. 62. @Curio_Research TESTING YOUR SURVEYS 62 https://c1.staticflickr.com/1/711/23173416364_e8f8ee489e_b.jpg
  63. 63. @Curio_Research USER TESTING
  64. 64. @Curio_Research USER TESTING ➤ Similar as user testing a prototype, but less formal ➤ Ideally, do a cognitive interview with a few people from the target population - Grab a few friends or coworkers who didn’t work on the survey ➤ Have them talk through taking the survey - Did they understand the intent of your questions? - Did they suggest other options? - How much time did it take? Less that 10 minutes? https://www.flickr.com/photos/notionscapital/10089497055
  65. 65. @Curio_Research PILOT STUDY
  66. 66. @Curio_Research PILOT STUDY ➤ A pilot study is like a dress rehearsal for surveys ➤ Recruit from the same population and look for pitfalls, mistakes, and misunderstandings in the recruiting, invitation, the survey itself, and the incentives ➤ When analyzing the results for: - Odd/unexpected answer patterns - Too many “other” entries - Completion rate and time to complete https://sd.keepcalm-o-matic.co.uk/i/keep-calm-its-only-dress-rehearsal.png
  67. 67. @Curio_Research A PROPER SENDOFF 67 https://upload.wikimedia.org/wikipedia/commons/0/0e/STS133_launch.jpg
  68. 68. @Curio_Research WHERE TO GET RESPONDENTS? ➤ Your own customer database - Avoid giving your customer base Survey Fatigue ➤ Panel or sample provider - Vet the quality of the sample - Ask the provider about their quality controls: • CAPTCHAs? • Proxy server detection? • Geo-IP validation? • Other authentication measures? https://www.flickr.com/photos/guiguibu91/2889883615
  69. 69. @Curio_Research HOW MANY RESPONDENTS DO YOU NEED? https://qualitypursuit.wordpress.com/statistical-tools/
  70. 70. @Curio_Research POPULATION, CL, AND ME DEFINITIONS ➤ Population = The total number of people in the group you are trying to study - The number of users of your product - The number of women in the US between the ages of 25 - 34 with college degrees ➤ Confidence Level = The probability your sample accurately reflects the population you are studying - 95% is the industry standard ➤ Margin of Error = The range, as a percentage, that your population’s responses will deviate from the answers of our sample - 5% is the industry standard
  71. 71. @Curio_Research HOW MANY RESPONDENTS DO YOU NEED? https://qualitypursuit.wordpress.com/statistical-tools/ My user base is 425. How many survey responses do I need for a 95% confidence level and a 5% margin of error?
  72. 72. @Curio_Research HOW MANY RESPONDENTS DO YOU NEED? https://qualitypursuit.wordpress.com/statistical-tools/ My user base is 425. How many survey responses do I need for a 95% confidence level and a 5% margin of error? 203
  73. 73. @Curio_Research HOW MANY RESPONDENTS DO YOU NEED? https://qualitypursuit.wordpress.com/statistical-tools/ My user base is 5,000. How many survey responses do I need for a 95% confidence level and a 5% margin of error?
  74. 74. @Curio_Research HOW MANY RESPONDENTS DO YOU NEED? https://qualitypursuit.wordpress.com/statistical-tools/ My user base is 500,000. How many survey responses do I need for a 95% confidence level and a 5% margin of error? 384
  75. 75. @Curio_Research HOW MANY RESPONDENTS DO YOU NEED? https://qualitypursuit.wordpress.com/statistical-tools/ My population size is about 5,000,000. How many survey responses do I need for a 95% confidence level and a 5% margin of error?
  76. 76. @Curio_Research HOW MANY RESPONDENTS DO YOU NEED? https://qualitypursuit.wordpress.com/statistical-tools/ My population size is about 5,000,000. How many survey responses do I need for a 95% confidence level and a 5% margin of error? 385
  77. 77. @Curio_Research THE MAGIC SAMPLE SIZE IS: 400
  78. 78. @Curio_Research THE INVITATION
  79. 79. @Curio_Research INVITATION TIPS ➤ Personalize your messages ➤ Keep email messages short and to the point ➤ Try using different forms of contact: ➤ Email, snail mail, text message ➤ Keep changing the message ➤ Strategically time the delivery of your invite ➤ Provide clear instructions ➤ Tell them how their input will be mutually beneficial https://commons.wikimedia.org/wiki/File:Invitation_from_the_National_Guard_of_Washington_to_the_opening_ball_for_the_officers_of_the_United_States_-_Front.jpg
  80. 80. @Curio_Research POINT OF CONTACT Who ya gonna call?
  81. 81. @Curio_Research MONITORING PROGRESS ➤ Response rate ➤ Completion rate ➤ Bounced emails ➤ Comments about the survey in the open ends? ➤ When and how often do you send a reminder? http://keytokorean.com/wp-content/uploads/2013/09/keep-calm-and-check-progress-5.png
  82. 82. @Curio_Research ANALYZING THE DATA 82 https://c1.staticflickr.com/3/2827/9423385629_171671f9c2_b.jpg
  83. 83. @Curio_Research HAVE A DATA ANALYSIS PLAN BEFORE YOU EVEN WRITE THE SURVEY 1. What are you objectives for the data? 2. What variables do you want to use in your cross tabs? 3. Additional data analysis? Regression? Conjoint? Text analytics? 4. Will you need specialized software to run the analysis? https://www.flickr.com/photos/76074333@N00/318034222
  84. 84. @Curio_Research CLEANING YOUR DATA ➤ Remove: - Extreme outliers - Respondents who didn’t answer all of the questions - Respondents who used red herring answers - Respondents who straight-lined ➤ Do you still have the sample size you need to continue? - No? Only remove the worst 20% of the iffy responses or get more respondents. https://static.pexels.com/photos/40862/lego-legomaennchen-males-workers-40862.jpeg
  85. 85. @Curio_Research WHAT DOES MY STANDARD DEVIATION MEAN? Low Number Homogeneous Data High Number Heterogeneous Data
  86. 86. @Curio_Research TOP 2 BOXES/PERCENT POSITIVE ➤ When analyzing scale data calculate the percentage of respondents calculate the percentage that chose the top 2 boxes - 5 point scale = top 2 boxes - 7 point scale = top 3 boxes It’s really bad It’s kind of bad Neutral It’s pretty good I’m having a great time Responses 1 24 19 59 78 % of Total 0% 13% 10% 33% 43% Top 2 Boxes 76%
  87. 87. @Curio_Research BUT WAIT, THERE’S MORE! 87 https://www.flickr.com/photos/tetsumo/3802078924
  88. 88. @Curio_Research FURTHER READING ➤Internet, Mail, and Mixed-Mode Surveys - The Tailored Design Method - Don A. Dillman, Jolene D. Smyth, and Leah Melani Christian ➤Questionnaire Design - Ian Brace ➤People Are Not Robots - Annie Pettit
  89. 89. @Curio_Research ONLINE SOURCES ➤ Twitter - #NewMR and @NewMR_News - Annie Pettit @LoveStats - Jeffrey Henning @JHenning - Ray Poynter @RayPoynter ➤ Websites - Pew Research Methods pewresearch.org/methods/ - Me curioresearch.net/portfolio/
  90. 90. @Curio_Research THANK YOU CurioResearch.net @Curio_Research

×