SURVEYS<br />FORMS<br />10 tips for a better survey<br />Caroline Jarrett<br />STC Summit 2011<br />
Caroline Jarretttwitter @cjforms		caroline.jarrett@effortmark.co.uk<br />2<br />
3<br />Successful<br />Survey = Questionnaire + Process<br />That involveslots of testing<br />
Outline<br />Tips about questions<br />Tips about process<br />Tips about testing<br />4<br />
5<br />120 characters<br />48 characters<br />
6<br />
Tech writing challenge: rewrite in 140 characters<br />XXX is conducting research to better understand and meet the needs ...
Tip 1<br />One introduction is enough<br />(and think 140 characters)<br />8<br />
Try answering this question, then tell me how you worked out your answer<br />9<br />
An example of scale effects:How many hours a day do you study?<br />Low scale<br />½ hour or less<br />From ½ to 1 hour<br...
An example of scale effects:How many hours a day do you study?<br />Low scale<br />Very lazy<br />A bit lazy<br />Middling...
Tip 2<br />Do not ask questions about unremarkable repetitive behaviours<br />12<br />
What’s missing from this question?<br />13<br />
Tip 3<br />Always include ‘other’ in category questions<br />Don’t know<br />Prefer not to answer<br />All of the above<br...
Watch the order of these two questions<br />15<br />
16<br />
Tip 4<br />Think of your questions as a conversation with your respondent<br />17<br />Jarrett, C. and Gaffney, G (2008) “...
18<br />
Outline<br />Tips about questions<br />Tips about process<br />Tips about testing<br />19<br />
Response relies on effort, reward, and trust<br />20<br />Perceivedeffort<br />Perceivedreward<br />Trust<br />Diagram fro...
How are response, effort and trust working here?<br />21<br />
In mail surveys, a token incentive works better than no incentive or a prize draw<br />22<br />
Tip 5<br />Try to find an incentive that gives an immediate reward<br />23<br />
24<br />
25<br />
26<br />
27<br />
Tip 6<br />One good incentive:allowing the respondents to express their views<br />28<br />
Write down your answers to this questionnaire<br />1.	How many user surveys have you run?<br />NONE    1 to 5   6 to 10   ...
Now try it as an interview.In pairs: use this is a script for an interview<br />1.	How many user surveys have you run?<br ...
Now tell each other a story about your experience relevant to these questions<br />1.	How many user surveys have you run?<...
Now let’s share data and stories<br />What numbers have we collected?<br />What open answers?<br />What stories?<br />32<b...
Tip 7<br />Interview some users to find out what they want to tell you<br />33<br />
Did you answer the same questionthat I asked?<br />34<br />1.	How many user surveys have you run?<br />
Does your household have access to running water?<br />35<br />Picture credits: James Mooney, Caroline Jarrett<br />
“Shared reference”: both sides interpret in the same way<br />36<br />Service area?<br />
Tip 8<br />Ask users for stories to explore vocabulary and concerns<br />37<br />Quesenbery, W and Brooks, K (2010) “Story...
38<br />© Caroline Jarrett and Effortmark Ltd<br />If you use this cartoon in another context, please credit Caroline Jarr...
What recommendation will you make based on your results?<br />What decision will you make?<br />What happens if …<br />The...
Tip 9<br />Try writing your presentation first<br />(before the questions)<br />40<br />Natalie Webb, conversation at UPA2...
Outline<br />Tips about questions<br />Tips about process<br />Tips about testing<br />41<br />
“If you don’t have time to do the pilot,you don’t have time to do the study”<br />Oppenheim, A. P. (1992) “Questionnaire D...
Tip 10<br />Run a pilot<br />43<br />
Caroline Jarrett<br />Twitter: @cjforms <br />Slideshare: cjformscaroline.jarrett@effortmark.co.uk<br />I’m a consultant, ...
Upcoming SlideShare
Loading in...5
×

10 tips for a better survey at STC2011

3,687

Published on

Ten tips for surveys: on questions, process, and testing your survey.
Books mentioned are listed here: http://rosenfeldmedia.com/uxzeitgeist/lists/cjforms/10-tips-for-a-better-survey-stc2011

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Published in: Design, Technology, Business
1 Comment
6 Likes
Statistics
Notes
No Downloads
Views
Total Views
3,687
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
42
Comments
1
Likes
6
Embeds 0
No embeds

No notes for slide

10 tips for a better survey at STC2011

  1. 1. SURVEYS<br />FORMS<br />10 tips for a better survey<br />Caroline Jarrett<br />STC Summit 2011<br />
  2. 2. Caroline Jarretttwitter @cjforms caroline.jarrett@effortmark.co.uk<br />2<br />
  3. 3. 3<br />Successful<br />Survey = Questionnaire + Process<br />That involveslots of testing<br />
  4. 4. Outline<br />Tips about questions<br />Tips about process<br />Tips about testing<br />4<br />
  5. 5. 5<br />120 characters<br />48 characters<br />
  6. 6. 6<br />
  7. 7. Tech writing challenge: rewrite in 140 characters<br />XXX is conducting research to better understand and meet the needs of the users of their site. Would you like to participate?<br />Responses are anonymous, view our privacy policy<br />7<br />
  8. 8. Tip 1<br />One introduction is enough<br />(and think 140 characters)<br />8<br />
  9. 9. Try answering this question, then tell me how you worked out your answer<br />9<br />
  10. 10. An example of scale effects:How many hours a day do you study?<br />Low scale<br />½ hour or less<br />From ½ to 1 hour<br />From 1 to 1 ½ hours<br />From 2 to 2 ½ hours<br />More than 2 ½ hours<br />High scale<br />2 ½ hours or less<br />From 2 ½ to 3 hours<br />From 3 ½ to 4 hours<br />From 4 to 4 ½ hours<br />More than 4 ½ hours<br />10<br />70%<br />29%<br />30%<br />71%<br />Schuman, H. and Presser, S. (1996) “Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context” <br />
  11. 11. An example of scale effects:How many hours a day do you study?<br />Low scale<br />Very lazy<br />A bit lazy<br />Middling<br />A bit diligent<br />Very diligent<br />High scale<br />Very lazy<br />A bit lazy<br />Middling<br />A bit diligent<br />Very diligent<br />11<br />70%<br />29%<br />30%<br />71%<br />Schuman, H. and Presser, S. (1996) “Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context” <br />
  12. 12. Tip 2<br />Do not ask questions about unremarkable repetitive behaviours<br />12<br />
  13. 13. What’s missing from this question?<br />13<br />
  14. 14. Tip 3<br />Always include ‘other’ in category questions<br />Don’t know<br />Prefer not to answer<br />All of the above<br />14<br />
  15. 15. Watch the order of these two questions<br />15<br />
  16. 16. 16<br />
  17. 17. Tip 4<br />Think of your questions as a conversation with your respondent<br />17<br />Jarrett, C. and Gaffney, G (2008) “Forms that work: Designing web forms for usability”<br />Janice (Ginny) Redish (2007) “Letting go of the words:Writing web content that works”<br />
  18. 18. 18<br />
  19. 19. Outline<br />Tips about questions<br />Tips about process<br />Tips about testing<br />19<br />
  20. 20. Response relies on effort, reward, and trust<br />20<br />Perceivedeffort<br />Perceivedreward<br />Trust<br />Diagram from Jarrett, C, and Gaffney, G (2008) “Forms that work: Designing web forms for usability” inspired by Dillman, D.A. (2000) “Internet, Mail and Mixed Mode Surveys: The Tailored Design Method”<br />
  21. 21. How are response, effort and trust working here?<br />21<br />
  22. 22. In mail surveys, a token incentive works better than no incentive or a prize draw<br />22<br />
  23. 23. Tip 5<br />Try to find an incentive that gives an immediate reward<br />23<br />
  24. 24. 24<br />
  25. 25. 25<br />
  26. 26. 26<br />
  27. 27. 27<br />
  28. 28. Tip 6<br />One good incentive:allowing the respondents to express their views<br />28<br />
  29. 29. Write down your answers to this questionnaire<br />1. How many user surveys have you run?<br />NONE 1 to 5 6 to 10 more than 10<br />2. What is your top tip for a better user survey, based on experience of writing or answering?<br />__________________________________<br />__________________________________<br />29<br />
  30. 30. Now try it as an interview.In pairs: use this is a script for an interview<br />1. How many user surveys have you run?<br />NONE 1 to 5 6 to 10 more than 10<br />2. What is your top tip for a better user survey, based on experience of writing or answering?<br />__________________________________<br />__________________________________<br />30<br />
  31. 31. Now tell each other a story about your experience relevant to these questions<br />1. How many user surveys have you run?<br />NONE 1 to 5 6 to 10 more than 10<br />2. What is your top tip for a better user survey, based on experience of writing or answering?<br />__________________________________<br />__________________________________<br />31<br />
  32. 32. Now let’s share data and stories<br />What numbers have we collected?<br />What open answers?<br />What stories?<br />32<br />
  33. 33. Tip 7<br />Interview some users to find out what they want to tell you<br />33<br />
  34. 34. Did you answer the same questionthat I asked?<br />34<br />1. How many user surveys have you run?<br />
  35. 35. Does your household have access to running water?<br />35<br />Picture credits: James Mooney, Caroline Jarrett<br />
  36. 36. “Shared reference”: both sides interpret in the same way<br />36<br />Service area?<br />
  37. 37. Tip 8<br />Ask users for stories to explore vocabulary and concerns<br />37<br />Quesenbery, W and Brooks, K (2010) “Storytelling in User Experience Design” <br />
  38. 38. 38<br />© Caroline Jarrett and Effortmark Ltd<br />If you use this cartoon in another context, please credit Caroline Jarrett and Effortmark Ltd.<br />
  39. 39. What recommendation will you make based on your results?<br />What decision will you make?<br />What happens if …<br />The users all agree with your gut feel?<br />The users all disagree with your gut feel?<br />The users don’t care either way?<br />39<br />
  40. 40. Tip 9<br />Try writing your presentation first<br />(before the questions)<br />40<br />Natalie Webb, conversation at UPA2010<br />
  41. 41. Outline<br />Tips about questions<br />Tips about process<br />Tips about testing<br />41<br />
  42. 42. “If you don’t have time to do the pilot,you don’t have time to do the study”<br />Oppenheim, A. P. (1992) “Questionnaire Design, Interviewing and Attitude Measurement “<br />And in each subsequent edition<br />Dillman, D.A. (2000) “Internet, Mail and Mixed Mode Surveys: The Tailored Design Method”<br />And in each subsequent edition<br />Gillham, B (2000) “Developing a Questionnaire”<br />And in the second edition<br />Jarrett, C (2010) “I wish I’d run a pilot”<br />42<br />
  43. 43. Tip 10<br />Run a pilot<br />43<br />
  44. 44. Caroline Jarrett<br />Twitter: @cjforms <br />Slideshare: cjformscaroline.jarrett@effortmark.co.uk<br />I’m a consultant, hire me:<br />Consultancy: www.effortmark.co.uk<br />Training: www.usabilitythatworks.com<br />Free stuff:<br />Forms advice: www.formsthatwork.com<br />Editing: www.editingthatworks.com<br />Columns: www.uxmatters.com “Good Questions” <br />www.usabilitynews.com “Caroline’s Corner” <br />44<br />
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×