0
Better UX surveysHow to ask better questions, andhow to assess UX using surveysA workshop led by Caroline Jarrett
Introductions (I‟m Caroline Jarrett)• Your name and role• A random thing about yourself                                   ...
Agenda   Introductions         What is a survey?         How to ask better questions         Break         How to assess u...
Is this a survey or something else?• Work in pairs• Review this survey• Decide whether it is a survey or something else   ...
Survey = Questionnaire + Process                                   5
A basic survey process               Goals          Users              Build           Deploy          Analyse            ...
Today we‟re starting withquestions users can answer   Goals           Users             Build           Deploy          An...
Then we‟re going on to goals,specifically for satisfaction surveys   Goals           Users             Build           Dep...
We won‟t have time for much on these topics   Goals           Users             Build           Deploy          Analyse• E...
Forms and questionnaires differin how the answers are used Form:                       Questionnaire: answers used individ...
Survey =Betterquestions       Questionnaire                       + Process                                   11
There are four steps to answer a question          Read and         understand the            question                    ...
ReadFindJudgePlace  13
Review these two questions from a survey• Think about the four steps of answering a question:   –   Read and understand th...
Any ideas?             15
The approximate curve of forgetting          High                                              Major life eventQualityof d...
This survey asks some „how often‟ questions• Where do these questions fit on the „curve of forgetting‟?   – Major life eve...
“Place the answer” is mostly about using the right widget to collect the answer  Use                                    Fo...
Do these questions have the right widgets?• Review the type of widgets used for these questions• Are they the right ones?•...
Grids are often full of problems at all four steps                                                 20
Grids are a major cause of survey drop-out          Total incompletes across the main section of the questionnaire        ...
But it‟s the topic that matters most          Total incompletes across the main section of the questionnaire              ...
Response relies oneffort, reward, and trust                  Perceived                                  Perceived         ...
An interesting subject helps in all the areas      Interesting           Interesting      topics                  subject ...
Compare survey invitations• Which survey invitation is the best? Why?• (Optional harder exercise: write a better invitatio...
Your answers to this surveyare important for our work      But what‟s in it for      me? And I‟m really      ready for a c...
Agenda   Introductions         What is a survey?         How to ask better questions         Break         How to assess u...
Survey = questionnaire + process                   © Caroline Jarrett and Effortmark Ltd   28
We‟ve got a lot of different goals to consider                       What the                     organisation            ...
The survey process goals are about what wewant to achieve with the survey   Goals           Users             Build       ...
Usability is about people doing things easilyThe extent to which a productcan be used by specified usersto achieve specifi...
ISO 9241-210 definition of user experience2.15 user experiencepersons perceptions and responses resulting from the useand/...
Or putting it more simply:user experience is the satisfaction bit of usabilityThe extent to which a productcan be used by ...
Satisfaction is a complex matterCompared experience to what?               Resulting thoughts(nothing)                    ...
Example: bronze medal winners tendto be happier than silver medal winners Nathan Twaddle, Olympic Bronze Medal Winner in B...
Memorable experiences are also complex• Think about the experience of  helping to run a stand at an exhibition   – What wo...
Memorable experiences are also complex• We have some candidate areas to explore• Let‟s compare with some actual questions ...
The challenge of UX and surveys:which bit to measure?The extent to which a productcan be used by specified usersto achieve...
Find out about users‟ goalsTip                                39
Ask about recent vividTip   experience                                   40      Image credit: Fraser Smith
Interview firstTip                        41
Remember mydefinition?      Survey = Questionnaire              + Process                           42
Bonus        SuccessfulTip        Survey = Questionnaire                + Process             That involves             lo...
Caroline JarrettTwitter @cjformshttp://www.slideshare.net/cjformscarolinej@effortmark.co.uk                               ...
More resources onhttp://www.slideshare.net/cjforms                                    45
Upcoming SlideShare
Loading in...5
×

Better UX Surveys part 1

2,241

Published on

The first part of a workshop on user experience surveys. Topics: (1) how to improve the questions in surveys and (2) how to assess UX using a survey.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Published in: Design
1 Comment
10 Likes
Statistics
Notes
  • Hello my dear,i am Sylvia,please i have something very important to discuss with you and i will be very grateful,if you can email me back soonest at sylvia10b10@hotmail.com i will be waiting to
    hear from you soon.Regards
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
2,241
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
49
Comments
1
Likes
10
Embeds 0
No embeds

No notes for slide

Transcript of "Better UX Surveys part 1"

  1. 1. Better UX surveysHow to ask better questions, andhow to assess UX using surveysA workshop led by Caroline Jarrett
  2. 2. Introductions (I‟m Caroline Jarrett)• Your name and role• A random thing about yourself 2
  3. 3. Agenda Introductions What is a survey? How to ask better questions Break How to assess user experience using surveys Wrap up 3
  4. 4. Is this a survey or something else?• Work in pairs• Review this survey• Decide whether it is a survey or something else 4
  5. 5. Survey = Questionnaire + Process 5
  6. 6. A basic survey process Goals Users Build Deploy Analyse • Establish • Interview • Final version • Run the • Extract“Let‟s do your goals users about of questions survey from useful ideasa survey” for the the topics in • Build the approach to • Share with survey your survey questionnaire follow-up others Questions Questions Some you need users can Questionnaire Data Insightquestions answers to answer 6
  7. 7. Today we‟re starting withquestions users can answer Goals Users Build Deploy Analyse• Establish • Interview • Final version • Run the • Extract your goals users about of questions survey from useful ideas for the the topics in • Build the approach to • Share with survey your survey questionnaire follow-up othersQuestions Questions you need users can Questionnaire Data Insightanswers to answer 7
  8. 8. Then we‟re going on to goals,specifically for satisfaction surveys Goals Users Build Deploy Analyse• Establish • Interview • Final version • Run the • Extract your goals users about of questions survey from useful ideas for the the topics in • Build the approach to • Share with survey your survey questionnaire follow-up othersQuestions Questions you need users can Questionnaire Data Insightanswers to answer 8
  9. 9. We won‟t have time for much on these topics Goals Users Build Deploy Analyse• Establish • Interview • Final version • Run the • Extract your goals users about of questions survey from useful ideas for the the topics in • Build the approach to • Share with survey your survey questionnaire follow-up othersQuestions Questions you need users can Questionnaire Data Insightanswers to answer 9
  10. 10. Forms and questionnaires differin how the answers are used Form: Questionnaire: answers used individually answers aggregated 10
  11. 11. Survey =Betterquestions Questionnaire + Process 11
  12. 12. There are four steps to answer a question Read and understand the question Find an answer Judge whether the answer fits Place theAdapted from Tourangeau, R., Rips, L. J. and Rasinski, K. A. (2000) answer“The psychology of survey response”
  13. 13. ReadFindJudgePlace 13
  14. 14. Review these two questions from a survey• Think about the four steps of answering a question: – Read and understand the question – Find the answer – Judge whether the answer fits – Place the answer• Any problems with either question?• If so, which step(s) are problematic? 14
  15. 15. Any ideas? 15
  16. 16. The approximate curve of forgetting High Major life eventQualityof data Low Unremarkable, repetitive Occasional, salient Recent Long ago 16 Time since event
  17. 17. This survey asks some „how often‟ questions• Where do these questions fit on the „curve of forgetting‟? – Major life event – Occasional, salient – Unremarkable, repetitive• Do we think the answers are useful? 17
  18. 18. “Place the answer” is mostly about using the right widget to collect the answer Use For Radio buttons A single known answer Check boxes Multiple known answers Text boxes Unknown answers “knownx answers” you‟ve interviewed users until you‟ve found all their answersAllen Miller, S. J. and Jarrett, C. (2001) “Should I use a drop-down?” 18http://www.formsthatwork.com/files/Articles/dropdown.pdf
  19. 19. Do these questions have the right widgets?• Review the type of widgets used for these questions• Are they the right ones?• How confident are we that the answer options reflect the answers that users are likely to have? 19
  20. 20. Grids are often full of problems at all four steps 20
  21. 21. Grids are a major cause of survey drop-out Total incompletes across the main section of the questionnaire (after the introduction stage) 5% 5% Subject Matter 15% 35% Media Downloads Survey Length Large Grids Open Questions 20% Other 20%Source: Database of 3 million+ web surveys conducted by Lightspeed Research/Kantar 21Quoted in Coombe, R., Jarrett, C. and Johnson, A. (2010) “Usability testing of market research surveys” ESRA Lausanne
  22. 22. But it‟s the topic that matters most Total incompletes across the main section of the questionnaire (after the introduction stage) 5% 5% Subject Matter 15% 35% Media Downloads Survey Length Large Grids Open Questions 20% Other 20%Source: Database of 3 million+ web surveys conducted by Lightspeed Research/Kantar 22Quoted in Coombe, R., Jarrett, C. and Johnson, A. (2010) “Usability testing of market research surveys” ESRA Lausanne
  23. 23. Response relies oneffort, reward, and trust Perceived Perceived effort reward TrustDiagram from Jarrett, C, and Gaffney, G (2008) “Forms that work: Designing web forms for usability” 23inspired by Dillman, D.A. (2000) “Internet, Mail and Mixed Mode Surveys: The Tailored Design Method”
  24. 24. An interesting subject helps in all the areas Interesting Interesting topics subject = take intrinsically less effort rewarding Shared interests inspire trust 24
  25. 25. Compare survey invitations• Which survey invitation is the best? Why?• (Optional harder exercise: write a better invitation) 25
  26. 26. Your answers to this surveyare important for our work But what‟s in it for me? And I‟m really ready for a coffee. 26
  27. 27. Agenda Introductions What is a survey? How to ask better questions Break How to assess user experience using surveys Wrap up 27
  28. 28. Survey = questionnaire + process © Caroline Jarrett and Effortmark Ltd 28
  29. 29. We‟ve got a lot of different goals to consider What the organisation wants to achieve Our aims in What the user doing a survey wants to do 29
  30. 30. The survey process goals are about what wewant to achieve with the survey Goals Users Build Deploy Analyse• Establish • Interview • Final version • Run the • Extract your goals users about of questions survey from useful ideas for the the topics in • Build the approach to • Share with survey your survey questionnaire follow-up othersQuestions Questions you need users can Questionnaire Data Insightanswers to answer 30
  31. 31. Usability is about people doing things easilyThe extent to which a productcan be used by specified usersto achieve specified goals witheffectiveness, efficiency and satisfactionin a specified context of use(ISO 9241:11 1998) This assumes that we agree on the goals 31
  32. 32. ISO 9241-210 definition of user experience2.15 user experiencepersons perceptions and responses resulting from the useand/or anticipated use of a product, system or serviceNOTE 1 User experience includes all the users emotions, beliefs, preferences, perceptions, physicaland psychological responses, behaviours and accomplishments that occur before, during and afteruse.NOTE 2 User experience is a consequence of brand image, presentation, functionality, systemperformance, interactive behaviour and assistive capabilities of the interactive system, the usersinternal and physical state resulting from prior experiences, attitudes, skills and personality, and thecontext of use.NOTE 3 Usability, when interpreted from the perspective of the users personal goals, can include thekind of perceptual and emotional aspects typically associated with user experience. Usability criteriacan be used to assess aspects of user experience. 32
  33. 33. Or putting it more simply:user experience is the satisfaction bit of usabilityThe extent to which a productcan be used by specified usersto achieve specified goals witheffectiveness, efficiency and satisfactionin a specified context of use(ISO 9241:11 1998) As the user, I get to choose the goals 33
  34. 34. Satisfaction is a complex matterCompared experience to what? Resulting thoughts(nothing) IndifferenceExpectations Better / worse / differentNeeds Met / not met / mixtureExcellence (the ideal product) Good / poor quality (or „good enough‟)Fairness Treated equitably / inequitablyEvents that might have been Vindication / regretAdapted from Oliver, R. L. (1996)“Satisfaction: A Behavioral Perspective on the Consumer” 34
  35. 35. Example: bronze medal winners tendto be happier than silver medal winners Nathan Twaddle, Olympic Bronze Medal Winner in Beijing Matsumoto D, & Willingham B (2006). The thrill of victory and the agony of defeat: spontaneous expressions of medal winners of the 35 2004 Athens Olympic Games. Photo credit: peter.cipollone, Flickr
  36. 36. Memorable experiences are also complex• Think about the experience of helping to run a stand at an exhibition – What would you expect to happen? – What would you need to happen? – What would the ideal experience be? – How would you expect to be treated compared to other people at the event? – If you didn‟t do this, what else might have happened? 36
  37. 37. Memorable experiences are also complex• We have some candidate areas to explore• Let‟s compare with some actual questions 37
  38. 38. The challenge of UX and surveys:which bit to measure?The extent to which a productcan be used by specified usersto achieve specified goals witheffectiveness, efficiency and satisfactionin a specified context of use(ISO 9241:11 1998) ???
  39. 39. Find out about users‟ goalsTip 39
  40. 40. Ask about recent vividTip experience 40 Image credit: Fraser Smith
  41. 41. Interview firstTip 41
  42. 42. Remember mydefinition? Survey = Questionnaire + Process 42
  43. 43. Bonus SuccessfulTip Survey = Questionnaire + Process That involves lots of testing 43
  44. 44. Caroline JarrettTwitter @cjformshttp://www.slideshare.net/cjformscarolinej@effortmark.co.uk 44
  45. 45. More resources onhttp://www.slideshare.net/cjforms 45
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×