Better UX Surveys part 1
Upcoming SlideShare
Loading in...5
×
 

Better UX Surveys part 1

on

  • 2,512 views

The first part of a workshop on user experience surveys. Topics: (1) how to improve the questions in surveys and (2) how to assess UX using a survey.

The first part of a workshop on user experience surveys. Topics: (1) how to improve the questions in surveys and (2) how to assess UX using a survey.

Statistics

Views

Total Views
2,512
Views on SlideShare
2,509
Embed Views
3

Actions

Likes
10
Downloads
48
Comments
1

1 Embed 3

https://twitter.com 3

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Hello my dear,i am Sylvia,please i have something very important to discuss with you and i will be very grateful,if you can email me back soonest at sylvia10b10@hotmail.com i will be waiting to
    hear from you soon.Regards
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Better UX Surveys part 1 Better UX Surveys part 1 Presentation Transcript

  • Better UX surveysHow to ask better questions, andhow to assess UX using surveysA workshop led by Caroline Jarrett
  • Introductions (I‟m Caroline Jarrett)• Your name and role• A random thing about yourself 2
  • Agenda Introductions What is a survey? How to ask better questions Break How to assess user experience using surveys Wrap up 3
  • Is this a survey or something else?• Work in pairs• Review this survey• Decide whether it is a survey or something else 4
  • Survey = Questionnaire + Process 5
  • A basic survey process Goals Users Build Deploy Analyse • Establish • Interview • Final version • Run the • Extract“Let‟s do your goals users about of questions survey from useful ideasa survey” for the the topics in • Build the approach to • Share with survey your survey questionnaire follow-up others Questions Questions Some you need users can Questionnaire Data Insightquestions answers to answer 6
  • Today we‟re starting withquestions users can answer Goals Users Build Deploy Analyse• Establish • Interview • Final version • Run the • Extract your goals users about of questions survey from useful ideas for the the topics in • Build the approach to • Share with survey your survey questionnaire follow-up othersQuestions Questions you need users can Questionnaire Data Insightanswers to answer 7
  • Then we‟re going on to goals,specifically for satisfaction surveys Goals Users Build Deploy Analyse• Establish • Interview • Final version • Run the • Extract your goals users about of questions survey from useful ideas for the the topics in • Build the approach to • Share with survey your survey questionnaire follow-up othersQuestions Questions you need users can Questionnaire Data Insightanswers to answer 8
  • We won‟t have time for much on these topics Goals Users Build Deploy Analyse• Establish • Interview • Final version • Run the • Extract your goals users about of questions survey from useful ideas for the the topics in • Build the approach to • Share with survey your survey questionnaire follow-up othersQuestions Questions you need users can Questionnaire Data Insightanswers to answer 9
  • Forms and questionnaires differin how the answers are used Form: Questionnaire: answers used individually answers aggregated 10
  • Survey =Betterquestions Questionnaire + Process 11
  • There are four steps to answer a question Read and understand the question Find an answer Judge whether the answer fits Place theAdapted from Tourangeau, R., Rips, L. J. and Rasinski, K. A. (2000) answer“The psychology of survey response”
  • ReadFindJudgePlace 13
  • Review these two questions from a survey• Think about the four steps of answering a question: – Read and understand the question – Find the answer – Judge whether the answer fits – Place the answer• Any problems with either question?• If so, which step(s) are problematic? 14
  • Any ideas? 15
  • The approximate curve of forgetting High Major life eventQualityof data Low Unremarkable, repetitive Occasional, salient Recent Long ago 16 Time since event
  • This survey asks some „how often‟ questions• Where do these questions fit on the „curve of forgetting‟? – Major life event – Occasional, salient – Unremarkable, repetitive• Do we think the answers are useful? 17
  • “Place the answer” is mostly about using the right widget to collect the answer Use For Radio buttons A single known answer Check boxes Multiple known answers Text boxes Unknown answers “knownx answers” you‟ve interviewed users until you‟ve found all their answersAllen Miller, S. J. and Jarrett, C. (2001) “Should I use a drop-down?” 18http://www.formsthatwork.com/files/Articles/dropdown.pdf
  • Do these questions have the right widgets?• Review the type of widgets used for these questions• Are they the right ones?• How confident are we that the answer options reflect the answers that users are likely to have? 19
  • Grids are often full of problems at all four steps 20
  • Grids are a major cause of survey drop-out Total incompletes across the main section of the questionnaire (after the introduction stage) 5% 5% Subject Matter 15% 35% Media Downloads Survey Length Large Grids Open Questions 20% Other 20%Source: Database of 3 million+ web surveys conducted by Lightspeed Research/Kantar 21Quoted in Coombe, R., Jarrett, C. and Johnson, A. (2010) “Usability testing of market research surveys” ESRA Lausanne
  • But it‟s the topic that matters most Total incompletes across the main section of the questionnaire (after the introduction stage) 5% 5% Subject Matter 15% 35% Media Downloads Survey Length Large Grids Open Questions 20% Other 20%Source: Database of 3 million+ web surveys conducted by Lightspeed Research/Kantar 22Quoted in Coombe, R., Jarrett, C. and Johnson, A. (2010) “Usability testing of market research surveys” ESRA Lausanne
  • Response relies oneffort, reward, and trust Perceived Perceived effort reward TrustDiagram from Jarrett, C, and Gaffney, G (2008) “Forms that work: Designing web forms for usability” 23inspired by Dillman, D.A. (2000) “Internet, Mail and Mixed Mode Surveys: The Tailored Design Method”
  • An interesting subject helps in all the areas Interesting Interesting topics subject = take intrinsically less effort rewarding Shared interests inspire trust 24
  • Compare survey invitations• Which survey invitation is the best? Why?• (Optional harder exercise: write a better invitation) 25
  • Your answers to this surveyare important for our work But what‟s in it for me? And I‟m really ready for a coffee. 26
  • Agenda Introductions What is a survey? How to ask better questions Break How to assess user experience using surveys Wrap up 27
  • Survey = questionnaire + process © Caroline Jarrett and Effortmark Ltd 28
  • We‟ve got a lot of different goals to consider What the organisation wants to achieve Our aims in What the user doing a survey wants to do 29
  • The survey process goals are about what wewant to achieve with the survey Goals Users Build Deploy Analyse• Establish • Interview • Final version • Run the • Extract your goals users about of questions survey from useful ideas for the the topics in • Build the approach to • Share with survey your survey questionnaire follow-up othersQuestions Questions you need users can Questionnaire Data Insightanswers to answer 30
  • Usability is about people doing things easilyThe extent to which a productcan be used by specified usersto achieve specified goals witheffectiveness, efficiency and satisfactionin a specified context of use(ISO 9241:11 1998) This assumes that we agree on the goals 31
  • ISO 9241-210 definition of user experience2.15 user experiencepersons perceptions and responses resulting from the useand/or anticipated use of a product, system or serviceNOTE 1 User experience includes all the users emotions, beliefs, preferences, perceptions, physicaland psychological responses, behaviours and accomplishments that occur before, during and afteruse.NOTE 2 User experience is a consequence of brand image, presentation, functionality, systemperformance, interactive behaviour and assistive capabilities of the interactive system, the usersinternal and physical state resulting from prior experiences, attitudes, skills and personality, and thecontext of use.NOTE 3 Usability, when interpreted from the perspective of the users personal goals, can include thekind of perceptual and emotional aspects typically associated with user experience. Usability criteriacan be used to assess aspects of user experience. 32
  • Or putting it more simply:user experience is the satisfaction bit of usabilityThe extent to which a productcan be used by specified usersto achieve specified goals witheffectiveness, efficiency and satisfactionin a specified context of use(ISO 9241:11 1998) As the user, I get to choose the goals 33
  • Satisfaction is a complex matterCompared experience to what? Resulting thoughts(nothing) IndifferenceExpectations Better / worse / differentNeeds Met / not met / mixtureExcellence (the ideal product) Good / poor quality (or „good enough‟)Fairness Treated equitably / inequitablyEvents that might have been Vindication / regretAdapted from Oliver, R. L. (1996)“Satisfaction: A Behavioral Perspective on the Consumer” 34
  • Example: bronze medal winners tendto be happier than silver medal winners Nathan Twaddle, Olympic Bronze Medal Winner in Beijing Matsumoto D, & Willingham B (2006). The thrill of victory and the agony of defeat: spontaneous expressions of medal winners of the 35 2004 Athens Olympic Games. Photo credit: peter.cipollone, Flickr
  • Memorable experiences are also complex• Think about the experience of helping to run a stand at an exhibition – What would you expect to happen? – What would you need to happen? – What would the ideal experience be? – How would you expect to be treated compared to other people at the event? – If you didn‟t do this, what else might have happened? 36
  • Memorable experiences are also complex• We have some candidate areas to explore• Let‟s compare with some actual questions 37
  • The challenge of UX and surveys:which bit to measure?The extent to which a productcan be used by specified usersto achieve specified goals witheffectiveness, efficiency and satisfactionin a specified context of use(ISO 9241:11 1998) ???
  • Find out about users‟ goalsTip 39
  • Ask about recent vividTip experience 40 Image credit: Fraser Smith
  • Interview firstTip 41
  • Remember mydefinition? Survey = Questionnaire + Process 42
  • Bonus SuccessfulTip Survey = Questionnaire + Process That involves lots of testing 43
  • Caroline JarrettTwitter @cjformshttp://www.slideshare.net/cjformscarolinej@effortmark.co.uk 44
  • More resources onhttp://www.slideshare.net/cjforms 45