How to find out about the usability of your web site using a survey by @cjforms

5,330 views
4,920 views

Published on

Workshop at UX Cambridge 2012 led by Caroline Jarrett. We agreed that you can't do a usability test using a questionnaire alone, but that you can find out things about usability using surveys.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Published in: Design

How to find out about the usability of your web site using a survey by @cjforms

  1. How to find out aboutthe usabilityof your web siteusing a surveyUX Cambridge 2012 workshop led by Caroline Jarrett
  2. I got asked this question recently.What are your views? “ Can a usability test just use a questionnaire, no observation? ” 2
  3. “Can a usability test just use a questionnaire,no observation?” << NO• We agreed that a usability test must have observation, otherwise it’s not a usability test• Recommended readingSteve Krug“Rocket Surgery Carol M.Made Easy” Barnum “Usability Testing Essentials” Workshop results 3
  4. I rejected the idea of using a questionnairealone. But let’s ponder this question. “How to find out about the usability of your web site using a survey” 4
  5. UX people typically encounter surveysin these three ways1. Post-test / post-task surveys e.g. SUS2. Someone is going to do a survey anyway http://www.slideshare.net/cjforms/ better-ux-surveys-at-ucd2012-by-cjforms3. Triangulating between Today’s topic, survey data and this time focusing data from elsewhere on usability 5
  6. Let’s unpick the standard definition.Which bits can we assess using a survey?The extent to which a productcan be used by specified usersto achieve specified goalswith effectiveness, efficiencyand satisfactionin a specified context of use(ISO 9241:11 1998) 6
  7. We picked out many details in the definition that we could explore in surveys Examples from the discussion: • “Specified users” - demographics, attitudes, history • “Specified goals” – what do users want to achieve? • “Satisfaction” – What did they expect? – What is their assessment after the experience? Workshop resultsCorbin, J. M. and Strauss, A.L. (2008)“Basics of qualitative research: techniques and 7procedures for developing grounded theory”
  8. Example: the product.Is that the whole website, a section, a page?The extent to which a productcan be used by specified usersto achieve specified goalswith effectiveness, efficiencyand satisfactionin a specified context of use(ISO 9241:11 1998) 8
  9. Used properly, NPS® (Net Promoter Score) takes “the product” up another level – to brand “Would you recommend us to a friend?” Image credit: Richard Vince, geograph.org.uk Image credit: Rodney Burton, geograph.org.uk Reichheld, F. F. (2006) 9Image credit:Cambridge County Council The ultimate question: driving good profits and true growth
  10. But sometimes NPS® gets muddled up withother things 10
  11. Ask one question at a timeTip 11 Image credit: Shutterstock
  12. My favourite bit to look at with surveys:what is the user’s concept of the goal?The extent to which a productcan be used by specified usersto achieve specified goalswith effectiveness, efficiencyand satisfactionin a specified context of use(ISO 9241:11 1998) What the user wants to do
  13. Find out about users’ goalsTip 13
  14. There are other goals to consider What the organisation wants to achieve Our aims in What the user doing a survey wants to do 14
  15. A set of questions starts to become a surveywhen we know our goals for the survey Goals Users Build Deploy Analyse• Establish • Interview • Final version • Run the • Extract your goals users about of questions survey from useful ideas for the the topics in • Build the approach to • Share with survey your survey questionnaire follow-up othersQuestions Questions you need users can Questionnaire Data Insightanswers to answer 15
  16. Exploratory: Don’t know what is out there. Send and hope.Image credit: 16Shutterstock
  17. Comparative:Explore trends, look at before-and-after, trackers Image credit: Shutterstock 17
  18. Descriptive: Do know what is out there. Go and count them. 18Image credit:Caroline Jarrett
  19. Modelling: Find factors that show cause and effectImage credit: 19Shutterstock
  20. Example: The 1946 Cohort Study combines comparative, descriptive, and modelling Cumulative death rates by fathers social class at birth in the 1946 birth cohort studyAdapted from Kuh, D., Hardy, R., Langenberg, C., Richards, M. and Wadsworth, M. E. J. (2002), 20Mortality in adults aged 26-54 years related to socioeconomic conditions in childhood and adulthood: post war birth cohort study.
  21. Exploratory, comparative, descriptive,modelling? Let’s discuss (from a pop-up survey) 21
  22. We found plenty to say about the details of the questions –but this is a COMPARATIVE survey, so keeping questionsthe same whenever it is used is crucial Workshop results 22
  23. The ideal: everything in balance Good Comparisons questions 23
  24. To get maximum insight from your survey,follow all the steps. Goals Users Build Deploy Analyse• Establish • Interview • Final version • Run the • Extract your goals users about of questions survey from useful ideas for the the topics in • Build the approach to • Share with survey your survey questionnaire follow-up othersQuestions Questions you need users can Questionnaire Data Insightanswers to answer 24
  25. Interview before buildTip 25
  26. Bonus SuccessfulTip Survey = Questionnaire + Process That involves lots of testing 26
  27. Caroline JarrettTwitter @cjformshttp://www.slideshare.net/cjformscarolinej@effortmark.co.uk 27
  28. More resources onhttp://www.slideshare.net/cjforms 28

×