Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Web Survey and Forms Usability
Design & Testing
Jennifer Romano Bergstrom
August 13, 2014
AAPOR webinar
@romanocog @AAPOR
2
@romanocog @AAPOR
What we design for…and reality
Krug, S. (2000) Don’t Make Me Think. Pearson Education *NEW 3rd Edition...
3
Placement of Instructions
4
http://www.pewinternet.org/three-technology-revolutions/
http://www.pewinternet.org/fact-sheets/mobile-technology-fact-s...
5
@romanocog @AAPOR
The % of cell phone owners who use their cell phone to…
http://www.pewinternet.org/2013/09/19/cell-pho...
6
@romanocog @AAPOR
Nielsen: The Cross-Platform Report, Quarter 2, 2012-US
7
@romanocog @AAPOR
Users read what they need to read
He, Siu, Strohl, & Chaparro (2014). Mobile. In Romano Bergstrom & Sc...
Usability = “the extent
to which a product can
be used by specified
users to achieve
specified goals with
effectiveness,
e...
9 User Experience Design (P. Morville): http://semanticstudios.com/publications/semantics/000029.php
@romanocog @AAPOR
Use...
10
@romanocog @AAPOR
When to test
11
https://www.youtube.com/watch?v=GrV2SZuRPv0
Low-Fi Paper Prototypes
@romanocog @AAPOR
12
https://www.youtube.com/watch?v=GrV2SZuRPv0
Low-Fi Paper Prototypes
@romanocog @AAPOR
13 https://uxmag.com/articles/eight-lessons-in-mobile-usability-testing
@romanocog @AAPOR
Mobile Low-Fi Paper Prototypes
14
SUBJECTIVE
+ Satisfaction & difficulty ratings
+ Verbal responses
+ Moderator follow up
+ Real-time +/- dial
@romanocog...
15
OBSERVATIONAL
+ Time on page/task
+ Reaction time
+ Selection/click behavior
+ Success/fail rate
+ Conversion rate
SUBJ...
16
OBSERVATIONAL
+ Time on page/task
+ Reaction time
+ Selection/click behavior
+ Success/fail rate
+ Conversion rate
IMPL...
@romanocog @AAPOR
Why should we measure implicit?
Traditional UX research is
good at explaining what
people say and do, not
what they think and feel.
@romanocog @AAPOR
Why ...
Explicit data
19
*Satisfaction Questionnaire: Please rate how difficult it was to log in on this device. 1= not difficult ...
Observational data
20
Details page about Military
Physician Assistant career, only
accessible through Military section.
De...
21
0
20
40
60
80
100
120
Page 1 Page 2 Page 3
Time(seconds)
Time spent on each page of the instructions before working on ...
2222
Trouble with scrolling.
22
@romanocog @AAPOR
Implicit Data
23
Modern eye tracking
Modern Eye Tracking
Copyright©2013TheNielsenCompany.Confidentialandproprietary.
24
Example Methodology
Participants:
• N = 74 | Average Age = ...
Jarrett, C., & Gaffney, G. (2009) Forms that Work: Designing Web Forms for Usability. Morgan Kaufmann
@romanocog #AAPOR
25...
Jarrett, C., & Gaffney, G. (2009) Forms that Work: Designing Web Forms for Usability. Morgan Kaufmann
Good forms = Good UX...
@romanocog @AAPOR
What can we learn from
eye tracking forms and
surveys?
28
@romanocog @AAPOR
Jarrett & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom & Schall (Eds). Eye Trackin...
29
@romanocog @AAPOR
Romano Bergstrom, J. C. & Strohl, J. (2014). Improving government websites and surveys with usability...
30
Users focus on the questions.
31
@romanocog @AAPOR
Romano, J. C. & Chen, J. M. (2011). A usability and eye-tracking evaluation of four versions of the o...
32
@romanocog @AAPOR
Error messages must
be useful.
33
@romanocog @AAPOR
Error messages must
be useful.
He, Siu, Strohl, & Chaparro (2014). Mobile. In Romano Bergstrom & Scha...
34
@romanocog @AAPOR
Error messages must
be useful.
• “How do I advance to the next screen?”
• “It seems like it's stuck o...
35
@romanocog @AAPOR
Users do not pay
attention to
multiple error
messages.
Romano, J. C. & Chen, J. M. (2011). A usabilit...
36
@romanocog @AAPOR
Navigation must be intuitive.
Bristol, K., Romano Bergstrom, J. & Link, M. (2014). Eye Tracking the U...
37
@romanocog @AAPOR
It must be clear
how to get
started.
Users must click “Add a School” to start. The “Add a school”
but...
@romanocog @AAPOR
Eye tracking is really cool!
Should I always include it
in my research?
@romanocog @AAPOR
Eye tracking is really cool!
Should I always include it
in my research?
@romanocog @AAPOR
Eye tracking is really cool!
Probably not
41
@romanocog @AAPOR
When NOT to track
Jarrett & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom &
Schall ...
42
@romanocog @AAPOR
Slot-In answers
Jarrett & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom &
Schall (E...
43
@romanocog @AAPOR
Gathered answers
Jarrett & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom &
Schall (...
44
@romanocog @AAPOR
Created answers
Jarrett & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom &
Schall (E...
45
@romanocog @AAPOR
Third-party answers
Jarrett & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom &
Schal...
46
Jarrett & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experien...
See Bristol, K., Romano Bergstrom, J. & Link, M. (2014). Eye Tracking the User Experience of a Smartphone and Web Data
Col...
Thank you!
• Twitter: @forsmarshgroup
• LinkedIn: http://www.linkedin.com/company/fors-marsh-group
• Blog: http://www.fors...
Upcoming SlideShare
Loading in …5
×

Web Survey and Forms Usability Design & Testing

3,415 views

Published on

Published in: Technology
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Web Survey and Forms Usability Design & Testing

  1. 1. Web Survey and Forms Usability Design & Testing Jennifer Romano Bergstrom August 13, 2014 AAPOR webinar @romanocog @AAPOR
  2. 2. 2 @romanocog @AAPOR What we design for…and reality Krug, S. (2000) Don’t Make Me Think. Pearson Education *NEW 3rd Edition (2014)*
  3. 3. 3 Placement of Instructions
  4. 4. 4 http://www.pewinternet.org/three-technology-revolutions/ http://www.pewinternet.org/fact-sheets/mobile-technology-fact-sheet/ @romanocog @AAPOR Technology Revolutions %ofAmericanadults 90% of American adults have a cell phone 58% have a smartphone 42% have an tablet 32% have an e-reader
  5. 5. 5 @romanocog @AAPOR The % of cell phone owners who use their cell phone to… http://www.pewinternet.org/2013/09/19/cell-phone-activities-2013/
  6. 6. 6 @romanocog @AAPOR Nielsen: The Cross-Platform Report, Quarter 2, 2012-US
  7. 7. 7 @romanocog @AAPOR Users read what they need to read He, Siu, Strohl, & Chaparro (2014). Mobile. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.
  8. 8. Usability = “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” ISO 9241-11 @romanocog @AAPOR + emotions and perceptions = UX
  9. 9. 9 User Experience Design (P. Morville): http://semanticstudios.com/publications/semantics/000029.php @romanocog @AAPOR User Experience
  10. 10. 10 @romanocog @AAPOR When to test
  11. 11. 11 https://www.youtube.com/watch?v=GrV2SZuRPv0 Low-Fi Paper Prototypes @romanocog @AAPOR
  12. 12. 12 https://www.youtube.com/watch?v=GrV2SZuRPv0 Low-Fi Paper Prototypes @romanocog @AAPOR
  13. 13. 13 https://uxmag.com/articles/eight-lessons-in-mobile-usability-testing @romanocog @AAPOR Mobile Low-Fi Paper Prototypes
  14. 14. 14 SUBJECTIVE + Satisfaction & difficulty ratings + Verbal responses + Moderator follow up + Real-time +/- dial @romanocog @AAPOR UX Data
  15. 15. 15 OBSERVATIONAL + Time on page/task + Reaction time + Selection/click behavior + Success/fail rate + Conversion rate SUBJECTIVE + Satisfaction & difficulty ratings + Verbal responses + Moderator follow up + Real-time +/- dial @romanocog @AAPOR UX Data
  16. 16. 16 OBSERVATIONAL + Time on page/task + Reaction time + Selection/click behavior + Success/fail rate + Conversion rate IMPLICIT + Eye tracking + Electrodermal activity (EDA) + Behavioral analysis + Verbalization analysis + Implicit associations + Pupil dilation + Facial expression analysis SUBJECTIVE + Satisfaction & difficulty ratings + Verbal responses + Moderator follow up + Real-time +/- dial @romanocog @AAPOR UX Data
  17. 17. @romanocog @AAPOR Why should we measure implicit?
  18. 18. Traditional UX research is good at explaining what people say and do, not what they think and feel. @romanocog @AAPOR Why should we measure implicit?
  19. 19. Explicit data 19 *Satisfaction Questionnaire: Please rate how difficult it was to log in on this device. 1= not difficult at all to 5= extremely difficult. “Love the picture in the middle of it.” “It looks very clean and very simple.” “It looks pretty organized, it's a nice design.” When asked how they would save information, four of six participants said they would bookmark the page or take a screenshot of the information. Only two mentioned that they would use the site functionality to save for later use. 83% 9% 9% Percentage of Difficulty Ratings* 1 & 2 3 >=4 @romanocog @AAPOR Subjective Data 0 3 6 9 Participant Ratings Likelihood to Recommend Not likely at all or Slightly likely Moderately Likely Very Likely 0 3 6 9 Participant Ratings Likelihood to Use Not likely at all or Slightly likely Moderately Likely Very Likely *Satisfaction Questionnaire: How likely would you be to use this site in the future? How likely would you be to recommend this site to a friend?
  20. 20. Observational data 20 Details page about Military Physician Assistant career, only accessible through Military section. Details page about Civilian Physician Assistant career. Six participants clicked on Salary & Wages when looking for states with the most jobs before looking under Job Growth. • Five participants thought they had completed the task once they found the Salary & Wages map on the wrong page. 0 5 10 15 20 Time on Page Seconds Average time on landing page @romanocog @AAPOR Observational Data First click data
  21. 21. 21 0 20 40 60 80 100 120 Page 1 Page 2 Page 3 Time(seconds) Time spent on each page of the instructions before working on form. Aggregate fixation count heat map across all participants, Page 1. Participants looked at ‘Purpose of Form’ section the most often. Implicit Data @romanocog @AAPOR Romano Bergstrom, J. C. & Strohl, J. (2014). Improving government websites and surveys with usability testing: A comparison of methodologies. Proceedings from the Federal Committee on Statistical Methodology (FCSM) Conference, Nov 2013, Washington, DC.
  22. 22. 2222 Trouble with scrolling. 22 @romanocog @AAPOR Implicit Data
  23. 23. 23 Modern eye tracking Modern Eye Tracking
  24. 24. Copyright©2013TheNielsenCompany.Confidentialandproprietary. 24 Example Methodology Participants: • N = 74 | Average Age = 37 • Mix of gender, ethnicity, income • Random assignment to diary condition • New, Old, Prototype, Bilingual Usability Testing session: • Participants read a description of the study. • The moderator gave instructions and calibrated the eye tracker. • Participants completed Steps 1-5 in the diary at their own pace. • End-of-session satisfaction questionnaire • Debriefing interview Eye Tracker Moderators worked from another room. Control Room Slide from: Walton, L., Romano Bergstrom, J., Hawkins, D. & Pierce, C. (2014). User Experience and Eye-Tracking Study: Paper Diary Design Decisions. Paper presentation at the American Association for Public Opinion Research (AAPOR) Conference, Anaheim, CA, May 2014.
  25. 25. Jarrett, C., & Gaffney, G. (2009) Forms that Work: Designing Web Forms for Usability. Morgan Kaufmann @romanocog #AAPOR 25 Poor forms lead to: • Increased errors • Lower response rate • Higher nonresponse • Loss of time • Increased frustrations • Poor user experiences.
  26. 26. Jarrett, C., & Gaffney, G. (2009) Forms that Work: Designing Web Forms for Usability. Morgan Kaufmann Good forms = Good UX @romanocog #AAPOR 26
  27. 27. @romanocog @AAPOR What can we learn from eye tracking forms and surveys?
  28. 28. 28 @romanocog @AAPOR Jarrett & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann. People read pages with questions on them differently than other pages. The F-shaped eye-tracking pattern of the block of text at the top of the page is completely different from the eye-tracking pattern on the question and answer spaces at the bottom of the page.
  29. 29. 29 @romanocog @AAPOR Romano Bergstrom, J. C. & Strohl, J. (2014). Improving government websites and surveys with usability testing: A comparison of methodologies. Proceedings from the Federal Committee on Statistical Methodology (FCSM) Conference, Nov 2013, Washington, DC. Fixation count gaze plot shows the participant looks back and forth between the two input options People don’t read instructions.
  30. 30. 30 Users focus on the questions.
  31. 31. 31 @romanocog @AAPOR Romano, J. C. & Chen, J. M. (2011). A usability and eye-tracking evaluation of four versions of the online National Survey for College Graduates (NSCG): Iteration 2. Statistical Research Division (Study Series SSM2011-01). US Census Bureau. Available online at <http://www.census.gov/srd/papers/pdf/ssm2011-01.pdf>. Users focus on the questions. Participants did not read the text on the log-in page, but they read the text on the very next page.
  32. 32. 32 @romanocog @AAPOR Error messages must be useful.
  33. 33. 33 @romanocog @AAPOR Error messages must be useful. He, Siu, Strohl, & Chaparro (2014). Mobile. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.
  34. 34. 34 @romanocog @AAPOR Error messages must be useful. • “How do I advance to the next screen?” • “It seems like it's stuck on the screen.” Gaze Plot: After getting an error message, the participant had to search all over the screen to find the missing field. He, Siu, Strohl, & Chaparro (2014). Mobile. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.
  35. 35. 35 @romanocog @AAPOR Users do not pay attention to multiple error messages. Romano, J. C. & Chen, J. M. (2011). A usability and eye-tracking evaluation of four versions of the online National Survey for College Graduates (NSCG): Iteration 2. Statistical Research Division (Study Series SSM2011-01). US Census Bureau. Available online at <http://www.census.gov/srd/papers/pdf/ssm2011-01.pdf>.
  36. 36. 36 @romanocog @AAPOR Navigation must be intuitive. Bristol, K., Romano Bergstrom, J. & Link, M. (2014). Eye Tracking the User Experience of a Smartphone and Web Data Collection Tool. Paper presentation at the AAPOR Conference, Anaheim, CA, May 2014 Intuitive ‘Next’ button location • “Where and how you click is a bit counter-intuitive. [It’s] not super obvious which button to click to get to next sections.” • “I feel like the ‘Next’ should be at the bottom and not the top.” Non-intuitive ‘Next’ button location
  37. 37. 37 @romanocog @AAPOR It must be clear how to get started. Users must click “Add a School” to start. The “Add a school” button does not look like an actionable button. It is not clear that this is how to start. The form did not look like it was clickable. Users were unsure how to proceed once they had successfully brought up two schools side-by-side. Romano Bergstrom, J. C. & Strohl, J. (2014). Improving government websites and surveys with usability testing: A comparison of methodologies. Proceedings from the Federal Committee on Statistical Methodology (FCSM) Conference, Nov 2013, Washington, DC.
  38. 38. @romanocog @AAPOR Eye tracking is really cool!
  39. 39. Should I always include it in my research? @romanocog @AAPOR Eye tracking is really cool!
  40. 40. Should I always include it in my research? @romanocog @AAPOR Eye tracking is really cool! Probably not
  41. 41. 41 @romanocog @AAPOR When NOT to track Jarrett & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.
  42. 42. 42 @romanocog @AAPOR Slot-In answers Jarrett & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.
  43. 43. 43 @romanocog @AAPOR Gathered answers Jarrett & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.
  44. 44. 44 @romanocog @AAPOR Created answers Jarrett & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.
  45. 45. 45 @romanocog @AAPOR Third-party answers Jarrett & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.
  46. 46. 46 Jarrett & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann. @romanocog @AAPOR
  47. 47. See Bristol, K., Romano Bergstrom, J. & Link, M. (2014). Eye Tracking the User Experience of a Smartphone and Web Data Collection Tool. Paper presentation at the AAPOR Conference, Anaheim, CA, May 2014 for more about the UX across devices. • Not necessarily the same UX across all devices • Different issues occur with different modes • Match user expectations and survey objectives • Conduct UX testing with real users • Use eye tracking to understand the UX…cautiously. @romanocog @AAPOR 47 Assess the UX across devicesAssess the UX across devices.
  48. 48. Thank you! • Twitter: @forsmarshgroup • LinkedIn: http://www.linkedin.com/company/fors-marsh-group • Blog: http://www.forsmarshgroup.com/all/ Jennifer Romano Bergstrom @romanocog jbergstrom@forsmarshgroup.com AAPOR Webinar | August 13, 2014

×