Improving Forms with User Experience Testing and Eye Tracking


Published on

talk at AAPOR 2014

Published in: Education, Technology, Design

Improving Forms with User Experience Testing and Eye Tracking

  1. 1. Improving Federal Forms with User Experience Testing and Eye Tracking Jennifer Romano Bergstrom, Jon Strohl, Andrew Hale, Sarah Keaton May 17, 2014 AAPOR | Anaheim, CA @romanocog #AAPOR
  2. 2. 2 @romanocog #AAPOR + emotions and perceptions = UX Usability = “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” ISO 9241-11 2
  3. 3. 3 User Experience Design (P. Morville): User Experience @romanocog #AAPOR 3
  4. 4. 4 OBSERVATION AL + Ethnography + Time to complete task + Reaction time + Selection/click behavior + Ability to complete tasks + Accuracy IMPLICIT + Facial expression analysis + Eye tracking + Electrodermal activity (EDA) + Behavioral analysis + Linguistic analysis of verbalizations + Implicit associations + Pupil dilation EXPLICI T+ Post-task satisfaction questionnaires + In-session difficulty ratings + Verbal responses + Moderator follow up + Real-time +/- dial UX Data 4 @romanocog #AAPOR
  5. 5. Traditional UX research is good at explaining what people say and do, not what they think and feel. Why should we measure implicit? @romanocog #AAPOR
  6. 6. 6 Modern eye tracking Modern Eye Tracking
  7. 7. Copyright©2013TheNielsenCompany.Confidentialandproprietary. 7 Example Methodology Participants: • N = 74 | Average Age = 37 • Mix of gender, ethnicity, income • Random assignment to diary condition • New, Old, Prototype, Bilingual Usability Testing session: • Participants read a description of the study. • The moderator gave instructions and calibrated the eye tracker. • Participants completed Steps 1-5 in the diary at their own pace. • End-of-session satisfaction questionnaire • Debriefing interview Eye Tracker Moderators worked from another room. Control Room Slide from Walton, Romano Bergstrom, Hawkins & Peirce – AAPOR 2014
  8. 8. 8 User Experience Design (P. Morville): @romanocog #AAPOR Plain language *Public Law 111-274. Retrieved from • President Obama signed the Plain Writing Act of 2010 requiring federal agencies to use “clear Government communication that the public can understand and use.” – Find what they need – Understand what they find – Use what they find to meet their needs 8
  9. 9. Jarrett, C., & Gaffney, G. (2009) Forms that Work: Designing Web Forms for Usability. Morgan Kaufmann Government forms matter! @romanocog #AAPOR 9 Poor forms lead to: • Increased errors • Lower response rate • Higher nonresponse • Loss of time • Increased frustrations • Poor user experiences.
  10. 10. Jarrett, C., & Gaffney, G. (2009) Forms that Work: Designing Web Forms for Usability. Morgan Kaufmann Good forms = good UX @romanocog #AAPOR 10
  11. 11. Issue: Instructions are not read, and the starting point is not clear *Hochheiser, H. & Shneiderman, B. (2000). Performance benefits of simultaneous over sequential menus as task complexity increases. International Journal of Human-Computer Interaction, 12(2), 173-192. UX Best Practice: Use step-by-step menus for simple forward-moving tasks.* “It’s not intuitive at all for where I’d go to find information.” Users select their desired job to plan a career path. • Keyword search • Select from a series of dropdowns. UX Findings: • Instructions are not read • Users enter their current job, not desired job. • It is not clear that users have the option to either search or enter job details when entering a starting job. Fixation count gazeplot shows the participant looks back and forth between the two input options @romanocog #AAPOR 11
  12. 12. Issue: The layout does not encourage left-to-right processing UX Best Practice: Allow users to enter information in ways that are logical to them. @romanocog #AAPOR 12 Version 1 PrototypeVersion 2 Users enter their household TV viewing for one week.
  13. 13. Issue: The location of navigation buttons cause confusion *Niemela, M. & Saarinen, J. (2000). Visual search for grouped versus ungrouped icons in a computer interface. Human Factors, 42(4), 630-635. UX Best Practice: Clearly differentiate navigation elements from one another.* Users enter the job title or keyword that they want to search for and click the arrow pointed to the right to run the search. UX Finding: • The “Save this Search” button is closer to the box where the job title is entered than the arrow icon that runs the search. • Some participants incorrectly clicked on “Save and Search” to continue. After clicking on the Save This Search button, one participant said: “Hmmm, create a new account and I actually don’t want to create a new account…Maybe I can just scroll down and have a look here.” @romanocog #AAPOR 13
  14. 14. Issue: It is not clear how to get started and enter data on the form 1: You want to compare the average cost of a public university to the average cost of a private university. Using this site, approximately how much more is a 4-year private university likely to cost? 2: You’re deciding between the University of Virginia Main Campus and Georgetown, and you’re interested in graduation rates. Which university graduates a higher percentage of students who enroll? UX Best Practice: Ensure clickable parts of the form appear clickable. Users must click “Add a School” to get started. They select a college or university and enter financial data. UX Findings: • The “Add a school” button does not look like an actionable button. It is not clear that this is how to start. • The form did not look like it was clickable. • Users were unsure how to proceed once they had successfully brought up two schools side- by-side. 10 17 2 15 10 4 Task Accuracy Public/Private Task1 Graduation Rate Task2 Success Fail n/a @romanocog #AAPOR 14
  15. 15. Issue: The text on the home page is dense, and the form is not visible. UX Best Practice: Ensure that the location of page elements do not create the illusion that users have reached the bottom of a page when they have not.1 Facilitate scanning by using clear headings, short phrases and sentences, and short paragraphs. 2 Users enter information into the form and are shown their cancer risk. UX Findings: • Users do not read the dense text. • The form to enter information is below the fold, and the layout creates the illusion that the bottom of the page has been reached. “It seems like a lot of information. I would skip past it and use the tool.” “I see blah, blah, blah. A lot of information and I don’t really care for it.” “I didn’t see it because I didn’t scroll down. If I saw this on the first page, I would have been much happier.” 1: Ivory, M.Y., Sinha, R.R., & Hearst, M.A. (2000, June). Preliminary findings on quantitative measures for distinguishing highly rated information-centric web pages. Proceedings of the 6th Conference on Human Factors and the Web. Retrieved November 2005, from 2: Morkes, J. & Nielsen, J. (1997). Concise, SCANNABLE, and objective: How to write for the Web. Retrieved November 2005, from Rosenholtz, R., Li, Y., Mansfield, J. & Jin, Z. (2005). @romanocog #AAPOR 15
  16. 16. See Bristol, Romano Bergstrom & Link, AAPOR 2014 for more about the UX across devices. • Not necessarily the same UX across all devices • Different issues occur with different modes • Match user expectations and survey objectives • Conduct UX testing with real users Assess the UX across devices. @romanocog #AAPOR 16
  17. 17. Thank you! • Twitter: @forsmarshgroup, • LinkedIn: • Blog: Jennifer Romano Bergstrom @romanocog AAPOR 2014| Anaheim, CA
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.