Unifying the UX of a Survey Across Multiple Devices (MoDevEast 2013)


Published on

The visual design of surveys and other types of online data collection tools impacts how users perceive, understand and navigate the instrument as well as the responses they provide. Two key considerations that impact how users experience online data collection tools are the device they are using (e.g., smartphone, tablet, computer) and the method of interaction (e.g., website, app or both). When designing online data collection tools, creating a common user experience across different devices and methods of interaction is important to create a consistent user experience and to minimize measurement differences.

In this talk, we will compare the user experience across four different combinations of device and method of interaction of a survey: (1) desktop PC-website, (2) smartphone-app, (3) tablet-app, and (4) tablet-website. Through performance and eye-tracking data, we identify UX elements that must be unified across all devices as well as elements that might need customization for difference devices or methods of interaction.

Published in: Technology, Design
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Unifying the UX of a Survey Across Multiple Devices (MoDevEast 2013)

  1. 1. Unifying the User Experience of a Survey Across Multiple Devices Jennifer Romano Bergstrom, David Hawkins, Jon Strohl December 12, 2013 MoDevEast | McLean, VA @forsmarshgroup @romanocog
  2. 2. About this talk 1. Usability and User Experience Intro 2. Web Survey Design 3. Implications from Eye Tracking on a Mobile and Desktop Tool @forsmarshgroup @romanocog 2
  3. 3. What is Usability? “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” ISO 9241-11 @forsmarshgroup @romanocog 3
  4. 4. Usability vs. User Experience (UX) Whitney’s 5 Es of Usability Peter’s User Experience Honeycomb @forsmarshgroup @romanocog 4 The 5 Es to Understanding Users (W. Quesenbery): http://www.wqusability.com/articles/getting-started.html User Experience Design (P. Morville): http://semanticstudios.com/publications/semantics/000029.php
  5. 5. What People do on the Web @forsmarshgroup @romanocog 5 Krug, S. Don’t Make Me Think
  6. 6. UX Design Failures • • • Poor planning “It’s all about me.” (Redish: filing cabinets) Human cognitive limitations • Memory and Perception • (activity) • Primacy • Recency • Chunking • Patterns @forsmarshgroup @romanocog 6
  7. 7. Patterns @forsmarshgroup @romanocog 7
  8. 8. Patterns @forsmarshgroup @romanocog 8
  9. 9. Patterns @forsmarshgroup @romanocog 9
  10. 10. Patterns @forsmarshgroup @romanocog 10
  11. 11. Mental models and repeating behaviors @forsmarshgroup @romanocog 11
  12. 12. Mental models and repeating behaviors @forsmarshgroup @romanocog 12
  13. 13. Web Survey Design
  14. 14. Why is Design Important? • No interviewer present to correct/advise • Visual presentation affects responses • While the Internet provides many ways to enhance surveys, design tools may be misused • Respondents extract meaning from how question and response options are displayed • Design may distract from or interfere with responses • Design may affect data quality @forsmarshgroup @romanocog 14
  15. 15. Why is Design Important? @forsmarshgroup @romanocog 15 http://www.cc.gatech.edu/gvu/user_surveys/
  16. 16. Why is Design Important? Many surveys are long (> 30min) Long surveys have higher nonresponse rates Length affects quality Respondents are more tech savvy today and use multiple technologies • It is not just about reducing respondent burden and nonresponse • We must increase engagement • High-quality design = trust in the designer • • • • @forsmarshgroup @romanocog 16 Adams & Darwin, 1982; Dillman et al., 1993; Haberlein & Baumgartner, 1978
  17. 17. @forsmarshgroup @romanocog 17 http://www.pewinternet.org/Static-Pages/TrendData-(Adults)/Device-Ownership.aspx
  18. 18. @forsmarshgroup @romanocog 18 Nielsen: The Cross-Platform Report, Quarter 2, 2012-US
  19. 19. Check Boxes in drop-down menus @forsmarshgroup @romanocog 19
  20. 20. Radio Buttons on Mobile @forsmarshgroup @romanocog 20
  21. 21. Useful Error Messages @forsmarshgroup @romanocog 21
  22. 22. Error Messages Across Devices @forsmarshgroup @romanocog 22
  23. 23. Consistency Across Devices • Predictable – User can anticipate what the system will do • Dependable – System fulfills user’s expectations • Habit-forming – System encourages behavior • Transferable – Habits in one context can transfer to another • Natural – Consistent with user’s knowledge @forsmarshgroup @romanocog 23
  24. 24. Implications from Eye Tracking on a Mobile and Desktop Tool
  25. 25. UX Testing Methodology Regardless of the platform that the tool was tested on, participants completed tasks in this order on Device 1, 2, and 3. Device 1 Account Registration Tutorial Homepage Impressions @forsmarshgroup @romanocog 25 Device 2 Device 3 Log in Log in 1st entry 3rd entry 2nd entry Check Entries
  26. 26. What is beautiful is usable! Two-thirds of participants (24 of 36) had positive feedback about the appearance of the homepage. 16 Initial Impressions of Homepage* 14 12 • • • “I like the colors, and my name is up there so it’s directed to me.” “It’s very clean and visually pleasing. The color choices- aqua and blue- it’s very calming.” “It seems like icons are very big and clear on what they are.” 10 8 6 4 2 0 Colorful/Attractive Clean/Straightforward @forsmarshgroup @romanocog *Verbal responses to Moderator’s Guide Q4: Without making any clicks, can you tell me what are your initial impressions of the homepage? 26 Tractinsky, N., Katz, A. S., & Ikar, D. (2000). What is beautiful is usable. Interacting with computers, 13(2), 127-145.
  27. 27. Logical entry order The account creation process had a logical flow from one question to the next. • • • “The questions were simple and were easy to answer.” “I was able to figure everything out with no trouble.” “It’s very easy and straightforward to fill out.” 40 Logical Flow Ratings* Number of Responses 35 30 25 <=2 2 1& 20 3 3 4& >=4 5 15 10 5 0 Participant Responses* @forsmarshgroup @romanocog *Device 1 Satisfaction Questionnaire Q4: Please rate the flow from one question to the next when entering your demographic and household information. 1= not logical at all to 5= extremely logical 27
  28. 28. Entering is straightforward and easy. The TV entry process was “not difficult at all” or “only slightly difficult.” Entries were “moderately to extremely accurate” in capturing TV viewing behavior. 5 Average Ratings of TV Entry Process 4 3 2 1 0 TV Entry Difficulty* • • • TV Entry Accuracy** “It’s easy to click with your finger, and the prompts are easy to follow.” “There were direct questions and direct options.” “The screen is short; it's a not a multi-page app. I can enter it all in one.” Tablet app Heat Maps: After being instructed to enter a TV show, fixations were concentrated on the ‘Enter Viewing’ icon, suggesting that the functionality of that icon is very clear. @forsmarshgroup @romanocog 28 Android app *Moderator’s Guide Q13 & Q25: On a scale of 1 to 5, how difficult was it to input your TV viewing entry, and why? 1 = not difficult at all to 5= extremely difficult **Device 2 & 3 Satisfaction Questionnaire Q6: Please rate how accurately this app/site, when used on this device, captures your actual TV viewing . 1= not accurate at all to 5= extremely accurate
  29. 29. Unified experience from paper to mobile/desktop Prototype Grid Order of Entry Paper Website/App 1. Name of Show 2. Station Name 3. Station Name/Number 4. Viewing Type 29 2. Name of Show 3. Channel Number @forsmarshgroup @romanocog 1. Viewing Type 4. Time
  30. 30. Issue: Icons are not clickable and do not match user expectations. Logo should link to the homepage, similar to the way logos work on websites. “I’m going to click on it, but it's not doing anything. That is annoying.” • “Do these not work?” • Homepage on the iPad website does not display the clickable links below each icon. Clicks along the bottom of the PC homepage showing that participants expected the icons to be working links. UX Best Practice: Reduce the amount of non-clickable real estate on the homepage. Focus on making icons and images clickable. This will help the homepage feel more intuitive as users expect icons and images to function as active links. @forsmarshgroup @romanocog 30
  31. 31. Issue: Error messages do not promote recovery. Gaze Plot: After getting an error message, the participant had to search all over the screen to find the missing field. • • “How do I advance to the next screen?” “It seems like it's stuck on the screen.” UX Best Practice: Specifically explain the error, and place the messages near where the error occurs so the user can quickly fix the error and move on. @forsmarshgroup @romanocog 31 M
  32. 32. Issue: The location of the ‘Next’ Button is not intuitive. Intuitive ‘Next’ button location Non-intuitive ‘Next’ button location • “Where and how you click is a bit counter-intuitive. [It’s] not super obvious which button to click to get to next sections.” • “I feel like the ‘Next’ should be at the bottom and not the top.” UX Best Practice: Users expect consistency and for navigation buttons to be close to the last entry field.* Place all ‘Next’ and navigation buttons near the last entry fields. @forsmarshgroup @romanocog * Couper, M.P., Baker, R., & Mechling, J. (2011). Placement and Design of Navigation Buttons in Web Surveys. Survey Practice, 4(1). 32
  33. 33. Issue: The ‘Back’ button confuses participants. • • • “I was expecting ‘Next,’ but I guess I have to push ‘Back.’” “It was hard to get confidence that I was proceeding in the right way.” “It didn't seem to flow, if you will—you had to go back and forth.” UX Best Practice: Make the location of the ‘Next’ button consistent across all devices by placing it in the lower right corner. This will assure users that their selections will be saved and promote a sense of linear progression. @forsmarshgroup @romanocog 33
  34. 34. Issue: The iPad website keyboard blocks the screen. 10 Tablet Web Users: Least Favorite Device Number of iPad Web Participants 9 8 7 6 5 4 3 2 1 0 Tablet PC Smart Phone UX Best Practice: Design the layout of the page such that important functions are always visible while the keyboard is open. @forsmarshgroup @romanocog 34
  35. 35. Issue: Workload is high when searching for entries. UX Best Practice: Follow users’ mental model: Arrange previous entries with the most recent at the top. @forsmarshgroup @romanocog 35
  36. 36. Assess the UX across devices • • • • Not necessarily the same UX across all devices Different issues occur on different devices Match user expectations and product objectives Conduct UX testing @forsmarshgroup @romanocog 36
  37. 37. Thank you! • • • Twitter: @forsmarshgroup LinkedIn: http://www.linkedin.com/company/fors-marsh-group Blog: www.forsmarshgroup.com/index.php/blog Jennifer Romano Bergstrom @romanocog jbergstrom@forsmarshgroup.com MoDevEast