UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

1,003 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,003
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

  1. 1. UX Assessment Techniques Jen Romano Bergstrom December 11, 2013 NOVA UX @forsmarshgroup @romanocog
  2. 2. Measuring the UX “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” ISO 9241-11 + emotions @forsmarshgroup @romanocog 2 • How does it work for the end user? • What does the user expect? • How does it make the user feel?
  3. 3. Why is it important? • Put it in the hands of the end user. • Things may seem straightforward to you but maybe not to your users. @forsmarshgroup @romanocog 3
  4. 4. Why is it important? • Put it in the hands of the end user. • Things may seem straightforward to you but maybe not to your users. @forsmarshgroup @romanocog 4
  5. 5. Why is it important? • Put it in the hands of the end user. • Things may seem straightforward to you but maybe not to your users. @forsmarshgroup @romanocog 5
  6. 6. Where to test LABORATORY • Controlled environment • All participants have the same experience • Record and communicate from control room • Observers watch from control room and provide additional probes (via moderator) in real time • • Incorporate physiological measures (e.g., eye tracking, EDA) No travel costs @forsmarshgroup @romanocog 6 REMOTE • Participants in their natural environments (e.g., home, work) • Use video chat (moderated sessions) or online programs (unmoderated) IN THE FIELD • Participants tend to be more comfortable in their natural environments • Recruit hard-to-reach populations (e.g., children, doctors) • Conduct many sessions quickly • Moderator travels to various locations • Recruit participants in many locations (e.g., states, countries) • Bring equipment (e.g., eye tracker) • Natural observations
  7. 7. Where to test LABORATORY • Controlled environment • All participants have the same experience • Record and communicate from control room • Observers watch from control room and provide additional probes (via moderator) in real time • • Incorporate physiological measures (e.g., eye tracking, EDA) No travel costs @forsmarshgroup @romanocog 7 REMOTE • Participants in their natural environments (e.g., home, work) • Use video chat (moderated sessions) or online programs (unmoderated) IN THE FIELD • Participants tend to be more comfortable in their natural environments • Recruit hard-to-reach populations (e.g., children, doctors) • Conduct many sessions quickly • Moderator travels to various locations • Recruit participants in many locations (e.g., states, countries) • Bring equipment (e.g., eye tracker) • Natural observations
  8. 8. How to test ONE-ON-ONE SESSIONS • • • In-depth feedback from each participant FOCUS GROUPS • No group think Can allow participants to take their own route and explore freely Participants may be more comfortable with others • Interview many people quickly • Opinions collide • No interference • Peer review • Remote in participant’s environment • Qualitative • Flexible scheduling • Qualitative and Quantitative @forsmarshgroup @romanocog 8 SURVEYS • Representative • Large sample sizes • Collect a lot of data quickly • No interviewer bias • No scheduling sessions • Quantitative analysis
  9. 9. When to test @forsmarshgroup @romanocog 9
  10. 10. What to measure EXPLICIT OBSERVATIONAL + Post-task satisfaction questionnaires + In-session difficulty ratings + Verbal responses + Moderator follow up + Real-time +/- dial + Ethnography + Time to complete task + Reaction time + Selection/click behavior + Ability to complete tasks + Accuracy IMPLICIT + Facial expression analysis + Eye tracking + Electrodermal activity (EDA) + Behavioral analysis + Linguistic analysis of verbalizations + Implicit associations + Pupil dilation @forsmarshgroup @romanocog 10
  11. 11. Case 1 • Problems: What do users want? Does the new design work? • Methods: Focus groups, one-on-one interviews, in-lab usability testing with eye tracking @forsmarshgroup @romanocog 1. Participant repeatedly fixated the upper right hand corner. Participant said that he/she was looking for a search tool on the page. The search tool was in a disappearing banner on the page. 2. Participants had similar fixation counts across bottom links, indicating uncertainty of where to click to get started. 11
  12. 12. Case 1 • Problems: What do users want? Does the new design work? • Methods: Focus groups, one-on-one interviews, in-lab usability testing with eye tracking @forsmarshgroup @romanocog 1. Participant repeatedly fixated the upper right hand corner. Participant said that he/she was looking for a search tool on the page. The search tool was in a disappearing banner on the page. 2. Participants had similar fixation counts across bottom links, indicating uncertainty of where to click to get started. 12
  13. 13. Case 2 • Problem: What parts of the form do people actually read? • Method: In-lab usability testing with eye tracking @forsmarshgroup @romanocog 13
  14. 14. Case 2 Participants did not read the instructions in their entirety (page 1: left; page 3: right); rather, they skimmed and then moved on to the form where they needed to enter information. 120 Time (seconds) 100 80 60 40 20 Aggregate fixation count heat map across all participants, Page 1. Participants looked at ‘Purpose of Form’ section the most often. @forsmarshgroup @romanocog 14 0 Page 1 Page 2 Page 3 Length of time spent on each page of the instructions before working on form.
  15. 15. Case 3 Prototype Grid @forsmarshgroup @romanocog 15 Old Grid
  16. 16. Case 4 Gaze Plot: After getting an error message, the participant had to search all over the screen to find the missing field. Smartphone “How do I advance to the next screen?” “It seems like it's stuck on the screen.” @forsmarshgroup @romanocog 16 Tablet
  17. 17. Many ways to evaluate the UX • • • • Surveys Focus groups In-person one-on-one with eye tracking Analytics @forsmarshgroup @romanocog 17
  18. 18. Obstacles to UX Testing • There is no time. – Start early in development process – One morning a month with 3 users – Krug – 12 people in 3 days – Anderson Reimer – 12 people in 2 days – Lebson & Romano Bergstrom • Can’t find representative users – Everyone is important – Travel – Remote testing • We don’t have a lab – Test anywhere @forsmarshgroup @romanocog 18
  19. 19. Thank you! • • • Twitter: @forsmarshgroup LinkedIn: http://www.linkedin.com/company/fors-marsh-group Blog: www.forsmarshgroup.com/index.php/blog Jennifer Romano Bergstrom @romanocog jbergstrom@forsmarshgroup.com NOVA UX

×