Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
UX Assessment Techniques

Jen Romano Bergstrom
December 11, 2013
NOVA UX
@forsmarshgroup @romanocog
Measuring the UX
“the extent to which a
product can be used by
specified users to
achieve specified goals
with effectivene...
Why is it important?
• Put it in the hands of the end user.
• Things may seem straightforward to you but maybe not to
your...
Why is it important?
• Put it in the hands of the end user.
• Things may seem straightforward to you but maybe not to
your...
Why is it important?
• Put it in the hands of the end user.
• Things may seem straightforward to you but maybe not to
your...
Where to test
LABORATORY
•

Controlled environment

•

All participants have the
same experience

•

Record and
communicat...
Where to test
LABORATORY
•

Controlled environment

•

All participants have the
same experience

•

Record and
communicat...
How to test
ONE-ON-ONE SESSIONS
•

•
•

In-depth feedback from
each participant

FOCUS GROUPS
•

No group think
Can allow ...
When to test

@forsmarshgroup @romanocog

9
What to measure
EXPLICIT

OBSERVATIONAL

+ Post-task satisfaction
questionnaires
+ In-session difficulty ratings
+ Verbal ...
Case 1
• Problems: What do users want? Does the new design work?
• Methods: Focus groups, one-on-one interviews, in-lab
us...
Case 1
• Problems: What do users want? Does the new design work?
• Methods: Focus groups, one-on-one interviews, in-lab
us...
Case 2
• Problem: What parts of the form do people actually read?
• Method: In-lab usability testing with eye tracking

@f...
Case 2
Participants did not read the
instructions in their entirety (page 1:
left; page 3: right); rather, they skimmed
an...
Case 3
Prototype Grid

@forsmarshgroup @romanocog

15

Old Grid
Case 4
Gaze Plot: After getting an error message, the participant
had to search all over the screen to find the missing fi...
Many ways to evaluate the UX
•
•
•
•

Surveys
Focus groups
In-person one-on-one with eye tracking
Analytics

@forsmarshgro...
Obstacles to UX Testing
• There is no time.
– Start early in development process
– One morning a month with 3 users – Krug...
Thank you!
•
•
•

Twitter: @forsmarshgroup
LinkedIn: http://www.linkedin.com/company/fors-marsh-group
Blog: www.forsmarshg...
Upcoming SlideShare
Loading in …5
×

UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

1,352 views

Published on

Published in: Technology, Business
  • Be the first to comment

UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

  1. 1. UX Assessment Techniques Jen Romano Bergstrom December 11, 2013 NOVA UX @forsmarshgroup @romanocog
  2. 2. Measuring the UX “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” ISO 9241-11 + emotions @forsmarshgroup @romanocog 2 • How does it work for the end user? • What does the user expect? • How does it make the user feel?
  3. 3. Why is it important? • Put it in the hands of the end user. • Things may seem straightforward to you but maybe not to your users. @forsmarshgroup @romanocog 3
  4. 4. Why is it important? • Put it in the hands of the end user. • Things may seem straightforward to you but maybe not to your users. @forsmarshgroup @romanocog 4
  5. 5. Why is it important? • Put it in the hands of the end user. • Things may seem straightforward to you but maybe not to your users. @forsmarshgroup @romanocog 5
  6. 6. Where to test LABORATORY • Controlled environment • All participants have the same experience • Record and communicate from control room • Observers watch from control room and provide additional probes (via moderator) in real time • • Incorporate physiological measures (e.g., eye tracking, EDA) No travel costs @forsmarshgroup @romanocog 6 REMOTE • Participants in their natural environments (e.g., home, work) • Use video chat (moderated sessions) or online programs (unmoderated) IN THE FIELD • Participants tend to be more comfortable in their natural environments • Recruit hard-to-reach populations (e.g., children, doctors) • Conduct many sessions quickly • Moderator travels to various locations • Recruit participants in many locations (e.g., states, countries) • Bring equipment (e.g., eye tracker) • Natural observations
  7. 7. Where to test LABORATORY • Controlled environment • All participants have the same experience • Record and communicate from control room • Observers watch from control room and provide additional probes (via moderator) in real time • • Incorporate physiological measures (e.g., eye tracking, EDA) No travel costs @forsmarshgroup @romanocog 7 REMOTE • Participants in their natural environments (e.g., home, work) • Use video chat (moderated sessions) or online programs (unmoderated) IN THE FIELD • Participants tend to be more comfortable in their natural environments • Recruit hard-to-reach populations (e.g., children, doctors) • Conduct many sessions quickly • Moderator travels to various locations • Recruit participants in many locations (e.g., states, countries) • Bring equipment (e.g., eye tracker) • Natural observations
  8. 8. How to test ONE-ON-ONE SESSIONS • • • In-depth feedback from each participant FOCUS GROUPS • No group think Can allow participants to take their own route and explore freely Participants may be more comfortable with others • Interview many people quickly • Opinions collide • No interference • Peer review • Remote in participant’s environment • Qualitative • Flexible scheduling • Qualitative and Quantitative @forsmarshgroup @romanocog 8 SURVEYS • Representative • Large sample sizes • Collect a lot of data quickly • No interviewer bias • No scheduling sessions • Quantitative analysis
  9. 9. When to test @forsmarshgroup @romanocog 9
  10. 10. What to measure EXPLICIT OBSERVATIONAL + Post-task satisfaction questionnaires + In-session difficulty ratings + Verbal responses + Moderator follow up + Real-time +/- dial + Ethnography + Time to complete task + Reaction time + Selection/click behavior + Ability to complete tasks + Accuracy IMPLICIT + Facial expression analysis + Eye tracking + Electrodermal activity (EDA) + Behavioral analysis + Linguistic analysis of verbalizations + Implicit associations + Pupil dilation @forsmarshgroup @romanocog 10
  11. 11. Case 1 • Problems: What do users want? Does the new design work? • Methods: Focus groups, one-on-one interviews, in-lab usability testing with eye tracking @forsmarshgroup @romanocog 1. Participant repeatedly fixated the upper right hand corner. Participant said that he/she was looking for a search tool on the page. The search tool was in a disappearing banner on the page. 2. Participants had similar fixation counts across bottom links, indicating uncertainty of where to click to get started. 11
  12. 12. Case 1 • Problems: What do users want? Does the new design work? • Methods: Focus groups, one-on-one interviews, in-lab usability testing with eye tracking @forsmarshgroup @romanocog 1. Participant repeatedly fixated the upper right hand corner. Participant said that he/she was looking for a search tool on the page. The search tool was in a disappearing banner on the page. 2. Participants had similar fixation counts across bottom links, indicating uncertainty of where to click to get started. 12
  13. 13. Case 2 • Problem: What parts of the form do people actually read? • Method: In-lab usability testing with eye tracking @forsmarshgroup @romanocog 13
  14. 14. Case 2 Participants did not read the instructions in their entirety (page 1: left; page 3: right); rather, they skimmed and then moved on to the form where they needed to enter information. 120 Time (seconds) 100 80 60 40 20 Aggregate fixation count heat map across all participants, Page 1. Participants looked at ‘Purpose of Form’ section the most often. @forsmarshgroup @romanocog 14 0 Page 1 Page 2 Page 3 Length of time spent on each page of the instructions before working on form.
  15. 15. Case 3 Prototype Grid @forsmarshgroup @romanocog 15 Old Grid
  16. 16. Case 4 Gaze Plot: After getting an error message, the participant had to search all over the screen to find the missing field. Smartphone “How do I advance to the next screen?” “It seems like it's stuck on the screen.” @forsmarshgroup @romanocog 16 Tablet
  17. 17. Many ways to evaluate the UX • • • • Surveys Focus groups In-person one-on-one with eye tracking Analytics @forsmarshgroup @romanocog 17
  18. 18. Obstacles to UX Testing • There is no time. – Start early in development process – One morning a month with 3 users – Krug – 12 people in 3 days – Anderson Reimer – 12 people in 2 days – Lebson & Romano Bergstrom • Can’t find representative users – Everyone is important – Travel – Remote testing • We don’t have a lab – Test anywhere @forsmarshgroup @romanocog 18
  19. 19. Thank you! • • • Twitter: @forsmarshgroup LinkedIn: http://www.linkedin.com/company/fors-marsh-group Blog: www.forsmarshgroup.com/index.php/blog Jennifer Romano Bergstrom @romanocog jbergstrom@forsmarshgroup.com NOVA UX

×