SlideShare a Scribd company logo
1 of 19
Download to read offline
UX Assessment Techniques

Jen Romano Bergstrom
December 11, 2013
NOVA UX
@forsmarshgroup @romanocog
Measuring the UX
“the extent to which a
product can be used by
specified users to
achieve specified goals
with effectiveness,
efficiency, and
satisfaction in a
specified context of
use.” ISO 9241-11
+ emotions

@forsmarshgroup @romanocog

2

• How does it work for the
end user?
• What does the user
expect?
• How does it make the user
feel?
Why is it important?
• Put it in the hands of the end user.
• Things may seem straightforward to you but maybe not to
your users.

@forsmarshgroup @romanocog

3
Why is it important?
• Put it in the hands of the end user.
• Things may seem straightforward to you but maybe not to
your users.

@forsmarshgroup @romanocog

4
Why is it important?
• Put it in the hands of the end user.
• Things may seem straightforward to you but maybe not to
your users.

@forsmarshgroup @romanocog

5
Where to test
LABORATORY
•

Controlled environment

•

All participants have the
same experience

•

Record and
communicate from
control room

•

Observers watch from
control room and provide
additional probes (via
moderator) in real time

•

•

Incorporate physiological
measures (e.g., eye
tracking, EDA)
No travel costs

@forsmarshgroup @romanocog

6

REMOTE
•

Participants in their
natural environments
(e.g., home, work)

•

Use video chat
(moderated sessions)
or online programs
(unmoderated)

IN THE FIELD
•

Participants tend to be
more comfortable in
their natural
environments

•

Recruit hard-to-reach
populations (e.g.,
children, doctors)

•

Conduct many sessions
quickly

•

Moderator travels to
various locations

•

Recruit participants in
many locations (e.g.,
states, countries)

•

Bring equipment (e.g.,
eye tracker)

•

Natural observations
Where to test
LABORATORY
•

Controlled environment

•

All participants have the
same experience

•

Record and
communicate from
control room

•

Observers watch from
control room and provide
additional probes (via
moderator) in real time

•

•

Incorporate physiological
measures (e.g., eye
tracking, EDA)
No travel costs

@forsmarshgroup @romanocog

7

REMOTE
•

Participants in their
natural environments
(e.g., home, work)

•

Use video chat
(moderated sessions)
or online programs
(unmoderated)

IN THE FIELD
•

Participants tend to be
more comfortable in
their natural
environments

•

Recruit hard-to-reach
populations (e.g.,
children, doctors)

•

Conduct many sessions
quickly

•

Moderator travels to
various locations

•

Recruit participants in
many locations (e.g.,
states, countries)

•

Bring equipment (e.g.,
eye tracker)

•

Natural observations
How to test
ONE-ON-ONE SESSIONS
•

•
•

In-depth feedback from
each participant

FOCUS GROUPS
•

No group think
Can allow participants to
take their own route and
explore freely

Participants may be
more comfortable with
others

•

Interview many people
quickly

•

Opinions collide

•

No interference

•

Peer review

•

Remote in participant’s
environment

•

Qualitative

•

Flexible scheduling

•

Qualitative and
Quantitative

@forsmarshgroup @romanocog

8

SURVEYS
•

Representative

•

Large sample sizes

•

Collect a lot of data
quickly

•

No interviewer bias

•

No scheduling sessions

•

Quantitative analysis
When to test

@forsmarshgroup @romanocog

9
What to measure
EXPLICIT

OBSERVATIONAL

+ Post-task satisfaction
questionnaires
+ In-session difficulty ratings
+ Verbal responses
+ Moderator follow up
+ Real-time +/- dial

+ Ethnography
+ Time to complete task
+ Reaction time
+ Selection/click behavior
+ Ability to complete tasks
+ Accuracy

IMPLICIT
+ Facial expression analysis
+ Eye tracking
+ Electrodermal activity (EDA)
+ Behavioral analysis
+ Linguistic analysis of verbalizations
+ Implicit associations
+ Pupil dilation
@forsmarshgroup @romanocog

10
Case 1
• Problems: What do users want? Does the new design work?
• Methods: Focus groups, one-on-one interviews, in-lab
usability testing with eye tracking

@forsmarshgroup @romanocog
1. Participant repeatedly fixated the upper right hand corner. Participant said that he/she was looking for a search tool on the
page. The search tool was in a disappearing banner on the page.
2. Participants had similar fixation counts across bottom links, indicating uncertainty of where to click to get started.

11
Case 1
• Problems: What do users want? Does the new design work?
• Methods: Focus groups, one-on-one interviews, in-lab
usability testing with eye tracking

@forsmarshgroup @romanocog
1. Participant repeatedly fixated the upper right hand corner. Participant said that he/she was looking for a search tool on the
page. The search tool was in a disappearing banner on the page.
2. Participants had similar fixation counts across bottom links, indicating uncertainty of where to click to get started.

12
Case 2
• Problem: What parts of the form do people actually read?
• Method: In-lab usability testing with eye tracking

@forsmarshgroup @romanocog

13
Case 2
Participants did not read the
instructions in their entirety (page 1:
left; page 3: right); rather, they skimmed
and then moved on to the form where
they needed to enter information.

120

Time (seconds)

100
80
60
40
20

Aggregate fixation count heat map across all
participants, Page 1. Participants looked at
‘Purpose of Form’ section the most often.

@forsmarshgroup @romanocog

14

0
Page 1

Page 2

Page 3

Length of time spent on each page of the instructions
before working on form.
Case 3
Prototype Grid

@forsmarshgroup @romanocog

15

Old Grid
Case 4
Gaze Plot: After getting an error message, the participant
had to search all over the screen to find the missing field.

Smartphone

“How do I advance to the next screen?”
“It seems like it's stuck on the screen.”
@forsmarshgroup @romanocog

16

Tablet
Many ways to evaluate the UX
•
•
•
•

Surveys
Focus groups
In-person one-on-one with eye tracking
Analytics

@forsmarshgroup @romanocog

17
Obstacles to UX Testing
• There is no time.
– Start early in development process
– One morning a month with 3 users – Krug
– 12 people in 3 days – Anderson Reimer
– 12 people in 2 days – Lebson & Romano Bergstrom
• Can’t find representative users
– Everyone is important
– Travel
– Remote testing
• We don’t have a lab
– Test anywhere
@forsmarshgroup @romanocog

18
Thank you!
•
•
•

Twitter: @forsmarshgroup
LinkedIn: http://www.linkedin.com/company/fors-marsh-group
Blog: www.forsmarshgroup.com/index.php/blog

Jennifer Romano Bergstrom
@romanocog
jbergstrom@forsmarshgroup.com
NOVA UX

More Related Content

What's hot

Guest lecture: Designing mobile apps
Guest lecture: Designing mobile appsGuest lecture: Designing mobile apps
Guest lecture: Designing mobile appsJohn Rooksby
 
Remote Audits in a COVID-19 World
Remote Audits in a COVID-19 WorldRemote Audits in a COVID-19 World
Remote Audits in a COVID-19 WorldGreenlight Guru
 
Getting the most from guerrilla usability testing
Getting the most from guerrilla usability testingGetting the most from guerrilla usability testing
Getting the most from guerrilla usability testingNexer Digital
 
Web Survey and Forms Usability Design & Testing
Web Survey and Forms Usability Design & TestingWeb Survey and Forms Usability Design & Testing
Web Survey and Forms Usability Design & TestingJennifer Romano Bergstrom
 
How to Fail at Building Websites - DrupalCorn 2014
How to Fail at Building Websites - DrupalCorn 2014How to Fail at Building Websites - DrupalCorn 2014
How to Fail at Building Websites - DrupalCorn 2014Ian Lintner
 
Predictive Analytics with UX Research Data: Yes We Can!
Predictive Analytics with UX Research Data: Yes We Can!Predictive Analytics with UX Research Data: Yes We Can!
Predictive Analytics with UX Research Data: Yes We Can!UXPA Boston
 
Getting business people and developers to listen to testers
Getting business people and developers to listen to testersGetting business people and developers to listen to testers
Getting business people and developers to listen to testersgojkoadzic
 
Beyond co-creation, be radically transparent
Beyond co-creation, be radically transparentBeyond co-creation, be radically transparent
Beyond co-creation, be radically transparentAnouschka Scholten
 
Agile Testers: Becoming a key asset for your team
Agile Testers: Becoming a key asset for your teamAgile Testers: Becoming a key asset for your team
Agile Testers: Becoming a key asset for your teamgojkoadzic
 
Design in Sight workshop presentation
Design in Sight workshop presentationDesign in Sight workshop presentation
Design in Sight workshop presentationStuart Chapman
 
UX in 10 Minutes - Usability Testing - Presented at SEMPO
UX in 10 Minutes - Usability Testing - Presented at SEMPOUX in 10 Minutes - Usability Testing - Presented at SEMPO
UX in 10 Minutes - Usability Testing - Presented at SEMPOCarol Smith
 
Designing for unfamiliar interfaces
Designing for unfamiliar interfacesDesigning for unfamiliar interfaces
Designing for unfamiliar interfacesDaniel Soltis
 
Sneaking in Good User Experience Without a UX Budget - anthonydpaul - WordCam...
Sneaking in Good User Experience Without a UX Budget - anthonydpaul - WordCam...Sneaking in Good User Experience Without a UX Budget - anthonydpaul - WordCam...
Sneaking in Good User Experience Without a UX Budget - anthonydpaul - WordCam...Anthony D. Paul
 

What's hot (20)

Guest lecture: Designing mobile apps
Guest lecture: Designing mobile appsGuest lecture: Designing mobile apps
Guest lecture: Designing mobile apps
 
Remote Audits in a COVID-19 World
Remote Audits in a COVID-19 WorldRemote Audits in a COVID-19 World
Remote Audits in a COVID-19 World
 
Getting the most from guerrilla usability testing
Getting the most from guerrilla usability testingGetting the most from guerrilla usability testing
Getting the most from guerrilla usability testing
 
Rapid Prototyping
Rapid PrototypingRapid Prototyping
Rapid Prototyping
 
Web Survey and Forms Usability Design & Testing
Web Survey and Forms Usability Design & TestingWeb Survey and Forms Usability Design & Testing
Web Survey and Forms Usability Design & Testing
 
How to Fail at Building Websites - DrupalCorn 2014
How to Fail at Building Websites - DrupalCorn 2014How to Fail at Building Websites - DrupalCorn 2014
How to Fail at Building Websites - DrupalCorn 2014
 
Predictive Analytics with UX Research Data: Yes We Can!
Predictive Analytics with UX Research Data: Yes We Can!Predictive Analytics with UX Research Data: Yes We Can!
Predictive Analytics with UX Research Data: Yes We Can!
 
Michael
MichaelMichael
Michael
 
Getting business people and developers to listen to testers
Getting business people and developers to listen to testersGetting business people and developers to listen to testers
Getting business people and developers to listen to testers
 
Irrational Products
Irrational ProductsIrrational Products
Irrational Products
 
Beyond co-creation, be radically transparent
Beyond co-creation, be radically transparentBeyond co-creation, be radically transparent
Beyond co-creation, be radically transparent
 
Agile Testers: Becoming a key asset for your team
Agile Testers: Becoming a key asset for your teamAgile Testers: Becoming a key asset for your team
Agile Testers: Becoming a key asset for your team
 
Gokul90
Gokul90Gokul90
Gokul90
 
Karthik
KarthikKarthik
Karthik
 
Indrajith
IndrajithIndrajith
Indrajith
 
Tests are Nothing, Testing is Everything
Tests are Nothing, Testing is EverythingTests are Nothing, Testing is Everything
Tests are Nothing, Testing is Everything
 
Design in Sight workshop presentation
Design in Sight workshop presentationDesign in Sight workshop presentation
Design in Sight workshop presentation
 
UX in 10 Minutes - Usability Testing - Presented at SEMPO
UX in 10 Minutes - Usability Testing - Presented at SEMPOUX in 10 Minutes - Usability Testing - Presented at SEMPO
UX in 10 Minutes - Usability Testing - Presented at SEMPO
 
Designing for unfamiliar interfaces
Designing for unfamiliar interfacesDesigning for unfamiliar interfaces
Designing for unfamiliar interfaces
 
Sneaking in Good User Experience Without a UX Budget - anthonydpaul - WordCam...
Sneaking in Good User Experience Without a UX Budget - anthonydpaul - WordCam...Sneaking in Good User Experience Without a UX Budget - anthonydpaul - WordCam...
Sneaking in Good User Experience Without a UX Budget - anthonydpaul - WordCam...
 

Similar to UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

Launch With Confidence! Integrate UX Research Throughout Development
Launch With Confidence! Integrate UX Research Throughout DevelopmentLaunch With Confidence! Integrate UX Research Throughout Development
Launch With Confidence! Integrate UX Research Throughout DevelopmentJennifer Romano Bergstrom
 
Easy & Effective Usability Testing at CodeMash 2012
Easy & Effective Usability Testing at CodeMash 2012Easy & Effective Usability Testing at CodeMash 2012
Easy & Effective Usability Testing at CodeMash 2012Carol Smith
 
More Than Usability
More Than UsabilityMore Than Usability
More Than UsabilityRazan Sadeq
 
The what, when, why and how of usability
The what, when, why and how of usabilityThe what, when, why and how of usability
The what, when, why and how of usabilitySimpleUsability
 
How to Conduct Usability Studies: A Librarian Primer
How to Conduct Usability Studies: A Librarian PrimerHow to Conduct Usability Studies: A Librarian Primer
How to Conduct Usability Studies: A Librarian PrimerTao Zhang
 
UXprobe workshop at Dare Festival 2016
UXprobe workshop at Dare Festival 2016UXprobe workshop at Dare Festival 2016
UXprobe workshop at Dare Festival 2016UXprobe
 
Usability Testing for Survey Research:How to and Best Practices
Usability Testing for Survey Research:How to and Best PracticesUsability Testing for Survey Research:How to and Best Practices
Usability Testing for Survey Research:How to and Best Practicesegeisen
 
Usability study
Usability studyUsability study
Usability studyLi Bin
 
Conducting Remote Unmoderated Usability Testing: Part 1 - RemoteUX Training W...
Conducting Remote Unmoderated Usability Testing: Part 1 - RemoteUX Training W...Conducting Remote Unmoderated Usability Testing: Part 1 - RemoteUX Training W...
Conducting Remote Unmoderated Usability Testing: Part 1 - RemoteUX Training W...UserZoom
 
Learn Learning + Prototype Testing
Learn Learning + Prototype TestingLearn Learning + Prototype Testing
Learn Learning + Prototype TestingDave Hora
 
UsabilityMatters_Usability_Testing_Introduction_Workshop
UsabilityMatters_Usability_Testing_Introduction_WorkshopUsabilityMatters_Usability_Testing_Introduction_Workshop
UsabilityMatters_Usability_Testing_Introduction_WorkshopUsability Matters
 
Website Usability & Eye-tracking by Marco Pretorious (Certified Usability Ana...
Website Usability & Eye-tracking by Marco Pretorious (Certified Usability Ana...Website Usability & Eye-tracking by Marco Pretorious (Certified Usability Ana...
Website Usability & Eye-tracking by Marco Pretorious (Certified Usability Ana...DrupalCape
 
Remote usability testing and remote user research for usability
Remote usability testing and remote user research for usabilityRemote usability testing and remote user research for usability
Remote usability testing and remote user research for usabilityUser Vision
 
UX Research in an Agile World
UX Research in an Agile WorldUX Research in an Agile World
UX Research in an Agile WorldHirajaved10
 
Designing and deploying mobile user studies in the wild: a practical guide
Designing and deploying mobile user studies in the wild: a practical guideDesigning and deploying mobile user studies in the wild: a practical guide
Designing and deploying mobile user studies in the wild: a practical guideKaren Church
 
Unifying the UX of a Survey Across Multiple Devices (MoDevEast 2013)
Unifying the UX of a Survey Across Multiple Devices (MoDevEast 2013)Unifying the UX of a Survey Across Multiple Devices (MoDevEast 2013)
Unifying the UX of a Survey Across Multiple Devices (MoDevEast 2013)Jennifer Romano Bergstrom
 
Game design 2 (2013): Lecture 11 - User Feedback in Game Design
Game design 2 (2013): Lecture 11 - User Feedback in Game DesignGame design 2 (2013): Lecture 11 - User Feedback in Game Design
Game design 2 (2013): Lecture 11 - User Feedback in Game DesignDavid Farrell
 
Presented to Japan MRA: Qualitative Research in the US - 10 trends & move tow...
Presented to Japan MRA: Qualitative Research in the US - 10 trends & move tow...Presented to Japan MRA: Qualitative Research in the US - 10 trends & move tow...
Presented to Japan MRA: Qualitative Research in the US - 10 trends & move tow...Dorrie Paynter
 

Similar to UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013) (20)

Launch With Confidence! Integrate UX Research Throughout Development
Launch With Confidence! Integrate UX Research Throughout DevelopmentLaunch With Confidence! Integrate UX Research Throughout Development
Launch With Confidence! Integrate UX Research Throughout Development
 
Easy & Effective Usability Testing at CodeMash 2012
Easy & Effective Usability Testing at CodeMash 2012Easy & Effective Usability Testing at CodeMash 2012
Easy & Effective Usability Testing at CodeMash 2012
 
More Than Usability
More Than UsabilityMore Than Usability
More Than Usability
 
Try It! Learn to Improve Guides and Websites Using Design Research Methods
Try It! Learn to Improve Guides and Websites Using Design Research MethodsTry It! Learn to Improve Guides and Websites Using Design Research Methods
Try It! Learn to Improve Guides and Websites Using Design Research Methods
 
The what, when, why and how of usability
The what, when, why and how of usabilityThe what, when, why and how of usability
The what, when, why and how of usability
 
How to Conduct Usability Studies: A Librarian Primer
How to Conduct Usability Studies: A Librarian PrimerHow to Conduct Usability Studies: A Librarian Primer
How to Conduct Usability Studies: A Librarian Primer
 
UXprobe workshop at Dare Festival 2016
UXprobe workshop at Dare Festival 2016UXprobe workshop at Dare Festival 2016
UXprobe workshop at Dare Festival 2016
 
Usability Testing for Survey Research:How to and Best Practices
Usability Testing for Survey Research:How to and Best PracticesUsability Testing for Survey Research:How to and Best Practices
Usability Testing for Survey Research:How to and Best Practices
 
Intro to Lean UX with UserTesting
Intro to Lean UX with UserTestingIntro to Lean UX with UserTesting
Intro to Lean UX with UserTesting
 
Usability study
Usability studyUsability study
Usability study
 
Conducting Remote Unmoderated Usability Testing: Part 1 - RemoteUX Training W...
Conducting Remote Unmoderated Usability Testing: Part 1 - RemoteUX Training W...Conducting Remote Unmoderated Usability Testing: Part 1 - RemoteUX Training W...
Conducting Remote Unmoderated Usability Testing: Part 1 - RemoteUX Training W...
 
Learn Learning + Prototype Testing
Learn Learning + Prototype TestingLearn Learning + Prototype Testing
Learn Learning + Prototype Testing
 
UsabilityMatters_Usability_Testing_Introduction_Workshop
UsabilityMatters_Usability_Testing_Introduction_WorkshopUsabilityMatters_Usability_Testing_Introduction_Workshop
UsabilityMatters_Usability_Testing_Introduction_Workshop
 
Website Usability & Eye-tracking by Marco Pretorious (Certified Usability Ana...
Website Usability & Eye-tracking by Marco Pretorious (Certified Usability Ana...Website Usability & Eye-tracking by Marco Pretorious (Certified Usability Ana...
Website Usability & Eye-tracking by Marco Pretorious (Certified Usability Ana...
 
Remote usability testing and remote user research for usability
Remote usability testing and remote user research for usabilityRemote usability testing and remote user research for usability
Remote usability testing and remote user research for usability
 
UX Research in an Agile World
UX Research in an Agile WorldUX Research in an Agile World
UX Research in an Agile World
 
Designing and deploying mobile user studies in the wild: a practical guide
Designing and deploying mobile user studies in the wild: a practical guideDesigning and deploying mobile user studies in the wild: a practical guide
Designing and deploying mobile user studies in the wild: a practical guide
 
Unifying the UX of a Survey Across Multiple Devices (MoDevEast 2013)
Unifying the UX of a Survey Across Multiple Devices (MoDevEast 2013)Unifying the UX of a Survey Across Multiple Devices (MoDevEast 2013)
Unifying the UX of a Survey Across Multiple Devices (MoDevEast 2013)
 
Game design 2 (2013): Lecture 11 - User Feedback in Game Design
Game design 2 (2013): Lecture 11 - User Feedback in Game DesignGame design 2 (2013): Lecture 11 - User Feedback in Game Design
Game design 2 (2013): Lecture 11 - User Feedback in Game Design
 
Presented to Japan MRA: Qualitative Research in the US - 10 trends & move tow...
Presented to Japan MRA: Qualitative Research in the US - 10 trends & move tow...Presented to Japan MRA: Qualitative Research in the US - 10 trends & move tow...
Presented to Japan MRA: Qualitative Research in the US - 10 trends & move tow...
 

More from Jennifer Romano Bergstrom

Processing Speed and Vocabulary are Related to Older Adults' Internet Experie...
Processing Speed and Vocabulary are Related to Older Adults' Internet Experie...Processing Speed and Vocabulary are Related to Older Adults' Internet Experie...
Processing Speed and Vocabulary are Related to Older Adults' Internet Experie...Jennifer Romano Bergstrom
 
Eye Tracking the UX of Mobile: What You Need to Know
Eye Tracking the UX of Mobile: What You Need to KnowEye Tracking the UX of Mobile: What You Need to Know
Eye Tracking the UX of Mobile: What You Need to KnowJennifer Romano Bergstrom
 
Unbiased Methods to Understand the User Experience
Unbiased Methods to Understand the User ExperienceUnbiased Methods to Understand the User Experience
Unbiased Methods to Understand the User ExperienceJennifer Romano Bergstrom
 
User-Centered Research on the Paying for College Website and Tools - EDUI 2014
User-Centered Research on the Paying for College Website and Tools - EDUI 2014User-Centered Research on the Paying for College Website and Tools - EDUI 2014
User-Centered Research on the Paying for College Website and Tools - EDUI 2014Jennifer Romano Bergstrom
 
Improving Forms with User Experience Testing and Eye Tracking
Improving Forms with User Experience Testing and Eye TrackingImproving Forms with User Experience Testing and Eye Tracking
Improving Forms with User Experience Testing and Eye TrackingJennifer Romano Bergstrom
 
Eye Tracking the User Experience of Mobile: What You Need to Know
Eye Tracking the User Experience of Mobile: What You Need to KnowEye Tracking the User Experience of Mobile: What You Need to Know
Eye Tracking the User Experience of Mobile: What You Need to KnowJennifer Romano Bergstrom
 
Usable Government Forms and Surveys: Best Practices for Design (from MoDevGov)
Usable Government Forms and Surveys: Best Practices for Design (from MoDevGov)Usable Government Forms and Surveys: Best Practices for Design (from MoDevGov)
Usable Government Forms and Surveys: Best Practices for Design (from MoDevGov)Jennifer Romano Bergstrom
 
Age-Related Differences in Search Strategy and Performance When Using a Data-...
Age-Related Differences in Search Strategy and Performance When Using a Data-...Age-Related Differences in Search Strategy and Performance When Using a Data-...
Age-Related Differences in Search Strategy and Performance When Using a Data-...Jennifer Romano Bergstrom
 
Age Differences in the Knowledge and Usage of QR Codes
Age Differences in the Knowledge and Usage of QR CodesAge Differences in the Knowledge and Usage of QR Codes
Age Differences in the Knowledge and Usage of QR CodesJennifer Romano Bergstrom
 
Making your Brand Pinteresting to Youth - ESOMAR 2013
Making your Brand Pinteresting to Youth - ESOMAR 2013Making your Brand Pinteresting to Youth - ESOMAR 2013
Making your Brand Pinteresting to Youth - ESOMAR 2013Jennifer Romano Bergstrom
 
Beyond Eye Tracking: Using User Temperature, Rating Dials, and Facial Analysi...
Beyond Eye Tracking: Using User Temperature, Rating Dials, and Facial Analysi...Beyond Eye Tracking: Using User Temperature, Rating Dials, and Facial Analysi...
Beyond Eye Tracking: Using User Temperature, Rating Dials, and Facial Analysi...Jennifer Romano Bergstrom
 
Introduction to Web Survey Usability Design and Testing
Introduction to Web Survey Usability Design and TestingIntroduction to Web Survey Usability Design and Testing
Introduction to Web Survey Usability Design and TestingJennifer Romano Bergstrom
 
Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Me...
Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Me...Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Me...
Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Me...Jennifer Romano Bergstrom
 
Benefits of Concurrent Cognitive and Usability Testing
Benefits of Concurrent Cognitive and Usability TestingBenefits of Concurrent Cognitive and Usability Testing
Benefits of Concurrent Cognitive and Usability TestingJennifer Romano Bergstrom
 

More from Jennifer Romano Bergstrom (20)

Processing Speed and Vocabulary are Related to Older Adults' Internet Experie...
Processing Speed and Vocabulary are Related to Older Adults' Internet Experie...Processing Speed and Vocabulary are Related to Older Adults' Internet Experie...
Processing Speed and Vocabulary are Related to Older Adults' Internet Experie...
 
User-Centered Design of Forms and Surveys
User-Centered Design of Forms and SurveysUser-Centered Design of Forms and Surveys
User-Centered Design of Forms and Surveys
 
Eye Tracking the UX of Mobile: What You Need to Know
Eye Tracking the UX of Mobile: What You Need to KnowEye Tracking the UX of Mobile: What You Need to Know
Eye Tracking the UX of Mobile: What You Need to Know
 
UXPA2015 sponsorship prospectus
UXPA2015 sponsorship prospectusUXPA2015 sponsorship prospectus
UXPA2015 sponsorship prospectus
 
Unbiased Methods to Understand the User Experience
Unbiased Methods to Understand the User ExperienceUnbiased Methods to Understand the User Experience
Unbiased Methods to Understand the User Experience
 
Benchmarking Usability Performance
Benchmarking Usability PerformanceBenchmarking Usability Performance
Benchmarking Usability Performance
 
User-Centered Research on the Paying for College Website and Tools - EDUI 2014
User-Centered Research on the Paying for College Website and Tools - EDUI 2014User-Centered Research on the Paying for College Website and Tools - EDUI 2014
User-Centered Research on the Paying for College Website and Tools - EDUI 2014
 
Improving Forms with User Experience Testing and Eye Tracking
Improving Forms with User Experience Testing and Eye TrackingImproving Forms with User Experience Testing and Eye Tracking
Improving Forms with User Experience Testing and Eye Tracking
 
Eye Tracking the User Experience of Mobile: What You Need to Know
Eye Tracking the User Experience of Mobile: What You Need to KnowEye Tracking the User Experience of Mobile: What You Need to Know
Eye Tracking the User Experience of Mobile: What You Need to Know
 
Usable Government Forms and Surveys: Best Practices for Design (from MoDevGov)
Usable Government Forms and Surveys: Best Practices for Design (from MoDevGov)Usable Government Forms and Surveys: Best Practices for Design (from MoDevGov)
Usable Government Forms and Surveys: Best Practices for Design (from MoDevGov)
 
The UX of Social Media
The UX of Social MediaThe UX of Social Media
The UX of Social Media
 
Age-Related Differences in Search Strategy and Performance When Using a Data-...
Age-Related Differences in Search Strategy and Performance When Using a Data-...Age-Related Differences in Search Strategy and Performance When Using a Data-...
Age-Related Differences in Search Strategy and Performance When Using a Data-...
 
Age Differences in the Knowledge and Usage of QR Codes
Age Differences in the Knowledge and Usage of QR CodesAge Differences in the Knowledge and Usage of QR Codes
Age Differences in the Knowledge and Usage of QR Codes
 
Making your Brand Pinteresting to Youth - ESOMAR 2013
Making your Brand Pinteresting to Youth - ESOMAR 2013Making your Brand Pinteresting to Youth - ESOMAR 2013
Making your Brand Pinteresting to Youth - ESOMAR 2013
 
Remote Testing Rocks! Ignite at UXPA 2013
Remote Testing Rocks! Ignite at UXPA 2013Remote Testing Rocks! Ignite at UXPA 2013
Remote Testing Rocks! Ignite at UXPA 2013
 
Beyond Eye Tracking: Using User Temperature, Rating Dials, and Facial Analysi...
Beyond Eye Tracking: Using User Temperature, Rating Dials, and Facial Analysi...Beyond Eye Tracking: Using User Temperature, Rating Dials, and Facial Analysi...
Beyond Eye Tracking: Using User Temperature, Rating Dials, and Facial Analysi...
 
Introduction to Web Survey Usability Design and Testing
Introduction to Web Survey Usability Design and TestingIntroduction to Web Survey Usability Design and Testing
Introduction to Web Survey Usability Design and Testing
 
UX Fundamentals
UX FundamentalsUX Fundamentals
UX Fundamentals
 
Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Me...
Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Me...Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Me...
Effects of Age and Think-Aloud Protocol on Eye-Tracking Data and Usability Me...
 
Benefits of Concurrent Cognitive and Usability Testing
Benefits of Concurrent Cognitive and Usability TestingBenefits of Concurrent Cognitive and Usability Testing
Benefits of Concurrent Cognitive and Usability Testing
 

Recently uploaded

Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5DianaGray10
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Valere | Digital Solutions & AI Transformation Portfolio | 2024
Valere | Digital Solutions & AI Transformation Portfolio | 2024Valere | Digital Solutions & AI Transformation Portfolio | 2024
Valere | Digital Solutions & AI Transformation Portfolio | 2024Alexander Turgeon
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"
UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"
UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"DianaGray10
 
100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimizationarrow10202532yuvraj
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 

Recently uploaded (20)

Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5UiPath Studio Web workshop series - Day 5
UiPath Studio Web workshop series - Day 5
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Valere | Digital Solutions & AI Transformation Portfolio | 2024
Valere | Digital Solutions & AI Transformation Portfolio | 2024Valere | Digital Solutions & AI Transformation Portfolio | 2024
Valere | Digital Solutions & AI Transformation Portfolio | 2024
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"
UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"
UiPath Clipboard AI: "A TIME Magazine Best Invention of 2023 Unveiled"
 
100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 

UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

  • 1. UX Assessment Techniques Jen Romano Bergstrom December 11, 2013 NOVA UX @forsmarshgroup @romanocog
  • 2. Measuring the UX “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” ISO 9241-11 + emotions @forsmarshgroup @romanocog 2 • How does it work for the end user? • What does the user expect? • How does it make the user feel?
  • 3. Why is it important? • Put it in the hands of the end user. • Things may seem straightforward to you but maybe not to your users. @forsmarshgroup @romanocog 3
  • 4. Why is it important? • Put it in the hands of the end user. • Things may seem straightforward to you but maybe not to your users. @forsmarshgroup @romanocog 4
  • 5. Why is it important? • Put it in the hands of the end user. • Things may seem straightforward to you but maybe not to your users. @forsmarshgroup @romanocog 5
  • 6. Where to test LABORATORY • Controlled environment • All participants have the same experience • Record and communicate from control room • Observers watch from control room and provide additional probes (via moderator) in real time • • Incorporate physiological measures (e.g., eye tracking, EDA) No travel costs @forsmarshgroup @romanocog 6 REMOTE • Participants in their natural environments (e.g., home, work) • Use video chat (moderated sessions) or online programs (unmoderated) IN THE FIELD • Participants tend to be more comfortable in their natural environments • Recruit hard-to-reach populations (e.g., children, doctors) • Conduct many sessions quickly • Moderator travels to various locations • Recruit participants in many locations (e.g., states, countries) • Bring equipment (e.g., eye tracker) • Natural observations
  • 7. Where to test LABORATORY • Controlled environment • All participants have the same experience • Record and communicate from control room • Observers watch from control room and provide additional probes (via moderator) in real time • • Incorporate physiological measures (e.g., eye tracking, EDA) No travel costs @forsmarshgroup @romanocog 7 REMOTE • Participants in their natural environments (e.g., home, work) • Use video chat (moderated sessions) or online programs (unmoderated) IN THE FIELD • Participants tend to be more comfortable in their natural environments • Recruit hard-to-reach populations (e.g., children, doctors) • Conduct many sessions quickly • Moderator travels to various locations • Recruit participants in many locations (e.g., states, countries) • Bring equipment (e.g., eye tracker) • Natural observations
  • 8. How to test ONE-ON-ONE SESSIONS • • • In-depth feedback from each participant FOCUS GROUPS • No group think Can allow participants to take their own route and explore freely Participants may be more comfortable with others • Interview many people quickly • Opinions collide • No interference • Peer review • Remote in participant’s environment • Qualitative • Flexible scheduling • Qualitative and Quantitative @forsmarshgroup @romanocog 8 SURVEYS • Representative • Large sample sizes • Collect a lot of data quickly • No interviewer bias • No scheduling sessions • Quantitative analysis
  • 10. What to measure EXPLICIT OBSERVATIONAL + Post-task satisfaction questionnaires + In-session difficulty ratings + Verbal responses + Moderator follow up + Real-time +/- dial + Ethnography + Time to complete task + Reaction time + Selection/click behavior + Ability to complete tasks + Accuracy IMPLICIT + Facial expression analysis + Eye tracking + Electrodermal activity (EDA) + Behavioral analysis + Linguistic analysis of verbalizations + Implicit associations + Pupil dilation @forsmarshgroup @romanocog 10
  • 11. Case 1 • Problems: What do users want? Does the new design work? • Methods: Focus groups, one-on-one interviews, in-lab usability testing with eye tracking @forsmarshgroup @romanocog 1. Participant repeatedly fixated the upper right hand corner. Participant said that he/she was looking for a search tool on the page. The search tool was in a disappearing banner on the page. 2. Participants had similar fixation counts across bottom links, indicating uncertainty of where to click to get started. 11
  • 12. Case 1 • Problems: What do users want? Does the new design work? • Methods: Focus groups, one-on-one interviews, in-lab usability testing with eye tracking @forsmarshgroup @romanocog 1. Participant repeatedly fixated the upper right hand corner. Participant said that he/she was looking for a search tool on the page. The search tool was in a disappearing banner on the page. 2. Participants had similar fixation counts across bottom links, indicating uncertainty of where to click to get started. 12
  • 13. Case 2 • Problem: What parts of the form do people actually read? • Method: In-lab usability testing with eye tracking @forsmarshgroup @romanocog 13
  • 14. Case 2 Participants did not read the instructions in their entirety (page 1: left; page 3: right); rather, they skimmed and then moved on to the form where they needed to enter information. 120 Time (seconds) 100 80 60 40 20 Aggregate fixation count heat map across all participants, Page 1. Participants looked at ‘Purpose of Form’ section the most often. @forsmarshgroup @romanocog 14 0 Page 1 Page 2 Page 3 Length of time spent on each page of the instructions before working on form.
  • 15. Case 3 Prototype Grid @forsmarshgroup @romanocog 15 Old Grid
  • 16. Case 4 Gaze Plot: After getting an error message, the participant had to search all over the screen to find the missing field. Smartphone “How do I advance to the next screen?” “It seems like it's stuck on the screen.” @forsmarshgroup @romanocog 16 Tablet
  • 17. Many ways to evaluate the UX • • • • Surveys Focus groups In-person one-on-one with eye tracking Analytics @forsmarshgroup @romanocog 17
  • 18. Obstacles to UX Testing • There is no time. – Start early in development process – One morning a month with 3 users – Krug – 12 people in 3 days – Anderson Reimer – 12 people in 2 days – Lebson & Romano Bergstrom • Can’t find representative users – Everyone is important – Travel – Remote testing • We don’t have a lab – Test anywhere @forsmarshgroup @romanocog 18
  • 19. Thank you! • • • Twitter: @forsmarshgroup LinkedIn: http://www.linkedin.com/company/fors-marsh-group Blog: www.forsmarshgroup.com/index.php/blog Jennifer Romano Bergstrom @romanocog jbergstrom@forsmarshgroup.com NOVA UX