SlideShare a Scribd company logo
1 of 26
Need a little usability?
What you can learn from usability testing
Whitney Quesenbery
Center for Civic Design
2
What questions do you
want to answer with a
usability test?
3
Usability testing answers questions, so we have to
understand the question we are asking.
If we want to know:
Which design helps users complete tasks more
effectively?
We are asking a question that is
Comparative, quantitative (errors, time)
(and why they made those mistakes)
If we want to know:
Do voters understand how to mark their ballot?
We are asking a question that is abut
Mental models, variability, qualitative insights
(and how prevalent each model is)
4
Helpful models for thinking about usability
A’s Ability, aptitude, attitude
(what users bring to the interaction)
E’s Efficient, effective, engaging, error-tolerant,
easy to learn
(dimensions of usability)
Use Useful, usable (operable), desirable
(context of use)
5
The As help you think about diverse participants
Attitude
motivation, emotion, risk tolerance, persistence, optimism, tolerance for
frustration
Aptitude
current knowledge, ability to make inferences or innovate solutions,
expertise, habits
Ability
physical and
cognitive attributes
Source: Dana Chisnell "Character Creator" http://www.slideshare.net/danachisnell/character-creator
6
Different users – and different contexts of use – can
change user needs
7
5Es help you plan usability tests
Usability Goal Types of usability testing needed
Evaluate tasks for how accurately they were
completed and how often they produce errors
Effective
Error-tolerant
Easy to learn Control how much instruction is given to test
participants, or recruit participants with different
levels of knowledge
Construct task scenarios to create situations
with potential problems
Engaging User satisfaction surveys to gauge acceptance
review logs for „time on site‟
Efficient Time tasks with realistic tasks and working
versions of the software
8
The recipe for planning a usability test
Location and context: formal informal
Recruiting: defined opportunistic
Activities: instructed tasks free tasks
Questions: structured unstructured
Data collection: observation task/data only
Results quantitative qualitative
9
A few examples of questions
and how we answered them
These examples all come from work on
elections and civic design..but the
principles apply to any project.
10
Comparing two designs
Can a design improve user performance?
Plain vs. traditional language
About this project
 45 participants
 3 lab setting, 8 days
 2 moderators
 Instructions for voting
 Careful observation
 Satisfaction survey
We learned about
 Voter accuracy
 Order effects
 Voter preference
Results
 Statistical analysis
 Participant preference
11
Comparing two designs
Can a minor difference have a major impact?
Left aligned or right aligned
names
About this project
 100 participants
 2 locations, 1 day
 4 moderators + 6 others
 Instructions for voting
 Minimal observation
We learned about
 Voter preference
 Time to vote
 Number of errors
Results
 Statistical analysis of marked
ballots
12
Testing a new design
What problems might voters have with a new ballot design?
About this project
 1 of 5 similar tests
 10 participants
 1 locations, 1 day
 2 moderators + 1 others
 Self-defined tasks
 Observation
 Post-task questionnaire
We learned about
 Navigation on the ballot
 Instructions
Results
 Found pattern of errors for
one interaction
 Time on task
13
Exploring new ideas
Can we extend a design to new audiences?
http://anywhereballot.org
http://civicdesigning.org/featured-story/rapid-responsive-radical-the-anywhere-ballot-is-born/
About this project
 Research project
 Paper prototyping
 Low literacy
 We learned about
 What they understsand
 Navigation
 Instructions
 Interactions
 Results
 Simplified everything for a
clear, universal design
14
Answers quickly and inexpensively
Can we find participants "where they are?"
http://anywhereballot.org
http://civicdesigning.org/featured-story/rapid-responsive-radical-the-anywhere-ballot-is-born/
About this project
 13 participants
 Observed and interviewed as
they filled in a new voter
registration form
 We learned about
 How people filled in the form
 Results
 Improved voter registration
forms
This project and poster were done by Jenny Greeve, the Design Fellow at the
Washington Department of Elections
15
Learning about comprehension
Can voters explain and act on that understanding?
About this project
 20 participants
 4 locations, 1 day
 8 moderators
 Self-defined tasks
 Minimal observation
We learned about
 Navigation on the ballot
 Instructions
 Mental models of counting
 Whether there‟s a problem
Results
 Qualitative insights
16
Gauging opinion
Which accessible voting system do voters prefer?
About this project
 496 surveys; multi-disabilit
 11 systems
 Multiday “voting fair”
They learned about
 Voter assessment of usability
 Voter comments about
accessibility
 Differences by disability
Results
 Statistical analysis survey
results
 Qualitative analysis of
comments
This was a project of the Oregon Department of Elections
17
Planning a
usability program
18
Look for a balance of methods by approach and
context of use
19
Early in
Project
Many Users
Few Users
Comparative
benchmarking
Summative testing
Card sorting
Satisfaction surveys
Market research
Iterative evaluation of
prototypes
Evaluation
of Designs
Ethnography
Use the right tool at the right time
Stakeholder
interviews
User observation
Walkthroughs
Expert
Reviews
A-B Testing
20
Look for the 'cadence' of what kind of usability work
you do at different times
Seeing the Elephant: Defragmenting User Research by Lou Rosenfeld
http://alistapart.com/article/seeing-the-elephant-defragmenting-user-research
21
Things to watch out for when
planning a usability test
22
Recruiting for diversity
Gender
Race/ethnicity
Income/education
Disability
23
Treating participants with respect
Language and cultural issues
Managing context
Participant protections and informed consent
24
Avoiding bias
Leading questions
Closed vs. open questions
Sampling bias
Moderator influence
Observer (or moderator reactions)
25
Whitney Quesenbery
whitneyq@wqusability.com
www.wqusability.com
@whitneyq
Center for Civic Design
whitneyq@centerforcivicdesign
centerforcivicdesign.org
@ChadButterfly
26
Storytelling for User Experience
with Kevin Brooks
Global UX
with Daniel Szuc
A Web for Everyone
with Sarah Horton
http://rosenfeldmedia.com/books/a-web-for-everyone/

More Related Content

What's hot

Aula 17 criatividade
Aula 17   criatividadeAula 17   criatividade
Aula 17 criatividadeLuiz Siles
 
Revisão assistente administrativo
Revisão   assistente administrativoRevisão   assistente administrativo
Revisão assistente administrativoProfe Raul
 
Design Sprint: metodologia na prática
Design Sprint: metodologia na práticaDesign Sprint: metodologia na prática
Design Sprint: metodologia na práticaMatina Moreira
 
Processo Unificado(RUP)
Processo Unificado(RUP)Processo Unificado(RUP)
Processo Unificado(RUP)elliando dias
 
Grokking Techtalk #45: First Principles Thinking
Grokking Techtalk #45: First Principles ThinkingGrokking Techtalk #45: First Principles Thinking
Grokking Techtalk #45: First Principles ThinkingGrokking VN
 
A importância da arquitetura de software
A importância da arquitetura de softwareA importância da arquitetura de software
A importância da arquitetura de softwareAdriano Tavares
 
Elicitacion
ElicitacionElicitacion
Elicitacionaruedaj
 
Design thinking - Aula 1 MFN 2014 UFRJ/FIRJAN - Bizzie
Design thinking - Aula 1 MFN 2014 UFRJ/FIRJAN - BizzieDesign thinking - Aula 1 MFN 2014 UFRJ/FIRJAN - Bizzie
Design thinking - Aula 1 MFN 2014 UFRJ/FIRJAN - BizzieRafaella Cavalca
 
Design Thinking | Preparação de pesquisa
Design Thinking | Preparação de pesquisaDesign Thinking | Preparação de pesquisa
Design Thinking | Preparação de pesquisaColaborativismo
 
Requerimientos de-cliente-y-desarrollador
Requerimientos de-cliente-y-desarrolladorRequerimientos de-cliente-y-desarrollador
Requerimientos de-cliente-y-desarrolladorAnibal Ulibarri
 
Análise e Projeto de Sistemas com UML e Java
Análise e Projeto de Sistemas com UML e JavaAnálise e Projeto de Sistemas com UML e Java
Análise e Projeto de Sistemas com UML e Javaarmeniocardoso
 
Inovação
InovaçãoInovação
Inovaçãolaikboy
 
Design Thinking
Design Thinking Design Thinking
Design Thinking Provectus
 

What's hot (20)

Aula 17 criatividade
Aula 17   criatividadeAula 17   criatividade
Aula 17 criatividade
 
engenharia-de-requisitos
engenharia-de-requisitosengenharia-de-requisitos
engenharia-de-requisitos
 
9.diseño de la arquitectura
9.diseño de la arquitectura9.diseño de la arquitectura
9.diseño de la arquitectura
 
Design Sprint
Design SprintDesign Sprint
Design Sprint
 
Revisão assistente administrativo
Revisão   assistente administrativoRevisão   assistente administrativo
Revisão assistente administrativo
 
Apresentação em Gestão de Stakeholders
Apresentação em Gestão de StakeholdersApresentação em Gestão de Stakeholders
Apresentação em Gestão de Stakeholders
 
Design Sprint: metodologia na prática
Design Sprint: metodologia na práticaDesign Sprint: metodologia na prática
Design Sprint: metodologia na prática
 
Taller Agile Inception Deck
Taller Agile Inception DeckTaller Agile Inception Deck
Taller Agile Inception Deck
 
A Startup Enxuta
A Startup EnxutaA Startup Enxuta
A Startup Enxuta
 
Processo Unificado(RUP)
Processo Unificado(RUP)Processo Unificado(RUP)
Processo Unificado(RUP)
 
Grokking Techtalk #45: First Principles Thinking
Grokking Techtalk #45: First Principles ThinkingGrokking Techtalk #45: First Principles Thinking
Grokking Techtalk #45: First Principles Thinking
 
A importância da arquitetura de software
A importância da arquitetura de softwareA importância da arquitetura de software
A importância da arquitetura de software
 
Elicitacion
ElicitacionElicitacion
Elicitacion
 
Design thinking - Aula 1 MFN 2014 UFRJ/FIRJAN - Bizzie
Design thinking - Aula 1 MFN 2014 UFRJ/FIRJAN - BizzieDesign thinking - Aula 1 MFN 2014 UFRJ/FIRJAN - Bizzie
Design thinking - Aula 1 MFN 2014 UFRJ/FIRJAN - Bizzie
 
Design Thinking | Preparação de pesquisa
Design Thinking | Preparação de pesquisaDesign Thinking | Preparação de pesquisa
Design Thinking | Preparação de pesquisa
 
Requerimientos de-cliente-y-desarrollador
Requerimientos de-cliente-y-desarrolladorRequerimientos de-cliente-y-desarrollador
Requerimientos de-cliente-y-desarrollador
 
Análise e Projeto de Sistemas com UML e Java
Análise e Projeto de Sistemas com UML e JavaAnálise e Projeto de Sistemas com UML e Java
Análise e Projeto de Sistemas com UML e Java
 
Inovação
InovaçãoInovação
Inovação
 
Design Thinking
Design Thinking Design Thinking
Design Thinking
 
Aula 05 - Stakeholders
Aula 05 - StakeholdersAula 05 - Stakeholders
Aula 05 - Stakeholders
 

Similar to What you can learn from usability testing

Usability Testing: Making it fast, good, and cheap
Usability Testing: Making it fast, good, and cheapUsability Testing: Making it fast, good, and cheap
Usability Testing: Making it fast, good, and cheapWhitney Quesenbery
 
Experience Research Best Practices - UX Meet Up Boston 2013 - Dan Berlin
Experience Research Best Practices - UX Meet Up Boston 2013 - Dan BerlinExperience Research Best Practices - UX Meet Up Boston 2013 - Dan Berlin
Experience Research Best Practices - UX Meet Up Boston 2013 - Dan BerlinMad*Pow
 
Experience Research Best Practices
Experience Research Best PracticesExperience Research Best Practices
Experience Research Best PracticesDan Berlin
 
Determining Requirements In System Analysis And Dsign
Determining Requirements In System Analysis And DsignDetermining Requirements In System Analysis And Dsign
Determining Requirements In System Analysis And DsignAsaduzzaman Kanok
 
Depending on our users
Depending on our usersDepending on our users
Depending on our userserinemeyer
 
How NOT to Aggregrate Polling Data
How NOT to Aggregrate Polling DataHow NOT to Aggregrate Polling Data
How NOT to Aggregrate Polling DataDataCards
 
Straight from the user's mouth: Usability testing UNC's LibGuides
Straight from the user's mouth: Usability testing UNC's LibGuidesStraight from the user's mouth: Usability testing UNC's LibGuides
Straight from the user's mouth: Usability testing UNC's LibGuidesSarah Joy Arnold
 
Depending on our users final
Depending on our users finalDepending on our users final
Depending on our users finalerinemeyer
 
Brightfind world usability day 2016 full deck final
Brightfind world usability day 2016   full deck finalBrightfind world usability day 2016   full deck final
Brightfind world usability day 2016 full deck finalBrightfind
 
Ces 2013 towards a cdn definition of evaluation
Ces 2013   towards a cdn definition of evaluationCes 2013   towards a cdn definition of evaluation
Ces 2013 towards a cdn definition of evaluationCesToronto
 
Visualization for public policy (2016)
Visualization for public policy (2016)Visualization for public policy (2016)
Visualization for public policy (2016)Arthi Krishnaswami
 
An Introduction to the World of User Research
An Introduction to the World of User ResearchAn Introduction to the World of User Research
An Introduction to the World of User ResearchMethods
 
PROGRAM DEVELOPMENT- nudrat mufti
PROGRAM DEVELOPMENT- nudrat muftiPROGRAM DEVELOPMENT- nudrat mufti
PROGRAM DEVELOPMENT- nudrat muftiNUDRAT MUFTI
 
Presentation Survey Research Sana.pptx
Presentation Survey Research Sana.pptxPresentation Survey Research Sana.pptx
Presentation Survey Research Sana.pptxZainabRamzan7
 
Conducting a survey
Conducting a surveyConducting a survey
Conducting a surveyGMOORE2013
 
Explaining recommendations: design implications and lessons learned
Explaining recommendations: design implications and lessons learnedExplaining recommendations: design implications and lessons learned
Explaining recommendations: design implications and lessons learnedKatrien Verbert
 
Preference and Desirability Testing: Measuring Emotional Response to Guide De...
Preference and Desirability Testing: Measuring Emotional Response to Guide De...Preference and Desirability Testing: Measuring Emotional Response to Guide De...
Preference and Desirability Testing: Measuring Emotional Response to Guide De...Paul Doncaster
 

Similar to What you can learn from usability testing (20)

Usability Testing: Making it fast, good, and cheap
Usability Testing: Making it fast, good, and cheapUsability Testing: Making it fast, good, and cheap
Usability Testing: Making it fast, good, and cheap
 
Experience Research Best Practices - UX Meet Up Boston 2013 - Dan Berlin
Experience Research Best Practices - UX Meet Up Boston 2013 - Dan BerlinExperience Research Best Practices - UX Meet Up Boston 2013 - Dan Berlin
Experience Research Best Practices - UX Meet Up Boston 2013 - Dan Berlin
 
Experience Research Best Practices
Experience Research Best PracticesExperience Research Best Practices
Experience Research Best Practices
 
Determining Requirements In System Analysis And Dsign
Determining Requirements In System Analysis And DsignDetermining Requirements In System Analysis And Dsign
Determining Requirements In System Analysis And Dsign
 
Better UX Surveys part 1
Better UX Surveys part 1Better UX Surveys part 1
Better UX Surveys part 1
 
Depending on our users
Depending on our usersDepending on our users
Depending on our users
 
How NOT to Aggregrate Polling Data
How NOT to Aggregrate Polling DataHow NOT to Aggregrate Polling Data
How NOT to Aggregrate Polling Data
 
Role of research in ux
Role of research in uxRole of research in ux
Role of research in ux
 
Straight from the user's mouth: Usability testing UNC's LibGuides
Straight from the user's mouth: Usability testing UNC's LibGuidesStraight from the user's mouth: Usability testing UNC's LibGuides
Straight from the user's mouth: Usability testing UNC's LibGuides
 
Depending on our users final
Depending on our users finalDepending on our users final
Depending on our users final
 
Brightfind world usability day 2016 full deck final
Brightfind world usability day 2016   full deck finalBrightfind world usability day 2016   full deck final
Brightfind world usability day 2016 full deck final
 
Ces 2013 towards a cdn definition of evaluation
Ces 2013   towards a cdn definition of evaluationCes 2013   towards a cdn definition of evaluation
Ces 2013 towards a cdn definition of evaluation
 
Visualization for public policy (2016)
Visualization for public policy (2016)Visualization for public policy (2016)
Visualization for public policy (2016)
 
An Introduction to the World of User Research
An Introduction to the World of User ResearchAn Introduction to the World of User Research
An Introduction to the World of User Research
 
PROGRAM DEVELOPMENT- nudrat mufti
PROGRAM DEVELOPMENT- nudrat muftiPROGRAM DEVELOPMENT- nudrat mufti
PROGRAM DEVELOPMENT- nudrat mufti
 
Presentation Survey Research Sana.pptx
Presentation Survey Research Sana.pptxPresentation Survey Research Sana.pptx
Presentation Survey Research Sana.pptx
 
Conducting a survey
Conducting a surveyConducting a survey
Conducting a survey
 
Explaining recommendations: design implications and lessons learned
Explaining recommendations: design implications and lessons learnedExplaining recommendations: design implications and lessons learned
Explaining recommendations: design implications and lessons learned
 
User centered expert reviews
User centered expert reviewsUser centered expert reviews
User centered expert reviews
 
Preference and Desirability Testing: Measuring Emotional Response to Guide De...
Preference and Desirability Testing: Measuring Emotional Response to Guide De...Preference and Desirability Testing: Measuring Emotional Response to Guide De...
Preference and Desirability Testing: Measuring Emotional Response to Guide De...
 

More from Whitney Quesenbery

Designing for difference: Are you failing at the most important design challenge
Designing for difference: Are you failing at the most important design challengeDesigning for difference: Are you failing at the most important design challenge
Designing for difference: Are you failing at the most important design challengeWhitney Quesenbery
 
Storytelling: Finding the insights in research data
Storytelling: Finding the insights in research data Storytelling: Finding the insights in research data
Storytelling: Finding the insights in research data Whitney Quesenbery
 
Storytelling makes research data come to life
Storytelling makes research data come to lifeStorytelling makes research data come to life
Storytelling makes research data come to lifeWhitney Quesenbery
 
From barrier-free to delightful
From barrier-free to delightfulFrom barrier-free to delightful
From barrier-free to delightfulWhitney Quesenbery
 
Plain language is accessibility for content
Plain language is accessibility for contentPlain language is accessibility for content
Plain language is accessibility for contentWhitney Quesenbery
 
From barrier-free to delightful
From barrier-free to delightfulFrom barrier-free to delightful
From barrier-free to delightfulWhitney Quesenbery
 
Accessible UX: Beyond the checklist to great experiences
Accessible UX: Beyond the checklist to great experiencesAccessible UX: Beyond the checklist to great experiences
Accessible UX: Beyond the checklist to great experiencesWhitney Quesenbery
 
Usability testing for accessible UX
Usability testing for accessible UXUsability testing for accessible UX
Usability testing for accessible UXWhitney Quesenbery
 
Security insights & issues for poll workers
Security insights & issues for poll workersSecurity insights & issues for poll workers
Security insights & issues for poll workersWhitney Quesenbery
 
Plain language (accessibility for content)
Plain language (accessibility for content)Plain language (accessibility for content)
Plain language (accessibility for content)Whitney Quesenbery
 
The power of stories: creating empathy and connection
The power of stories: creating empathy and connectionThe power of stories: creating empathy and connection
The power of stories: creating empathy and connectionWhitney Quesenbery
 
Accessibility as Innovation: Creating accessible user experiences
Accessibility as Innovation: Creating accessible user experiencesAccessibility as Innovation: Creating accessible user experiences
Accessibility as Innovation: Creating accessible user experiencesWhitney Quesenbery
 
Using Personas and Stories Effectively - Workshop for TriUXPA
Using Personas and Stories Effectively - Workshop for TriUXPAUsing Personas and Stories Effectively - Workshop for TriUXPA
Using Personas and Stories Effectively - Workshop for TriUXPAWhitney Quesenbery
 

More from Whitney Quesenbery (20)

Designing for difference: Are you failing at the most important design challenge
Designing for difference: Are you failing at the most important design challengeDesigning for difference: Are you failing at the most important design challenge
Designing for difference: Are you failing at the most important design challenge
 
Storytelling: Finding the insights in research data
Storytelling: Finding the insights in research data Storytelling: Finding the insights in research data
Storytelling: Finding the insights in research data
 
Storytelling makes research data come to life
Storytelling makes research data come to lifeStorytelling makes research data come to life
Storytelling makes research data come to life
 
From barrier-free to delightful
From barrier-free to delightfulFrom barrier-free to delightful
From barrier-free to delightful
 
Accessibility Basics
Accessibility BasicsAccessibility Basics
Accessibility Basics
 
Writing Great Alt Text
Writing Great Alt TextWriting Great Alt Text
Writing Great Alt Text
 
Plain language is accessibility for content
Plain language is accessibility for contentPlain language is accessibility for content
Plain language is accessibility for content
 
From barrier-free to delightful
From barrier-free to delightfulFrom barrier-free to delightful
From barrier-free to delightful
 
Accessible UX: Beyond the checklist to great experiences
Accessible UX: Beyond the checklist to great experiencesAccessible UX: Beyond the checklist to great experiences
Accessible UX: Beyond the checklist to great experiences
 
Usability testing for accessible UX
Usability testing for accessible UXUsability testing for accessible UX
Usability testing for accessible UX
 
Accessibility as innovation
Accessibility as innovationAccessibility as innovation
Accessibility as innovation
 
Security insights & issues for poll workers
Security insights & issues for poll workersSecurity insights & issues for poll workers
Security insights & issues for poll workers
 
Writing Great Alt Text
Writing Great Alt TextWriting Great Alt Text
Writing Great Alt Text
 
Don't Make Me Read
Don't Make Me ReadDon't Make Me Read
Don't Make Me Read
 
Plain language (accessibility for content)
Plain language (accessibility for content)Plain language (accessibility for content)
Plain language (accessibility for content)
 
The power of stories: creating empathy and connection
The power of stories: creating empathy and connectionThe power of stories: creating empathy and connection
The power of stories: creating empathy and connection
 
Personas for Accessible UX
Personas for Accessible UXPersonas for Accessible UX
Personas for Accessible UX
 
Accessibility as Innovation: Creating accessible user experiences
Accessibility as Innovation: Creating accessible user experiencesAccessibility as Innovation: Creating accessible user experiences
Accessibility as Innovation: Creating accessible user experiences
 
Usability meets accessibility
Usability meets accessibilityUsability meets accessibility
Usability meets accessibility
 
Using Personas and Stories Effectively - Workshop for TriUXPA
Using Personas and Stories Effectively - Workshop for TriUXPAUsing Personas and Stories Effectively - Workshop for TriUXPA
Using Personas and Stories Effectively - Workshop for TriUXPA
 

Recently uploaded

办理学位证(SFU证书)西蒙菲莎大学毕业证成绩单原版一比一
办理学位证(SFU证书)西蒙菲莎大学毕业证成绩单原版一比一办理学位证(SFU证书)西蒙菲莎大学毕业证成绩单原版一比一
办理学位证(SFU证书)西蒙菲莎大学毕业证成绩单原版一比一F dds
 
cda.pptx critical discourse analysis ppt
cda.pptx critical discourse analysis pptcda.pptx critical discourse analysis ppt
cda.pptx critical discourse analysis pptMaryamAfzal41
 
306MTAMount UCLA University Bachelor's Diploma in Social Media
306MTAMount UCLA University Bachelor's Diploma in Social Media306MTAMount UCLA University Bachelor's Diploma in Social Media
306MTAMount UCLA University Bachelor's Diploma in Social MediaD SSS
 
办理(UC毕业证书)查尔斯顿大学毕业证成绩单原版一比一
办理(UC毕业证书)查尔斯顿大学毕业证成绩单原版一比一办理(UC毕业证书)查尔斯顿大学毕业证成绩单原版一比一
办理(UC毕业证书)查尔斯顿大学毕业证成绩单原版一比一z xss
 
Passbook project document_april_21__.pdf
Passbook project document_april_21__.pdfPassbook project document_april_21__.pdf
Passbook project document_april_21__.pdfvaibhavkanaujia
 
(办理学位证)约克圣约翰大学毕业证,KCL成绩单原版一比一
(办理学位证)约克圣约翰大学毕业证,KCL成绩单原版一比一(办理学位证)约克圣约翰大学毕业证,KCL成绩单原版一比一
(办理学位证)约克圣约翰大学毕业证,KCL成绩单原版一比一D SSS
 
Call Girls Aslali 7397865700 Ridhima Hire Me Full Night
Call Girls Aslali 7397865700 Ridhima Hire Me Full NightCall Girls Aslali 7397865700 Ridhima Hire Me Full Night
Call Girls Aslali 7397865700 Ridhima Hire Me Full Nightssuser7cb4ff
 
定制(CQU文凭证书)中央昆士兰大学毕业证成绩单原版一比一
定制(CQU文凭证书)中央昆士兰大学毕业证成绩单原版一比一定制(CQU文凭证书)中央昆士兰大学毕业证成绩单原版一比一
定制(CQU文凭证书)中央昆士兰大学毕业证成绩单原版一比一Fi ss
 
2024新版美国旧金山州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
2024新版美国旧金山州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree2024新版美国旧金山州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
2024新版美国旧金山州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degreeyuu sss
 
办理学位证加州州立大学洛杉矶分校毕业证成绩单原版一比一
办理学位证加州州立大学洛杉矶分校毕业证成绩单原版一比一办理学位证加州州立大学洛杉矶分校毕业证成绩单原版一比一
办理学位证加州州立大学洛杉矶分校毕业证成绩单原版一比一Fi L
 
Unveiling the Future: Columbus, Ohio Condominiums Through the Lens of 3D Arch...
Unveiling the Future: Columbus, Ohio Condominiums Through the Lens of 3D Arch...Unveiling the Future: Columbus, Ohio Condominiums Through the Lens of 3D Arch...
Unveiling the Future: Columbus, Ohio Condominiums Through the Lens of 3D Arch...Yantram Animation Studio Corporation
 
办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书
办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书
办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书zdzoqco
 
Business research proposal mcdo.pptxBusiness research proposal mcdo.pptxBusin...
Business research proposal mcdo.pptxBusiness research proposal mcdo.pptxBusin...Business research proposal mcdo.pptxBusiness research proposal mcdo.pptxBusin...
Business research proposal mcdo.pptxBusiness research proposal mcdo.pptxBusin...mrchrns005
 
FiveHypotheses_UIDMasterclass_18April2024.pdf
FiveHypotheses_UIDMasterclass_18April2024.pdfFiveHypotheses_UIDMasterclass_18April2024.pdf
FiveHypotheses_UIDMasterclass_18April2024.pdfShivakumar Viswanathan
 
Dubai Calls Girl Tapes O525547819 Real Tapes Escort Services Dubai
Dubai Calls Girl Tapes O525547819 Real Tapes Escort Services DubaiDubai Calls Girl Tapes O525547819 Real Tapes Escort Services Dubai
Dubai Calls Girl Tapes O525547819 Real Tapes Escort Services Dubaikojalkojal131
 
PORTFOLIO DE ARQUITECTURA CRISTOBAL HERAUD 2024
PORTFOLIO DE ARQUITECTURA CRISTOBAL HERAUD 2024PORTFOLIO DE ARQUITECTURA CRISTOBAL HERAUD 2024
PORTFOLIO DE ARQUITECTURA CRISTOBAL HERAUD 2024CristobalHeraud
 
CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10
CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10
CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10uasjlagroup
 
原版美国亚利桑那州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
原版美国亚利桑那州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree原版美国亚利桑那州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
原版美国亚利桑那州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degreeyuu sss
 
How to Empower the future of UX Design with Gen AI
How to Empower the future of UX Design with Gen AIHow to Empower the future of UX Design with Gen AI
How to Empower the future of UX Design with Gen AIyuj
 
Top 10 Modern Web Design Trends for 2025
Top 10 Modern Web Design Trends for 2025Top 10 Modern Web Design Trends for 2025
Top 10 Modern Web Design Trends for 2025Rndexperts
 

Recently uploaded (20)

办理学位证(SFU证书)西蒙菲莎大学毕业证成绩单原版一比一
办理学位证(SFU证书)西蒙菲莎大学毕业证成绩单原版一比一办理学位证(SFU证书)西蒙菲莎大学毕业证成绩单原版一比一
办理学位证(SFU证书)西蒙菲莎大学毕业证成绩单原版一比一
 
cda.pptx critical discourse analysis ppt
cda.pptx critical discourse analysis pptcda.pptx critical discourse analysis ppt
cda.pptx critical discourse analysis ppt
 
306MTAMount UCLA University Bachelor's Diploma in Social Media
306MTAMount UCLA University Bachelor's Diploma in Social Media306MTAMount UCLA University Bachelor's Diploma in Social Media
306MTAMount UCLA University Bachelor's Diploma in Social Media
 
办理(UC毕业证书)查尔斯顿大学毕业证成绩单原版一比一
办理(UC毕业证书)查尔斯顿大学毕业证成绩单原版一比一办理(UC毕业证书)查尔斯顿大学毕业证成绩单原版一比一
办理(UC毕业证书)查尔斯顿大学毕业证成绩单原版一比一
 
Passbook project document_april_21__.pdf
Passbook project document_april_21__.pdfPassbook project document_april_21__.pdf
Passbook project document_april_21__.pdf
 
(办理学位证)约克圣约翰大学毕业证,KCL成绩单原版一比一
(办理学位证)约克圣约翰大学毕业证,KCL成绩单原版一比一(办理学位证)约克圣约翰大学毕业证,KCL成绩单原版一比一
(办理学位证)约克圣约翰大学毕业证,KCL成绩单原版一比一
 
Call Girls Aslali 7397865700 Ridhima Hire Me Full Night
Call Girls Aslali 7397865700 Ridhima Hire Me Full NightCall Girls Aslali 7397865700 Ridhima Hire Me Full Night
Call Girls Aslali 7397865700 Ridhima Hire Me Full Night
 
定制(CQU文凭证书)中央昆士兰大学毕业证成绩单原版一比一
定制(CQU文凭证书)中央昆士兰大学毕业证成绩单原版一比一定制(CQU文凭证书)中央昆士兰大学毕业证成绩单原版一比一
定制(CQU文凭证书)中央昆士兰大学毕业证成绩单原版一比一
 
2024新版美国旧金山州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
2024新版美国旧金山州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree2024新版美国旧金山州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
2024新版美国旧金山州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
 
办理学位证加州州立大学洛杉矶分校毕业证成绩单原版一比一
办理学位证加州州立大学洛杉矶分校毕业证成绩单原版一比一办理学位证加州州立大学洛杉矶分校毕业证成绩单原版一比一
办理学位证加州州立大学洛杉矶分校毕业证成绩单原版一比一
 
Unveiling the Future: Columbus, Ohio Condominiums Through the Lens of 3D Arch...
Unveiling the Future: Columbus, Ohio Condominiums Through the Lens of 3D Arch...Unveiling the Future: Columbus, Ohio Condominiums Through the Lens of 3D Arch...
Unveiling the Future: Columbus, Ohio Condominiums Through the Lens of 3D Arch...
 
办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书
办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书
办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书
 
Business research proposal mcdo.pptxBusiness research proposal mcdo.pptxBusin...
Business research proposal mcdo.pptxBusiness research proposal mcdo.pptxBusin...Business research proposal mcdo.pptxBusiness research proposal mcdo.pptxBusin...
Business research proposal mcdo.pptxBusiness research proposal mcdo.pptxBusin...
 
FiveHypotheses_UIDMasterclass_18April2024.pdf
FiveHypotheses_UIDMasterclass_18April2024.pdfFiveHypotheses_UIDMasterclass_18April2024.pdf
FiveHypotheses_UIDMasterclass_18April2024.pdf
 
Dubai Calls Girl Tapes O525547819 Real Tapes Escort Services Dubai
Dubai Calls Girl Tapes O525547819 Real Tapes Escort Services DubaiDubai Calls Girl Tapes O525547819 Real Tapes Escort Services Dubai
Dubai Calls Girl Tapes O525547819 Real Tapes Escort Services Dubai
 
PORTFOLIO DE ARQUITECTURA CRISTOBAL HERAUD 2024
PORTFOLIO DE ARQUITECTURA CRISTOBAL HERAUD 2024PORTFOLIO DE ARQUITECTURA CRISTOBAL HERAUD 2024
PORTFOLIO DE ARQUITECTURA CRISTOBAL HERAUD 2024
 
CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10
CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10
CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10
 
原版美国亚利桑那州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
原版美国亚利桑那州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree原版美国亚利桑那州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
原版美国亚利桑那州立大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
 
How to Empower the future of UX Design with Gen AI
How to Empower the future of UX Design with Gen AIHow to Empower the future of UX Design with Gen AI
How to Empower the future of UX Design with Gen AI
 
Top 10 Modern Web Design Trends for 2025
Top 10 Modern Web Design Trends for 2025Top 10 Modern Web Design Trends for 2025
Top 10 Modern Web Design Trends for 2025
 

What you can learn from usability testing

  • 1. Need a little usability? What you can learn from usability testing Whitney Quesenbery Center for Civic Design
  • 2. 2 What questions do you want to answer with a usability test?
  • 3. 3 Usability testing answers questions, so we have to understand the question we are asking. If we want to know: Which design helps users complete tasks more effectively? We are asking a question that is Comparative, quantitative (errors, time) (and why they made those mistakes) If we want to know: Do voters understand how to mark their ballot? We are asking a question that is abut Mental models, variability, qualitative insights (and how prevalent each model is)
  • 4. 4 Helpful models for thinking about usability A’s Ability, aptitude, attitude (what users bring to the interaction) E’s Efficient, effective, engaging, error-tolerant, easy to learn (dimensions of usability) Use Useful, usable (operable), desirable (context of use)
  • 5. 5 The As help you think about diverse participants Attitude motivation, emotion, risk tolerance, persistence, optimism, tolerance for frustration Aptitude current knowledge, ability to make inferences or innovate solutions, expertise, habits Ability physical and cognitive attributes Source: Dana Chisnell "Character Creator" http://www.slideshare.net/danachisnell/character-creator
  • 6. 6 Different users – and different contexts of use – can change user needs
  • 7. 7 5Es help you plan usability tests Usability Goal Types of usability testing needed Evaluate tasks for how accurately they were completed and how often they produce errors Effective Error-tolerant Easy to learn Control how much instruction is given to test participants, or recruit participants with different levels of knowledge Construct task scenarios to create situations with potential problems Engaging User satisfaction surveys to gauge acceptance review logs for „time on site‟ Efficient Time tasks with realistic tasks and working versions of the software
  • 8. 8 The recipe for planning a usability test Location and context: formal informal Recruiting: defined opportunistic Activities: instructed tasks free tasks Questions: structured unstructured Data collection: observation task/data only Results quantitative qualitative
  • 9. 9 A few examples of questions and how we answered them These examples all come from work on elections and civic design..but the principles apply to any project.
  • 10. 10 Comparing two designs Can a design improve user performance? Plain vs. traditional language About this project  45 participants  3 lab setting, 8 days  2 moderators  Instructions for voting  Careful observation  Satisfaction survey We learned about  Voter accuracy  Order effects  Voter preference Results  Statistical analysis  Participant preference
  • 11. 11 Comparing two designs Can a minor difference have a major impact? Left aligned or right aligned names About this project  100 participants  2 locations, 1 day  4 moderators + 6 others  Instructions for voting  Minimal observation We learned about  Voter preference  Time to vote  Number of errors Results  Statistical analysis of marked ballots
  • 12. 12 Testing a new design What problems might voters have with a new ballot design? About this project  1 of 5 similar tests  10 participants  1 locations, 1 day  2 moderators + 1 others  Self-defined tasks  Observation  Post-task questionnaire We learned about  Navigation on the ballot  Instructions Results  Found pattern of errors for one interaction  Time on task
  • 13. 13 Exploring new ideas Can we extend a design to new audiences? http://anywhereballot.org http://civicdesigning.org/featured-story/rapid-responsive-radical-the-anywhere-ballot-is-born/ About this project  Research project  Paper prototyping  Low literacy  We learned about  What they understsand  Navigation  Instructions  Interactions  Results  Simplified everything for a clear, universal design
  • 14. 14 Answers quickly and inexpensively Can we find participants "where they are?" http://anywhereballot.org http://civicdesigning.org/featured-story/rapid-responsive-radical-the-anywhere-ballot-is-born/ About this project  13 participants  Observed and interviewed as they filled in a new voter registration form  We learned about  How people filled in the form  Results  Improved voter registration forms This project and poster were done by Jenny Greeve, the Design Fellow at the Washington Department of Elections
  • 15. 15 Learning about comprehension Can voters explain and act on that understanding? About this project  20 participants  4 locations, 1 day  8 moderators  Self-defined tasks  Minimal observation We learned about  Navigation on the ballot  Instructions  Mental models of counting  Whether there‟s a problem Results  Qualitative insights
  • 16. 16 Gauging opinion Which accessible voting system do voters prefer? About this project  496 surveys; multi-disabilit  11 systems  Multiday “voting fair” They learned about  Voter assessment of usability  Voter comments about accessibility  Differences by disability Results  Statistical analysis survey results  Qualitative analysis of comments This was a project of the Oregon Department of Elections
  • 18. 18 Look for a balance of methods by approach and context of use
  • 19. 19 Early in Project Many Users Few Users Comparative benchmarking Summative testing Card sorting Satisfaction surveys Market research Iterative evaluation of prototypes Evaluation of Designs Ethnography Use the right tool at the right time Stakeholder interviews User observation Walkthroughs Expert Reviews A-B Testing
  • 20. 20 Look for the 'cadence' of what kind of usability work you do at different times Seeing the Elephant: Defragmenting User Research by Lou Rosenfeld http://alistapart.com/article/seeing-the-elephant-defragmenting-user-research
  • 21. 21 Things to watch out for when planning a usability test
  • 23. 23 Treating participants with respect Language and cultural issues Managing context Participant protections and informed consent
  • 24. 24 Avoiding bias Leading questions Closed vs. open questions Sampling bias Moderator influence Observer (or moderator reactions)
  • 25. 25 Whitney Quesenbery whitneyq@wqusability.com www.wqusability.com @whitneyq Center for Civic Design whitneyq@centerforcivicdesign centerforcivicdesign.org @ChadButterfly
  • 26. 26 Storytelling for User Experience with Kevin Brooks Global UX with Daniel Szuc A Web for Everyone with Sarah Horton http://rosenfeldmedia.com/books/a-web-for-everyone/

Editor's Notes

  1. A tool to help you focus on the attributes that matterAccount for what the user brings to the tableIdentify both center and outliers
  2. The 'Anywhere Ballot'Designed through iterative testing with low literacy participants
  3. On April 4th, 2005, citizens with disabilities from all over the state tested voting systems and completed surveys documenting their reactions to each system. A total of 496 surveys were completed.The survey results on the following page will be one of several factors used to help the Oregon Secretary of State decide which voting system or systems to purchase to give citizens with disabilities the opportunity to vote privately and independently. Survey was a modified SUS – 12 questionsI would like to use this voting system in an election. There were too many steps in using this voting system.I thought this voting system was easy to use. The instructions for this voting system were difficult to understand. The buttons or touch screen were easy to use. It was hard to move around the ballot with this system. I think that most people could learn to use this voting system very quickly.I found this voting system awkward or difficult to use. I felt very confident that my vote was cast correctly with this voting system. I would need help each time I used this voting system. The ballot text was easy to read or hear. Casting my vote was easy.