Need a little usability?

Need a little usability?What you can learn from usability testingWhitney QuesenberyDana ChisnellUsability in Civic Life

2What questions do you have?

3Usability testing answers questions, so we have tounderstand the question we are askingIf we want to know: Which ballot design helps voters be more accurate?We are asking a question that is Comparative, quantitative (errors, time) (and why they made those mistakes)If we want to know: Do voters understand how to mark their ballot?We are asking a question that is abut Mental models, variability, qualitative insights (and how prevalent each model is)

4Helpful models for thinking about usabilityA’s Ability, aptitude, attitude (what voters bring to the election)E’s Efficient, effective, engaging, error-tolerant, easy to learn (dimensions of usability)Use Useful, usable (operable), desirable (how we define success)

5The recipe for planning a usability testLocation and context: formal informalRecruiting: defined opportunisticActivities: instructed tasks free tasksQuestions: structured unstructuredData collection: observation task/data onlyResults quantitative qualitative

6A few examples

7Which instructions work better?Traditional ballot instructions vs. plain language version About this project  45 participants  3 lab setting, 8 days  2 moderators  Instructions for voting  Careful observation  Satisfaction survey We learned about  Voter accuracy  Order effects  Voter preference Results  Statistical analysis  Participant preference

8Which layout works better?Left aligned vs. right aligned names? About this project  100 participants  2 locations, 1 day  4 moderators + 6 others  Instructions for voting  Minimal observation We learned about  Voter preference  Time to vote  Number of errors Results  Statistical analysis of marked ballots

9What do voters do with unusual options?How do voters react to “double-vote” options? About this project  200 participants  5 locations, 1 day  12 moderators + 5 others  Self-defined tasks  Minimal observation We learned about  Voter awareness of party  Types of marks and errors  Need for voter ed. Results  Inspection of ballots for marking patterns

10What problems might voters have with a new ballot?What problems might voters have with a new ballot design? About this project  1 of 5 similar tests  10 participants  1 locations, 1 day  2 moderators + 1 others  Self-defined tasks  Observation  Post-task questionnaire We learned about  Navigation on the ballot  Instructions Results  Found pattern of errors for one interaction  Time on task

11What do voters think about different systems?Which accessible voting system do voters prefer? About this project  496 surveys; multi-disabilit  11 systems  Multiday “voting fair” They learned about  Voter assessment of usability  Voter comments about accessibility  Differences by disability Results  Statistical analysis survey results  Qualitative analysis of comments

12Do voters understand how their votes are counted?Can voters explain and act on that understanding? About this project  20 participants  4 locations, 1 day  8 moderators  Self-defined tasks  Minimal observation We learned about  Navigation on the ballot  Instructions  Mental models of counting  Whether there’s a problem Results  Qualitative insights

13Do voters understand a new voting system?What is their reaction to new procedures? About this project  Focus group  Mock election  1 moderators They learned about  Attitudes and reactions Results  Qualitative insights

14Things to watch out for whenplanning a usability test

15Treating participants with respectLanguage and cultural issuesManaging contextParticipant protections and informed consent

16Avoiding biasLeading questionsClosed vs. open questionsSampling biasModerator influenceObserver (or moderator reactions)

17Thoughts?

18Whitney Quesenberywhitney@wqusability.comDana Chisnelldana@usabilityworks.netField guides to ensuring voter intenthttp://tinyurl.com/ensuringvoterintentUsability in Civic Lifehttp://usabilityinciviclife.orgCivic Designhttp://civicdesigning.orgLEO Usability Testing Kithttp://www.usabilityinciviclife.org/voting/leo-testing-kit/

http://civicdesigning.org	229
http://tweetedtimes.com	2
http://paper.li	1

Need a little usability?

by Whitney Quesenbery

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 232

Statistics

Need a little usability? Presentation Transcript