Need a little usability?What you can learn from usability testingWhitney QuesenberyDana ChisnellUsability in Civic Life
2What questions do you have?
3Usability testing answers questions, so we have tounderstand the question we are askingIf we want to know:        Which b...
4Helpful models for thinking about usabilityA’s   Ability, aptitude, attitude      (what voters bring to the election)E’s ...
5The recipe for planning a usability testLocation and context:       formal     informalRecruiting:                defined...
6A few examples
7Which instructions work better?Traditional ballot instructions vs. plain language version                                ...
8Which layout works better?Left aligned vs. right aligned names?                                About this project        ...
9What do voters do with unusual options?How do voters react to “double-vote” options?                                About...
10What problems might voters have with a new ballot?What problems might voters have with a new ballot design?             ...
11What do voters think about different systems?Which accessible voting system do voters prefer?                           ...
12Do voters understand how their votes are counted?Can voters explain and act on that understanding?                      ...
13Do voters understand a new voting system?What is their reaction to new procedures?                            About this...
14Things to watch out for whenplanning a usability test
15Treating participants with respectLanguage and cultural issuesManaging contextParticipant protections and informed consent
16Avoiding biasLeading questionsClosed vs. open questionsSampling biasModerator influenceObserver (or moderator reactions)
17Thoughts?
18Whitney Quesenberywhitney@wqusability.comDana Chisnelldana@usabilityworks.netField guides to ensuring voter intenthttp:/...
Upcoming SlideShare
Loading in …5
×

Need a little usability?

2,621 views

Published on

Presentation on what you can learn from usability testing and projects in ballot and elections. Dana Chisnell and Whitney Quesenbery at EVN 2012

Published in: Technology, News & Politics
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,621
On SlideShare
0
From Embeds
0
Number of Embeds
279
Actions
Shares
0
Downloads
12
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide
  • On April 4th, 2005, citizens with disabilities from all over the state tested voting systems and completed surveys documenting their reactions to each system. A total of 496 surveys were completed.The survey results on the following page will be one of several factors used to help the Oregon Secretary of State decide which voting system or systems to purchase to give citizens with disabilities the opportunity to vote privately and independently. Survey was a modified SUS – 12 questionsI would like to use this voting system in an election. There were too many steps in using this voting system.I thought this voting system was easy to use. The instructions for this voting system were difficult to understand. The buttons or touch screen were easy to use. It was hard to move around the ballot with this system. I think that most people could learn to use this voting system very quickly.I found this voting system awkward or difficult to use. I felt very confident that my vote was cast correctly with this voting system. I would need help each time I used this voting system. The ballot text was easy to read or hear. Casting my vote was easy.
  • Need a little usability?

    1. 1. Need a little usability?What you can learn from usability testingWhitney QuesenberyDana ChisnellUsability in Civic Life
    2. 2. 2What questions do you have?
    3. 3. 3Usability testing answers questions, so we have tounderstand the question we are askingIf we want to know: Which ballot design helps voters be more accurate?We are asking a question that is Comparative, quantitative (errors, time) (and why they made those mistakes)If we want to know: Do voters understand how to mark their ballot?We are asking a question that is abut Mental models, variability, qualitative insights (and how prevalent each model is)
    4. 4. 4Helpful models for thinking about usabilityA’s Ability, aptitude, attitude (what voters bring to the election)E’s Efficient, effective, engaging, error-tolerant, easy to learn (dimensions of usability)Use Useful, usable (operable), desirable (how we define success)
    5. 5. 5The recipe for planning a usability testLocation and context: formal informalRecruiting: defined opportunisticActivities: instructed tasks free tasksQuestions: structured unstructuredData collection: observation task/data onlyResults quantitative qualitative
    6. 6. 6A few examples
    7. 7. 7Which instructions work better?Traditional ballot instructions vs. plain language version About this project  45 participants  3 lab setting, 8 days  2 moderators  Instructions for voting  Careful observation  Satisfaction survey We learned about  Voter accuracy  Order effects  Voter preference Results  Statistical analysis  Participant preference
    8. 8. 8Which layout works better?Left aligned vs. right aligned names? About this project  100 participants  2 locations, 1 day  4 moderators + 6 others  Instructions for voting  Minimal observation We learned about  Voter preference  Time to vote  Number of errors Results  Statistical analysis of marked ballots
    9. 9. 9What do voters do with unusual options?How do voters react to “double-vote” options? About this project  200 participants  5 locations, 1 day  12 moderators + 5 others  Self-defined tasks  Minimal observation We learned about  Voter awareness of party  Types of marks and errors  Need for voter ed. Results  Inspection of ballots for marking patterns
    10. 10. 10What problems might voters have with a new ballot?What problems might voters have with a new ballot design? About this project  1 of 5 similar tests  10 participants  1 locations, 1 day  2 moderators + 1 others  Self-defined tasks  Observation  Post-task questionnaire We learned about  Navigation on the ballot  Instructions Results  Found pattern of errors for one interaction  Time on task
    11. 11. 11What do voters think about different systems?Which accessible voting system do voters prefer? About this project  496 surveys; multi-disabilit  11 systems  Multiday “voting fair” They learned about  Voter assessment of usability  Voter comments about accessibility  Differences by disability Results  Statistical analysis survey results  Qualitative analysis of comments
    12. 12. 12Do voters understand how their votes are counted?Can voters explain and act on that understanding? About this project  20 participants  4 locations, 1 day  8 moderators  Self-defined tasks  Minimal observation We learned about  Navigation on the ballot  Instructions  Mental models of counting  Whether there’s a problem Results  Qualitative insights
    13. 13. 13Do voters understand a new voting system?What is their reaction to new procedures? About this project  Focus group  Mock election  1 moderators They learned about  Attitudes and reactions Results  Qualitative insights
    14. 14. 14Things to watch out for whenplanning a usability test
    15. 15. 15Treating participants with respectLanguage and cultural issuesManaging contextParticipant protections and informed consent
    16. 16. 16Avoiding biasLeading questionsClosed vs. open questionsSampling biasModerator influenceObserver (or moderator reactions)
    17. 17. 17Thoughts?
    18. 18. 18Whitney Quesenberywhitney@wqusability.comDana Chisnelldana@usabilityworks.netField guides to ensuring voter intenthttp://tinyurl.com/ensuringvoterintentUsability in Civic Lifehttp://usabilityinciviclife.orgCivic Designhttp://civicdesigning.orgLEO Usability Testing Kithttp://www.usabilityinciviclife.org/voting/leo-testing-kit/

    ×