This document summarizes usability testing of ballot paper prototypes for tablets over 4 days with 18 participants and 16 iterations. It describes iterations tested for instructions, supplemental content, straight party voting, multi-candidate contests, judge retention contests, referenda, ballot measures, write-ins, and reviewing and casting votes. The testing aimed to refine designs for tiny screens and minimal instructions. Further usability testing of digital prototypes was planned for mid-January 2013.

1. Usability test of
ballot paper prototypes for
tablet -
design iterations
University of Baltimore
Drew Davies
Dana Chisnell
Kathryn Summers
Megan McKeever
November 11-14, 2012

2. Iteration 1

3. Iteration 2 Localizes the ballot for the testing location.

4. Iteration 3
Reorients to vertical
from horizontal,
getting ready for tiny
screens.

5. Instructions to voters

6. Iteration 1 Appears after the title screen. Text comes from NIST research.

7. Iteration 2 Corrects emphasis in the first paragraph.

8. Iteration 3 Added illustration, which did not seem to help.

9. Iteration 4
Moved from horizontal
to vertical changes the
proportions, which
feels different.

10. Iteration 5
Added a simpler
illustration back into
the vertical version.

11. Help version
Here we added a
pointer to the
supplemental content
that acts like context
help.

12. Straight party voting

13. Iteration 1 The NIST ballot includes straight-party.

14. Iteration 2 Expands explanation of straight-party voting.

15. Iteration 2, selected state

16. Iteration 3 Paring down the instructions substantially.

17. Iteration 4 Simplifying wording, and adding paragraph breaks.

18. Iteration 5
Vertical layout
enforces order. We
added (i) buttons for
supplemental content.

19. Vote for 1

20. Iteration 1 Showing the number of choices remaining was helpful.

21. Iteration 1, selected state

22. Iteration 2 Correcting the instructions for changing votes.

23. Leads to
information
Information
about the
about
contest
marking the
ballot for
this contest
Position
statements
from the
candidates
Iteration 3 Removing most instructions and adding entry to supplemental content.

24. Supplemental content, iteration 1

25. Supplemental content, iteration 1

26. Supplemental content, iteration 1

27. Iteration 4
Vertical, with minimal
instructions.

28. Supplemental
content,
iteration 2
In addition to going
vertical, we added
simplified illustrations.

29. Supplemental
content,
iteration 2

30. Multi-candidate
contests

31. Iteration 1

32. Iteration 1, selected state Avoiding a testing artifact, we changed party names.

33. Iteration 2 Correcting the instructions for changing votes.

34. Iteration 2, content scrolled down

35. Iteration 2, content scrolled to end

36. Iteration 3
Vertical format,
minimal instructions.

37. Iteration 3,
content
scrolled

38. Iteration 4

39. Iteration 4,
scrolled
We still have questions
about how scrolling
should work.

40. Judge retention
contests

41. Iteration 1

42. Iteration 2 Changing the order of and shortening instructions.

43. Iteration 3
Vertical, minimalist.

44. Referenda

45. Iteration 1 Note “Yes” or “No.”

46. Iteration 2, and selected state

47. Iteration 3 Correcting the instructions on changing votes. Switching to “For” and
“Against.” The literature is unclear about whether it’s better than “Yes” and “No.”

48. Iteration 4
Staying with “For” and
“Against” for the
vertical version posed
no problems.

49. Ballot measures

50. Iteration 1

51. Iteration 2 Correcting the instructions on changing votes.

52. Iteration 3 Localizing the state the ballot measure applies to.

53. Iteration 4
Pared-down
instructions, and room
for a question that can
be answered “Yes” or
“No.”

54. Write-ins

55. Iteration 1 Instructions from the NIST study.

56. Iteration 1

57. Iteration 2
Vertical version, with
minimal instructions,
focusing on what
would invalidate the
vote.

58. Iteration 2

59. Supplemental
content,
iteration 1
Touching the (i) leads
to the instructions that
used to be on the
surface screen.

60. Reviewing votes

61. Iteration 1

62. Note the
use of the
word,
“choices”
Iteration 1, scrolled down (page 2)

63. Iteration 1, scrolled down (page 3)

64. Iteration 1, scrolled to the end

65. Iteration 2
Vertical format, with
wording change in the
scroll button.

66. Iteration 2,
scrolled down
(page 2)

67. Iteration 2,
scrolled down
(page 3)

68. Iteration 2,
scrolled down
(page 4)

69. Iteration 3,
scrolled down
(page 4)

70. Iteration 4,
scrolled down
(page 2)
Revelation! “Choices”
means there are still
more choices to make.
Wording changes both
in the title and the
button label.

71. Iteration 4,
scrolled down
(page 3)
The wording changes
and simple instructions
helped participants
understand the
purpose and use of
the Review.

72. Supplemental
content,
iteration 1

73. Changing a vote

74. Iteration 1

75. Iteration 2
Changing the button
label to telegraph the
navigation.

76. Iteration 3

77. Iteration 4

78. Casting the vote

79. Iteration 1

80. Iteration 1

81. Iteration 2
We’re hoping that the
vertical format will
scale well to tiny
screens.

82. Iteration 2

83. 18 participants
4 days
16 versions

84. Next:
Usability testing of
digital prototypes
mid-January 2013