UKUPA Jan 10 Clare Barnett and Caroline Jarrett: Election Ballot Usability
Upcoming SlideShare
Loading in...5
×
 

UKUPA Jan 10 Clare Barnett and Caroline Jarrett: Election Ballot Usability

on

  • 1,092 views

The Electoral Commission was concerned about whether the design of ballot papers was making it difficult for electors to vote accurately....

The Electoral Commission was concerned about whether the design of ballot papers was making it difficult for electors to vote accurately.

Spurred by this, the Electoral Commission commissioned User Vision and Effortmark to conduct usability tests with a range of voters in Scotland, England, Wales and Northern Ireland.

In their talk, Clare and Caroline will provide insight on how they approached the project and what they learned about ballots and about running paper testing.

Statistics

Views

Total Views
1,092
Views on SlideShare
1,091
Embed Views
1

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 1

http://www.slideshare.net 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Goal: Develop draft design standards
  • multi-member first past the post paper where the voter can choose up to 3 candidates, as there are 3 vacancies to be filled.
  • Northern Ireland in 2005 , concerns were voiced about voter confusion. This was attributed in part to the similarity in appearance of the local government and UK parliamentary ballot papers , which added to the general confusion caused by different voting systems being used for each ballot. National Assembly for Wales in 2007 . The Commission noted that many voters commented on a ‘two feet long’ ballot paper for the regional list elections. Some local Returning Officers used their discretion to print a more manageable-sized ballot paper with two columns. However, this meant some voters cast their vote on ballot papers of different appearance to others, owing to the inconsistencies that arose between constituencies within an electoral region. Scotland 2007 , combined parliamentary and local government elections where a relatively high number of ballot papers were rejected (i.e. they were not included in the votes which were counted to determine the result because they were not properly completed) which attracted criticism of the design of the ballot papers used at those elections.
  • Examples of a participant, aged 52, who thought it was important that he voted as it was his only chance to have a voice.

UKUPA Jan 10 Clare Barnett and Caroline Jarrett: Election Ballot Usability UKUPA Jan 10 Clare Barnett and Caroline Jarrett: Election Ballot Usability Presentation Transcript

  • Electoral Commission project TM
  • Clare Barnett and Caroline Jarrett
    • Clare is a usability consultant with User Vision
    • Caroline is a usability consultant / forms expert
    • Other people involved:
      • Louise Ferguson advised the Electoral Commission on usability issues
      • Will Reburn of the Electoral Commission was our main contact for this project
      • Chris Rourke & Rob Van Tol from User Vision worked with us.
  • And now, a word from Caroline’s sponsor
    • Redish (2007)
    • “ Letting go of the words: writing web content that works”
    • Morgan Kaufmann
    Jarrett and Gaffney (2009) “ Forms that work: Designing web forms for usability” Morgan Kaufmann
  • How would you cast your vote?
  • Outline: we tested ballots in four countries
    • Background to the testing
    • How we approached the testing
    • A brief description of the main findings
      • What makes voting hard or easy
      • The detailed design findings
    • How we analysed the results
    • Offline and online testing, differences and similarities
  • This project was a response to problems in 2007
    • After the 2007 Scottish elections, and other elections, concerns grew about the usability of election materials.
    • The Electoral Commission commissioned usability testing of:
      • existing voting materials,
      • the use of registered party descriptions on ballot papers.
    • Focus groups were also organised by Brahm.
    • The end goal was to feed into the development of draft design standards creating consistency.
  • The participants represented a cross-section of UK voters
    • Location
      • Scotland, England, Wales, Northern Ireland
      • rural, suburban, and urban locations
    • Age
      • from 18 to 67
    • Voting experience
      • never voted, occasional voters, frequent voters
    • Different voting methods
      • Polling station & postal
    • Social classes
    • Educational levels
      • from no qualifications to degrees
    • Nationality & first language
      • Including Welsh speakers.
  • Outline: we tested ballots in four countries
    • Background to the testing
    • How we approached the testing
    • A brief description of the main findings
      • What makes voting hard or easy
      • A selection of the detailed design findings
    • How we analysed the results
    • Offline and online testing, differences and similarities
  • The test was an ordinary task-based test
    • Each participant had to complete a selection of voting tasks, varied by location. Two examples:
      • In London: General election, Mayoral election, London Assembly, European election, Local election by postal vote
      • In Scotland: General election, Scottish Parliament, European election, Local election
    • For each voting task :
      • we asked who they were going to vote for
      • we gave them the appropriate materials
      • they voted
      • we briefly discussed the experience of that particular vote
      • we offered a short distraction task between votes
    • We had a ‘beauty competition’ of the materials to look at the finer details.
  • We co-ordinated our approach
    • We created a generic crib sheet that worked for all the different voting patterns
    • We used the first test in London as a pilot
      • Caroline facilitated
      • Rob and the clients watched
      • We debriefed immediately
    • We passed our experiences along:
      • Rob tested in Wales
      • Caroline tested in Cornwall
      • Rob and Clare worked together in Scotland
      • Clare tested in Northern Ireland
    • We all met in Scotland to debrief later.
  • We used a distraction task between votes
      • As the voting process is a fairly repetitive task, we provided distraction tasks between voting.
  • Each session required skills in paper juggling
    • For a typical participant we had:
      • 4 ballot papers
      • 3 station posters
      • 3 booth posters
      • 1 set of postal instructions
        • Postal voting statement (or declaration of identity in N Ireland)
        • Textual instructions
        • Illustrated instructions
        • 2 envelopes
      • our generic crib sheet template for notes.
  • Outline
    • Background to the testing
    • How we approached it
    • A brief description of the main findings
      • What makes voting hard or easy
      • A selection of the detailed design findings
    • How we analysed the results
    • Offline and online testing, differences and similarities
  • Two big factors: voting experience and the voting task(s)
    • Participants found many plausible (though incorrect) interpretations of the materials.
    • We found that:
      • Voting experience is important .
      • The structure of the election has a major effect .
      • The complexity of the postal voting process caused difficulty .
  • Voting experience is crucial to voting success
    • If offered the choice, even new voters skip the instructions
    • If asked to read the instructions, new voters read more carefully
    • Even if they had read the instructions, new voters became confused and some made mistakes
    Extract of Ballot Paper completed by a new voter
  • Participant deals with a complex election as if it were an ordinary one
    • Example
  • Voting is harder than you realise: He votes the way he has done for years
  • Am I voting for a candidate or a party?
    • This participant tried to vote for an individual
    • The instruction to ‘vote for a party or an individual’ can be misinterpreted to mean ‘vote for an individual within the party’.
  • Why do I have more than one vote?
    • London Mayor
      • Voter can indicate first and second choice
      • “ Can I vote twice for Boris?”
      • “ Do I have to vote for someone else?”
    • Scottish Local Election
      • You can vote for as many as you like
      • “ Why would I vote for more than one?”
    • Welsh (& English) Local Election
      • “ Why do I have so many votes?”
      • What happens if the voter does not use them all?
  • Outline
    • Background to the testing
    • How we approached it
    • A brief description of the main findings
      • What makes voting hard or easy
      • A selection of the detailed design findings
    • How we analysed the results
    • Offline and online testing, differences and similarities
  • Why have the candidate’s addresses on the ballot?
    • Some participants thought this was unnecessary
    • Others worried about the potential danger to the candidate
    • Some were concerned that it might sway floating voters
  • Logos are important. Some of them are missing
    • Some participants didn’t vote for parties without logos as they didn’t look “official”.
  • Numbers next to candidates are not useful
    • Numbering on the ballot is unhelpful
      • There is a risk of circling the number instead of voting properly.
    • Numbering within the party is also unhelpful
      • Potential for error in voting for a candidate rather than a party.
  • Candidates names are in an unfamiliar format
    • Some ballots have:
      • SURNAME First names
    • Other ballots have:
      • SURNAME Full name
    • And other ballots have even more:
  • Party names are important, but sometimes hard to find
    • Scottish Conservative & Unionist Party
    • Conservatives
    • Conservative Party
    • Welsh Conservatives
    • Welsh Conservative Party
    • The Conservative Party
    • Scottish Labour Party
    • The Scottish Labour Party
    • Labour Party
    Party names are included in the brackets with the candidates name and address.
  • Instructions on the ballot were often poor
    • Some ballots did not have important instructions:
      • Title of the ballot
      • How to vote
    • Some ballots had instructions, but not easy to see
      • London elections white-on-black instruction
    • If participants saw this instruction then they liked it:
  • English and Welsh should be kept aligned
    • Welsh and English together double the information
    • There was no consistency over which came first
      • Participants do not mind which, slight preference for Welsh first
    • They do not want to see separate English and Welsh forms
    • Translations into Welsh were not exactly the same
    • Mixing Welsh and English is confusing
  • Example of mixing Welsh and English
  • Speak in voters’ language
    • Give it a simple title.
    • Don’t bunch important instructions into one sentence.
    • Don’t double information by repeating it.
  • Polling station notices have instructions that are hard to understand
    • What does ‘spoil a ballot’ mean?
    • What’s a compartment?
    • What’s this about showing your vote to the presiding officer?
    • Instructions have numbers alongside them (good) but often more than one item included for a number (bad)
    • Why tell voters to leave the polling station immediately?
  • The poster in the booth should tell you what to do in the booth
    • If they need it before, put it on a poster
    • If they need it in the booth, tell them in the booth
      • What type of mark to make
        • “ Do I use a cross or a tick?”
      • The rules about the numbers of votes you have
      • What to do if you make a mistake.
    • If they need it after voting, put it after
      • Fold/don’t fold,
      • Show to member of staff.
    • If they don’t need it at all, scrap it!
      • “ Leave the voting station immediately”
  • Outline
    • Background to the testing
    • How we approached it
    • A brief description of the main findings
      • What makes voting hard or easy
      • A selection of the detailed design findings
    • How we analysed the results
    • Offline and online testing, differences and similarities
  • Collating analysis was a joint task
    • There were 3 testers, 45 participants and 4 countries.
    • We entered our findings per country into an Excel spread sheet
    • Caroline came to Edinburgh
      • We each had 30 mins to talk about our particular test results for each country
      • This allowed us to see whether issues were country specific or UK wide
      • Each issue was written onto a post it and placed on a white board under country specific or UK wide
      • We then removed any duplicated issues from the board
    • The Electoral commission wanted a summary presentation of our main findings, followed up with a report detailing a full list of the issues and recommendations.
  • We wrote the full report with client involvement
    • The Electoral Commission wanted to make the report a public document.
    • Due to experiences with previous reports, the Electoral Commission did not want any figures referred to in the report.
      • “ In a small qualitative study, it is important to focus on the problems that were identified by participants rather than the numbers who found those problems.“
      • “ We also did not consider how difficult or easy our recommendations may be to act on for legislative or any other reason.“
    • We split issues into categories. These were agreed with the Electoral Commission.
    • The post-its from the analysis meeting were used, and split into categories, flowing as much as possible with the journey that voters take.
  • Full report
    • Electoral commission website:
    • http://www.electoralcommission.org.uk/document-summary?assetid=77687
  • Outline
    • Background to the testing
    • How we approached it
    • A brief description of the main findings
      • What makes voting hard or easy
      • The detailed design findings
    • How we analysed the results
    • Offline and online testing, differences and similarities
  • Offline and online testing: similarities
    • The process followed was the same for online testing.
    • Flapping about of paper is similar to users being unable to find information online: Users are having trouble finding something.
    • People skipped instructions, even if they were only a few words long.
    • Users seemed to ‘get into’ the idea of the test quickly and treat it as realistic.
  • Offline and online testing: differences
    • “ Feel appeal” – you actually interact with the materials
      • You can draw straight onto the materials.
      • Paper is an physical product and can be bulky.
    • On paper, signatures really matter.
    • Recording is harder:
      • One venue: good quality video – of the participant’s head
      • Other places: used a web cam, but quality not very good.
  • What next?
    • The Electoral Commission has published their guidelines – Making your Mark:
    • http://www.dopolitics.org.uk/making-your-mark
  • Question time Caroline Jarrett [email_address] 01525 370379 Clare Barnett [email_address] 0131 225 0850