We are testing a new beta website for gov.scot go to new site

Testing of the Ballot Paper for the 2012 Local Government Elections in Scotland RF 16/2011

Listen

Ipsos MORI Scotland undertook three rounds of user testing to inform the design of the ballot paper for the 2012 Local Government elections. In the first round, a draft design provided by the Scottish Government was tested with a sample of 102 people recruited to be representative of the electorate on key characteristics, eight people who had English as an Additional Language and eight people who had lower levels of literacy skills. In each of the second and third rounds, amended versions were tested with a further 30 people.

Main Findings

  • Awareness and understanding of the Single Transferable Vote ( STV) system used in Local Government elections in Scotland was extremely low. This underpins many of the issues that emerged in testing.
  • There are limits to what a ballot paper alone - however well designed - can do to counter this. The voter education campaign in advance of the 2012 elections should focus on raising awareness and understanding of STV.
  • Overall, the first round of testing suggested that the draft ballot paper worked well in many respects. Most participant liked the design of the draft ballot paper: it was frequently praised as being simple, clear and straightforward.
  • In the first round of testing, almost half of the ballot papers that would be accepted by returning officers were, in some way, problematic. These can be grouped into three broad categories:
    • o ballot papers that would probably have been filled in differently had the participants had a better understanding of their options (such as participants being unclear that they could vote for as few or as many candidates as they wish and participants not understanding that they could vote for more than one candidate per party)
    • o ballot papers that were not completed in line with the instructions (such as participants using a single 'X')
    • o ballot papers that did not fully match participants' stated intentions (such as participants failing to spot a candidate for a party they wanted to vote for or making a mistake with the ordering of their choices).
  • The potential impact of these issues will vary and some are more problematic than others.
  • Based on the findings from the first round of testing, a number of changes were made to the ballot paper and then retested. These changes focused on making the instructions more concise and easier to read, in order to increase the proportion of people who read them, and to give information about the number of candidates to be elected in each ward.
  • Testing suggested that the final revised version was an improvement on the original draft. It is recommended that this version (with the minor amendment of the whole of the second line of instructions being in bold) be put before Parliament.
  • If possible, polling station staff should remind voters to read the instructions when they hand over the ballot paper.
  • Polling station notices and staff should encourage voters to ask for assistance if they want any guidance on how to complete their ballot paper.

Background

In October 2010, the Office of the Chief Researcher at the Scottish Government, on behalf to the Election team in the National Conversation, Referendum and Elections Division, commissioned Ipsos MORI Scotland to conduct testing of a draft ballot paper to be used in the 2012 Local Government Elections.

Aims and methods

The overall aim of the testing was to assess the clarity and usability of the draft ballot paper design, identify possible improvements and make recommendations for the final ballot paper to be put to Parliament prior to the election.

Three rounds of testing were undertaken. In the first round, the draft design provided by the Scottish Government was tested with a sample of 102 people recruited to be representative of the electorate on key characteristics, eight people who had English as an Additional Language and eight people who had lower levels of literacy skills.

The second and third rounds of testing used ballot papers amended in response to the first and second rounds of testing. Each round involved testing with a further 30 research participants, recruited to be broadly representative of the Scottish electorate.

All testing was undertaken using the same method. After a brief introduction, participants were given the draft ballot paper and asked to complete it as they would if it was the day of the Local Government elections. After casting their 'vote', a semi-structured cognitive interview was carried out to explore whether participants completed the paper according to their intentions, how they found the experience, and any aspects of the ballot paper design that presented difficulties for them.

Findings from the first round of testing

Awareness and understanding of the STV voting system used in Local Government elections in Scotland was extremely low. It was also clear that very few participants were aware that council wards are multi-member and that more than one candidate would be elected. This lack of understanding was evident throughout fieldwork and underpins many of the issues that emerged in testing.

Overall, the first round of testing suggested that the draft ballot paper works well in many respects. Most participants liked the design of the draft ballot paper: it was frequently praised as being simple, clear and straightforward.

Almost half of the ballot papers that would be accepted by returning officers were, in some way, problematic. These can be grouped into three broad categories:

  • ballot papers that would probably have been filled in differently had the participants had a better understanding of their options (such as participants being unclear that they could vote for as few or as many candidates as they wish and participants not understanding that they could vote for more than one candidate per party)
  • ballot papers that were not completed in line with the instructions (such as participants using a single 'X')
  • ballot papers that did not fully match participants' stated intentions (such as participants failing to spot a candidate for a party they wanted to vote for or making a mistake with the ordering of their choices).

The potential impact of these issues will vary and some are more problematic than others. For example, ballot papers that were not completed in line with the instructions but that captured intentions correctly would increase the number of ballots that are not automatically processed, and are referred to the Returning Officer for verification, but would have no impact on the result. In contrast, problems related to a lack of understanding of the voting options, or a lack of understanding of how many candidates can be ranked, could have an impact on results.

Unsurprisingly, mistakes and misunderstandings were more common among those who did not read the instructions or just skimmed them very quickly.

Findings from the second and third rounds of testing

Based on the findings from the first round of testing, a number of changes were made to the ballot paper and then retested. These changes focused on making the instructions more concise and easier to read, in order to increase the proportion of people who read them, and to give information about the number of candidates to be elected in each ward.

Given sample sizes involved, statistically significant differences in the incidence of specific problems would not be expected. This was primarily a qualitative exercise and the research was looking for some indication that the changes had helped address the problems they were designed to address - and had not introduced, or worsened, other problems.

Many of the amendments were designed to encourage more people to read the instructions and there did seem to be a slight improvement with slightly fewer people ignoring the instructions altogether. Probably linked to this, there was a slight reduction in the number of people who assumed this was a 'normal' election and put a single X or a single tick.

Adding "Three[/Four] of the candidates listed below will be elected" to the instructions was designed to provide people with at least some basic information about the voting system. There was a slight improvement in the number of people who understood that they could vote for more than one candidate per party. The addition of this line did not appear to cause any confusion to participants.

Fewer participants given version 2 or 3 ranked just one candidate, more ranked two, three or four candidates and fewer ranked five candidates. In terms of the impact on the election results and voters taking advantage of the STV system, this pattern suggests that version 2 and version 3 are an improvement on version 1.

Recommended ballot paper design

Our recommended ballot paper design is shown below. This is the version that was used in the third round of testing (with the minor amendment of the whole of the second line of instructions being in bold) and is very similar to the version used in the second round of testing.

The testing indicates that this design will work well: the remaining problems relate to the perennial issue of people not reading instructions carefully and to the very low levels of awareness and understanding of STV.

Amendments to the ballot paper have countered these problems to some extent: making the instructions more concise and easier to read has increased the proportion of people who read them and the addition of information about the number of candidates to be elected means that voters are at least aware of this aspect of the system. However, as noted after the first round of testing, there is a limit to what the ballot paper alone can do.

Other recommendations

If possible, polling station staff should remind voters to read the instructions when they hand over the ballot paper (e.g. saying something along the lines of "Do take your time and read the instructions at the top here - because this is different from some other elections and you use numbers rather than a cross").

Polling station notices and staff should encourage voters to ask for assistance if they want any guidance on how to complete their ballot paper.

As noted above, many of the problems that emerged in the testing were linked to the very low levels of awareness and understanding of STV among the Scottish electorate. These problems did not generally result in obvious errors and ballot papers that would be rejected. However, they had an impact on the way participants used their vote and the extent to which they were able to take advantage of the STV system - and they have the potential to impact on election results.

The design of the ballot paper alone cannot solve these problems: they can only be addressed by a much wider programme of voter education. We would therefore strongly recommend that the voter education campaign in advance of the 2012 elections focuses on raising awareness and understanding of STV.

Final recommended ballot paper design (not to scale)

Final recommended ballot paper design (not to scale)

Actual dimensions: width 157.5mm, depth of instructions 52mm, depth of each candidate row 22 mm, voting box and party logo both 16mm square. Font size: title 12 pt, instructions 13 pt, candidate names 14 pt, addresses and party names 12 pt.

This document, along with full research report of the project, and further information about social and policy research commissioned and published on behalf of the Scottish Government, can be viewed on the Internet at: http://www.scotland.gov.uk/socialresearch . If you have any further queries about social research, please contact us at socialresearch@scotland.gsi.gov.uk or on 0131-244 7560.