Voices in Justice: Parole Reform in Scotland: Consultation Analysis

2. About the data preparation and analysis

2.1 This chapter provides details of issues that had to be considered and / or addressed in preparation for the analysis.

Consultation questionnaire

2.2 As noted in Chapter 1, there were two versions of the consultation questionnaire. As well as the online questionnaire (available on Citizen Space), a modified pen-and-paper version was also made available within the prison estate (see Annex 1) to allow those in custody to take part in the consultation.^[3] This version of the consultation questionnaire is referred to in this report as the ‘prison questionnaire’.

2.3 It is well established that literacy levels within prison populations are often lower than in the general population. The prison questionnaire was therefore designed to be simpler and easier to understand. Changes made to the wording of both the questions and response options may have affected how some respondents interpreted and answered the questions. As a result, responses to the prison questionnaire are not always directly comparable with responses to the online questionnaire. This is discussed further in relation to specific questions at relevant points in the report.

Data quality – prison questionnaires

2.4 The data input of the prison questionnaires was undertaken in the first instance by the Scottish Government and reviewed by the independent analytical team as part of a quality assurance process.

2.5 This process highlighted a range of data quality issues in relation to the prison questionnaire. Key concerns are set out below. The following section (on data cleaning procedures) explains how these issues were addressed.

2.6 The data quality issues identified were as follows:

• The handwritten comments provided in response to open questions were often illegible and / or difficult to understand due to spelling / grammatical errors. In some cases, analysts had to interpret or infer the intended meaning of comments.
• In many cases, respondents made comments in the text boxes provided but did not select any of the available tick-box options.
• The design of the online questionnaire meant that, for most of the closed questions, respondents could only select a single option from the list of possible responses. However, those completing the prison questionnaire could – and often did – select multiple response options. In most cases when these responses were input by the Scottish Government to Citizen Space, only the first of the multiple options selected was input.
• The handwritten comments provided by respondents did not always align with the tick-box they had selected.

2.7 It was also noted at this stage that there were a large number of comments – across almost all questions – which simply indicated that the respondent was not sure, didn’t know, or had no opinion in relation to the question under consideration.

Data cleaning procedures

2.8 The data cleaning process for the online questionnaire responses involved checking whether there were any duplicate or multiple responses from a single respondent and identifying any campaign responses (see Chapter 3).

2.9 The process for reviewing the data input for each prison questionnaire involved two analysts working together. One read out the information that had been entered into the database, while the other checked that it matched the original form. The data cleaning process then involved the following steps:

• Where the respondent had written comments but had not selected a tick-box option, the analysts agreed which of the tick-box response options represented the ‘best fit’ and added this to the database.
• Where the respondent had selected multiple response options, but the online questionnaire had required the selection of a single option, the analysts agreed either on a single response option which represented the ‘best fit’ to the comments, or used a new response option (created by the analysts), namely ‘multiple options selected’, and added this to the database. If this new option was used, a note was inserted in the comments box for the question detailing the respondent’s original (multiple) selections.
• Where the respondent had ticked multiple response options for questions where multiple response options were allowed in the online questionnaire, the analysts checked that all the options selected were entered correctly into the database.

2.10 In some cases, changes were made to the comments that had been entered into the database. This was done if it became clear during the data cleaning process that text that had been entered did not fully match the corresponding text on the original form (often because the handwriting was not easy to read) and did not properly capture the views of the respondent. In these cases, the original entry was revised, based on the analysts’ best interpretation of what the respondent had written.^[4]

Categorising individuals for comparative analysis

2.11 Question 43 asked respondents to assign themselves to one of 11 categories:

• I am the victim or survivor of a crime
• I am a family member or friend of the victim or survivor of a crime
• I have served time in prison, but have not served time on parole
• I have served time in prison, including serving part of my sentence on parole in the community
• I am a family member or friend of someone who has served time in prison, but they did not serve time on parole
• I am a family member or friend of someone who has served time on parole
• I am a professional who works in the justice sector
• I am a solicitor
• I work with victim support organisations
• I am a member of the public with an interest in this area
• Other, please specify.

2.12 The online questionnaire allowed respondents to choose one option only from the list presented. However, respondents choosing ‘other’ could provide any information they felt was relevant. Those completing a prison questionnaire could select as many options as they wished and could also write in any other relevant information.

2.13 For the purpose of analysis, the information provided at Question 43 was used to categorise the individuals who responded to the consultation, as described in the remainder of this section.

2.14 To enable comparative analysis, the original set of 11 categories offered in the questionnaire was reduced to seven (7). This involved combining some of the original categories and adding a new ‘not known’ category. See Table 2.1 for a description of the seven categories.

Table 2.1 Description of revised set of categories

Category / Description

Individual with experience of custody (current or past)

Those identifying themselves as having experience of custody, irrespective of whether they had spent time on parole

Professional in the justice sector

Those identifying themselves as solicitors, working with victim support organisations, or other professionals working in the justice sector (includes Parole Board panel members, prison officers and social workers)

Family / friend of someone with experience of custody

Those identifying themselves as a family member or friend of someone with experience of custody, irrespective of whether the person had spent time on parole

Family / friend of victim

Those identifying themselves as a family member / friend of a victim / survivor of crime

Victim

Those identifying themselves as a victim / survivor of crime

Member of public

Those identifying themselves as a member of the public with an interest in this area

Not known

Those for whom there was insufficient information to allocate them to any of the categories above

2.15 Where necessary, the following rules were applied to allocate each individual respondent to a single category only:

• In cases where multiple options were selected, if the respondent selected an option or provided information that indicated that they were either currently in custody or had previously been in custody they were categorised as ‘individual with experience of custody’.
• In cases where no option was selected, the response was read in full to establish whether a category from the list could be assigned. If insufficient information was provided to enable the respondent to be assigned to any of the first six categories, they were assigned to the ‘not known’ category.

2.16 This simplified set of seven categories is used in the qualitative analysis in this report.

Analysis and presentation of findings

2.17 This report is based on an analysis of the responses to the consultation questions.

2.18 The analysis explored the views of three groups: (i) organisations, (ii) individuals with no experience of custody (which includes individuals categorised as ‘not known’), and (iii) individuals with experience of custody (current or past). Note that most respondents in the first two groups completed the online questionnaire, while most respondents in the third group completed the prison questionnaire (see Chapter 3, paragraph 3.7).

2.19 Given the different questionnaires completed and the quality issues with the responses to the prison questionnaire, the views of the third group are presented separately in the report. Specifically, in each chapter the views of organisations and individuals with no experience of custody are presented first, followed by the views of individuals with experience of custody.

2.20 Frequency analysis was undertaken in relation to the responses to all the closed consultation questions and the findings are presented throughout this report.

2.21 Findings relating to the closed questions are presented in table form for the first two groups (that is, organisations and individuals with no experience of custody). The tables in the report show numbers and percentages. This allows readers to quickly understand the balance of opinion for each question. However, it should be borne in mind that these percentages (particularly in relation to organisational respondents) are based on small numbers.

2.22 For the third group of respondents (individuals with experience of custody) findings related to the closed questions are presented as simple counts rather than as percentages, given the data quality issues outlined at paragraphs 2.4 to 2.7.

2.23 Not all respondents answered all the closed questions – and so the totals shown for each question relate to the number of respondents who answered that question.

2.24 Qualitative analysis has also been undertaken of the comments submitted in response to each question. The aim of this analysis was to identify and report the main themes and most common points in the responses to each question (or group of questions), and to explore areas of agreement and disagreement among respondents.

A caveat about the findings

2.25 As with all consultations, the views of those who have responded cannot be seen as representing the views of the wider population. Individuals (and organisations) who have a keen interest in a topic – and the capacity to respond – are more likely to participate in a consultation than those who do not. This self-selection means that the views of consultation participants cannot be generalised to the wider population.

2.26 For this reason, the approach to consultation analysis is primarily qualitative in nature. Its main purpose is not to identify how many people held particular views, but rather to understand the full range of views expressed and any concerns that respondents may have. The qualitative analysis also helps in understanding the responses to the closed questions and gaining deeper insight into people’s views.