Understanding extremism in Scotland: public perceptions and experiences

2. Methodology

2.1 Overview

As noted in the introduction, an evidence review carried out by the Scottish Government (2023c) identified a lack of existing research on extremism in Scotland. At the outset of this project a further Rapid Evidence Review was carried out, the focus of which was on methods and questions which have been used to explore views from the public on extremism in studies elsewhere. While the purpose of this Rapid Evidence Review was to inform the design of the research instruments for this work, it also confirmed that significant evidence gaps exist in relation to what is known about extremism in Scotland. In particular, no research was identified which had previously explored the public's views on, or experiences of, extremism in Scotland.

To address the research aim and questions, a mixed-methods approach was adopted, incorporating quantitative and qualitative elements. The quantitative research involved a large-scale survey, while the qualitative research involved focus groups and interviews. The quantitative research was undertaken first, so that the qualitative research could be used to explore any findings of interest or contradictory results in greater detail. The research design was subject to an in-depth ethical review process involving the research team and an ethics committee within the University of St Andrews.

This report details the findings from the primary research, with additional context added, where relevant and available, from secondary sources identified by the Rapid Evidence Review and further searches for relevant literature.

This section of the report details each stage of the research in turn, before concluding with a discussion of the limitations of the approach adopted.

2.2 Rapid Evidence Review

The main purpose of the Rapid Evidence Review was to ascertain how the public have been asked about their views on extremism in other studies, to inform the data collection tools for the primary research. The review also explored what is already known about public views on extremism in Scotland, building on the evidence review carried out by the Scottish Government (2023c).

As a rapid review of evidence, this stage was not as exhaustive as a systematic review or a literature review but was included to gain an overview of existing evidence on the topic and inform the design of the primary research.

To conduct the review, the research team utilised the subject expertise of the Handa CSTPV, including their knowledge of existing research on this topic. This research was reviewed, including reviewing the bibliographies of identified studies for further evidence. Additional searches of journal publications were also carried out using the key terms 'public', 'extremism' and 'data'.

The review covered five main areas. Firstly, building on a review of definitions of extremism used in other countries carried out by the Scottish Government (2023d), terminology adopted by governments elsewhere was explored. Variation in definitions was noted, for example the references to violence within the definitions adopted in Australia and New Zealand (Australian Government, 2022; Counter-Terrorism Coordination Committee, 2019). These definitions informed the development of the qualitative data collection tools, as participants were asked for their views on them during the interviews.

The review then explored categorisations of different types of extremism developed by academics. A notable study identified was that carried out by Michalski (2019) who examined 8,000 terrorist attacks that took place in the UK and the US between 1970 and 2017 and classified them according to their underlying motivation. The categories developed by Michalski (2019) informed the development of questions relating to the public's views on the threat posed by different types of extremism.

Thirdly, the review explored key findings of recent, relevant research which had been conducted in Scotland. The review confirmed that while the academic literature published in English language on extremism is extensive, primary research on this topic in Scotland is lacking. The few studies that have been conducted in Scotland (Blackwood, Hopkins, & Reicher, 2013; Brooke, 2018; Morris & Meloy, 2020) provide relevant insights, but none were identified which have specifically sought to explore public perceptions and experiences of extremism. Further, these studies are largely based on small samples, meaning their findings cannot be easily generalised to Scotland as a whole.

Fourthly, the review explored previous variables from large-scale surveys which have been carried out with members of the public in Great Britain only, or in England and Wales only. Some of these surveys explicitly asked questions about extremism, or related concepts such as terrorism. This allowed the research team to draw on variables which had been previously tested and administered successfully when devising the survey questionnaire for this work.

Lastly, the review explored key sources relating to the UK and Scottish policy context regarding extremism and terrorism. These sources were valuable for the context sections of this report (for example see Redgrave et al., 2020; Wilkinson and van Rij, 2019).

In total the review drew evidence from 20 academic articles, six datasets and their survey questions, and six non-academic publications. The Rapid Evidence Review was used as a working document throughout the project and was updated and referred to during the primary research instrument design and qualitative and quantitative analysis.

2.3 Primary research preparation and ethics

Given the sensitive nature of the research topic, this project was designed with consideration to many ethical sensitivities including:

• Protecting against potential harm to participants, such as minimising distress caused by discussion of extremism.
• Enabling participants to speak freely without risk of reprisals or upsetting others.
• Ensuring there were as few barriers as possible to participation.
• Incorporating a diverse range of perspectives.
• Avoiding placing an unnecessary burden on participants.
• Protecting the identity of participants and ensuring that their views were not attributed to them.
• Mitigating against potential harm to wider groups, for example if the research were to result in stereotyping or associations with any communities with extremism.
• Protecting against potential harm to the researchers.
• Ensuring a safeguarding process was in place.
• The ethics process consisted of:
• An initial ethics assessment conducted by the Scottish Government prior to commissioning the research.
• Discussion of ethical issues at the inception of the research in relation to the proposed methods and approach.
• An extensive ethics review conducted by the University of St Andrews ethics committee.
• Preparation of ethics-related materials for participants, including an information sheet, sources of support, consent forms and privacy notices.

The participant materials were designed to inform participants of the purposes of the research, data procedures and data handling, and gain their informed and ongoing consent. Participants were assured before, during and after their involvement that they could opt out at any time.

Given a focus of this research was on exploring public perceptions and experiences of extremism, the possibility that a participant might disclose a Prevent concern relating to someone known to them was considered. A safeguarding process was developed with Police Scotland and put in place for all researchers to follow. In practice, the need to implement this did not arise during this research.

2.4 Quantitative research

The main aim of the quantitative research was to allow for inferences to be drawn about the views, attitudes, beliefs, and experiences of people across Scotland, including between those with different demographic characteristics. As has been noted, the Rapid Evidence Review identified a lack of robust data from a large sample of the Scottish population relating to this topic. While some Great Britain-wide polling surveys have included respondents from Scotland (for example YouGov, 2020), the small sample size for Scotland in such surveys precludes analysis by variables such as attitudes and demographic characteristics. This research therefore sought to capture the views of a large, representative sample of the Scottish population which would allow for subgroup analysis to be conducted.

Administration

A large-scale survey was issued by Survation between 9 and 25 May 2022. As noted above, an ethical consideration for this research was to ensure potential barriers to participation were reduced as far as possible, including digital inclusion. With this in mind, the survey was administered both online through Survation's Scotland panel, and also by telephone using Survation's expert, trained interviewers.

Sample

The survey aimed to collect 1,500 online responses and 500 telephone responses. The online sample was constructed by randomly selecting members of Survation's online panel with a valid postcode from Scotland. Membership of this panel is on a voluntary basis and respondents are not paid to complete surveys. Panel members come from a range of demographic backgrounds and geographic areas, including those living in more remote and rural areas. The telephone sample was constructed by randomly selecting members of Survation's telephone panel. Telephone interviewing has the potential to reach participants who may not take part in online research, such as people who are not confident in using online platforms (Fricker, 2016) or do not have internet access.

Respondents were given no prior knowledge of the contents of the survey before completion, to reduce the risk that people with stronger or polarising views on the research topic would be disproportionately motivated to respond.

There were 2,071 responses to the survey from residents of Scotland aged 16 and over, including 1,568 online responses and 503 telephone responses (see Appendix B for full sample details). Both samples were asked identical questionnaires and as such, the data were combined in one dataset.

The margin of error for the data, based on a nationally representative survey of the adult population of Scotland, is 2% at the 99% confidence level.^[3]

Variables

The survey included 12 questions, the majority of which were closed questions with answer categories provided. Two open text questions were also included so respondents could describe their own experiences in more detail, where applicable.

A further five questions gathered information on demographic characteristics, including the religion, ethnicity, and political affiliation of survey respondents. See Appendix A for all questions.

Survation provided additional demographic information about respondents. These were panel variables, not gathered as part of the survey, but collected and updated as standard by Survation. These included respondents' sex, age, Scottish Parliamentary Region,^[4] household income and education level.

Education level is comprised of four groups based on attainment: one to four Standard Grades (any grade) or equivalent; five or more Standard Grades (grade A to C) or equivalent including intermediate apprenticeships; two or more Advanced Highers or equivalent; and Higher Education qualifications, including college or university qualifications. The findings section of this report includes short-hand descriptors of these four groups.

Age was broken down into the following bands: 16-34, 35-44, 45-54, 55-64, and 65+. Additionally, responses were broken down into the following Scottish Parliamentary Regions: Highlands and Islands, Mid Scotland and Fife, North East Scotland, Lothian, South Scotland, Glasgow, West Scotland, and Central Scotland. Finally, responses were broken down into the following annual household income categories: up to £19,999, £20,000-£39,999, and over £40,000.

Analysis

On completion of the survey, data were weighted to the profile of all adults in Scotland aged 16+. Data were weighted by age, sex, Scottish Parliamentary Region and 2021 voting at the Scottish Parliament election. Targets for the weighted data were derived from Office for National Statistics Data and the results of the 2021 Scottish Parliament Election.

Descriptive statistics were conducted for the closed questions in the survey. These included frequencies of responses to the different answer options. In the case of multiple response questions, the sum of frequencies will not add to 100%. In the case of single response questions, the sum of frequencies will largely add to 100%, allowing for rounding.

In some cases, results for composite values are included. For example, 'any agreement' combining 'strongly agree' and 'agree' and 'any disagreement' for 'strongly disagree' and 'disagree'. These are clearly flagged as composites in the report, and all answer options can be consulted in Appendix A. Composites were also created based upon demographic questions to aid analysis. In particular, to identify respondents identifying as belonging to a religion a composite variable was created combining: Buddhist, Church of Scotland, Roman Catholic, Other Christian, Hindu, Jewish, Muslim, Pagan, Sikh, and Another religion or body. A composite BAME variable was also created by combining: African, Scottish African or British African, Asian, Scottish Asian or British Asian, Caribbean or Black, and Mixed or multiple ethnic groups.

Cross tabulation was used to explore the relationship between two variables. Significance testing (a two-sample z-test) was performed on each cross tabulation to determine the likelihood of the differences observed between subgroups happening by chance.

Results are only reported as statistically significant for confidence intervals over 95%, where the test conducted is valid and the sub-samples of both groups are sufficient (due to the large number of responses gathered all of the subgroups of each classification question have a sample size which is greater than 30, meaning all were large enough to report on). It should be noted that even when statistical tests show there is a significant difference between groups, the level of difference can be small. Therefore, this report couples the frequency results for subgroups with any reporting of significance.

As noted above two open text questions were included in the survey. The results to these questions are displayed as word clouds. Word clouds are visual representations of textual data that are otherwise difficult to analyse. The word clouds were created with the aid of R software using the most common words within the open text responses. The bigger and bolder the word appears, the more often it is mentioned within the open text responses.

2.5 Qualitative research

The qualitative research aimed to explore public understandings, perceptions, and experiences of extremism in Scotland in greater depth, and was conducted in two stages. Firstly, five focus groups were carried out, to maximise participation numbers and enable discussion between participants, including exchanging and building of views. The focus groups took place between 24 June and 2 July 2022. Eight in-depth, one-to-one interviews were then carried out with a selection of people who had taken part in the focus groups, to allow a smaller number of participants to explain their views and experiences in more detail. The interviews took place between 11 and 13 July 2022.

The focus groups and interviews were conducted online using the online platform Zoom. Online research, adopted more commonly during the COVID-19 pandemic when face-to-face research was not possible, brings some advantages. For example, the use of an online platform can make it easier for people from different parts of Scotland to participate, as well as people with disabilities and long-term health conditions who may have difficulty travelling to a physical location.

At the same time, where online approaches are used, it is important to be mindful of barriers to participation that might arise as a result of digital exclusion or data poverty. As such, when contacting interested participants the research team checked whether anyone selected needed help to access devices, data or Wi-Fi to participate. Although this was not the case, the team were requested, and did provide additional support for participants less familiar and confident with the platform in advance.

Following their participation, qualitative research participants were provided with an incentive payment of £30 per hour to compensate them for their time.

Recruitment

The telephone survey included a question to gauge interest in taking part in follow-up research. Those interested were asked to provide their contact details. This gained expressions of interest largely from people aged over 55 years.

This approach was supplemented by additional recruitment undertaken by a professional recruitment agency Taylor McKenzie. Taylor McKenzie was provided with a pre-screening questionnaire based upon the survey, which enabled the research team to ensure they held data about the characteristics of participants and their views in relation to extremism in advance of the qualitative research.

Participant characteristics

As explained, the qualitative phase of the research consisted of five focus groups and eight follow-up interviews with a subgroup of focus group participants. There were between four and seven participants in each focus group, making the total number of participants in the qualitative research 26.

Quotas were set for the focus groups in order to ensure diversity in the demographic characteristics of participants. These included minimum numbers of females, younger people, people identifying as religious and participants who were not from white ethnic groups. In addition, minimum quotas were set for location with regards to rural areas and for low SIMD postcodes (Scottish Government, 2020; Scottish Government, 2022). The demographic breakdown of the 26 participants is shown in Appendix C.

Appendix C also indicates those participants who took part in both focus groups and then follow-up interviews. Interview selection was based on interest to take part, gathered following each focus group from individual participants, and their availability to take part during the interview period. Moreover, participants were approached on the basis that they had collectively expressed a range of viewpoints and perspectives during the survey or recruitment survey, and the focus groups. Lastly, albeit with a small sample of eight people, efforts were made to include participants of different sexes, ages, religions and ethnicities, and from different areas of Scotland.

Discussion guides

The focus group and interview discussion guides were informed by a range of sources (see Appendix D for summaries of the discussion guides). Firstly, the Rapid Evidence Review provided key topics to potentially explore with the public. The evidence reviews carried out by the Scottish Government (2023c; 2023d) on definitions of extremism and the extent and nature of extremism in Scotland were also used to determine key topics of interest. Discussion guides from the stakeholder and public sector practitioner strands of the research programme (Scottish Government 2023a; 2023b) were also considered to enable comparison of findings across the three projects. Initial analysis of the survey also produced results to be explored in more depth through qualitative methods.

Question order bias was considered in the drafting of discussion guides, which is where participants are primed by the words and ideas presented earlier. This was minimised by asking general unaided questions before more specific questioning and prompts. The language within the discussion guides was also chosen to mitigate acquiescence bias, where a participant demonstrates a tendency to agree with what is presented by a moderator – for example, reassurance that there were no 'right' or 'wrong' answers. The questions raised and scenarios shared were designed to prompt further reflection and discussion than could be possible in response to the survey.

In both the focus groups and interviews, the researchers focused on phrasing questions to show it was possible for participants to answer in a way that might not be considered socially desirable, with participants reassured that their views were welcome. Social desirability bias is where participants answer questions in a way that they think would be viewed favourably by others, such as the interviewer, other participants in the focus groups or the research commissioner (also known as sponsor bias).

The phrasing of questions was not the only mitigation against response bias. Participants were assured about the handling of their data, including that their anonymity would be protected and that they would not be identified in reporting. This assurance was given in the information leaflets and privacy notices provided to participants, as well as verbally by the researchers.

All of the researchers were experienced and trained in conducting qualitative research on challenging topics. All qualitative research activities included a researcher from the Handa CSTPV who had specialist experience and expertise in conducting primary research on the topic of extremism and the related topic of terrorism.

Focus groups

The focus groups lasted 90 minutes. They took place on a range of days and times in order to encourage participation from people with different commitments and routines.

The focus group discussions were jointly facilitated by researchers from the Diffley Partnership and the Handa CSTPV, with a member of the Diffley Partnership asking the main questions and a member of the Handa CSTPV asking follow-up questions.

It was crucial for the research team to enable participants to speak freely without risk of reprisal in order to give them the opportunity to share their honest perceptions. At the same time, it was important to facilitate the focus group in a way that precluded one participant upsetting another through their choice of words or their views. In practice there was discussion and debate, but no apparent or reported stress or upset. The researchers followed up with participants individually to ensure they were not negatively impacted by their participation, and to offer support if necessary.

Results from the survey influenced the split of focus groups. Higher proportions of females than males responded 'don't know' to many survey questions. Therefore, a decision was taken to include a female-only focus group, allowing participants in this focus group time to reflect upon uncertainty and express their views. Furthermore, analysis of the survey indicated differences in responses between age groups. Therefore, the decision was taken to also include focus groups split by age.

The five focus groups therefore consisted of:

1. Over 55 years, mixed sex

2. Under 55 years, mixed sex

3. Mixed age groups, mixed sex

4. Mixed age groups, female only

5. Mixed age groups, mixed sex

Interviews

As noted above, subsequent to the focus groups eight interviews were conducted with participants who had already taken part in a focus group. As with the focus groups, these took place over Zoom at times to suit the participants. Each interview lasted around an hour.

The interviews provided an opportunity to follow up with individuals with a diverse range of perspectives and experiences. An interview setting can be more comfortable for individuals than a group setting, depending on individual preferences. The researchers conducting the interviews were mindful of the need to allow participants time to consider and express their views, whilst ensuring they did not feel pressured or burdened by their participation.

Analysis

Following the completion of qualitative fieldwork, transcripts were compiled, studied in-depth and thematically analysed to identify patterns in the data. From this analysis process a series of themes emerged, and a systematic multi-stage coding process was undertaken to code particular qualitative contributions in relation to the research topics. This process allowed the researchers to identify key insights, to draw out important nuances within the qualitative material, and to relate findings to the research questions set out at the start of the research.

To ensure the identity of participants was protected during the analysis stage, personal details such as full names, addresses, and contact details were only held by the Diffley Partnership. Across the research team the first names and key demographic details of participants were known and referred to within analysis.

2.6 Limitations and methodological notes

This research has several limitations. Biases are present in all forms of research, and the research team sought to minimise these in relation to this project as far as possible as described above. In addition to those outlined above, it should also be noted that when asked for views about past events, responses are subject to recall bias. This means that people sometimes forget certain things or do not remember past events accurately. The retelling of past events can be influenced by a variety of factors, such as a research participant's state of mind or more recent events that change attitudes or memories in hindsight.

In relation to the qualitative research specifically, it is important to note that the purpose of qualitative research is not to achieve a sample that is representative of the wider population in a statistical sense, which was the aim of the survey (Silverman, 2021). Rather, the aim of the qualitative research was to capture the views and experiences of a small sample of the public in detail. To elicit as broad a range of views as possible, quotas were set to ensure diversity in the characteristics of participants (including minimum numbers of females, people identifying as religious and people from BAME groups). However, because of the small sample size, quotas could not be set to ensure representation of the full range of demographic groups that reside in Scotland (see Appendix C for the demographic characteristics of the qualitative participants). For example, although there was a quota for people identifying as religious, the final sample only contained participants identifying as belonging to Christian denominations and not identifying with any religion. Further research may wish to explore the views and experiences of particular demographic groups in more detail than has been possible in this research.

A further point of note relates to the fact that no definition of extremism has been adopted in this research. Knight and Keatley (2020) highlight that in the absence of an agreed definition of extremism, studies often rely on different definitions and understandings, meaning they cannot be readily and robustly compared. In this research, no definition of extremism was presented to participants; rather, the study sought to explore how participants themselves understand and define this concept, before exploring their perceptions of the threat and experiences of extremism. This means that the comparability of this research to other studies, which rely on particular definitions of extremism, is limited.