Refugee integration - role of local authorities: research

2. Methodology

This chapter provides an overview of the research methodology behind this report. The research approach can be divided into three main components:

• A survey was developed and distributed to staff and partners in all 32 of Scotland’s local authorities. The survey questions aimed to gather insights about the different perspectives, realities, and challenges involving refugee integration and humanitarian protection work across Scotland, drawing out the breadth of the work, and the contrasts between different local authorities.
• Case studies were carried out in three local authorities, involving more in-depth engagement with how local authorities deliver integration work in Scotland.
• An online workshop was held with local authority resettlement officers to discuss the provisional findings and recommendations of the research.

Survey research

Researchers developed and conducted an online survey to capture the diversity of experiences across all 32 of Scotland’s local authorities and explore specific issues and policy areas faced by local authorities in supporting refugee integration. The survey was designed to be completed by those with direct experience of working on the development and implementation of refugee integration and humanitarian protection activities at the local level. This included people directly employed by the council, as well as partner organisations in the wider community (charities, grassroots community organisations, and the private sector).

Survey design

The survey was developed and assembled over the course of several weeks. This involved producing a series of draft survey questions, which fell under five different themes and took the form of one of three question types (either a binary response, a Likert scale, or a written response). There was a rigorous process of review and reformulation of questions within the immediate IPPR research team, in liaison with COSLA and the Scottish Government.

Once the IPPR research team and COSLA and Scottish Government colleagues agreed on the full suite of questions to be presented to respondents, they then translated the survey into an appropriate and functional electronic format. The survey was created through the software Survey Hero (chosen because of variety of question types and readability), which allowed for the survey to be circulated by electronic link to participants, answered, and data collected and collated centrally.

The survey itself was broken down into five main research themes:

1. Gaining an understanding of the current picture on the ground in the delivery of humanitarian protection programmes and refugee integration.

2. Uncovering where local authorities have found successes and faced challenges.

3. Examining how different experiences compare across Scotland’s 32 local authorities.

4. Determining the impact of policy and legislation on local authorities’ refugee integration strategies.

5. Exploring how local authorities might change and adapt their approach in the future.

Consisting of 48 questions in total, these included a mixture of multiple-choice questions, many on a 5-point Likert scale, and open-ended questions which allowed respondents to enter figures or provide more detailed written responses. Some of the questions sought to establish particular facts or figures relating to a local authority’s refugee strategy. Others focused on how individual respondents rated different aspects of integration particular to their respective local authority, or gauged how respondents evaluated national policy and legislation. For a full list of the questions in the survey, please see Appendix A.

The questions were designed to be answered from different local perspectives. Respondents could choose not to answer questions where they lacked relevant knowledge or experience, either by leaving a question blank or selecting ‘Don’t know’ when a question required a response to continue. Certain questions, however, drew upon information which could only be reasonably expected to be answered by local authority employees (e.g. question 3 asked how many refugees have been resettled in a local authority in the last 12 months, and question 4 asked for the number of ‘full-time equivalent staff’ working directly for their local council on the resettlement and integration of refugees and people seeking asylum. The survey was designed so that those two questions could only be answered by respondents who selected in the initial establishing questions ‘Local government’ as their employer. This was to avoid respondents working outside local government, who are not privy to this kind of information, inputting answers.

Fieldwork

With the help of partners in COSLA and the Scottish Government, researchers asked resettlement leads across all 32 local authorities to distribute the electronic survey to individuals and organisations involved in different aspects of resettlement and integration work within their respective local authority, including those working directly for local authorities and those working for partner organisations. Responses were collected between the end of July and mid-October 2022, with a target to gather a minimum of 100 total responses from across Scotland. As the survey was conducted during the summer holidays and during a time of intense pressure facing local authorities responding to the Ukraine crisis, the timeline was extended beyond the anticipated two months to achieve the 100-response target and get responses from all 32 local authorities.

During the period the survey was live, the researchers regularly monitored the response rates from different local authorities to identify which local authorities were submitting complete responses, and which were lagging (proportionate to the size and resource available to local authorities). At a minimum, researchers established that all local authorities should submit one complete response from someone who is directly employed by the local authority, and could be relied upon to provide accurate information. Where that was not possible, this was flagged to colleagues at COSLA and the Scottish Government who reached out to the relevant local authority and encouraged potential respondents to complete the survey.

Responses

At the close of the survey, researchers recorded a total of 103 responses registered from across Scotland. The vast majority of respondents (approximately 80%) were employed in local government. 11% worked in the third sector and 2% worked in local community organisations. A further 7% worked in ‘other’ types of organisations, including in health, education, and faith organisations.

All local authorities except one, the City of Edinburgh, were able to provide at least one submission from a respondent directly employed by the council. In order to complement the survey, virtual interviews were conducted with officers from five local authorities, including the City of Edinburgh, either as a substitute for completing the survey or to add context and detail to existing incomplete survey responses. The questions in these interviews mirrored those in the survey in order to capture similar information, as well as allowing for some more in-depth answers to give a more detailed insight into council officers’ perspectives.

In terms of the geographical spread of responses, more responses were received from local authorities with larger, urban populations – notably Glasgow and Dundee, where more than 10 people responded to the survey – while every other local authority, aside from the City of Edinburgh for which there were no survey responses, ranged between one and five responses each.

Analysis

The analysis of the survey results varied according to the type of survey question. For survey questions investigating specific factual issues restricted to local authority staff– e.g. whether a local authority has a dedicated refugee integration strategy, or the number of refugees resettled in a local authority within the last 12 months – researchers calculated a single response for each local authority based on the individual responses provided. In some cases, individual responses from the same local authority contradicted each other. In these cases, researchers chose the answer given by the respondent who, firstly, was employed within the relevant council, and secondly would be expected, given their job title, to have the greatest knowledge of the particular issue. Where it was difficult to make this judgement, an average figure was taken from the respondents employed by the council to give an approximation.

Other questions in the survey asked how individual local authorities performed in certain areas. Respondents were asked to provide an answer quantified on a 4- or 5-point Likert scale. For instance, some questions sought to quantify the level of support provided by the local authority in specific areas relating to resettlement and integration from ‘0 = No support’ to ‘4 = A great deal of support’. Others asked respondents to, on a 4-point scale, assess local authority coordination with different institutions . For analysis of these questions, researchers wanted to ensure each local authority was weighted evenly, given some local authorities had more responses than others. A mean value was therefore calculated for all the responses submitted within a single local authority, producing a single local authority response for each relevant question.

For multiple-choice questions not specifically about the delivery of services by the local authority – e.g. those relating to the impact of policies and legislation – researchers analysed the results at an individual response level, without accounting for disparities in the number of responses between local authorities. This was because these questions did not focus on the support provided by local authorities across Scotland, so it was determined that it was not necessary to ensure that each local authority was weighted evenly in the analysis.

For the open-ended survey questions, written questions were posed to allow respondents to provide additional context and detail to preceding multiple choice, factual, or Likert-scale questions. At the close of the survey, researchers were able to extract written responses into a separate word processing document per question. Acting as a log of written responses, full submissions by individual respondents could be cross referenced against the profession of the person making the submission, and by their associated local authority. This allowed for analysis of responses, question-by-question, complementing the analysis of non-written responses.

Case study research

Site selection

The qualitative research was carried out between June and November 2022 in three case study areas. The case study research sought to illustrate an in-depth and nuanced picture of humanitarian protection and refugee integration across Scotland. With advisory input from COSLA and Scottish Government, a shortlisting process aimed to select three areas that were diverse in i) their geography, ii) their histories of resettlement and migration, and iii) their approach to humanitarian protection and refugee integration.

Initially, it was intended that the researchers would work with local authority areas that broadly fell under the following three categories: ‘rural’, ‘urban’ and ‘partnership working’. The first two categories reflected the ambition to select areas that were geographically diverse and that had different histories of resettlement and migration. The third sought to spotlight the partnership approach taken to humanitarian protection and refugee integration by neighbouring local authority areas. In selecting candidates to approach for inclusion in the case study research, consideration was also given to known capacity and workload challenges, as well as existing research taking place in particular areas, to avoid overburdening these local authorities. For this reason, for instance, Glasgow was not approached to be the ‘urban’ case study area.

Fieldwork

Following selection on the above criteria, local authorities were invited by email to participate in the research and/or to attend an introductory meeting with a researcher to learn more about the project and the requirements of the research. Both the shortlisted options for the ‘rural’ and ‘urban’ areas agreed to participate in the research – Aberdeenshire for the former and Dundee for the latter. However, due to capacity and workload issues, brought about by the complexity of humanitarian protection programmes at the time of the fieldwork, the local authorities identified as candidates for the ‘partnership working’ category were unable to participate in the case study research. In order to secure a third case study area, an open invitation to participate was shared with all local authorities. This led to the participation of Na h-Eileanan Siar, a valuable addition for the opportunity to understand humanitarian protection and refugee integration work in a more remote context.

For each of the three case studies, the researchers planned to conduct:

• One-to-one interviews with approximately six stakeholders within each respective local authority with senior responsibility for the delivery of refugee integration and humanitarian protection work.
• A focus group for each case study area with a maximum of six people with direct experience of working on the ground in refugee integration.
• Four one-to-one interviews with refugees and/or people seeking asylum living in the case study area and who have experience of receiving support within the local authority area.

In the first instance, stakeholders with senior responsibility for refugee integration in each case study area were identified by COSLA, and researchers were introduced to these individuals to set up an online interview (via Teams). Further participants were identified through a snowball approach, with all participants asked if they could identify other people either with senior responsibility for refugee integration or who were involved in the direct delivery of refugee integration work. This proved a successful method and highlighted the close-knit nature of much refugee integration work, as multiple participants suggested inviting the same individuals to participate. Prior to interview, all participants received an information sheet and were asked to complete an online form to record their consent for participating (see Appendix B). All stakeholder interviews were conducted online via Teams, while interviews with refugees were held either via Teams or on the phone.

Refugees were recruited by first asking stakeholders to ask people that they worked with whether they would be interested in speaking with a researcher about their experiences of integration in Scotland. Contact details of those who agreed were shared with the research team and interview dates arranged directly with participants. Participants received a copy of the information sheet (see Appendix C), were asked if they would like an interpreter present at interview, and were offered a £40 supermarket voucher as a thank-you for their contribution to the research.

In Aberdeenshire, researchers held six interviews with eight stakeholders, one focus group with five stakeholders, and four interviews with five people with lived experience. In Dundee, researchers held five stakeholder interviews, one focus group with five stakeholders, and interviewed four people with lived experience. In Na h-Eileanan Siar, researchers held five interviews with stakeholders, and interviewed one person with lived experience. In total, across the three case study areas, researchers spoke with 28 stakeholders and interviewed 10 resettled refugees. Stakeholders interviewed included those responsible for policy and/or operational matters and people delivering support on the ground. The research team faced challenges in recruiting a greater number of people with lived experience in Na h-Eileanan Siar due to the small numbers of people resettled there. Similarly, a smaller pool of people working on refugee integration on the islands meant that they were unable to recruit stakeholders to a focus group.

Research tools

Topic guides were developed through an iterative process, with a draft version shared with COSLA and the Scottish Government for input and feedback. The topic guides broadly covered:

• What refugee integration provision looked like in the case study area
• Identification of successes and challenges
• The unique attributes of the case study area that support or pose a challenge to successful refugee integration
• The impact of policy and legislation on refugee integration
• Suggestions for changes that could promote improved refugee integration outcomes.

Interviews were semi-structured, and the topic guides were intended to aid discussion rather than be prescriptive. Following early interviews, researchers made the decision to reduce the number of questions in the interview, in order to avoid repetition and to prevent the interviews from being overly long (in light of the workload pressures on many of those being interviewed). For the full topic guide see Appendix D. For those taking part in a focus group the interview questions were slightly adapted and shortened to accommodate multiple participants.

Refugees who were interviewed were asked about their experiences of integration support, challenges and successes faced, suggestions for policy and practice, and their hopes for the future. There was a focus on understanding who had supported them in their integration journey so far, how they felt about the support or services received, and the extent to which integration support matched up with their expectations. For further details see Appendix E.

Ethical considerations

A number of ethical measures were put in place to support and protect the rights and dignity of participants throughout the research process. In particular, for participating refugees, researchers took the following steps:

• Offered to translate or interpret the information sheet and privacy notice into their preferred language, to ensure consent to participate was informed and freely given.
• Asked partner organisations supporting researchers with recruitment to identify people who they anticipated would be appropriate to invite to participate, in particular taking into consideration any acute difficulties or challenges they were currently facing.
• Agreed that participants who had identified support needs during the course of the interview should receive a follow-up call from the referring organisation to discuss these further (with permission of the participant).
• Reiterated the rights of participants at the beginning of the interview, particularly in relation to being able to withdraw without penalty and that they were free to decide not to answer a question should they prefer. Participants received an information sheet assuring them that researchers were independent, that only anonymised data would be shared with Scottish Government, and that their participation would in no way impact on services they receive. They were also advised to take a break should they need one.
• Informed participants of the safeguarding protocol that researchers would follow, in line with IPPR’s safeguarding policy.

For further details about the ethical considerations and measures put in place, see the ‘ethics statement’ in Appendix F.

Analysis

Analysis of interviews and focus group transcripts were conducted by the research team using an a-priori code list drawn from the topic guide and with reference to the overall research aims. The research team discussed their individual analyses as a group and compared findings in each case study area against one another to understand common themes, differences and key learnings across the case study areas.

Policy workshop

Towards the end of the project in October 2022 a virtual workshop was held with refugee resettlement staff from across different local authorities in Scotland. This workshop helped to test out initial research findings and inform some of the policy implications discussed in the final chapter of this report. The workshop was divided into two halves, with each half comprised of a briefing of the research, followed by a ‘breakout discussion’. For breakout discussions the participants were divided into three groups and invited to respond to findings and questions posed by one of the researchers who hosted the discussion.

The first portion touched on the successes and challenges faced by local authorities around ESOL provision, skills and employability support, housing, and how local authorities’ models of resettlement responded to recent pressures on Syria, Covid-19 and Ukraine. The session also fed back to participants what the research found regarding how varying geographies impacted resettlement efforts. During the breakout session participants were asked what surprised them about the research findings and what from the perspective of their local authority was missing.

The second portion turned to what the research team had determined to be the implications for policy moving forward. Several ideas were presented to participants, among them that resettlement teams should be granted core status within local authorities, and a more goal-oriented New Scots Refugee Integration Strategy with a mechanism to hold local authorities to account on New Scots. Following the briefing, participants were asked what they thought of the various policy implications and recommendations presented, what other recommendations the research team should consider, and hopes for the next iteration of the strategy.

Researchers, alongside colleagues from COSLA who were also in attendance during the discussion sessions, took note of the proceedings and what participants shared to inform the policy discussion and final recommendations in this report.

Methodological limitations

There are a number of methodological limitations of the survey which are important to note. First, the survey does not capture a representative sample of respondents and so should not be interpreted as reflecting the opinion of the total population of staff members of local authority and partner organisations working on refugee integration across Scotland.

Second, in spite of considerable efforts, researchers struggled to secure the desired response rate, and some responses were incomplete. Moreover, among the responses received, there was an uneven number of responses received per local authority. As a result, the survey results should be understood as illustrative of the different experiences of local authorities and partner organisations in delivering humanitarian protection programmes and refugee integration work and not as fully representative of the whole population of local authority staff and partner activities across Scotland. While researchers have worked hard to ensure all local authorities are represented in the survey – and have in some cases corrected skews towards certain local authorities through weighting the results as discussed above – the limitations of the fieldwork should be taken into account when interpreting the quantitative findings.

On the case study research, and as noted above, key limitations arose in relation to the third case study area. First, researchers were unable to recruit the ideal third case study which would have highlighted the partnership working of two or more neighbouring local authorities. Second, within Na h-Eileanan Siar, researchers were unable to engage the proposed number of stakeholders and refugees. This is likely due to the relatively small size and scale of refugee integration work on the islands, and the smaller numbers of refugees resettled there. In addition, delays in securing this third case study area meant that researchers had less time to undertake recruitment in this location. Despite this limitation, interviews with stakeholders point to similar challenges and successes – which suggest that researchers did manage to get a fair understanding of the experiences of refugee integration in this local authority area.

Finally, limitations are likely to have arisen from the Covid-safety measures that researchers were required to adhere to during the project, which prevented researchers from visiting case study areas to conduct research. While this is a relatively small limitation, rapport and trust with participants and the richness of data may have been impacted by conducting interviews and focus groups online or via telephone compared with in-person.