Fair Start Scotland Evaluation Report - Year 6: Phone Survey with Service Participants - March 2026

Chapter Two: Methodology

Research aims

A telephone survey was carried out with FSS participants, based on the methodology from previous waves of research. This quantitative research aims to broadly explore FSS service participants’ experiences and associated outcomes for Year 6 (2023-24 cohort) compared to Year 4 and 5 (2021-22 cohort), and previous cohorts where methodologically reliable. This research incorporated a particular focus on previously identified groups who have significant barriers to employment and are further from the labour market. This includes participants from families at highest risk of child poverty, participants from a minority ethnic background, disabled participants, participants living in a deprived area and those living in a rural area.

Survey methodology

FSS participants who took part in the surveys are referred to as “cohorts” according to the calendar year they joined FSS. From the Wave 2 survey onwards, each survey had a longitudinal element, meaning that a proportion of earlier cohorts were recontacted. Figure 2 below shows the number of participants who were surveyed over the five survey waves and the sample sizes.

Note: Figure 2 shows the number of participants surveyed at each Wave. It does not indicate how these sample sizes were calculated or what proportion they represent of the total FSS population. Details on sampling are provided in the Technical Appendix.

Table 1 below compares the different waves of the survey with reference to estimated average time gap between the point at which participants first joined FSS and when they completed the survey. It shows that while waves 1 to 4 have a similar time gap of around one year, Wave 5 has a longer time gap, at around a year and a half. This means that some findings may not be reliably compared between waves, most notably employment status and work outcomes. This is because participants interviewed at Wave 5 will have had more time after starting FSS support to change their work situation compared to participants interviewed at waves 1 to 4. That said, some measures are not expected to be affected by this longer time gap, including the profile of FSS participants and experiences of support. Throughout this report, comparisons to previous waves have been made based on whether measures can reasonably be expected not to be affected by this longer time gap. A comprehensive list of reported measures alongside this reasoning is provided in the Technical Appendix.

Table 1: Comparison of Fair Start Scotland start timing and survey timing for all cohorts

For Wave 5, a sample of 469 participants from the 2023-24 cohort completed the survey. 200 participants from the 2021-22 cohort (who had previously completed the survey in Wave 4) completed the survey in Wave 5. The Technical Appendix provides details around the number of eligible participants, the number who were contacted, how participants were approached to participate, and further details like average survey length.

6,655 participants belonging to the 2023-24 cohort were identified to be contacted. Specifically, these were participants who started FSS between October 2023 and March 2024, to provide the closest comparison to previous waves (as show in Table 1, the maximum time gap between starting FSS and completing the survey is comparable to the 2021-22 cohort)^[12]. An additonal 750 participants belonging to the 2021-2022 cohort (i.e. those who took part in the Wave 4 survey) were identified for the purpose of the longitudinal part of the survey.

This report details findings from Wave 5 of the survey that was carried out in June and July 2025. Chapters 3-7 focus predominantly on participants from the 2023-24 cohort. Within each chapter comparisons are made between the current 2023-24 cohort and past cohorts (i.e. the 2018, 2019, 2020 and 2021-22 cohorts) where relevant and where methodologically reliable.

Chapter 8 focuses on the experiences of the 2021-22 cohort who were first surveyed between December 2022 and January 2023 as part of the Wave 4 survey. This chapter compares this cohort’s answers from the Wave 4 survey and the Wave 5 survey, in order to assess key measures over time and examine longitudinal outcomes. It should be noted that the base size for the 2021-22 cohort (200 participants) is low, so subgroup comparisons should be treated with caution.

Differences between the different year groups, survey waves and subgroups within them have been tested, and only those that are significant are reported on. Generally, questions where the base size was lower than 30 were not used. For a full explanation of the significance testing undertaken, and a full list of subgroups tested, please refer to the Technical Appendix.

A note on weighting

Survey answers from both the 2023-24 and 2021-22 cohorts were weighted to bring the data back in line with the original population proportions of FSS starters for this cohort, and to correct for any non-response bias^[13].

A note on eligibility for participation in Fair Start Scotland from 2021-22 onwards

In response to the labour market context during 2021-22, and participant feedback collected during the Year 3 Evaluation, some changes were made to the participation and eligibility criteria for 2021-22 onwards, as follows:

• a reduction of the criteria for “length of time unemployed” from 24 months to 12 months
• allowing a “right of return” to previous participants; we have termed this group “re-joiners” in this report
• continuing to allow participants to “pause” their engagement with the service

As a result, the profile of individuals surveyed as part of the 2021-22 and 2023-24 cohorts reflects these changes. Just over a quarter (26 per cent) of the 2021-22 cohort were re-joiners, having received support from the service perviously, whilst just under one-fifth (19 per cent) of the 2023-24 cohort were re-joiners. More information on re-joiners can be found in Chapter 3.

Limitations of the research

This research has a number of limitations, highlighted below, that should be considered when interpreting the findings.

Comparisons of findings

As shown in Table 1 earlier in this chapter, comparisons of findings between the 2023-24 cohort and previous cohorts should be approached with caution, owing to the longer time gap for the 2023-24 cohort between when participants joined FSS and when they completed the survey. In particular, comparisons of employment status and work outcomes (including quality of work measures) have not been made to previous cohorts, as the time gap difference is expected to impact the findings, and any differences cannot be reliably attributed to changes in experiences of FSS delivery. A comprehensive list of reported measures, alongside this reasoning is provided in the Technical Appendix.

Attribution to Fair Start Scotland service support

FSS participants who were in work at the time of the survey were not asked how they obtained their current job. It is therefore not possible to say whether those who were in work at the time of the survey were placed in their current job through FSS, or whether they obtained their job independently (including after having left the service). This limitation applies to both the survey of new FSS participants and the longitudinal survey.

Differences between subgroups of participants

Differences between subgroups of participants and between different waves of research were assessed using statistical significance tests. Whilst statistical significance indicates whether differences are unlikely to have occurred by chance, it does not show the size or practical importance of those differences. Effect sizes that provide information on the practical significance of findings and measure the magnitude of differences, were not analysed as part of this research. Therefore, whilst some differences are statistically significant, their real world impact may be small.

Subgroup comparisons

The number of participants interviewed in Wave 5 was lower than in previous waves of research. This means that subgroup comparisons are necessarily more limited. The statistical significance testing conducted in this evaluation takes sample size into account, so any differences between subgroups are reliable differences. However, smaller sample sizes mean that some subgroups could not be compared, obscuring potential differences in the population.

Income/earnings analysis

Analysis is based on self‑reported earnings rather than PAYE data, and therefore provides indicative rather than definitive estimates of earning the National Living Wage, the Real Living Wage and the National Minimum Wage.