Scottish Mentoring and Leadership Programme: evaluation - final report
Scottish Mentoring and Leadership Programme (SMLP) supports care-experienced and disadvantaged young people through MCR Pathways mentoring and Columba 1400 leadership. The evaluation found improved confidence, attainment, school retention and post-school outcomes, with MCR providing strong value for money.
Chapter 2: Methods
Methodology
This report uses qualitative and quantitative data to explore process, outcome and Value for Money findings, addressing the evaluation questions in relation to the reach, implementation and impact of the programme.
A short overview of the methods used is provided in this chapter, and further details for each can be found in previously published reports which are signposted.
Qualitative fieldwork
The qualitative fieldwork was undertaken in two waves:
- Wave one (November 2023 – February 2024): 75 qualitative depth interviews were conducted with pupils, parents/carers, MCR mentors[14], Columba 1400 delivery staff, and school staff, in relation to either the Columba 1400 Young Peoples’ Leadership Academy or MCR Pathways, as well as with local authority and national stakeholder representatives. More details on this wave of research can be found in the interim report.
- Wave two (November 2025 – February 2026): where possible, the same pupils, parents/carers and teachers who took part in wave one of the research were re-interviewed two years later. Interviews were also undertaken with different local authority and national stakeholder representatives. Additional interviews were also undertaken with young people who had taken part in a Columba 1400 YPLA with a place-based element, along with staff from the partner organisations and Columba 1400 staff involved in the delivery of the place-based approach. In total, 51 interviews were conducted in wave two.
Four schools were visited in the first wave of qualitative research: two taking part in MCR Pathways and two taking part in the Columba 1400 YPLA. The schools were selected with the aim of achieving a mix in terms of local authority area, geographic location, rurality, proportion of care experienced pupils, proportion of pupils living in the most deprived areas (based on the Scottish Index of Multiple Deprivation - SIMD), school size, and denomination. MCR schools were also selected on the basis that they had begun the scheme in 2021/22 or earlier in order that there were pupils at the school who had taken part in the group work element as well as those who had taken part in mentoring. Further details of how the schools were selected and pupils recruited in wave one can be found in the interim report. Note that minority ethnic pupils were included in the qualitative research but the numbers were insufficient to identify any clear differences between them and non-ethnic minority pupils.
All four schools were revisited in wave two to explore longer-term impacts with individuals who had consented to recontact. Those who had left school since wave one were invited to take part in a virtual interview via video call, however, very few young people responded to this invite.
In addition, two additional case studies were conducted in wave two to explore the Columba 1400 YPLA with a place-based element. A list of examples was provided by Columba 1400 staff, and case studies were selected on the basis that they took place in different locations, were recently completed, and had reached distinct groups of young people.
Representatives from local authorities and national stakeholders who were involved in implementation or oversight of the interventions(s) were suggested by the Scottish Government, MCR Pathways and Columba 1400. Different local authority representatives and national stakeholders were approached in each wave to provide a wider range of perspectives. Stakeholder interviews included:
- 10 national stakeholders (eight in wave one and two in wave two), including Scottish Government officials and non-departmental bodies associated with the programme
- Seven local authority representatives in areas delivering either one or both of the interventions (four in wave one and three in wave two)
- Seven MCR Pathways or Columba 1400 staff members (six in wave one and one in wave two).
MCR schools
MCR fieldwork covered two schools, each visited twice. A summary of interviews by wave is provided in the table below.
MCR schools summary
| Wave | Participant group | Details | Number of interviews |
|---|---|---|---|
| One (2023/24) | Pupils | 11 girls, 9 boys | 20 |
| One (2023/24) | Pupils | 10 in S2 (group work only), 10 in S4 (received mentoring but not done group work because MCR Pathways had not started when they were in S1/S2) | As above |
| One (2023/24) | Pupils | 9 in MCR Group One (care experienced young people), 11 in MCR Group Two (other vulnerabilities/reasons, e.g. being young carers or being at risk of becoming care experienced) | As above |
| One (2023/24) | School staff (depute headteachers, MCR Co-ordinators) | N/A | 4 |
| One (2023/24) | Mentors | N/A | 6 |
| One (2023/24) | Parents/carers | N/A | 2 |
| One (2023/24) | Total | Wave One Total | 32 |
| Two (2025/26) | Pupils (all except one were follow-up interviews and all were still receiving mentoring) | 5 girls, 2 boys | 7 |
| Two (2025/26) | Pupils | 2 in S4, 1 in S5, 3 in S6, 1 school leaver (at college but still receiving mentoring) | As above |
| Two (2025/26) | Pupils | 4 in MCR Group One, 3 in MCR Group Two | As above |
| Two (2025/26) | School staff (headteachers, depute headteachers, MCR Co-ordinators) | N/A | 5 |
| Two (2025/26) | Total | Wave Two Total | 12 |
Columba 1400 schools
Columba 1400 fieldwork covered two schools, each visited twice. Overall, 18 participants who took part in wave one also took part in wave two (including 12 pupils). Four pupils had moved or left school since taking part in wave one and did not respond to the wave two invitation. One additional pupil was interviewed in wave two who had not been interviewed in wave one.
A summary of interviews by wave is provided in the table below.
Columba 1400 schools summary
| Wave | Participant group | Details | Number of interviews |
|---|---|---|---|
| One (2023/24) | Pupils | 5 girls, 11 boys | 16 |
| One (2023/24) | Pupils | 6 in S3 (recently returned from the residential element),[15] 10 in S4 (been on the residential element the year before) | As above |
| One (2023/24) | Pupils | 1 care experienced young person, 6 young people selected due to low social confidence, 9 selected for other reasons | As above |
| One (2023/24) | School staff (headteachers, teachers) | N/A | 4 |
| One (2023/24) | Parents/carers | N/A | 4 |
| One (2023/24) | Total | Wave One Total | 24 |
| Two (2025/26) | Pupils (all follow-up interviews except one) | 4 girls, 9 boys | 13 |
| Two (2025/26) | Pupils | 6 in S5, 7 in S6 | As above |
| Two (2025/26) | Pupils | 1 care experienced young person, 4 young people selected due to low social confidence, 8 selected for other reasons | As above |
| Two (2025/26) | School staff (headteachers, teachers) | N/A | 3 |
| Two (2025/26) | Parents/carers | N/A | 3 |
| Two (2025/26) | Total | Wave Two Total | 19 |
Columba 1400 YPLA place-based case studies
The Columba 1400 YPLA place-based fieldwork in wave two involved two case studies. Over the two case studies, interviews were conducted with nine young people, one parent/carer, four staff members from the partner organisations, and one Columba 1400 staff member. In one case study, the YPLA had taken place two years prior to fieldwork, in 2024, with young people aged between 18 and 22 (all had left school when they took part in the YPLA, with some in college or working). In the other case study, the YPLA took place 18 months prior to fieldwork, between 2024 and 2025, with young people across several schools and year groups (between S1 and S5).
Data collection and analysis
The majority of interviews were conducted face-to-face at schools or at partner organisation offices (including interviews with young people, school or organisation staff, mentors, MCR coordinators and parents/carers in wave one). Interviews with Columba 1400 staff, local authority staff, national stakeholders and some interviews with parents and mentors were conducted by phone or video call.
Interviews explored different aspects of the SMLP, including perspectives on the programme aims and delivery, perceived impact of the programme, the extent to which these were sustained (in wave two), what had worked well and what could be improved. A more detailed overview of data collection and analysis can be found in the interim report.
For the analysis, data from interviews were summarised into thematic matrices using Excel, with each column representing a theme, and each row an individual interview, so that the data could be sorted in different ways for further analysis. These were developed by the research team and drew on the evaluation questions. These matrices were then reviewed to identify the full range of views and experiences under each theme. Additional columns were added for the wave two data collection to enable analysis of whether or not impacts were sustained and identification of any new impacts arising that had not been cited in wave one.
Quantitative impact assessment
As described above, a quantitative impact assessment of the MCR Pathways intervention on pupil outcomes was carried out, in addition to the qualitative research, for the MCR Pathways programme only. The estimated impact was based on data that linked records of participation in the intervention with pupil-level data held by the Scottish Government on attendance rates, attainment, staying-on rates, and post-school destinations. The analysis is based on those pupils who were in S4 between 2014/2015 and 2021/2022. (We were not able to include later cohorts as it takes time to see the results in S5/6 and for leaver destinatios).
The impact evaluation was conducted using five analytical approaches, with the three most robust approaches used to estimate the impact of the intervention on outcomes. Outcomes for participating pupils and schools were compared against outcomes for pupils and schools where there was no MCR Pathways participation, with the application of statistical matching methods ensuring comparisons were made between similar groups. Full technical details are provided in the SMLP Impact and Value for Money Evaluation report.
Value for Money assessment
The Value for Money assessment estimates the costs of delivering MCR Pathways against the monetised benefits, which includes the benefits of improved educational attainment, reduced unemployment (9 months after leaving school), and avoided school exclusions. This assessment also included the wellbeing gains of the volunteer mentors, acknowledging the role volunteers play in the MCR Pathways intervention in supporting the pupils they mentor on a one-to-one basis. Many of these benefits, such as the benefits of improved educational attainment to lifetime earnings, will be realised a considerable time after the evaluation period, as they are calculated over the pupil’s lifetime.
These estimates of monetary value underpin a Social Cost Benefit Analysis, which assesses the impact of the intervention on social welfare in monetary terms, with all relevant costs and benefits valued (where proportionate and possible) and non-monetised impacts considered.
Two metrics were calculated – the ratio of benefits to costs, the Benefit Cost Ratio (BCR), and the value of benefits minus the costs, the Net Present Value (NPV). All BCR and NPV estimates are consistent with HM Treasury Green Book Guidance on assessing Value for Money. Again, full technical details are provided in the SMLP Impact and Value for Money Evaluation report.