Scottish Mentoring and Leadership Programme: evaluation - final report

Executive summary

Background

The Scottish Mentoring and Leadership Programme (SMLP) was established to support and improve the outcomes of care experienced young people, and those in ‘tough realities’, for example, those on the cusp of care, young carers, those from the most deprived backgrounds and those suffering from a lack of self-confidence and/or anxiety.

Central to the SMLP is the idea that young people can be empowered by developing their relationships with family, friends and adults in the wider community, and that more relational forms of support will enhance wellbeing and develop the capabilities of young people and their families, thereby improving their short, medium and long-term outcomes.

SMLP comprises two elements: MCR Pathways, a school-based mentoring scheme, and Columba 1400 Young People’s Leadership Academy, which provides values-based leadership experiences for young people.

Following early evidence from an independent evaluation^[1] that MCR Pathways had a positive impact on outcomes and capabilities, the Scottish Government and the Hunter Foundation agreed to invest additional funding to expand MCR Pathways and roll it out over a six year period (from 2021/22 to 2026/27) to a much larger cohort of young people. Alongside this, Columba 1400 would deliver their values-based Young People’s Leadership Academies (YPLA) to young people in schools. It was envisaged that the two schemes would complement each other in many schools with, for example, schools involved in MCR being encouraged to participate in Columba 1400 as well – and vice versa.

In 2022/23 The Hunter Foundation took the decision to fund only the Columba 1400 element of the programme from 2023/4 onwards and this means funding for the MCR element of the programme is now provided exclusively from SG. As a consequence, in February 2023, ministers agreed to extend the programme to 2027/28 to give local authorities more flexibility around when/if they might join the programme and to maintain progress in 2023/24 against the original policy intention of offering the mentoring component to every school that wants it. Nevertheless, while the formal structure of the SMLP was discontinued, both MCR Pathways and Columba 1400 continue to deliver on the programme objectives.

Overview of MCR Pathways

The overall aim of the MCR Pathways^[2] intervention is to increase the number of young people in full-time work, college or university after leaving school (known as ‘positive destinations’). In doing so, the scheme aims to improve attainment, and staying-on rates for S5 and S6 pupils, as well as supporting young people to develop key life skills.

Schools have an MCR Pathways Co-ordinator, employed by and located in the school, who supports pupils and mentors, working closely with other school staff. In S1 and S2, young people typically take part in weekly group work sessions with the Co-ordinator. These sessions focus on building confidence, life skills and improving literacy and numeracy.

Pupils are then matched with a mentor from S3 onwards. This provides more one-to-one support. Mentors are volunteers from the local community with a range of different backgrounds who receive training from MCR.

Overview of Columba 1400 Young People’s Leadership Academy

Columba 1400’s Young People’s Leadership Academy^[3] (YPLA) provides values-based leadership experiences for young people. The YPLA involves three phases. The purpose of Phase One is to introduce the cohort to values-based leadership, to provide opportunities for the young people to get to know each other and to help them prepare for Phase Two. Phase Two is delivered through a residential element or a place-based element.

The residential element is typically delivered in partnership with a school year group (usually S2/S3) from one school, involving a week-long residential stay, at Columba 1400’s purpose-built Community and International Leadership Centre in Staffin on the Isle of Skye.

The place-based model is a more tailored approach, delivered in the community, designed to include young people for whom the residential element would not be accessible.

The purpose of Phase Three is to further embed values-based leadership by enabling young people to put the skills they discovered in Phases One and Two into practice. The young people decide on a community-based or school-based project that they want to wok on, develop a plan for this, and deliver it together.

Aims of the evaluation

The aim of the evaluation was to assess the effectiveness of the SMLP. There were three main strands to the evaluation, which explored:

• The delivery of the programme
• The impact on ‘hard’ outcomes (meaning attendance, attainment, staying-on rates and positive post-school destinations) and on capabilities (such as increased social confidence, increased health and wellbeing, and better relationships) which young people have said matter to them
• Value for Money – assessing the costs against monetised benefits (including the longer-term benefits to wider society)^.

The impact on ‘hard’ outcomes and Value for Money was assessed for MCR Pathways care experienced pupils only as it was not possible to construct a robust control group for non-care experienced pupils on MCR Pathways or for participants in the Columba 1400 YPLA.

Methods

Quantitative impact assessment

The estimated impact of the MCR Pathways programme on ‘hard’ outcomes was based on data that linked records of participation in the programme with pupil-level data held by the Scottish Government on attendance rates, attainment, staying-on rates, and post-school destinations. The analysis is based on those who were in S4 between 2014/2015 and 2021/2022.

The impact evaluation was conducted using five analytical approaches, with the three most robust approaches used to estimate the impact of the programme on outcomes. Outcomes for participating pupils and schools were compared against outcomes for pupils and schools where there was no MCR Pathways participation, with the application of statistical matching methods to ensure that comparisons were made between similar groups. Full technical details are provided in the SMLP Impact and Value for Money Evaluation report.

Value for Money assessment

The Value for Money assessment estimates the costs of delivering MCR Pathways against the monetised benefits found in the quantitative impact assessment, including the benefits of improved educational attainment, reduced unemployment, and avoided school exclusions. This assessment also included the wellbeing gains of volunteers in the intervention, acknowledging the role that volunteer mentors play in MCR Pathways. Many of these benefits, such as the benefits of improved educational attainment to lifetime earnings, will be realised a considerable time after the evaluation period.

These estimates of benefits underpin a Social Cost Benefit Analysis, which assesses the impact of the programme on social welfare, with all relevant costs and benefits valued in monetary terms (where proportionate and possible). Non-monetised benefits are also considered.

Two metrics were produced – the Benefit Cost Ratio (BCR), comparing the monetised value of benefits as a proportion of costs, and the Net Present Value (NPV), the discounted sum of benefits in monetary terms less the total costs, accounting for inflation.. All BCR and NPV estimates are consistent with HM Treasury Green Book Guidance on assessing Value for Money. Again, full technical details are provided in the SMLP Impact and Value for Money Evaluation report. The Value for Money assessment did not include the Columba 1400 element or the MCR Pathways element offered to non-care experienced pupils in ‘tough realities’. It was not possible to identify suitable control groups to assess the impact with sufficient robustness due to varying inclusion criteria across schools.

Qualitative process and impact assessment

The evaluation used qualitative depth interviews to explore both process and outcome findings in relation to the reach, implementation and impact of the programme on the capabilities of young people. The qualitative research was conducted over two waves (wave one took place from November 2023 to February 2024 and wave two took place two years later, from November 2025 to February 2026). In total, 126 participants were interviewed including young people taking part in the programme; their parents/carers; school staff; managers and delivery staff from MCR and Columba 1400; local authority representatives and national stakeholders. The wave two interviews with young people mostly followed up with the same young people who had taken part in wave one. This was largely to assess whether the benefits identified in wave one had been sustained.

The qualitative evidence presented in the report is valuable for understanding participants' experiences, perceptions and the mechanisms through which the interventions may have influenced outcomes. However, qualitative findings are not statistically representative and should not be interpreted as evidence of the scale or prevalence of impacts across participants. Nor can they, in isolation, establish causal attribution between the interventions and observed changes.

Key findings

Qualitative process findings: delivery of the interventions

School staff and wider stakeholders generally agreed that the interventions were reaching the young people who would benefit the most. Stakeholders highlighted the importance of school staff having the flexibility to decide which pupils take part, and trusting schools to make the right decisions.

However, there were some challenges to overcome (for both interventions) in reaching a greater number of care experienced young people who have lower attendance at school.

Both interventions have effectively engaged with and supported young people. In MCR Pathways, engagement is encouraged by a positive relationship between the school Co-ordinator and young people, and a thorough mentor matching process. In both the residential and place-based Columba 1400 YPLAs, the professionalism and skills of Columba 1400 staff were seen as key to this engagement.

Relationships with other initiatives

MCR was generally felt to have a positive working relationship with other initiatives supporting young people, specifically Developing the Young Workforce, Skills Development Scotland, and the Promise. However, some stakeholders compared MCR Pathways with other programmes (particularly Career Ready) and felt there was some duplication because these interventions were available to similar cohorts of young people and had similar aims.

Columba 1400 staff also mentioned engagement with other organisations supporting young people, including involvement with Developing the Young Workforce’s ‘World of Work’ annual event and conversations with Skills Development Scotland about developing a ‘leadership passport’.

Changes and improvements over time

There were examples from both interventions of improvements to delivery, based on MCR and Columba 1400 staff reflecting on how things could be better or based on feedback from schools, pupils and delivery staff.

For MCR, this included being more flexible with the model e.g. allowing Co-ordinators to cover more than one school and dropping the S1 and S2 group work to make this feasible. This reduced the costs for the school/local authority and made the intervention more sustainable. MCR are also expanding into further education colleges to allow existing mentoring relationships to continue post-school and thereby help sustain positive destinations.

For Columba 1400, the biggest addition has been the introduction of the place-based model, alongside the YPLA with a residential element. In wave one, it was recognised that the residential element of the YPLA was a barrier for those in challenging circumstances for whom going away would be impractical or inappropriate. Columba 1400 staff noted that the place-based academy (a new approach which involves activities within the local community and does not include a residential element) had been developed to engage with these young people. Evidence gathered from the place-based case studies in wave two suggested that this alternative approach to delivering the YPLA had been successful in reaching different groups of young people (i.e. those for whom the residential would not be accessible), enabling a wider range of young people to take part and benefit from the YPLA.

Has the programme been implemented as intended?

MCR Pathways was generally felt to have been implemented as intended.

It was also generally agreed that the Columba 1400 YPLAs have been implemented as intended. A number of key considerations were highlighted for ensuring successful delivery of the YPLAs, including:

• Setting clear expectations with key adults supporting the delivery of the YPLA, whether in schools or other organisations. This was felt to be critical to ensuring that the impacts on young people are sustained over the longer term.
• Making the most of Phase Three, as this was highlighted as a key stage for embedding and strengthening capabilities. Maintaining a flexible approach to Phase Three was also felt to be important, as different groups have different capacities for undertaking project work. Since 2023, Columba 1400 has been operating an enhanced Phase Three. This involves those who are not quite ready to lead others engaging in one-to-one coaching with a facilitator to continue to develop their leadership of self skills.

Qualitative impact evaluation

Columba 1400 Young People’s Leadership Academy impacts

Participants were very positive about the Columba 1400 Young People’s Leadership Academy (YPLA) overall, describing the impact it had as ‘powerful’, ‘affirming’, ‘life-changing’ and ‘transformational’.

Increased social confidence and communication skills were common themes when discussing the overall impacts of the Columba 14000 YPLA. Pupils attributed this increased confidence to the amount of group work that they experienced in Phases One and Two, which encouraged them to speak to their peers and helped them feel more comfortable in group settings. Staff also mentioned that some pupils had been chosen to take part because they were struggling with confidence, and they had observed significant changes in those pupils. These impacts had mostly been sustained at wave 2 and in some cases had strengthened over time.

Pupils also highlighted how the YPLA had improved their self-esteem and sense of belief in their abilities. They described having more confidence in themselves to contribute at school and to socialise or to try new things outside of school. Parents/carers and staff also observed more self-assuredness in the young people who had lacked this before taking part in Columba 1400’s YPLA. In general, these impacts had been sustained two years later.

As with the YPLA with a residential element, increased social confidence was one of the main impacts of the place-based model – and this was felt to have underpinned many of the other capabilities that young people felt the YPLA had improved, such as belief in their abilities and enhanced social capital.

MCR Pathways impacts

Qualitative research with pupils, parents, mentors and school staff explored the impact of the programme on the capabilities which young people have said matter to them. Increased confidence was a dominant theme when discussing the key impacts of MCR Pathways on young people, particularly in relation to increased social confidence and more belief in their own abilities. This increase in confidence was attributed, in part, to encouragement from mentors and MCR Co-ordinators. These individuals were described as ‘patient’, ‘non-judgemental’ and ‘reliable’ which allowed them to build strong relationships with pupils. Follow-up research with pupils two years later showed that these impacts were clearly sustained over time.

Other improvements as a result of the intervention, which different MCR pupils saw as having the most impact on them personally, were:

• Feeling less anxious
• Improved attendance at school
• Feeling calmer
• Feeling less angry
• Feeling more confident about doing well at a job in the future.

However, there were some pupils for whom mentoring seemed to make little difference. In some cases, pupils felt they already had the capabilities asked about and so their perspective was that mentoring had made no difference to them. Mentors also indicated that a few pupils struggled to open up to them and therefore they felt that these pupils received less benefit from the intervention.

Quantitative impact analysis

The quantitative impact analysis (undertaken for care experienced pupils only) revealed significant positive impacts on almost all of the educational and post-school outcomes considered as a result of taking part in MCR Pathways.

Exclusions: the analysis found a positive effect in terms of reducing school exclusions amongst the target group. The estimated share of care experienced pupils ever being excluded fell by seven to nine percentage points, depending on the analytical approach taken.

Staying-on Rates: the evaluation also suggests a positive impact on rates of staying-on at school past the end of compulsory education. All analytical approaches indicate a significant positive effect on pupils staying on to S5, as a result of participation in MCR Pathways, with estimated effect sizes ranging from eight to thirteen percentage points. The programme also positively impacts the rate of pupils staying on to S6.

Attainment: we estimate a positive effect on attainment, with the magnitude varying across different qualification levels. The impact of the programme is greater for less advanced qualifications. The estimated effect size range is:

• Six percentage points for achieving 1 or more Level-5 qualifications in English.
• Three to four percentage points for 1 or more Level-5 qualifications in Maths.
• Six to seven percentage points for achieving 3 or more Level-5 qualifications.
• Three to five percentage points for achieving 5 or more Level-5 qualifications.
• Three to four percentage points for achieving 3 or more Level-6 qualifications.

Post-School Destinations: the analysis suggests there was also a positive effect on pupils' post-school destinations, particularly by increasing participation in Further Education and training and reducing unemployment. The intervention’s impact on reducing unemployment is maintained over time, with an estimated effect size of a five percentage point reduction in being unemployed nine months after leaving school. The effect of the programme on increasing the proportion of care experienced pupils being in any positive outcomes post-school also remains considerable nine months after leaving (an estimated increase of 5-7 percentage points), although the size of the impact is smaller than at three months after leaving (an estimated increase of 7-8 percentage points).

Attendance: the evaluation did not find robust evidence that MCR Pathways has improved attendance rates overall.

Value for Money assessment of MCR Pathways for care experienced pupils

The Value for Money assessment built on the findings from the impact evaluation to estimate a Benefit Cost Ratio for the programme. Compared to an estimated cost of £18.8 million, the benefits from MCR Pathways outweigh the cost, with a Benefit Cost Ratio (BCR) of between 2.3:1 and 2.8:1, and a Net Present Value (NPV) of between £24.3 million and £34.6 million. This means that for every £1 spent on MCR Pathways, between £2.30 and £2.80 of benefits to society are created, with the benefits outweighing the costs by between £24.3 million and £34.6 million. This suggests that investment in the programme represents value for money when considering the societal and economic benefits. It should be noted that the monetisation of the benefits of MCR focus on the benefits to care experienced pupils and the benefits to mentors and therefore do not reflect wider beneficiaries (e.g. non-care experienced pupils, schools and the employers of mentors) from MCR Pathways. There is also a range of potential additional benefits that could not be monetised, which should also be given due consideration when reaching a judgement on value for money.

Have mentoring and leadership been embedded across the school system?

There was some evidence that the two interventions had embedded mentoring and leadership within the schools in which they were operating, with staff being keen for the schemes to continue. In relation to Columba 1400, it was commonly mentioned by school staff that the YPLA had helped to embed mentoring and leadership practices across their school. For example, one school’s participation in the Columba 1400 YPLA had helped embed more of a mentoring culture through changes to registration groups, encouraging discussions about values and shifting the approach to discipline away from ‘punitive’ towards more ‘restorative’ approaches. It was also noted, in the same school, that values-based approaches had been adopted among wider school staff (beyond those that took part in the YPLA).

Columba 1400 staff also reported that they often receive feedback from teachers about the (sometimes unexpected) impact that the YPLA has had on them professionally in terms of how they view themselves as leaders.

MCR Pathways, by 2021/22, had been embedded in all 30 Glasgow schools (mainly from 2014/15 onwards, after the first school came on board in 2007/8) and in 74 schools across the rest of Scotland, with these schools coming on board from 2017/18 onwards. There was an overall sense among participants that it is important for organisations like MCR to take the lead on embedding mentoring for young people in schools, since school staff have other priorities and do not have the capacity to do this kind of work.

There was, however, limited evidence that mentoring has been embedded more widely across the local authority areas we visited outside of Glasgow, rather it has been restricted to certain schools within each local authority. This is linked to the more limited roll-out of the SMLP programme as a whole, compared to what had been originally envisaged.

In relation to the programme as a whole, the speed and scale of rollout was less than originally intended. Stakeholders indicated that the main reason for the more limited rollout of the programme was the much less favourable fiscal climate which had developed since the start of the programme. In 2022/23 there was also a structural change, with The Hunter Foundation funding only the Columba 1400 element of the programme.

The vision of the two interventions complementing each other within schools has also not materialised. Cost is likely to be a major factor again, with schools/local authorities feeling that they cannot justify funding two schemes in the same school.

Given that the programme has not been rolled out at the scale and pace intended, and has reached considerably fewer schools and pupils than originally envisioned, we would conclude that mentoring and leadership have not yet been fully embedded across the Scottish school system as a whole. Scottish Ministers, however, remain committed to delivering the overall outcomes of the SMLP programme and will keep both elements under review, directly funding MCR Pathways and providing Columba 1400 with some core funding (for the development of Fairer Futures Partnerships) and match funding in partnership with The Hunter Foundation for their Head Teachers’ Leadership Academies.

Suggestions for improvement

The following are largely based on suggestions and comments made by participants interviewed for the evaluation.

At school/intervention delivery level:

• MCR should consider providing more information to parents/carers on the activities involved in the scheme and on the mentor that their child has been matched up with.
• Schools should aim to embed MCR Co-ordinators as much as possible (e.g. treating them as part of the pupil support team). It may also be worth considering whether this would be beneficial for other specialist support.
• MCR should consider the following suggestions from mentors about ways in which their early stage experiences could be improved: an opportunity to have a one-to-one conversation with an experienced mentor; more use of breakout rooms in online training; training sessions in the evening so more people can attend; information about what school is like for young people nowadays; and raising awareness of peer resources available for mentors.

At the Scottish Government/strategic level:

• Given the very positive findings on the impact and Value for Money of MCR Pathways, review the funding model (including exploring alternative sources of funding) to enable MCR Pathways to be rolled out further and sustained, and encourage uptake in other Local Authorities/schools by publicising the evaluation findings.
• Conduct an evaluability assessment of the Columba 1400 YPLA to assess the feasibility of a quantitative impact and Value for Money assessment, which could then inform funding decisions.
• Scottish Government has been keen to understand ways in which any duplication of effort across skills/employment-related activities can be avoided. We recommend Scottish Government undertake further review to ensure programmes are complementary and continue to meet the needs of young people requiring specialist support.

There are some further learning outcomes from the SMLP experience in relation to bringing different interventions/organisations together to deliver a joint programme. These include:

• considering first whether a joint programme is likely to add significant value, compared with the interventions being delivered separately
• considering how practical it is to bring them together
• allowing enough time – several months at least – for relationships between different delivery organisations to be developed before the delivery stage
• actively facilitating the development of a shared vision and shared values for the joint programme.