Scottish Mentoring and Leadership Programme: impact and value for money evaluation

1. Introduction

This report provides the results of the quantitative analysis undertaken as part of the evaluation of the Scottish Mentoring and Leadership Programme (SMLP). The SMLP includes both the MCR Pathways Young Scottish Talent Programme and the Columba 1400 Young Persons Leadership Academy (YPLA) - programmes aiming to improve outcomes for young people who are care experienced or in ˜tough realities.

Background

The Scottish Mentoring and Leadership Programme (SMLP) aims to support and improve the outcomes of care experienced young people, and those in ˜tough realities (such as those on the cusp of care, young carers, and those living in SIMD 1 or 2 areas). SMLP is one of several initiatives aiming to improve the wellbeing of children and young people. It is predicated on the idea that young people can be empowered by developing their relationships with family, friends and adults in the wider community, and that more relational forms of support will enhance wellbeing and develop young peoples capabilities, thereby improving their short-, medium- and long-term outcomes.

There are two strands to SMLP: MCR Pathways and Columba 1400 Young Peoples Leadership Academies:

• MCR Pathways is a school-based mentoring programme. The programme engages S1 and S2 pupils in group work and then matches them with a mentor from the local community, who meets with them in school once a week from S3 onwards. The programme aims to improve attainment and staying-on rates for S5 and S6 pupils, as well as supporting young people to develop key life skills. By doing so, the aim is to increase the number of young people in full-time work, college or university after leaving school, known as ˜positive destinations.
• Columba 1400 provides values-based leadership for young people. It is delivered in three phases over the course of S2/3 (starting with group work in school, followed by a residential trip to Skye, then a group project back at school). It aims to support young people to develop leadership skills, create better relationships and improve confidence and wellbeing.

The evaluation aims to assess the effectiveness of SMLP. There are three main strands to the evaluation, which explore:

• The delivery of the programme.
• The impact on measurable outcomes (attendance, exclusion, attainment, staying-on rates and positive post-school destinations) and on capabilities which young people have said matter to them (such as increased social confidence, increased health and wellbeing, and better relationships).
• Value for Money (VfM) - assessing the costs against monetised benefits (including the longer-term benefits to wider society).

This document provides the results of the second and third strands, the impact on measurable outcomes and Value for Money.

This analysis covers the MCR Pathways element of SMLP for Group 1 (care experienced) pupils only. The inclusion criteria for MCR Pathways Group 2 (˜tough realities) and Columba 1400 pupils vary across schools, and it was not possible to identify suitable control groups to assess impacts with sufficient robustness^[2].

The findings from the quantitative analysis feed into the Value for Money assessment of the costs against monetised benefits (including the longer-term benefits to wider society).

Five analytical approaches were employed to estimate the impact of the MCR Pathways programme (for care experienced pupils), namely:

• Approach A: Participants versus non-participants. A comparison of key outcomes of MCR participants with a matched control sample of care experienced pupils from non-participating schools.
• Approach B1: All care experienced pupils in participating schools vs care experienced pupils in all non-participating schools, again comparing key outcomes of interest among pupils who both did and did not participate in the programme in participating schools against a matched group of young people from schools not involved in the programme.
• Approach B2: Care experienced pupils in participating schools vs care experienced pupils in matched non-participating schools. This involves matching schools to ensure the control group is from schools that are similar to those that have adopted MCR.
• Approach C: Comparison between participating and non-participating schools, reconfiguring the analysis from individual-level to school/cohort-level. This involved analysis of average outcomes at the school-level for care experienced pupils.
• Approach D: A ˜staggered treatment design that compares outcomes at the school-level as with Approach C but for schools that adopted MCR earlier compared to those that adopted MCR later, on the basis that outcomes should be visible for the former sooner.

Chapter 2 provides background information on these approaches and on the data sources and implementation of the analysis. Further technical details are provided in the appendices.

Chapter 3 summarises the impact findings by each of the different outcomes.

Chapter 4 provides the Value for Money (VfM) analysis. Further technical details are provided in Appendix D.

Chapter 5 provides a concluding summary.