Scottish Mentoring and Leadership Programme: evaluation - final report

Chapter 1: Policy background and evaluation aims

Policy background

In July 2024, there were 11,844 children under 18 in care in Scotland.^[4] While education outcomes for care experienced children have improved considerably over the last 10-14 years, by 2021/22 (the latest year for which we have data) there was still a notable gap between them and their non-care experienced peers. For example, 27% of care experienced pupils achieved 3+ National 5 qualifications compared with 75% of non-care experienced pupils.^[5]

The Scottish Mentoring and Leadership Programme^[6] (SMLP) aims to support and improve the outcomes of care experienced young people, and those in ‘tough realities’. In the context of the programme, there are no set criteria for what qualifies as ‘tough realities’, but it is generally understood by school and programme staff to mean young people who might struggle to reach their full potential because of their family circumstances or personal issues. This might include those on the cusp of care, young carers, those from the most deprived backgrounds and those suffering from a lack of self-confidence and/or anxiety.

Central to the SMLP is the idea that young people can be empowered by developing their relationships with family, friends and adults in the wider community, and that more relational forms of support will enhance wellbeing and develop the capabilities of young people and their families, thereby improving their short, medium and long-term outcomes. SMLP comprises two elements: MCR Pathways (a school-based mentoring scheme) and the Columba 1400 Young People’s Leadership Academy (which provides values-based leadership experiences for young people). See Intervention Overviews below for more details on each.

SMLP is very much in line with The Promise made in 2020 to the country’s most vulnerable children following the Independent Care Review. The Promise sets out what Scotland needs to do ‘to make sure its most vulnerable children feel loved and have the childhood they deserve’. One of the foundations of The Promise is People:

“The children that Scotland cares for must be actively supported to develop relationships with people in the workforce and wider community, who in turn must be supported to listen and be compassionate in their decision-making and care.”^[7]

SMLP is one of several initiatives aiming to improve the wellbeing of children and young people, including others such as Developing the Young Workforce (DYW), and it aligns with the preventative agenda set out in the Scottish Government’s Tackling Child Poverty Delivery Plan 2026-2031, which states:

“We want every young person to have a positive and fulfilling experience at school. To make this possible, we must empower our headteachers to put the right support in place for young people and their families to help close the poverty-related attainment gap. Every child should be able to go to school feeling comfortable, confident and ready to learn.”^[8]

Following early evidence from an independent evaluation^[9] that MCR Pathways (one of the two strands of SMLP) had a positive impact on outcomes and capabilities, the Scottish Government and the Hunter Foundation agreed to invest additional funding to expand MCR Pathways and roll it out over a six year period (from 2021/22 to 2026/27) to a much larger cohort of young people. Alongside this, Columba 1400 would deliver their values-based Young People’s Leadership Academies (YPLA)to young people in schools. It was envisaged that the two schemes would complement each other in many schools with, for example, schools involved in MCR being encouraged to participate in Columba 1400 as well – and vice versa.

The funding model was intended to act as a catalyst: supporting take-up in areas where MCR and Columba 1400 were not embedded already – by providing part funding for three years^[10]. It was intended that this would give sufficient time to demonstrate impact, after which schools/local authorities (or other funders) would commit funding to sustain the programme.

Intervention overviews

MCR Pathways

MCR Pathways^[11] is a school-based mentoring scheme that seeks to improve the life chances of care experienced young people and those in ‘tough realities’. The overall aim of the intervention is to increase the number of young people in full-time work, college or university after leaving school (known as ‘positive destinations’). In doing so, the scheme aims to improve attainment, and staying-on rates for S5 and S6 pupils, as well as supporting young people to develop key life skills.

There are two broad groups of eligible pupils:

• Group one: young people who are currently or have previously been in care. These pupils are prioritised for support. These pupils are initially identified via the school’s information management system, known as SEEMis, and then each individual’s potential participation is discussed by the scheme Co-ordinator and school staff who know the pupil (e.g. guidance and pupil support staff).
• Group two: other young people who may benefit from the intervention (i.e. those in ‘tough realities’). These young people are identified by school staff.

All eligible young people are free to decide whether or not to take part in the whole intervention, or parts of it.

Schools have an MCR Pathways Co-ordinator, employed by and located in the school, who supports pupils and mentors, working closely with other school staff. In S1 and S2, young people typically take part in weekly group work sessions with the Co-ordinator. These sessions focus on building confidence, life skills and improving literacy and numeracy. It is intended that the Co-ordinator builds trust and an individual relationship with each pupil at an early stage in their secondary school journey. Co-ordinators can also support the transition of pupils from primary school (P7 to S1).

Pupils are then matched with a mentor from S3 onwards^[12]. This is intended to provide more one-to-one support. Mentors are volunteers from the local community with a range of different backgrounds who receive training from MCR. They are not required to have any specific qualifications or prior experience working with young people. Pupils and mentors are matched based on their personalities, skills and interests, and the pupils’ career aspirations.

Columba 1400

Columba 1400’s Young People’s Leadership Academy^[13] (YPLA) provides values-based leadership experiences for young people. The YPLA involves three phases, with Phase Two delivered through a residential element or a place-based element.

The residential element is typically delivered in partnership with a school year group (usually S2/S3) from one school, involving a week-long residential element, at Columba 1400’s purpose-built Community and International Leadership Centre in Staffin on the Isle of Skye as part of Phase Two. Some schools have ongoing engagement with Columba 1400 and have taken part in the scheme multiple times with different year groups.

The place-based model is a more tailored approach, designed to include young people for whom the residential element would not be accessible. The place-based model can be taken up either by a single school, by a local authority (involving several schools), or by any organisation working with young people.

An overview of each phase of the YPLA is provided below.

YPLA with a Residential Element

Phase One

In Phase One of the YPLA with a residential element, the selected pupils meet with Columba 1400 staff, typically over three to four sessions (ranging from a couple of hours to a full day), in the weeks leading up to the Phase Two residential element. The delivery of this phase is tailored, with more sessions, staggered sessions, or combined sessions, depending on the needs of each school. The purpose of Phase One is to introduce the cohort to values-based leadership, to provide opportunities for the young people to get to know each other and to help them prepare for the residential element. Pupils take part in a range of activities, including team-building exercises, discussions about leadership and mental health strategies, learning about Columba 1400’s core values, and a ‘values walk’ designed to provide an informal environment for young people to explore their own core values

Phase Two

In Phase Two, pupils, accompanied by three or four members of school staff, take part in the residential element on the Isle of Skye and stay at Columba 1400’s centre in Staffin for five days. From the point of arrival, with the ‘welcome meal’. there is a focus on high quality treatment and providing a safe space for the young people where they will be treated with respect.

Each day focuses on one of the Columba 1400 core values: Awareness, Focus, Creativity, Integrity, Perseverance and Service. The schedule includes a mix of group and individual indoor and outdoor sessions (including a search and rescue activity where pupils are split into two teams and have to find a dummy and transport it back to the centre), as well as discursive and more reflective work (e.g. journalling). Columba 1400 uses the surrounding, inspiring landscape for young people to put the leadership skills they are discovering into practice. The sessions are grounded in different leadership models and are designed to encourage young people to discover and live their core values.

Towards the end of the residential element, the group is introduced to Phase Three. This is a project focused on service for themselves and others, which they will work on when they return to school.

Phase Three

After returning from the residential element, the young people give a presentation to the school’s senior leadership team to summarise their experiences. Supported by the Columba 1400 team, the young people decide on a community-based or school-based service project that they want to work on, develop a plan for this, and deliver it together (sometimes as one group delivering one larger project or in smaller groups delivering two or more smaller projects). At this point, Columba 1400 staff typically take a step back to allow the young people to carry out their project – but they provide ongoing facilitation if needed. The purpose of Phase Three is to further embed values-based leadership by enabling young people to put the skills they discovered in Phases One and Two into practice. This includes a minimum of three to four sessions, culminating in a Celebration Graduation at the end to which parents and carers are invited.

Place-based YPLA model

A place-based model for delivering the Columba 1400 YPLA was developed from late 2020 onwards in recognition of the fact that the residential element, delivered in Phase Two, was not accessible for some pupils. The YPLA with a place-based element has been utilised as an alternative model for a range of young people and is designed by Columba 1400 to provide a more tailored approach to delivering the YPLA in a way that best suits the needs of the young people involved. Partner organisations are varied, but are typically local authorities or third sector organisations supporting young people. It is usually part funded by the commissioning organisation. Columba 1400 also helps with fundraising. As with the residential model, the place-based model is delivered using a phased structure:

Phase One

The young people are brought together for initial meetings and team-building sessions. Sometimes this involves bringing young people together from different schools or settings who have not met before.

Phase Two

Phase Two is the intensive place-based element which typically takes place over five days with sessions exploring values-based leadership. Under the place-based model, the location is typically within the local community and the duration and types of sessions are tailored depending on the needs of the group.

Phase Three

In Phase Three, Columba 1400 and the partner organisation continue to work with the group to support them in whatever they choose to do next. This could be a joint service project or something more informal. The aim of Phase Three is to help sustain the impact of the YPLA on young people’s confidence and self-esteem, and to help them embed the leadership skills they have discovered.

Evaluation aims

The aim of this evaluation was to assess the effectiveness of the Scottish Mentoring and Leadership Programme (SMLP), which includes both the Columba 1400 Young Peoples’ Leadership Academy and MCR Pathways interventions. There were three main strands to the evaluation, which explored:

• The delivery of the programme.
• The impact on ‘hard’ outcomes (i.e. objective and quantifiable outcomes), such as attendance, attainment, staying-on rates and positive post-school destinations. (This element was assessed for the MCR Pathways care experienced element only as it was not possible to construct a robust control group for the MCR Pathways pupils in ‘tough realities’ or for Columba 1400 YPLA pupils due to variability of the inclusion criteria.)
• The impact on young people’s capabilities, such as increased social confidence, increased health and wellbeing, and better relationships, which young people have said matter to them.
• Value for Money (VfM), assessing the costs of the intervention against monetised benefits (including the longer-term benefits to wider society). Again, this was assessed for MCR Pathways care experienced pupils only, as the Social Cost Benefit Analysis approach we used requires robust data on ‘hard’ outcomes, which was only available for MCR Pathways’ care experienced pupils.
• The evaluation also aimed to identify ways in which the outcomes of care experienced young people can be further improved through these programmes.

The research was guided by an Evaluation Framework (see Appendix 1), which sets out the evaluation questions and the primary sources of evidence used to answer each question.

To support evaluation of the programme, a Theory of Change was developed during the Evaluability Assessment undertaken in 2021. It sets out a theoretical underpinning for understanding how the programme is intended to impact on the capabilities of young people (e.g. social confidence, health and wellbeing, relationships). Split into ‘doing’ and ‘being’ capabilities, these were largely derived from what young people themselves said was important to them. The Theory of Change has been used in conjunction with the Evaluation Framework to guide the design of data collection materials, analysis and reporting. Both can be found in Appendix 1.

The structure of this report

The remainder of this report covers both qualitative and quantitative findings on the above evaluation questions.

The qualitative process and impact findings include both MCR Pathways and Columba 1400. It should be noted that the aim in qualitative research is not to achieve a sample that is statistically representative of the wider population, but to identify as much diversity of experience as possible. Estimates of prevalence based on qualitative data are therefore inappropriate and this report avoids quantifying language, such as ‘most’ or ‘a few’ when discussing findings from qualitative interviews. The value of qualitative research, in a process and impact evaluation such as this, is that it provides the perspective of those involved on what aspects of the process have worked well/less well, what the perceived impacts have been and how they think those impacts have been achieved. Qualitative research does not attempt to measure the extent of the impact but to set out the range of perceived impacts. On occasion, where there was a clear difference between the findings for girls and boys, we have noted this.

The quantitative impact and Value for Money findings provide robust quantitative data on the extent to which MCR Pathways (only) has had an impact on key outcomes for care experienced pupils. The estimated impact was based on data that linked records of participation in the intervention with pupil-level data held by the Scottish Government on attendance rates, attainment, staying-on rates, and post-school destinations. The Value for Money assessment then used estimates of the costs of delivering MCR Pathways (MCR provided an estimated average cost per school which was aggregated up) against the monetised benefits, which includes the benefits of improved educational attainment, reduced unemployment (9 months after leaving school), and avoided school exclusions.

Although there are commonalities between the MCR Pathways and Columba 1400 interventions, the report covers findings for each separately as they are different interventions, with different approaches and durations, and it would be inappropriate to compare them directly with each other.

Chapter 3 focuses on qualitative findings on the Columba 1400 Young People’s Leadership Academy (YPLA), summarising the perceived reach, implementation and impact of the intervention on the young people taking part. Chapter 4 focuses on the Columba 1400 place-based approach specifically. Chapter 5 explores MCR Pathways and includes qualitative findings as described above, as well as quantitative impact and Value for Money findings. Chapter 6 considers whether the individual interventions, and SMLP as a whole, have been implemented as intended and whether mentoring and leadership have been embedded across the school system. Chapter 7 sets out the conclusions and suggested improvements at the school/individual delivery level and at the Scottish Government/strategic level.