Scottish Mentoring and Leadership Programme: evaluation - final report
Scottish Mentoring and Leadership Programme (SMLP) supports care-experienced and disadvantaged young people through MCR Pathways mentoring and Columba 1400 leadership. The evaluation found improved confidence, attainment, school retention and post-school outcomes, with MCR providing strong value for money.
Appendices
Appendix 1: Theory of Change and Evaluation Framework
Theory of Change
SMLP Theory of Change
Diagram showing the SMLP Theory of Change for two programmes:
- MCR Mentoring
- Columba 1400 – Residential Leadership Academy
Inputs
- MCR Mentoring connecting to:
- Mentoring programme
- Pathways Coordinator
- Columba 1400 – Residential Leadership Academy connecting to:
- Engagement with schools
- Leadership Academy
Activities
- MCR Mentoring programme connecting to:
- Mentor training
- 1-2-1 mentoring sessions
- MCR Pathways Coordinator connecting to:
- Coordinator group and 1-2-1 sessions
- Young people who would benefit supported to participate
- Columba 1400 engagement with schools connecting to:
- Young people who would benefit supported to participate
- Relationship / planning with school
- Whole school day activity day
- Columba 1400 Leadership Academy connecting to:
- Residentials
- Community projects
What changes
'Being' capabilities
- Mentor / employer benefits: Mentor skills and confidence, Perceptions of young people
- Increased social confidence: Communication skills, Self-esteem
- Enhanced social capital: Social networks, Family relationships
- Increased academic confidence: Growth mindset
- Increased workplace knowledge and skills: Understanding world of work, Workplace confidence, Employability skills
- Higher aspirations: Young person, Family
- Improved health and wellbeing: Happiness, Anxiety
'Doing' capabilities
- Mentor actions: Sustained mentoring relationships
- Increased engagement in learning: Staying on rates, Attendance
- Increased attainment / achievement: Improvements in literacy / numeracy
- Stronger young person voice in schools: Participation in school decisions, Leadership skills in practice
- Improved relationships with peers, families and key adults: Relationship with mentor, Relationship with teachers, Family wellbeing
- Subject / course choice: Subject choice, Qualification level
- Engagement in wider activities: Participation in extracurricular activities, Participation in employability activities
Impact
- Increased attainment / achievement: Improvements in literacy / numeracy, SQA results, Wider achievements
- Sustained positive destinations: Progression to further / higher education, Sustained employment, Training
- Reduction in the Attainment Gap: Literacy, Numeracy, Wellbeing
- School benefits: Ripple effects for teachers and other pupils
- Employer benefits: Employee skills and confidence, Engagement with young people
Evaluation framework
| Evaluation questions (process) | Primary source(s) of evidence | 2022/3 (Year 1) | 2023/4 (Year 2) | 2024/5 (Year 3) | 2025/6 (Year 4) |
|---|---|---|---|---|---|
| Has the Programme been implemented as intended? | Qualitative fieldwork with stakeholders (SG, Hunter Foundation, MCR, Columba1400, LA Education Leads, school staff) | N/A | Oct 23-Jan 24 | N/A | Oct 25-Jan 26 |
| Has the Programme been implemented as intended? | Analysis of delivery monitoring information from MCR and Columba1400 | N/A | Dec 23- Jan 24 | N/A | Dec 25-Jan 26 |
| What has worked well and what can be improved in the implementation? | Qualitative fieldwork with young people, parents/carers, mentors, MCR Pathways Co-ordinators, employers, and other stakeholders (SG, Hunter Foundation, MCR, Columba1400, LA Education Leads, school staff) | N/A | Oct 23-Jan 24 | N/A | Oct 25-Jan 26 |
| To what extent has the Programme embedded mentoring and leadership across the school system? | Qualitative fieldwork with stakeholders (mainly LA Education Leads, school staff) | N/A | Oct 23-Jan 24 | N/A | Oct 25-Jan 26 |
| To what extent has the Programme engaged with and supported young people? | Qualitative fieldwork with young people, parents/carers, mentors, MCR Pathways Co-ordinators, Columba 1400 staff, school staff | N/A | Oct 23-Jan 24 | N/A | Oct 25-Jan 26 |
| To what extent has the programme developed a positive relationship with DYW, SDS and the Promise? | Qualitative fieldwork with representatives from DYW, SDS, the Promise, MCR, Columba 1400 | N/A | Oct 23-Jan 24 | N/A | Oct 25-Jan 26 |
| How is the programme delivery changing and improving over time? | Qualitative fieldwork with young people, parents/carers, mentors, MCR Pathways Co-ordinators, employers, and other stakeholders (SG, Hunter Foundation, MCR, Columba 1400, LA Education Leads, school staff) | N/A | Oct 23-Jan 24 | N/A | Oct 25-Jan 26 |
| Are participants being reached as intended? | Analysis of delivery information from MCR and Columba 1400 | N/A | Dec 23- Jan 24 | N/A | Dec 25-Jan 26 |
| Are participants being reached as intended? | Qualitative fieldwork with stakeholders (SG, Hunter Foundation, MCR, Columba1400, LA Education Leads, school staff) | N/A | Oct 23-Jan 24 | N/A | Oct 25-Jan 26 |
| Evaluation questions (impact) | Primary source(s) of evidence | 2022/3 (Year 1) | 2023/4 (Year 2) | 2024/5 (Year 3) | 2025/6 (Year 4) |
|---|---|---|---|---|---|
| To what extent has the programme supported the capabilities of young people? (i.e. The 'being' and 'doing' capabilities from the Theory of Change) | Qualitative fieldwork with young people, parents/carers, school staff | N/A | Oct 23-Jan 24 | N/A | Oct 25-Jan 26 |
| To what extent has the programme supported the capabilities of young people? (i.e. The 'being' and 'doing' capabilities from the Theory of Change) | Analysis of feedback data from MCR (short 4 question survey for mentors and YP to complete at every session currently being developed, will probably include questions on agency, problem solving and self-worth) and Columba 1400 YPLA survey (self-rated confidence, knowing own values, knowing what is important to me, feeling positive about future, motivation, leadership, working with others) and Columba 1400 WEMWBS (wellbeing) | N/A | 6 monthly from Autumn '23 to Autumn '25 | 6 monthly from Autumn '23 to Autumn '25 | 6 monthly from Autumn '23 to Autumn '25 |
| To what extent has the programme contributed to academic attainment of participating young people? | Quantitative analysis of data on attendance (SG data), attainment (SQA via ADR Scotland), staying- on rates (SG data) and positive destinations post-school (SDS data) for intervention group and control group | May-July 23 | May-July 23 | May-July 25 | N/A |
| Did the Programme achieve the expected outcomes? | Quantitative analysis of data on attendance, attainment and staying-on rates for intervention group and control group | May-July 23 | May-July 23 | May-July 25 | N/A |
| Did the Programme cause the difference? | Quantitative analysis of data on attendance, attainment and staying-on rates for intervention group and control group | May-July 23 | May-July 23 | May-July 25 | N/A |
| Have different groups been impacted in different ways, how and why? | Quantitative analysis of data on attendance, attainment, staying-on rates and positive destinations post-school for intervention group and control group; analysis of differences within the intervention group (e.g. by gender, level/type of care experience) | May-July 23 | May-July 23 | May-July 25 | N/A |
| Have different groups been impacted in different ways, how and why? | For the 'how' and 'why'? elements: qualitative fieldwork with young people, parents/carers, mentors, MCR Pathways Co-ordinators, Columba 1400 staff, school staff | N/A | Oct 23-Jan 24 | N/A | Oct 25-Jan 26 |
| Are young people achieving sustained positive destinations as a result of the Programme? | Quantitative analysis of positive destinations post-school | May-July 23 | May-July 23 | May-July 25 | N/A |
| Evaluation questions (value for money) | Primary source(s) of evidence | 2022/3 (Year 1) | 2023/4 (Year 2) | 2024/5 (Year 3) | 2025/6 (Year 4) |
|---|---|---|---|---|---|
| What are the costs of the programme and how does the cost of its work compare to initial expectations? | Cost data from SG, MCR, LAs and schools related to the funding and running of the programme: operational costs, staff costs, materials costs, direct funding awards. | N/A | May-Sept 23 | N/A | May-Sept 25 |
| How does the programme’s effectiveness compare to other interventions that could achieve similar outcomes? | Value for Money analysis based on quantitative impact data, costs and benchmarking against other interventions (where data available). (See note 1 for more details). | N/A | Aug-Dec 23 | N/A | Aug-Dec 25 |
| How can we robustly translate the programme’s initial impacts (e.g. on positive destinations) into longer term monetisable benefits for themselves and others? | Cost Benefit Analysis will focus primarily on academic/employment destinations associated with mentoring using data collated from educational datasets. The value for money analysis will apply different modeling approaches to estimate the lifetime monetary benefits to the individual depending on whether the impacts of mentoring were associated with academic or vocational qualifications/apprenticeship placements. Other outcomes like attendance can be monetised using indicative cost savings to the exchequer related to truancy. (See note 2 for more details). | N/A | Aug-Dec 23 | N/A | Aug-Dec 25 |
| How do the monetised benefits compare to costs, and how should we account for other important benefits that are too difficult to monetise? | Cost Benefit Analysis will focus primarily on academic/employment destinations associated with mentoring using data collated from educational datasets. The value for money analysis will apply different modeling approaches to estimate the lifetime monetary benefits to the individual depending on whether the impacts of mentoring were associated with academic or vocational qualifications/apprenticeship placements. Other outcomes like attendance can be monetised using indicative cost savings to the exchequer related to truancy. (See note 2 for more details). | N/A | Aug-Dec 23 | N/A | Aug-Dec 25 |
| Where and how can we monetize improved capabilities that do not immediately lead to direct economic benefits | Cost Benefit Analysis will focus primarily on academic/employment destinations associated with mentoring using data collated from educational datasets. The value for money analysis will apply different modeling approaches to estimate the lifetime monetary benefits to the individual depending on whether the impacts of mentoring were associated with academic or vocational qualifications/apprenticeship placements. Other outcomes like attendance can be monetised using indicative cost savings to the exchequer related to truancy. (See note 2 for more details). | N/A | Aug-Dec 23 | N/A | Aug-Dec 25 |
Information leaflet
Research about the Scottish Mentoring and Leadership Programme
The Scottish Government wants to know how well the Scottish Mentoring and Leadership Programme has been working and what difference, if any, it’s making for young people.
They have asked Ipsos Scotland (an independent research organisation) to speak to young people to help them find out. Before taking part, it’s important that you understand why this research is taking place and what it will involve.
How can I help?
They have asked Ipsos Scotland (an independent research organisation) to speak to young people to help them find out. Before taking part, it’s important that you understand why this research is taking place and what it will involve.
The Scottish Mentoring and Leadership Programme includes Young Scottish Talent (delivered by MCR Pathways) and the Columba 1400 Young Person’s Leadership Academy (YPLA).
You’ve been randomly picked to take part in the research because you take part in one or both of these programmes. We hope you can share your views!
What does the research involve?
Taking part involves a short in-person one-to-one interview with a researcher from Ipsos (photos on next page). It will last around 40 minutes or one school period. The interview will happen in June 2024 and will take place in school.
The interview will be relaxed and informal and you don’t need to prepare anything to take part. You can have a friend there if you would like.
For your information, we will also be interviewing some parents/carers to hear their views on the programme. If your parent/carer takes part, this will be a completely separate interview to yours, and they will not hear what you said in your interview.
The researcher will ask you about things like:
- What kind of activities you’ve taken part in as part of the Programme
- How you feel now compared to the start of the Programme
- If there’s anything you’ve done differently since the start of the Programme
- If there’s anything you would change about the Programme
There are no right or wrong answers. We’re just keen to hear about your experiences.
How will my information be used?
We would like to audio-record our discussions (but will only do so with your agreement). Your responses to the interview will be completely confidential and anonymous and will not be seen by anybody outside of Ipsos Scotland and the transcriber. All data will be destroyed after the evaluation is finished.
Ipsos will write a report of the findings for the Scottish Government. The report will be published and is likely to use quotes, but we will not name or identify anyone. If you do not want us to quote anything you say, please just let us know.
Do I have to take part?
No. It is entirely your choice whether or not you take part and you can change your mind at any time without giving us a reason. You can also choose not to answer individual questions and/or leave the discussion completely.
What are my rights?
The General Data Protection Regulation (GDPR) is a set of rules and regulations designed to protect the privacy and personal data of individuals. We have a legal duty to inform you about why we are collecting information from you for the research.
We provide this information in our Privacy Notice which should be read alongside this information sheet. It outlines what information we will collect and why, and your rights.
What happens now?
If you are under 16, we will need to ask your parent or carer for permission before we speak to you. We have a separate information sheet for them.
If you are happy to take part, please speak to the person who gave you this information leaflet. This might have been your teacher or Young Scottish Talent coordinator.
If there is anything we can do to make it easier for you to take part, please let us know using the contact details below.
Where can I get more information?
If you would like more information about the research, or have any questions, you can speak to your teacher or Young Scottish Talent coordinator. Or you can get in touch with Ipsos using the contact details below.
Call or message [researcher] on: [mobile number]
Email Ipsos at: Scotland-SMLP@ipsos.com
Appendix 3: capabilities and statements
I feel more confident in social situations (Increased social confidence)
I am more comfortable speaking to other people (communication skills)
I feel better about myself (self-esteem)
My relationships with the people I live with are better (enhanced social capital: family relationships and improved relationships with peers, families and key adults: family wellbeing)
I have more people I can talk to about things (enhanced social capital: social networks)
I feel happier (Improved health and wellbeing: happiness)
I feel less anxious (Improved health and wellbeing: anxiety)
I am getting on better with my friends (Improved relationships with peers, families and key adults)
I have more belief in my ability to do well in school/gain qualifications (increased academic confidence: growth mindset)
I am aiming higher in terms of what I would like to do after school (higher aspirations: young person)
I am thinking more about the subjects I need to choose at school to help me get the kind of job I might want in the future (Subject / course choice: subject choice)
I am doing more things that might help me get the kind of job I want in the future (engagement in wider activities: Participation in employability activities)
I plan to stay on longer at school than I did before (Increased engagement in learning: staying-on rates and subject / course choice: qualification level)
My attendance at school has improved (Increased engagement in learning: attendance)
I am doing better in my schoolwork (increased attainment/achievement)
I know more about the kinds of jobs I might like (understanding world of work)
I feel more confident about doing well at a job in the future (Understanding world of work: workplace confidence and employability skills)
I feel more involved in school decisions (stronger young person voice in schools: participation in school decisions)
At school, I have been able to put into place some of the leadership skills I have learnt (stronger young person voice in schools: leadership skills in practice)
I am getting on better with my teachers (improved relationships with peers, families and key adults: Relationship with teachers)
I find it easier to talk to teachers (improved relationships with peers, families and key adults: Relationship with teachers)
I am more comfortable speaking to my mentor than I was at the start of the mentoring (Improved relationships with peers, families and key adults: Relationship with mentor) – used in MCR interviews only.
I am doing more activities out of school (Engagement in wider activities: Participation in extracurricular activities)
How to access background or source data
The data collected for this :
- are available in more detail through Scottish Neighbourhood Statistics
- are available via an alternative route
- may be made available on request, subject to consideration of legal and ethical factors. Please contact for further information.
- cannot be made available by Scottish Government for further analysis as Scottish Government is not the data controller.