Mental Health and Wellbeing Leadership Board: monitoring and evaluation update paper

2. Evaluation

2.1 Evaluation Prioritisation by Public Health Scotland

Public Health Scotland (PHS) were asked to help decide which parts of the Mental Health and Wellbeing Strategy should be prioritised for evaluation. To do this, they ran two projects at the same time:

1. A Delphi panel survey – where experts were asked to identify top priorities across several rounds of questions.
2. An evaluation mapping survey – which gathered information on existing or planned evaluation work.

The aims of these projects were:

• To identify priority long-term outcomes from the published Strategy framework to focus evaluation resource upon;
• To identify priority population groups to focus evaluation resource upon;
• To identify ongoing or planned evaluation work being undertaken by others relevant to the Strategy and indicate gaps and opportunities for evaluation activity.

The details and findings from these projects are summarised below. A full report with complete details of both projects is available on request from the PHS team.

The two surveys ran concurrently, commencing on 8 January 2025. The Delphi closed on 20 February 2025, and the mapping survey on 21 February 2025.

2.2 Delphi survey update

2.2.1 Delphi Survey Summary

The Delphi survey asked experts to rank the importance of 12 long-term outcomes from the Strategy, and to highlight which population groups should be the focus for evaluation.

Top three outcomes identified as critical to evaluate:

• From the outset, all people have the material, social and emotional resources to ensure good mental health and wellbeing so they can cope during times of stress, or challenging life circumstances.
• Everyone has equitable access to support and services in the right place, at the right time wherever they are in Scotland. As far as possible, this should be delivered in a way that respects individual needs and preferences.
• From pre-birth onwards, more people have the foundations that enable them to develop and maintain healthy, nurturing, supportive relationships throughout their lives.

Panellists told us they chose these because they are seen as essential for building good mental health, shifting toward prevention, addressing unmet needs, and supporting early intervention.

Panellists also prioritised the outcomes that were seen as foundational for positive mental health, key for shifting towards preventative approaches, impactful on populations with greater unmet needs, and important for early intervention.

Panellists were also asked to prioritise a list of population groups for whom some targeted evaluation may be required. Panellists also had an opportunity to add any other population groups to the list during the survey. The following population groups were prioritised as ‘critical’ to target across the three rounds, fitting into 3 categories:

Universal Population Groups (whole population groups)

• Early years, children, young people
• Remote, rural, and island communities
• People experiencing loneliness/social isolation
• Non-economic migrants

Targeted Groups (populations at higher risk)

• People experiencing poverty/low-income
• People with a mental health related disability
• Care experienced people
• Care experienced young people
• People in contact with the criminal justice system

Indicated Groups (populations at very high risk)

• People experiencing mental health crisis
• People with severe and enduring mental health conditions
• People affected by suicide
• People affected by self-harm
• People experiencing homelessness
• People experiencing complexity around poverty, gender-based violence, and addiction in combination
• People engaged in harmful substance use
• Survivors of childhood abuse
• Survivors of domestic violence

Panellists prioritised these populations on the basis that they were most likely to: be vulnerable to mental health challenges; experience the most severe mental health related harms; be excluded from mental health systems; and be most likely to be reliant on mental health services.

2.3 Mapping Update

The mapping survey was sent to a wide range of stakeholders who might have knowledge of ongoing evaluation projects relevant to the Strategy’s outcomes. Participants were asked to provide key details about their projects. In total 20 completed submissions were received. These were mapped to the Strategy’s long-term outcomes and population groups. The various projects submitted in the mapping survey are described in detail in the full report, with reference to the outcomes and populations categorised as of critical importance above.

2.4 Next steps and expected outputs

The next step is to develop a full Evaluation Framework. This will be shaped by:

• The findings from the prioritisation work above
• An earlier technical report
• Work by the Scottish Government on monitoring, data and indicators

The Evaluation Framework will include:

• Key questions the evaluation will answer
• What data is available or still needed
• Phased plans for evaluation work
• Any limitations or challenges

Independent experts will review the framework to make sure it's strong and reliable. The Monitoring and Evaluation Group will review and agree the framework, which is expected to be ready by Summer 2025. The framework will be brought to the Leadership Board in August to support decision making in evaluating the Strategy outcomes.