Distress Brief Intervention - under 18s elements: evaluation

Methodology

To address the research aims and objectives, the Scottish Centre for Social Research, the University of Stirling, and Children in Scotland conducted a mixed method evaluation which involved a scoping phase, engagement with young people to inform the development of the research approach, interviews with young people who have accessed support from DBI and interviews and focus groups with stakeholders.

Scoping phase

The scoping phase included a review of data routinely collected by the DBI programme, scoping conversations and two online exploratory workshops with key stakeholders involved in the development and delivery of the DBI programme - with the aims of:

• Defining the intended outcomes of the service for children and young people
• Defining the pathways into DBI for children and young people
• Understanding the changes, challenges and barriers to delivering DBI for children and young people
• Understanding where DBI for under 18s fits in the wider landscape of support for children and young people.

The scoping phase culminated in a Theory of Change workshop which brought together strategic and delivery stakeholders to agree outcomes and develop a Theory of Change for the under 18s elements of DBI.

The output for the scoping phase was an options appraisal which set out the options for sampling and analysis of the routine DBI data and presented the draft Theory of Change (See Figure 2 in the supporting document). The Theory of Change was reviewed during the final reflections workshop (See Figure 3 in the supporting document).

Data from young people

Data was collected from young people in two ways: through routine data gathered by DBI Level 2 services—both directly from young people and about their engagement with the service—and through qualitative fieldwork involving young people.

Young people’s sample

This evaluation includes young people who have accessed DBI via three different pathways:

• Individuals aged 16 and 17 who were referred through the existing DBI pathways designed for adults—known as the adult pathway. These referrals came from frontline services such as the police, Scottish Ambulance Service, primary care, and accident and emergency departments.
• Young people aged 13 to 17^[1] who have accessed DBI via the test of change and were referred through the tripartite referral pathway.
• Young people aged 14 to 17 who have accessed DBI via the Compassionate Distress Response Service (CDRS) in Glasgow City and East Dunbartonshire.

The DBI Level 2 routine dataset contains data on all young people who have accessed DBI across Scotland. Where data is presented for all under 18s, this includes those who have been referred via the adult pathway, the test of change or CDRS. Where data is presented for those referred via the test of change, this includes those young people who were referred by schools or CAMHS. Due to low numbers of referrals from the CDRS, it is not possible to present sub-analysis for this group. Where those aged 18 have been referred by schools or CAMHS, these individuals are not included in the data tables.

The sample of young people who took part in interviews includes young people who have accessed DBI via all three different pathways. Due to the relatively small numbers of participants from each pathway, it is not possible to perform sub-analysis for these different pathways.

Analysis of routine DBI data

Aggregate data tables for the data routinely collected by DBI Level 2 services were supplied by Public Health Scotland for all under 18s referred to DBI between 31st May 2019 to 30th September 2024 (n=2219) and for all adults referred to DBI over the same period (n= 67989) for the purposes of comparison.

Interviews with young people

Ethical Approval

Ethical committee approval for the interviews with young people was granted from the University of Stirling General University Ethics Panel (Ref: GUEP 2024 18284 13746) in June 2024.

Recruitment

Young people were recruited via DBI Level 2 staff members. Individuals accessing DBI were provided by their DBI Level 2 staff member with an information sheet and link to a short animation explaining the evaluation. If the young person was interested in participating, the Level 2 staff member completed a Microsoft Form providing the young person’s consent to be contacted by the researcher. A researcher then contacted the young person directly to further explain the evaluation and arrange an interview if they wished to participate. Informed consent was gained through a signed consent form returned before the interview via post, email or WhatsApp/messenger image. Verbal consent was sought again at the start of the interview.

Sample

This research aimed to interview a purposive sample of 20 children and young people, from S3 (mostly 14-15 years old) up to 18 years of age, who had received a DBI Level 2 intervention and were referred through the test of change pathway or through other established routes (emergency services, primary care etc), from across five Local Authorities delivering DBI through school/CAMHS referrals (Lanarkshire, Glasgow, West Dunbartonshire, Highland and Aberdeen).

A total of 20 young people aged between 13-18 years (Table 1) were recruited between June 2024 - January 2025 from across the five different research areas. A small number of 13 year olds were included in the sample which reflects the extension of the age range by DBI as some young people in S3 were aged 13. Of these 20 young people, 17 (85%) were female and 3 (15%) were male. Although there were slightly fewer males in the interview sample than reflected in the national referral rates (Data Table 3.1), the number of males accessing DBI is substantially lower than females.

More young people were recruited from Lanarkshire (n=7) and West Dunbartonshire (n=6) than other areas: Glasgow/East Dunbartonshire (n=3), Highland (n=3), and Aberdeen (n=1) (Table 2). However, this is also indicative of the number of referrals for young people being larger in these areas (Data Table 1).

At the time of their interviews, 16 young people had completed their DBI intervention and 4 still had a few final appointments to go. Most of the young people (n=13) had been referred to DBI by someone in their school via the tripartite pathway, with other referrals being primarily within a health care setting via the CDRS and adult pathways (Table 3).

Fieldwork

Interviews were held either in-person at an agreed safe location (home / school / community space / café), by telephone, or WhatsApp text messages, according to the young person’s preference. Interviews followed the pre-defined semi-structured interview topic guide, with the researcher adopting a trauma informed approach, exercising sensitivity and consideration of the young person’s emotional state during the interview.

Young people exercised their choice of how they would like to be interviewed, with 9 being completed in person, 8 by telephone and 3 through WhatsApp text messaging. The interviews varied in duration from approximately 15 to 90 minutes, reflecting the mode of communication and the young person’s ability to engage.

In-person and telephone interviews were audio recorded and transcribed verbatim. WhatsApp text message interviews were downloaded and transferred to Microsoft Word Documents.

Young people were given a £20 voucher of their choice as a token of appreciation for their time.

Engagement with young people’s group

A small number of young people from Children in Scotland and Includem, a charity which works with young people (n=5), supported the researchers to develop recruitment materials and the interview topic guide through a mixture of on-line and in person meetings. Following data collection and analysis, a further group of young people (n=3) with experience of accessing mental health support services provided their reflections and feedback on the emerging findings from the interview data. All young people received a £20 voucher of their choice in appreciation for their time.

Interviews with stakeholders

Ethical approval for the stakeholder interviews was granted by NatCen’s Research Ethics Committee in February 2024.

Recruitment

Given established relationships between Level 1 referrers and Level 2 DBI teams through the test of change, Level 2 leads were asked to recruit 2-3 staff members for the DBI staff focus groups and to provide contact details of Level 1 referrers from schools and CAMHS. Strategic stakeholders such as those who are involved in the development and governance of the under 18s elements of DBI were identified through the options appraisal and the existing DBI network.

Parents of children and young people who had accessed DBI through school were recruited through school staff, who shared information about the research. With consent from parents, contact details were shared with the research team who then organised an interview with a parent.

All those who were invited to take part in an interview were sent an invitation by email with an attached information sheet which contained an embedded link to the privacy notice. Those interested in taking part were asked to contact the study team by email or telephone and a date/time for the interview was arranged.

Sample

Through the options appraisal conducted with key stakeholders, a recruitment and sampling approach was identified and agreed (Table 4). Interviews with stakeholders were conducted from December 2024 to March 2025, consisting of:

• Five focus groups (15 participants total) with DBI Level 2 services involved in the test of change. This included service leads and DBI practitioners who worked directly with young people.
• Ten interviews with Level 1 referrers from schools (n=8) and CAMHS (n=2)
• Six interviews with strategic stakeholders with key involvement in DBI
• One interview with a parent of a child receiving DBI

Fieldwork

Interview and focus group topic guides for use with stakeholders were developed in consultation with the Research Advisory Group convened for this research and agreed with the Scottish Government.

All interviews and focus groups with stakeholders were conducted online. Interviews took an average of 39 minutes and focus groups took an average of 85 minutes to complete. Interviews were audio-recorded and transcribed verbatim. No incentives were offered to stakeholders to take part in the research.

Qualitative analysis

Transcribed interview and focus group data were managed and analysed in Microsoft Excel using a framework approach for qualitative data analysis (Gale et al, 2013; Richie et al, 2014). The Framework approach used a 'matrix' to conduct case-and-theme-based analysis by identifying key areas or themes emerging from the data and conducting a process of charting, whereby data from each interview is summarised under each area/theme. Researchers met regularly to discuss themes and resolve any disagreements. The data were then analysed to identify the range of experiences and views, identifying similarities and differences.

Emerging findings from the qualitative interviews were presented to 3 young people with lived experience of accessing mental health support but not DBI. This discussion enabled the research team to develop a more comprehensive understanding of the topic through wider lived experience, leading to recommendations on how to facilitate translation of these findings into practice.

Limitations

A number of limitations were associated with this evaluation. Due to the scope and timescale, it was not possible to:

• Collect quantitative data from a large number of young people under 16.
• Add additional questions or scales to the routine DBI data.
• Compare outcomes between young people who received DBI and a control group who did not.
• Access individual-level DBI data, limiting analysis to aggregate tables.
• Assess medium- or long-term outcomes for those delivering or receiving DBI support.
• Explore whether different referral pathways lead to different outcomes or experiences.
• Evaluate the cost-effectiveness of DBI for under-18s.

There were also limitations in the qualitative data collection. The sample of young people was restricted by availability during the data collection period and reliance on DBI Level 2 staff to identify participants, which affected demographic representation and coverage across providers. Since DBI was extended to 16–17-year-olds in 2019, some stakeholders had difficulty recalling details when asked in 2024 to reflect on its implementation. Recruiting parents of young people who received DBI support also proved challenging.