Distress Brief Intervention - under 18s elements: evaluation

Discussion

The aims of this evaluation were to assess whether DBI can be an effective intervention for use with young people aged under 18, explore the changes which are required in the design and implementation of the DBI compared with its use in adults and understand the mechanisms of delivery and the contextual factors that contribute to its successful implementation.

In this section, we assess the evidence that the evaluation has provided in response to these aims, highlight key findings and discuss improvements and adaptations which could be made to the intervention. First, the delivery of the under 18s elements of DBI is explored, followed the potential future directions for DBI for under 18s including further roll out of the test of change, expansion to younger age groups and introduction of additional referral pathways. Finally, the limitations of the evaluation are discussed.

This evaluation draws together the views of young people who have accessed support from DBI and those who were involved in the design and delivery of the intervention. It also includes analysis of the data routinely collected by DBI Level 2 and workshop discussions with stakeholders to address the detailed research questions and meet the overall aims of the evaluation.

This evaluation provides valuable insight into the implementation of the under 18s elements of DBI and the short-term outcomes it has produced for those involved in accessing and delivering it at this early stage of its development. The delivery of DBI to 16 and 17 year olds as part of the adult programme, the CDRS and the test of change delivery of DBI to under 18s via new referral pathways are included in the scope of this evaluation.

Delivery of under 18s elements of DBI

Facilitators to implementation

Key facilitators for the implementation of the test of change included strong policy drivers, local perception of pressure on CAMHS, and DBI’s reputation as a reliable service.

Strong leadership and governance, in particular the partnership between the DBI Central team and the University of Glasgow in ensuring that the development of the programme was based in evidence, and the input of the Children and Young People’s Advisory Group, have been central to the successful implementation of the under 18s elements of DBI.

The policy environment regarding distress and children and young people’s mental health has also contributed to the development of the extension to 16 and 17 year olds, and the test of change. The tripartite structure of DBI in the test of change was also a perceived facilitator and the inclusion of CAMHS in the referral pathway provided reassurance to schools.

The pre-existing DBI network and structures, and that the test of change was implemented largely in the original DBI pilot sites where the service is well-developed and embedded, contributed to successful implementation in these areas. Where DBI has been implemented for under 16s without this structure and has instead been embedded into other pre-existing service structures, there have been significant deviations from the model used in other sites such as support being provided for a longer period and additional types of support being offered. It is not possible within the scope of this evaluation to compare the efficacy of these different models of delivery.

The strong relationships that have been developed between Level 2 services and those who refer into them from schools and CAMHS have also been crucial to the success of the delivery of DBI via the test of change. These relationships take a substantial amount of time and effort to develop and maintain, particularly in the context of staff turnover.

Buy-in at all levels, from those involved in leadership and governance to those delivering and referring into DBI, seemed to facilitate implementation, although it should be noted that those who were more engaged in delivering DBI were more likely to engage in the evaluation, meaning that the evaluation may not represent the views of those who were less involved or who struggled to implement the test of change in their schools.

The core principles of DBI - namely a compassionate approach, contact within 24 hours following referral and short-term community-based problem-solving support, wellness and distress management planning, supported connections and signposting - remain central for the delivery of DBI through the test of change.

The immediate and timely contact following referral was highly valued by Level 1 referrers and young people and the attempt to contact within 24 hours should be retained as a central component of the test of change.

Training for Level 1 referrers and Level 2 staff members

Both Level 1 referrers and Level 2 staff members gave positive feedback on the training they received, and this training helped them to build the skills and confidence to deliver DBI to under 18s. DBI is a valuable resource for Level 1 referrers in schools and CAMHS and is seen as filling a gap for immediate support for young people in distress.

Outcomes for young people, Level 1 referrers and Level 2 staff members

The findings of this evaluation provide evidence of promising practice, and indications that DBI is effective in meeting the intended outcomes identified for children and young people, Level 1 referrers and Level 2 staff members. The extension of DBI to 16 and 17 year olds as part of the national adult service has been well implemented and is now embedded in the service.

Young people received a compassionate response at Level 1 and Level 2, and uptake was high among young people, with over 80% taking up at least one supportive phone call. There is evidence that young people feel less distressed during DBI support compared to when they started. In some cases, they receive person-centred and flexible support that helps them develop tools and strategies to manage their distress, making them feel heard and validated.

Feedback from young people suggested that the paperwork and worksheets used in their sessions could present barriers to them being able to talk about the issues of most importance. Care is therefore needed in balancing the approach taken by Level 2 providers so that the young person has the opportunity to speak and discuss issues while also using tools and worksheets to ensure that support is truly person-centred.

Best practice should be shared on achieving a balance of using tools and worksheets with the time required for the young person to talk about the issues of most importance to them. This will support greater consistency of person-centred approaches across Level 2 providers. Where possible, young people should have the option of face-to-face sessions, in a location of their choosing, as this appears to be their preference.

Supported connections and signposting for young people does not appear to be working as it does for adults accessing DBI, and young people reported being signposted to services or apps that they would not use. This may be due to fewer services being available to young people, and that they are less likely to access services themselves due to a lack of confidence or ability.

Opportunities to support young people with accessing additional services should be explored, such as the Level 2 providers developing relationships with other services, making contact with services on the young person’s behalf or having a joint meeting with the young person and another service. In line with the principles of GIRFEC and reflecting a school’s duty of care to young people, it may be that schools are well placed to support young people in accessing further support, such as school counselling or input from an educational psychologist.

Development of Theory of Change

The initial Theory of Change presents our understanding of how DBI for under 18s was initially conceptualised and delivered. Following review, some minor amendments were made to the outcomes to improve clarity but there were no substantial changes made to the outcomes. These changes include:

• Level 2 outcome has been revised to read: ‘6a. DBI level 2 staff members contact the young person within 24 hours, engage with the young person referred and offer accessible support for up to 14 days, including developing a distress management plan’
• One outcome for young people has been removed as it was covered sufficiently by other outcomes. The removed outcome referred to young people being able to access ongoing support if needed

Where DBI fits alongside other services

From the qualitative evidence presented, young people are frequently being referred to DBI while on the waiting list for other services, indicating that DBI may provide an additional, rather than alternative service. This finding should be considered in the wider context of the principles of Getting it right for every child, Scotland’s commitment to provide all young people and their families with the right support at the right time, to ensure that young people are receiving the support which is most appropriate for them, rather than being referred to multiple different services.

DBI may be playing an invaluable role in reducing and managing distress when lengthy waiting times are encountered, and there may be a need for a specific service to address this gap in provision. However, the routine data collected by Level 2 does not capture whether young people are waiting to access another service. Therefore, it is not possible to know how commonly DBI is being used in this way. The information shared at referral by Level 1 referrers and the data collected by Level 2 services as part of the routine dataset should include a field to record whether a young person is on the waiting list for another service.

Suggested improvements to delivery of DBI for under 18s

DBI providers should consider the need to offer DBI for up to 21 or 28 days where appropriate, particularly for young people aged 15 or under who access DBI as part of the test of change pathway. The rationale for extending DBI in this way reflects both the evidence that the average length of the intervention is longer in young people than in adults, and the desire of stakeholders and young people for a longer period of support. Initial support periods of 28 days or 4-6 weeks were suggested by those who took part in the evaluation.

Additionally, data on the length of intervention indicated that over three-quarters of young people had completed their support within 3 weeks and almost 90% within four weeks. A number of factors are responsible for this, including the longer time taken to build rapport between young people and their Level 2 provider, and operational issues associated with the test of change such as Level 2 staff members only being available in school one day a week, with the result that young people may receive fewer sessions within 14 days compared with adults. However, care must be taken that the intervention remains brief and time limited to ensure that the fidelity to the core principles of DBI is maintained.

Future developments of DBI for under 18s

Further roll out of the test of change

The strong desire from referrers and providers that DBI should be implemented more widely to under 18s across Scotland via the tripartite referral pathway is acknowledged. While there are indications of promising outcomes for young people who access DBI and clear examples of good practice and recognising the importance of equity of access to services, this evaluation is not of the scope to state definitively that wider roll-out across Scotland would be successful. Unlike the evaluation of the DBI pilot in adults, there was not the opportunity for this evaluation to administer validated quantitative measures of distress with a large sample of DBI participants.

To generate further evidence of the effectiveness of DBI in under 18s, particularly those who access DBI via the tripartite pathway, it is suggested that a short, validated scale is introduced within the Level 2 dataset. This would allow the collection of more robust evidence on the impact that DBI has on young people. The opportunity for administering this at various time points, including longer-term follow-up should be explored. The Short Warwick–Edinburgh Mental Well-being Scale (SWEMWBS) or the Young Persons’s Clinical Outcomes in Routine Evaluation (YP-CORE) would be particularly suitable for this as both have been used extensively with young people in evaluations.

There should also be further consideration of the reasons for lower uptake and reluctance to refer to DBI in some schools as it was not possible to fully explore this in the evaluation and it would provide valuable information regarding barriers to further roll out.

While it may be possible to make some amendments to DBI and the way it is delivered in the future, it is important to maintain fidelity to the DBI model, as it has been recognised as being robust and working well by stakeholders in this and other evaluations.

If a wider rollout of DBI for under 18s is undertaken, it should be implemented incrementally and accompanied by a robust, independent evaluation. Such as evaluation should include a well-designed control group from the outset. One possible approach is a stepped wedge cluster randomised trial, which could help address the lack of a control group seen in this and previous DBI evaluations. This evaluation design involves introducing an intervention sequentially to a number of “clusters”, which in this case could be schools or local authority areas, while collecting data from all clusters so that each cluster contributes evaluation data under both control and intervention observation periods.

Regardless of the chosen evaluation design, there would be benefit in an independent outcome evaluation – a type of evaluation which assesses the changes brought about by the intervention and entails collection and analysis of data relating to defined outcomes – which builds in use of a control group. This more detailed and robust approach would add weight to the findings of this evaluation, which demonstrated that there was evidence of promising practice, and that DBI is effective in meeting the outcomes identified for children and young people, Level 1 referrers and Level 2 staff membersDifficulties related to the COVID-19 pandemic meant that involvement of young people in the design of the test of change was somewhat limited. Any future development of DBI for under 18s should be informed by engagement with young people and parents.

There is also a need to more fully understand where DBI for under 18s sits in the landscape of services that support children and young people with their wellbeing and mental health and how wider roll out would be managed at a local authority level, for example, how DBI might fit with the delivery of mental health and wellbeing support in general practice, for example, by MH nurses, Occupational Therapists, and child and wellbeing practitioners.

Expansion of DBI to younger age groups

Data from Level 1 referrers in schools indicate a clear need for immediate support for S1 and S2 pupils experiencing distress. However, it is unlikely that DBI would be suitable for this age group given the considerations of the evidence basis in the unpublished scoping review for limiting DBI to S3 and above, and the concerns voiced as to the complexity of distress in young pupils, their psychological maturity, their ability to engage with DBI in its current format, the age of consent and the likely need to involve families. Therefore, DBI in its current form should not be extended to those younger than S3 at this stage. Opportunities for a more family-based support provision which is available on a similar timescale should be explored.

Additional referral pathways

If additional referrers to DBI for under 18s, beyond schools and CAMHS, were to be considered, GPs were thought to be most appropriate and could provide valuable coverage over the school holiday periods. Non-school meeting places for DBI sessions should be explored for young people who are not referred via their school. While there is a desire from NHS 24 to refer under 16s to DBI it is acknowledged that there are a number of areas for consideration regarding the efficacy of doing so.

This evaluation found young people had a strong preference for face-to-face sessions and, given that there is not national coverage of DBI for under 16s, much of the support would have to take place virtually and it is unclear whether this would be effective. Therefore, opening the NHS 24 referral pathway for under 16s could only be implemented if DBI for this age group was more established across Scotland. Evidence submitted by NHS 24 following a meeting of the Scottish Parliament Health, Social Care and Sport Committee in June 2023 suggests that parents, carers and guardians tend to call NHS 24 on behalf of young people aged under 16. Therefore NHS 24 would need to develop effective ways of engaging directly with the young person to enable DBI Level 1 to take place.

Limitations

The evaluation’s main limitations are that it was not possible to conduct medium or longer-term follow up with young people or services to assess the longer-term outcomes for young people of accessing DBI, though it provides rich qualitative evidence regarding the participants’ views and experiences. It was not possible as part of this evaluation to collect quantitative data at a larger scale, or to compare outcomes among young people who accessed DBI with a control group of young people who did not receive the intervention.

It should be noted that the evaluation used an opt-in approach to recruitment and is therefore likely to have involved those with more supportive views and experiences. The perspective of those who did not access or engage with DBI, and of referrers from schools who do not refer into DBI, are therefore not included.

Recruitment of parents of young people who have received support from DBI to take part in the evaluation proved challenging and therefore this report offers limited perspectives on parents’ views and experiences, though it should be acknowledged that parents would not necessarily know whether a young person for whom they were responsible had been in contact with DBI, or the details of the support they received.

This evaluation did not explore the cost effectiveness of the under 18s elements of DBI. There is some indication that access to DBI reduces workload for school staff and may reduce inappropriate referrals to CAMHS. Clarity regarding the intended impact of DBI on service use and whether the service aims to generate any cost efficiencies in terms of reducing demand on other services such as CAMHS is required before cost effectiveness of the programme can be fully assessed.