Dementia post-diagnostic support evaluation: final report

2. Evaluation approach

Aims and approach

The ultimate aim of the evaluation was to provide impartial and independent evidence-backed analysis of the effectiveness and outcomes of PDS, including impacts on both those living with dementia and their care partners. This analysis will help inform future developments.

To achieve these objectives, IFF developed an integrated process and theory-based outcomes evaluation^[17] to address the research questions:

Process

• How is PDS currently delivered across Scotland? What inequalities and inconsistencies currently exist in provision?
• What were the experiences of those delivering (staff) and accessing PDS support (those living with dementia/their carers)? How does this vary across Scotland, service delivery model and people with different characteristics?
• What encourages or discourages people from taking up PDS when offered?
• What does effective and ineffective implementation and delivery look like? For whom and in what circumstances? And how does this vary by service delivery model?
• What is the role of wider community support?

Impact

• To what extent have the outcomes outlined in the Dementia Strategy been achieved? Do outcomes vary by service delivery model and/or across Scotland? How do outcomes vary for different groups (including marginalised groups, those with young onset dementia etc.)?
• Were there any unintended outcomes?
• What difference (if any) has PDS support made to those who received it? How does this compare to those that were offered but chose not to receive it?
• What were the key mechanisms/drivers that contributed to outcomes being achieved?

Learnings

• What can we learn by comparing and contrasting the different service models of PDS delivery, including the challenges, risks, best value, quality and opportunities from different approaches?
• What improvements could be made to support the future development of PDS across Scotland (including uptake and engagement)?
• What additional monitoring data would be beneficial to support ongoing understanding of people’s experiences of PDS experience and service quality?

Research methodology

Scoping and set-up

Our first step was to carry out a document review of background and contextual information related to PDS to improve our understanding of the support and what it aims to achieve. We also conducted a data mapping exercise to understand what quantitative data was available, its quality, its format, the access rights, and how it could benefit the evaluation. We reviewed a selection of 2024-25 monitoring returns^[18] from five HSCPs that Scottish Government provided.

In collaboration with Scottish Government and the Evaluation Assurance Group (EAG)^[19] we then developed a Logic Model to underpin the delivery of the evaluation. We also developed an evaluation framework, showing the evaluation objectives, the research questions, and the methods and sources for data. The final framework can be found in the Appendix.

Qualitative case studies

We engaged six HSCPs as case study areas for the evaluation. The choice of these HSCPs was led by SG, who selected HSCPs on the basis of having a good mix in terms of PDS commissioning arrangements (two delivered PDS internally, two commissioned external providers, and two used a hybrid of both approaches) and other factors such as a mix of geographies, population, rurality and deprivation levels.

Across the six case study areas, we spoke to 110 people in total:

• 12 HSCP/commissioned provider leads during 60-minute interviews
• 36 frontline practitioners (NHS PDS practitioners, Alzheimer Scotland Link Workers) during 90-minute focus groups
• 31 community/national stakeholders (these included charities and organisations that run activities or groups for people living with dementia in their local communities and people who work in roles related to dementia strategy or steering groups) during 90-minute focus groups
• 27 people living with dementia who access PDS (service users)/their care partners during 60-minute interviews
• 2 people living with dementia who had not taken up PDS during 60-minute interviews
• 2 supplementary interviews with participants who could provide additional insight into the experience of people living with dementia from minority ethnic communities.

Quantitative data

We analysed quantitative data from two sources:

• Public Health Scotland (PHS) data: PHS release an annual dementia post-diagnostic support publication on people newly diagnosed and referred to PDS by financial year of diagnosis. The data reported in this publication is based on quarterly submissions NHS Health Boards provide to PHS. The most recent publication was based on data for the financial year 2023-2024 and included: no. of people referred for PDS and rates per 10,000 population, waiting time from diagnosis to first contact, and achievement of each part of the LDP standard^[20] (by Integration Authority Area (IAA), gender, age and deprivation quintile).

For our evaluation, we also received additional information based on data for the financial year 2021-2022, including: PDS referral source, reason for PDS termination, rates of PDS refusal, delivery model (by IAA, NHS Board, gender, age, deprivation).

• Monitoring Returns sent to SG: These are forms filled in by HSCPs on an annual basis for the SG covering data related to PDS, including: total spend, staffing levels, service delivery details, no. of people who have been referred to PDS, levels of refusal, waiting times, achievement of the LDP standard, and details on impact measurement. The monitoring return data was based on the 2024-2025 financial year. Of the 31 HSCPs, we received 30 monitoring return forms (only Aberdeenshire was missing).

Evaluation considerations

While the aim of this report is to present findings on PDS from across Scotland, high-level detail on the direct views and experiences of PDS practitioners, strategic leads and service users can only be representative of the six case study areas in which we did deep-dive qualitative research. Likewise, we can only provide good practice examples from these six case study areas, and cannot capture good practice that might be happening elsewhere in Scotland. We have aimed as much as possible to situate our qualitative findings against the quantitative data we have access to, so that our analysis and recommendations can be applicable for looking across the whole country.

Where quantitative data was available in the most recent published data set from PHS (based on the 2023 - 2024 financial year), this was prioritised for analysis and use in the final report. If the data we needed to consider was not available in this latest published set, we either drew on the additional data PHS provided for us based on financial year 2021 - 2022 or the 2024 to 2025 monitoring returns sent to the SG.

We have looked at sub-group differences as part of our analysis, for example by commissioning arrangement. Where we observed differences, these have been reported in the main report and where these are not mentioned, it means we did not find any differences.

To recruit service users and their care partners (and those who were offered but declined PDS) to the research, we relied on HSCP staff themselves to identify suitable participants to speak to. This means there is a potential positive bias in the experiences of the service users we spoke to.

Relatedly, it was important for us to hear from service users with a range of experiences and contexts. However, despite the best efforts of us and HSCP staff, challenges were faced in engaging people from minority ethnic communities in the research. To address this, we spoke to a member of the PDS Evaluation Assurance Group who was from a minority ethnic background and had lived experience. We also spoke to a member of staff from Minority Ethnic Carers of People Project (MECOPP) based in Edinburgh to gain insight into the experiences of care partners of people living with dementia who received PDS. We additionally tried to recruit interview participants through our contact at MECOPP but were unsuccessful.

Further detail on our methodology and limitations can be found in the Appendix.