Self-directed Support Implementation Study 2018: report 2

6. Conclusions

The assessment of the evaluability of self-directed support looked at published previous evaluations of similar system-wide policies and strategies, reviewed data currently collected in Scotland against the SDS Change Map, and considered evidence from the case study research on potential issues for future evaluation,

The evidence from the literature review highlighted that evaluation of similar systems-wide policies or strategies is challenging - particularly economic evaluation - and almost entirely qualitative. The literature review found no evidence that anyone had ever evaluated ‘good conversations’. Several of the evaluations identified highlighted the need to be able to monitor change over time for individual supported people, their carers and families and service providers, and the need for longitudinal research.

The case studies revealed many of the same issues as the literature review, notably challenges associated with economic evaluation, and the need for longitudinal research.  They also identified that measures of uptake of the different options in a local authority is not a suitable proxy for genuine implementation of self-directed support and fidelity to its principles.

Current routine data collections are not sufficient to monitor or evaluate against the SDS Change Map and it is recommended that:

• longitudinal research is undertaken with people receiving social care starting from the time of initial assessment;
• a survey of staff who administer social care in local authorities is introduced;
• a slightly expanded version of the Self-directed Support User Experience Survey is used to include a high-level question on social care outcomes;
• a survey is conducted of independent and third sector organisations who hold service contracts with statutory services; and
• a reporting requirement should be introduced for local authorities to evidence and self-assess their approach to strategic planning and how they are facilitating creative solutions to achieve individualised outcomes.

Economic evaluation will remain a challenge with no quantitative counterfactual^[12] possible and as such should focus on the time taken by staff in local authorities to deliver choice and control and compare this to the outcomes collected in the expanded SDS User Experience Survey.