Counting the Cost of Choice and Control: Evidence for the costs of self-directed support in Scotland

APPENDIX 1: METHODOLOGY AND ANALYSIS

Sampling frame

The project used a flexible design, developing and refining costing methodologies which have been validated in other projects but whose reliability and applicability had not yet been robustly tested within this particular context (Bowes and Bell, 2007; Glendinning et al, 2000; Glendinning et al, 2008). We sampled three Scottish Local Authorities with a high volume of SDS users (in the form of DPs, as these were the easiest group on which to access data) crossing several user groups, with a mix of urban and rural provision, where it was possible to establish the availability and accessibility of the data we needed.

In sampling the non-DP users for comparability, we matched user groups, service users and levels of need as far as possible. In sampling our range of SDS users for the user-derived costs, we initially stratified this sample according to levels of need using Local Authority data and collected additional data on need levels from users themselves using a structured questionnaire and validated measures such as the GHQ IADL scores to ensure our findings were valid and reliable.

Data sources

User survey

A postal questionnaire was distributed to a total of 210 SDS users by three participating Local Authorities. The Local Authorities were provided with questionnaire packs containing a copy of the questionnaire and a pre-paid reply envelope, to which the Local Authorities added a covering letter and an address label. The survey instrument was prefaced by information about the study and the research team, following which there were eight sections of questions collecting: basic demographic information; data on service users' support needs; the value of their current SDS package; the support that they receive (whether paid for using SDS (DPs), using money from other sources, or unpaid support); use of their SDS package more generally; other costs associated with their use of SDS; service user views on the value of their SDS; and a section requesting contact details. We received 59 usable responses (28%).

Local Authority sample

Three participating Local Authorities were asked to provide the anonymised records of 'matched' samples of SDS (DP) users and users receiving care services arranged by the Local Authority. Users included in the samples were matched in terms of type of disability and basic demographic details such as sex and age. Local Authorities were requested to stratify the samples in terms of disability type but to select user records randomly within each type. The sample sizes varied between participating Local Authorities, based on numbers of SDS users and the authority's resources available to access the required information. Data requested as part of the anonymised records included: age; gender; marital status; household structure (if known); service user type; risk band; date current SDS / arranged package started; details of current SDS / arranged package in terms of numbers of hours and type of support; sources of funds included in package (DP, ILF etc); date of assessment; date of care plan; details of previous package(s). Supplying the requested data provided a challenge for all of the participating authorities, as individuals' records often spanned different computer and paper-based systems. No authority was able to supply all of the data requested. In total 265 usable anonymised records were returned.

Data gathered from Local Authorities was likely to miss some components and under-report the full costs of packages of care (both standard and SDS). Data gathered from individual users therefore gave a fuller picture of the range of costs to users and carers, but there were problems of recall or lack of data about full amounts, frequency and type of services and timings. We therefore used both data sources to allow for the strengths and weaknesses of each approach. We used both Local Authority data and user-provided data for the costs and service use associated with SDS packages. However, it was unrealistic to expect users to be able to provide accurate data concerning the costs and service use of standard care packages for comparison, so we relied on Local Authority data and a modified version of the costings used by Curtis (2009), adjusted for inflation and the local context.

Main qualitative sources

• Notes of meeting with Local Authority A (4 participants)
• Interview with Local Authority A Senior Managers (2 interviewees)
• Interview with Local Authority A Contracts Manager
• Notes of meeting with Local Authority B (4 participants)
• Notes of meeting with Local Authority C (2 participants)
• Interview with Local Authority C - Direct Payments Team Manager
• Interview with SDS lead in non participating Local Authority
• Interview with DP support organisation Local Authority B (2 interviewees)
• Interview with Provider 1 (voluntary sector, cross Local Authority)
• Interview with Provider 2 (voluntary sector, cross Local Authority) (4 interviewees)
• Interview with voluntary sector provider Local Authority B (2 interviewees)
• Interview with Managing Director Private Sector Provider Local Authority C
• User-specific voluntary sector provider Focus Group (8 participants)
• Interviews and email correspondence with Reference Group (2 users, 1 carer and 2 Local Authority care managers from non participating Local Authorities)
• DP policies and procedures, and internal evaluations from participating Local Authorities
• Workshop notes (15 participants, including reference and advisory group members, representatives from participating Local Authorities, provider organisations, local and national user advocacy groups)

Reliability and validity of instruments and results

We based all costing tools and measurements on previously tested and validated measures (both in our own previous research and in similar studies in other contexts).

We appointed a reference group comprising a range of stakeholders, including Local Authorities care and commissioning managers, provider managers, and SDS users who were not part of the sample. We used them to check the external validity and reliability of our costing tools and findings as it is our experience that even previously-validated costing and measurement tools cannot simply be replicated in different contexts.

As far as possible we developed costing methodologies that allowed for the comparability of data across Local Authorities, and that could be linked to the development and implementation of Resource Allocation Systems. This ensured the internal validity and reliability of the measures and findings, and provided a framework for the improvement of data gathering and costing for the future development of SDS services across different Local Authorities in Scotland.

However, our ability to do this was limited due the availability and accessibility of Local Authority-derived data and the findings in one Local Authority are not necessarily transportable across different contexts. To a certain extent our use of alternative costings methods based on user-derived data did mitigate against the limits of comparability due to data insufficiency and improve the internal reliability and validity of our findings. In addition, we checked the validity and reliability of our findings across Local Authorities with invited stakeholders (participants in the study and wider informants, including representatives from user and carer organisations, through our reference group and workshop).

Ethics and equalities issues

Prior to commencing fieldwork, ethical approval was sought from the Ethics Committee of the School of Applied Social Science at the University of Stirling, which conforms to the Economics and Social Sciences Research Committee's Research Ethics Framework. The research team adhered to the ethical guidelines of the Social Research Association and the Social Policy Association, which can be found at http://www.the-sra.org.uk/ethical.htm and www.social-policy.com respectively. The key ethical considerations which were relevant to this project included:

• Risk of harm to participants: The risks of this were low due to the non-invasive and non-sensitive nature of the research. Data from Local Authorities and users which was used for costings purposes was fully anonymised, and individual-level data from users was not fed back to care managers in an identifiable form. We anonymised the Local Authority sites in this report and will do so in other dissemination activities, although it was not possible to ensure internal anonymity for the study participants. However, we do not feel this impaired the reliability and validity of the research results. All service users were reimbursed for their time (with high street vouchers to reduce the risks associated with directly paying vulnerable adults). This resulted in fairly good response rates for the postal survey. Any areas unmet need that were uncovered during the course of the research were dealt with by users and carers being given information about services and support in their area wherever possible to ameliorate this risk. All the research team were Disclosure Scotland checked and complied with the necessary adult protection legislation.
• Confidentiality: Data was anonymised and kept completely confidential for the purposes of data handing, storage, and external and academic dissemination. We have anonymised the location of the Local Authority sites and participants. However, due to the nature of the methods used, some internal participants (particularly Local Authority and care provider managers) were able to identify each other: they were aware of this when consenting to take part in the research. Data provided by users was completely anonymised and not made available in identifiable format to the Local Authority.
• Equalities issues: Participants in this study were chosen to reflect their experiences, roles and expertise, rather than being representative of the population as a whole, so it was inappropriate to stratify the sample on the basis of age, gender or ethnicity. We did attempt to pay particular attention to the recruitment of black and ethnic minority users but this proved impossible due to the low numbers of respondents.

Glossary of terms

Community Care Assessment

In order to access social care services all disabled⁵ people in Scotland have to undergo an assessment of their needs. This is usually done by a social worker or other professional working on behalf of the Local Authority, and assesses whether the disabled person has needs which are considered 'eligible'. Local Authorities have discretion in deciding what these needs are, and whether they can afford to meet them.

Direct Payment (DP)

Anyone who is assessed as needing social care services can chose to take these in the form of a 'direct payment', which enables them to employ their own carers and purchase their own services.

Disability Living Allowance

DLA is a tax-free benefit for disabled children and adults to help with extra costs they may have because they are disabled. The equivalent benefit for over 65s is called Attendance Allowance. It is not administered by Local Authorities, and is due to be replaced with Personal Independence Payments.

Independent Living Fund

The ILF was set up as a national resource to enable disabled people to live independent lives in their community rather than in residential care. It is now closed to new applicants, but some long-standing users of social care still receive funding from this scheme.

Individual Budget (IB)

The Individual Budgets pilot programme was a cross-government initiative led by the Department of Health working closely with the Department for Work and Pensions, and Communities and Local Government. The pilot was conducted over two years 2006-2007 involving 13 Local Authorities. The evaluation of these pilots can be found in Glendinning et al (2008).

Individual Service Fund (ISF)

When someone wants to use their individual budget to buy supports from a provider, it is called an Individual Service Fund (ISF). Individual Service Funds mean that

• The money is held by the provider on the individual's behalf
• The person decides how to spend the money
• The provider is accountable to the person
• The provider commits to only spend the money on the individual's service and the management and support necessary to provide that service (not into a general pooled budget).⁶

Personal Assistant (PA)

This is the term used for a carer/supporter who is usually directly employed by a disabled person holding a direct payment.

Personalisation

Personalisation is a social care approach described by the Department of Health as meaning that "every person who receives support, whether provided by statutory services or funded by themselves, will have choice and control over the shape of that support in all care settings". (Dept of Health, 2008). While it is often associated with direct payments and personal budgets, under which service users can choose the services that they receive, personalisation also entails that services are tailored to the needs of every individual, rather than delivered in a one-size-fits-all fashion. It also encompasses the provision of improved information and advice on care and support for families, investment in preventive services to reduce or delay people's need for care and the promotion of independence and self-reliance among individuals and communities.⁷