Developing a Mental Health Experience of Services (MHES) Survey for Scotland

Chapter One: Introduction and methodology

Background

Service user experience is widely regarded as an important measure of health care quality, and there is evidence of an association between more positive service user experience and improved health care outcomes (Doyle, Lennox and Bell, 2013). Care experience surveys are undertaken by the Scottish Government for some areas of health and social care including cancer, inpatient and maternity care, alongside the Health and Care Experience survey (HACE).

Developing possibilities for a survey of mental health experience of services is an action in the delivery plan for the Scottish Government’s 2023 Mental Health and Wellbeing Strategy. It is envisaged that a survey would measure and report on whether the Scottish Government’s recently published Core Mental Health Standards are being met. These Standards set out what people should expect from mental health services, and that services should ensure that people receive the support they need at the right time and in the right place

The implementation of a Mental Health Experience of Services (MHES) survey in Scotland could provide data and evidence that informs and measures the progress of Scottish Government outputs including 1) the Core Mental Health Standards, 2) existing and forthcoming service specifications, and 3) outcomes within the Scottish Government’s Mental Health and Wellbeing Strategy and associated delivery plan.

The results from a MHES survey could also be used by health boards, service providers, and stakeholders for service design and improvement. Additionally, they could be used for national benchmarking with the rest of the UK, such as by comparing Scotland’s results against those from the Care Quality Commission’s (CQC) Community Mental Health Survey and contributing to the NHS UK Benchmarking network.

Purpose of the scoping study and summary of approach

The Scottish Government commissioned The Lines Between (TLB) to conduct research to inform the development of options for a potential survey of how people experience mental health services. Key objectives included proposing options for the MHES survey design and delivery and exploring possible survey content.

This report presents an analysis of three sets of data collected through qualitative interviews with stakeholders and group events, an online survey and an evidence review. Each of these are summarised below.

Qualitative research

Interviews were conducted with 26 stakeholders with experience conducting service user experience surveys, potential data users of a MHES survey, technical survey experts, and people with lived experience. The Scottish Government identified an initial list of stakeholders, and additional stakeholders identified in interviews were added to the sample. Interviewees were asked slightly differing questions in a semi-structured interview depending on the nature of their work.

12 interviewees provided insight from their experience of conducting surveys, seven others with insights from a policy perspective took part, six individuals representing organisations that might use data from a patient experience survey and two with lived experience of a mental health condition (one of which also has experience of conducting surveys so is counted twice).

Researchers attended three events, the Mental Health Foundation’s Diverse Experiences Advisory Panel, the Mental Health Equality and Human Rights Forum and the Mental Health Policy Huddle, which captured the perspectives of at least 17 individuals, resulting in at least 44 individuals contributing to this report. Event attendees brought a range of experiences, such as lived experience of poor mental health, caring for/family members of people with poor mental health, and forum members from a range of foundations and advocacy groups. Attendees were asked a shortened set of questions from the full discussion guide.

Online survey

A link to a short online survey was distributed by TLB and promoted by the Scottish Government to a range of stakeholders. The TLB survey received 105 respondents, some of whom may also have been interviewed. The roles of survey respondents are shown in Figure 1. Among the 17% of respondents who selected ‘other’, roles included healthcare workers, retirees, family/friends, lived experience/service users, carers and mental health professionals.

Evidence Review

A total of 34 existing mental health and other health and social care experience surveys from Scotland, the wider UK and abroad, including Ireland, USA, Canada, Australia and Denmark, were reviewed. These were identified through previous Scottish Government and Irish Government scoping reviews, which were shared with the research team as part of the development work behind this project, as well as a Google search. The review for this study involved analysing technical reports and methodological descriptions of thesurveys idenitfiied to gather details of how they were undertaken, including factors such as sample size, themes covered, settings involved (e.g. community and/or in-patient services), participant inclusion criteria, how the surveys were administered, approaches to inviting participation and issues around reporting and dissemination of results. We have provided brief details of where the surveys cited in the evidence review were conducted, and by whom. More information can be found in Appendix B.

Analysis approach

This report presents the themes identified in our primary data, supplemented with evidence review findings. The prevalence of themes is noted throughout. Quantitative data from the TLB survey is presented in the narrative or charts; responses to open survey questions were included in the qualitatitive thematic analysis. Areas of consensus, or lack of consensus, are noted throughout and consolidated in chapter summaries, which can be used to determine the next steps in developing a MHES survey.

When reviewing the analysis in this report, we ask the reader to consider that not all interviewees or TLB survey respondents answered every question. The analysis can, therefore, only be based on the responses provided. Additionally, a few individuals with lived experiences participated in the interviews. Given the need to maintain confidentiality, only limited specific examples of their perspectives are provided in this report. Developing and piloting a survey with those with lived experience would be a valuable next step.

Terminology use and acronyms

In this report, the survey run by TLB is referred to as the ‘TLB survey’, while the survey proposed by the Scottish Government is referred to as the ‘MHES survey’.

The term ‘service user’ is used to describe those who access and use mental health services. We acknowledge the challenges in using this term as it can exclude those who do not currently have access to services, and not all people accessing services may consider themselves to be service users. The term ‘mental health condition’ is used to describe those with a diagnosis and ‘poor mental health’ is used to describe those who do and do not have a diagnosis.

Acknowledgements

TLB and the Scottish Government would like to thank all interviewees and TLB survey respondents for their contributions to this project. While participation remains confidential, we are grateful to the Mental Health Foundation’s Diverse Experiences Advisory Panel, the Mental Health Equality and Human Rights Forum, and PLUS Perth and Kinross for their help in widening the scope of the research.

The National Care Experience Programme in Ireland is currently undertaking a similar process to develop its own mental health experience of services survey. We would like to acknowledge the assistance of this team in this project, which includes participating in interviews and providing their unpublished literature review.