Developing a Mental Health Experience of Services (MHES) Survey for Scotland

Chapter Nine: Conclusions and recommendations

This research explored key stakeholders’ views on the need, aspirations, and methodology for a potential MHES survey in Scotland. Participants expressed a broad range of perspectives on what an ideal survey might look like and how to ensure it is useful in supporting and improving the quality of mental health care in Scotland. However, multiple stakeholders acknowledged the need to balance their aspirations with feasibility in terms of time, budget, and ethical considerations. Below we summarise the complexities and conclude the chapter with our recommended approach, cost models, and next steps for the Scottish Government.

Areas with and without agreement

The research established many points of agreement about a possible survey:

• A MHES survey would be a valuable tool for better understanding experiences and improving mental health services in Scotland.
• The survey should ideally include anyone who has experience of using or attempting to access mental health services and support.
• Survey questions should include at least some, or ideally all, key aspects of using a service, e.g. referrals and access, waiting times, moving between different services, and overall experiences.
• A mix of closed and open questions is required to understand how effectively services are performing and how they could be improved.
• Survey questions should align with the Mental Health Core Standards.
• Data collection should be at a fixed point in time rather than continuous, and avoid holiday periods
• An online survey will be the quickest and most cost-effective approach.
• The survey should not be distributed directly by practitioners.
• Provide clear upfront information to participants that participation will not negatively impact their care, they can opt-out at any time, detail the purpose of the survey, and explain when confidentiality is not applied.
• Provide the survey in multiple formats and languages to support accessibility and ensure a diverse range of voices is included.
• Multiple actions be taken to minimise unintended harms and maintain confidentiality e.g. sensitive information not being placed on invites and not reporting on small service user populations.
• Access to local or service-level data is likely to be most helpful in driving service improvements and meeting stakeholders’ needs; however, overall results may only need to be published at a national level.

However, mixed views were evident about multiple areas of survey design. This reflects the range of stakeholder priorities, and the challenges associated with uncertainty about the likely scale of, and budget available for, a MHES survey.

Further consideration needs to be given to the following issues:

• Whether the carers, family members and friends of people accessing or attempting to access mental health services, or children and young people, are included in the survey sample.
• Which services should be included, with perspectives ranging from inpatient only, to community services only, to all mental health services.
• Whether to sample using existing databases or to sample by service use.
• The desired sample size, which will depend on who and what services are included in the survey. It was agreed that a larger sample gives more scope for sub-group analysis and reduces the risk of identification but will incur a larger cost.
• The length of the survey and estimated completion time - likely ranging between 5-30 minutes but ultimately would be dependent on the number of services included.
• The period a person is asked to reflect over within the survey.
• How frequently the survey should take place and how long it should be open.
• The scale of accessibility mitigations available to support data collection, and how to ensure that data collection does not become a burden on services.
• Whether to include incentives.
• Appropriate methods of data protection, storage, security, and de-identification of responses will need to be determined and provided on the survey privacy notice.

Recommended approach

Based on the findings of this research, and acknowledging the complexities described regarding a range of uncertainties about a possible MHES survey in Scotland, we recommend that the Scottish Government develop a small-scale survey focussed on a few services which span a range of mental health settings. Specifically, we recommend developing and piloting a MHES survey with services that were early adopters of the Mental Health Core Standards.

This pilot approach provides the opportunity to evaluate the survey methodology, implementation, value, and results. It will also provide a model for embedding a patient experience survey within services to drive improvement before a larger-scale survey is conducted.

Recommended Survey Elements and Associated Recommendations

• Aim: To better understand service user experiences of mental health services in Scotland, support service development, identify areas of good performance and improvement, and inform local planning.
• Services to Include: Inpatient services and adult secondary community services within NHS Boards that were early adopters of the Mental Health Core Standards.
• Participants: Adults with experience of using these services and support.
• Sampling Strategy: Draw the sample from details collected by services for the survey. Existing data collection structures used by early adopters may also be utilised.
• Sample Size: Include all service users to ensure a broad range of voices are captured, given the limited scope of services.
• Survey Length and Completion Time: Should take no more than 15 minutes to complete, balancing concentration retention and depth of findings.
• Survey Questions: Cover as many relevant themes as possible within the time limit. Priority topics include:
• Accessing services and support
• Waiting times and their impact
• Transitions between services
• Overall experiences
• Involvement in care
• Out-patient and in-patient care
• Staff interactions
• Information about diagnosis Include both qualitative and quantitative questions aligned with Core Standards. Develop and pilot questions with individuals with lived experience.
• Demographic Information: Collect essential demographic data with optional questions for additional details. Key demographics include:
• Age
• Long-term health conditions and disability
• Deprivation
• Location
• Ethnicity
• Working status
• Sex at birth
• Gender identity
• Period to Reflect Over: Most recent experience or the past year.
• Survey Frequency: Conduct annually.
• Time of Year: Administer at the same time each year, avoiding holiday periods.
• Length of Time Open: Keep the survey open for a minimum of 4 weeks.
• Data Collection Methodology: Primarily online, with paper options available for accessibility.
• Incentives: No incentives recommended.
• Survey Distribution: Distribute through services (e.g. email or reception staff), but not directly by practitioners.
• Risk of Unintended Harm: Ensure the survey is sensitive, uses appropriate terminology, clarifies that participation does not affect care, and includes signposting to support.
• Ethical Approval Requirements: Obtain ethics and privacy approvals.
• Consent and Feedback: Secure informed consent. Clearly communicate:
• Data will be anonymised
• Reporting will prevent identification
• Situations where confidentiality may not apply
• Procedures in place to manage these situations
• Accessibility and Minimisation of Barriers: Design the online survey for screen reader compatibility, use plain language, and provide translations or easy-read versions.
• Data Privacy and Governance: Implement robust data management processes to minimise unintended disclosure.
• Survey Reporting and Dissemination: Share data at local or service level to support improvement. Report broader findings nationally.
• Levels of Analysis: Analyse data by demographics, reach, and service type. Consider breakdowns at service level, NHS Board level, Health and Social Care Partnership (HSCP) level, and national level.

This recommendation has been made because it is an effective, pragmatic, and achievable way to develop and pilot a survey and gather evidence of service user experiences. However, other options could be considered depending on the budget available.

An initial task for a working group is to establish whether the option of piloting the survey with early adopter services is preferred and feasible. If not, other services will need to be chosen for a pilot before a survey is used more widely.

To aid these initial discussions, Table 2 presents three options to consider along with their advantages and disadvantages. Option B mirrors the recommended approach detailed in Table 2, while options A and C offer narrower and larger-scope surveys, respectively.