Developing a Mental Health Experience of Services (MHES) Survey for Scotland

Appendix C: Selected quotes

Chapter Two: The aims, purpose, aspirations and need for a MHES survey

“Everyone knows there are big systemic and structural challenges… A survey that just uncovers that some people aren't having a very good time in services and their outcomes could be better isn't any use to anyone because we could all say that now. And it's no use to staff in the mental health system because they already know that and they're already trying to work in a system that they know isn't the way they want it to be. So, going back to the purpose, I would really like to see it being supportive to service staff.”

Chapter Three: Survey scope and sampling

“If you're looking at mental health, you could go really narrow in terms of just those admitted to a psychiatric hospital… At the other end of the scale, have you been prescribed a drug for anxiety and depression?... There's just not enough time in the world to do that sort of thing. But at the same time, if you're looking just at psychiatric admissions, that's a very specific and very extreme outcome that probably doesn't relate to most people's experience of mental health services within the NHS. I would personally want it for everybody, but that's just not feasible.”

“Something that we do is we look at continuous assessment. So, we're always taking information in. We have 56,000 responses to our surveys, for example, so we have a huge pod of data that we can pull upon. I think if you are doing the survey, then it's actually how do you build the survey to be developed in such a way that's continuous, so that you’re building your sample bigger and bigger and bigger every time. But I suggest you probably have to work out how big does the sample [need to] be to give you an informed opinion. Ideally you want to get to thousands of patients to make a real, very good assessment. But I think we have to consider the kind of variability within the sample - ethnic backgrounds, cultural backgrounds, but also geography is a big factor as well.”

Chapter Four: Survey content

“It's easier to analyse quantitative information. But qualitative information is really where you get a lot of the really useful information that you can see what's working for people and what's not working for people.”

“It’s good to have consistent data that's comparable across borders and areas. But then, in order to do that, you have to ask exactly the same set of questions as somewhere else. Then you essentially massively limit what you can undertake and do.”

Chapter Five: MHES survey administration

“For some people it might be really risky for them, for example, to receive a phone call to their house or a letter through their door if they are accessing services and other members of their family are not aware. That could end up in a disclosure situation that someone really wasn't ready to deal with or even should be put in that position.”

Chapter Six: Ethical considerations, accessibility, and data governance

“PBPP has two levels of panels and if you go up to the higher level of panel you'll be under more scrutiny and then much more work. Also don't underestimate how long it'll take you to pull all that together and then get approval. So, finding a survey model that doesn't require you to process personal information is actually a lot less work than the survey that does require you to process personal information.”

Chapter Seven: Data analysis reporting and dissemination

“I think we could use that data, find out what the issues are, why people are not happy, you know, why work with boards, why are they not doing what they're meant to be doing? And then we could do some improvement work around specific issues. I think ministers would absolutely love it because we can say, okay, so we are meeting or nearly meeting the performance targets in terms of the waiting times, but also there's a high level of satisfaction with the services.”

Chapter Eight: Resource and budget considerations

“In theory, the best approach without worrying about money is to have a very, very large sample or everyone included in your survey... And so, it really comes down to cost. And if it's a postal survey the cost of stamps is the biggest thing... And their [cost] of scanning in the questionnaires that come back is a much lower proportion of that. So, if you want to do it more cheaply, it needs to be done by email or text… But you'd have to be in a situation where people don't just delete that information. A letter is, especially if it comes in an envelope with something on the envelope, seen to be a more formal and important method of communication. I know people don't answer surveys by letter either, but I think the difficulty with email or text is it will get pretty heavily discarded.”

“You're making your service more effective and more efficient. You're making it less burdensome for the patients. So, the return on that investment is going to be significant in terms of quality.”

“Is there a need to report by health board or is there just a need for national level figures? Because if you just need national level figures, you effectively just need to get some responses, and if you do a small survey of, say, about four thousand people or something like that, you can comfortably get a big enough sample to do national figures. If you need to do health board level figures or HSCP level figures then you need to get a big enough sample for each area and then that just multiplies your sample out and of course multiplies out your costs as well.”