Understanding survey nonresponse behaviours: evidence and practical solutions

2. Barriers and potential solutions: capability

The ‘capability’ element of the COM-B model relates to a person’s psychological and physical ability to engage in the behaviour. In the context of taking part in a survey, ‘capability’ might include whether a potential respondent understands what they are being asked to do and whether it is something they would be cognitively and physically able to do.

This research identified three main barriers relating to capability: low awareness; failure to open or read the advance letter (and therefore to understand what they are being asked to do); and accessibility concerns.

Low awareness

Summary

Low awareness of the three main Scottish Government surveys, or of government surveys and statistics in general, was seen by interviewers and survey leads as a barrier to capturing the attention of potential respondents. The general public research supported the view that awareness is low, and that when linked to a lack of appreciation of its purpose, this may be a barrier.

Awareness of surveys is something that is almost impossible to accurately measure (since surveys themselves would be the most obvious route for measuring this and would, by default, risk excluding those with the lowest levels of awareness). However, a lack of awareness was identified as the second most common reason for non-completion of the 2022 Scottish Census (in a doorstep exercise to follow-up non-responding households).^[5] There was also a belief among stakeholders interviewed for this study that there was a general societal lack of awareness or understanding of the purposes of government surveys and the importance of statistics to shaping and evaluating policy decisions. Prior awareness of specific Scottish Government surveys was certainly low among the members of the public interviewed for this study.

While not necessarily a critical barrier (the purpose of the advance letter and the interviewer calling is to introduce respondents to the survey and explain more about it), low general awareness of government surveys and a lack of understanding of their purpose or value (something we return to in chapter 5) can arguably contribute to a wider climate that makes potential respondents less receptive and the interviewer’s task more challenging.

Potential solutions

The main potential solution discussed by both professional stakeholders and general public participants was to advertise Scottish Government surveys and statistics more widely to raise public awareness. Suggestions for targeted advertising included:

• GP surgeries (for SHeS specifically – which could help increases its legitimacy)
• Council buildings
• Public transport, like buses and trains
• Social media.

A group with participants from South Asian backgrounds noted that they would be more likely to engage with social media posts if they were in the language they spoke at home, so tailoring social media adverts in different languages could help raise awareness among particular communities.

The cost of advertising would vary depending on the nature/scale of any campaign. The direct and immediate impact on response rates would likely be low overall, since few of those who hear about the surveys through advertising are likely to be invited to take part in them. However, it could, in the longer-term, have a role in promoting the overall value of government surveys and helping to establish a more positive public climate for them. Thinking longer-term, the Scottish Government could also work with NRS to identify opportunities to incorporate its main general population surveys with census communications, where appropriate. For example, the ‘thank you’ messages people receive after they complete the census could also reference helping government if people get requests to take part in other surveys for the Scottish Government or ONS.

ONS’s recent paper setting out their improvement plan for their economic statistics includes proposals to improve public recognition and views of official surveys and statistics, as part of an aspiration to drive a ‘refreshed citizen relationship’. However, this paper also recognises the wide uncertainty over the impact of this kind of marketing in its discussion of plans to try and drive a ‘refreshed citizen relationship’ with the organisation and its surveys, noting that:

“While citizen engagement, targeting marketing and communications campaigns are routinely employed by many organisations, this area of work within the plan has some of the highest levels of complexity and uncertainty. It remains a key area to improve and research, but realism is required as to the ability to cost-effectively reverse societal trends seen across the whole survey sector.” (ONS, 2025)

If the Scottish Government decides to develop similar plans to improve perceptions of its surveys and statistics, a similar degree of realism about its potential impacts is likely to be required.

Failure to open or read the advance letter

Summary

Survey interviewers suggested that few people open and read the advance letters before they call. This was supported by qualitative research with the general public, which indicated people struggle to distinguish them from ‘spam’ mail.

The three main Scottish Government general population surveys all send an advance letter with a study leaflet to all sampled addresses. This is intended to avoid ‘cold calling’, so that potential respondents are already aware the survey is happening and that an interviewer will be calling. The letters and leaflets also include some background on what is involved, and the purpose and uses of each survey.

However, the survey interviewers who took part in this study were dubious about whether many people actually opened or read the advance letters. This was confirmed by qualitative research with the public. Participants were told in advance that they would be sent a letter to open and read during focus groups (as the researchers wanted to see their ‘real time’ reactions to the content). Despite this, some participants had thrown away or misplaced the letter. It was suggested that the envelope looked like “spam”, specifically because of the lack of personalisation (letters are typically addressed to ‘The Householder’ as the sample is drawn from the Postcode Address File, which does not include names).

"I probably would not open it. You know, if this is supposed to be some sort of government survey, you'd think that they would have some idea of who lives here" (Public Group 4, aged 25+, rural areas)

Participants expressed mixed views about the logos featured in the survey materials. For some, the SHS and Official Statistics logos provided reassurance or stimulated curiosity about the contents. Others however, voiced concerns that commercial or other organisations use official-looking logos as a tactic to deceive people into engaging with them.

Those participants who thought they would open the letter indicated that there would be a very small window of opportunity to encourage them to read it and that they were likely to skim the letter at most (some participants did not spot that there was additional text on the reverse until this was pointed out to them). Aspects of the design or content that appealed to participants included:

• The header ‘Help improve public services in [your local area]’, which stood out for some as conveying how taking part was relevant to them. However, members of the stakeholder group noted potential tensions, as this wording might raise expectations that the survey is a more immediate route for feedback on issues with council services, rather than informing longer-term, wider government policy. SHeS interviewers also noted that the advance letter for the SHeS led some respondents to assume more of the survey was about the NHS, rather than their own health (which is the main focus).
• The reference number, telephone contact number, and guarantees of confidentiality were all seen as reassuring.
• The Scottish Government logo, which appears on the letter but not the envelope for the SHS, was reassuring to some participants. However, survey interviewers reported encountering potential respondents who said they did not want to engage in anything associated with the current Scottish Government. Survey interviewers also perceived an increase in individuals with anti-establishment or anti-politician views for whom any mention of the Scottish Government might be off-putting (a topic we return to in chapter 4).
• Generally, the writing style was felt to be easy to understand.

Alongside the features that participants found appealing, several aspects of the survey materials were viewed less positively. The letter was perceived as being too long, which some felt might discourage people from reading it in full. There was a general preference for the leaflet, which was considered more visually appealing and easier to digest. In addition, the inclusion of “an interviewer will call” was seen as off-putting for some, as they felt they had not agreed to this. Concerns were also raised about the accessibility of the letter to disabled or neurodivergent people; these are discussed in more detail below.

Potential solutions

Finding a way to personalise invitations – in particular, by adding names – would arguably be the most effective change here. However, given that changing the sample frame to allow for more personalised invitations is not currently feasible, redesigning the envelope and advance letter based on general public feedback presents the most suitable approach to encourage more people to open and read the survey materials. Any redesign should prioritise:

• Making the envelope stand out and distinguishing it from ‘spam’ mail or items perceived as irrelevant or negative (such as bills).
• Conveying information that will encourage people to ‘hear the interviewer out’ on the doorstep in as concise and engaging a manner as possible.
• Testing different logos or combinations of logos through experimental designs, due to the lack of consensus about their impact on response rates.

In addition, the content of the letters themselves could be reviewed and ideally shortened) in light of general public feedback (more detail on which is included in Appendix D). This review process could range from light touch, through to more expensive user testing and design processes. It is important to note, however, that advance letters for the SHS, SCJS and SHeS have already undergone regular review over time.

Survey interviewers emphasised the importance of having advance letters even if recipients do not read them, as these letters provide a reference point that helps establish legitimacy during doorstep interactions. At the same time, survey interviewers were sceptical that further improvements to the letters alone would substantially increase response rates, since skilled survey interviewers often succeed in persuading people even when letters have gone unnoticed.

Accessibility issues and concerns

Summary

Recent qualitative research has identified a range of accessibility issues and concerns that can prevent disabled people from taking part in surveys. This was supported by the views and experiences of disabled people interviewed for this study.

Several accessibility issues and concerns were identified as potential barriers that may prevent disabled people from taking part in surveys, starting with the advance materials. Disabled and neurodivergent participants raised concerns about the formatting, structure, length, and size of text on the survey advance letter:

“I probably would want to take part, but I would lose interest in this letter after the first line… the way that it's formatted tells me that they're not interested enough, in my opinion, to contact me in a way that's easy for me to process. So that would tell me right away, it's not for me.” (Interview 3, Neurodivergent person)

Beyond these initial hurdles, disabled participants identified further challenges related to taking part in the survey itself. These included concerns about whether the questionnaire would be accessible and understandable, whether survey interviewers would adopt measures (e.g. wear masks) to ensure a safe and comfortable environment, and whether there would be sufficient flexibility in scheduling of interviews to account for fluctuating symptoms.

“If [the interview] was like all in one chunk and it was going to be over an hour or something like that, A: I'd start to get fractious and B: I'd get quite fatigued, so I'd ask them to break it into chunks.” (Interview 4, Disabled person)

Concerns raised within this research reflect similar issues identified in ongoing work by ONS to explore barriers to participation among disabled people. This work found that even where advance letters mention that interviews can be conducted in alternative formats, potential respondents are not always clear if those accommodations will apply to them (for example, how self-complete sections are made accessible to those who are visually impaired, or whether sign language interpreters can be made available). Discomfort with having strangers in their home was also found to be higher among the disabled people they spoke to.^[6]

Potential solutions

Addressing the accessibility concerns discussed above needs to involve both communicating existing accessibility options more clearly, and identifying and removing any remaining barriers. Specific actions could include:

• Reviewing advance letters and leaflets to ensure they meet accessibility guidelines, such as that referenced in the UK Government’s guidance on inclusivity and accessibility in survey development. Various online tools also exist for checking different aspects of accessibility, such as the reading age at which particular documents are pitched.
• Commissioning an accessible design organisation to review advance letters and leaflets to ensure they are as accessible as they can be.
• Highlighting accessibility options more clearly to potential respondents, both in letters and on the doorstep. This could include, for example, highlighting options to schedule or split interviews into multiple parts, to arrange sign language interpreters, and for survey interviewers to wear masks if requested.
• Considering whether survey interviewers would benefit from additional training around disability awareness, so that they are better equipped to respond to any adjustments that may be needed to secure participation from disabled people.
• While not specifically mentioned in the current research, enabling people with cognitive impairments or learning disabilities to participate in surveys also requires strong respondent centred survey design to ensure that the questionnaire, materials and interviewer approach can meet their accessibility requirements.

As discussed in chapter 1, there is limited evidence that disabled people are statistically under-represented in social surveys, so the impact of the above actions on overall response rates may be relatively limited. However, from an ethical and inclusivity perspective removing barriers (both real and perceived) to participation is important. Moreover, ‘disabled people’ are not one homogenous group. Taking a broader perspective on removing accessibility barriers could improve representation from specific sub-groups of disabled and/or neurodiverse people, and thus improve the representativeness of the sample.

More radical solutions could involve more proactively offering different modes of participation to encourage those who may be uncomfortable (whether for reasons related to their disability or not) with having survey interviewers inside their home. This is discussed further in chapters 3 and 4.