Understanding survey nonresponse behaviours: evidence and practical solutions

Footnotes

A complete reference list is provided in the Survey Nonresponse Research: Appendices.

1 Key features of these three surveys are summarised in Appendix F, with links to further background where required.

2 Young people are commonly underrepresented in surveys, and analysis of the three Scottish Government surveys for this research showed that the age gap in terms of response has increased over time. From 2012 to 2019, overall those from minority ethnic groups excluding white minority backgrounds also became more underrepresented, though representation did improve between 2019 and 2022 (see Appendix B for more detailed analysis on this). While the research team did not find specific evidence of disabled people being statistically underrepresented in surveys (possibly because of the correlation between disability and age), various studies have identified accessibility barriers to disabled people’s participation in surveys (see Appendix A: Literature review for further discussion of this).

3 For further discussion of survey workforce issues in the post-pandemic period, see Charmna et al (2024) The post-pandemic role of face-to-face fieldworkers, Survey Futures.

4 With the exception of disabled participants and participants from Black and Asian ethnic backgrounds, as there was interest in whether they faced specific barriers to taking part regardless of willingness to do so.

5 Data Collection Exercises – reasons for not yet completing Scotland’s census and CCS hard refusals

6 Reported at ONS seminar for the Social Research Association (SRA), 5/12/24.

7 See for example Brick and Williams (2013) Explaining Rising Nonresponse Rates in Cross-Sectional Surveys. The ANNALS of the American Academy of Political and Social Science, 645(1), 36–59, and Smith, T.W. (1984). Estimating Nonresponse Bias with Temporary Refusals. Sociological Perspectives, 27, 473 – 489.

8 Data Collection Exercises – reasons for not yet completing Scotland’s census and CCS hard refusals

9 SCJS takes around 40 minutes, while the SHeS main interview varies depending on the number of people in the household, but lasts around 45 minutes for a one adult household, while the child interview lasts around 30 minutes (interviews for multiple adults in the same household are sometimes conducted concurrently to reduce the total time).

10 If this approach were adopted, there would be a need to consider how to disincentivise survey interviewers from offering the shorter ‘core’ interview routinely (for example, by setting pay for ‘core’ interviews at a much lower rate). Otherwise, it might lead to a significant loss of data.

11 See Ormston et al, 2024, for a more detailed discussion of the cost implications of moving from a purely face-to-face to a mixed mode design.