Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS): technical report 2018

Response rates

Overall, 199 out of an invited 325 schools participated in the survey, giving a school response rate of 61%. This equated to 1,036 out of an invited 1,821 classes and 21,650 out of a possible 23,791 pupils participated in the survey. All pupils who completed the survey were included in the analysis of results. However, due to differences in sampling, the schools completing the RCS boost were not included in assessment of the overall response rate, discussed in more detail below.

The class response rate was 57%, 1,036 classes out of the original sample of 1,821. Overall 21,650 pupils completed the questionnaire. Based on the class response forms^[6] sent out to participating schools, this equated to a pupil response rate of 91%.

Prior to 2002, the survey was conducted across the whole of the UK and not just in Scotland. While in previous years the response rate was calculated as the product of the school response rate and the pupil response rate, this changed in 2002 when the Scottish survey became separated from the English and Welsh survey. From this point on, the overall response rate was calculated as the product of the class response rate and the pupil response rate, with the exception of 2006. The overall response rate in 2018 was 52% (Table 1 and Figure 1).

The SALSUS response rate has declined over time (Table 1 and Figure 1). This mirrors the trend seen with other school surveys and population surveys. For instance, the response rate for the Smoking, Drinking and Drug Use among Young People in England 2016 survey (the latest available at the time of writing) was 26%, down from 55% ten years prior.^[7] Similar, if less substantial, declines have also been seen in population surveys.^[8]

Table 1: Response rates for SALSUS and predecessors: 1982-2018

Figure 1: Response rates for SALSUS: 2002-2018

Source SALSUS 2002-2018

N.B. The response rate in 2006 was calculated in a different way than other years of SALSUS.

It should be noted that Glasgow City Council originally agreed that their schools could be included in the SALSUS 2018 sample. However, as a result of their own health and wellbeing survey being delayed, they did not wish for their schools to be approached to participate in SALSUS whilst their survey was ongoing. We were not advised that they could not be recruited until after fieldwork in other areas had started. This had an impact on the response rate as Glasgow City Council schools were included in the sample, but ultimately did not take part.

Non-response

Most surveys are subject to possible bias due to non-response. Within this survey there were several possible reasons for non-response to occur: school and class non-response; pupil non-response; and item (question) non-response. The impacts of non-response bias can be addressed through the use of weights which is discussed later in further detail.

School and Class Non-Response

The extent to which school non-response leads to bias in the survey results will depend on the extent to which this leads to a systematic under-representation of schools with particular features, where those features are linked with the variables the survey measures. For example, smoking prevalence can be higher at schools with a high proportion of pupils living in areas of greater deprivation.

The overall school response rate was 61%. Table 2 presents a comparison of the sample with pupil census information^[9] to allow assessment of the existence of non-response bias. This shows that the sample was representative in terms of school denomination and whether the school was independent or not.

There was some under-representation of S4 pupils (46% in the sample, compared with 49% of the school pupil population) and over-representation of 13 year olds (54% in the sample, compared with 51% of the school pupil population).

Pupils in the sample were also more likely to be in rural areas than the population profile (21% in the sample, compared with 18% of the population) which could be indicative of bias.

There did not seem to be any other obvious differences between the schools that participated and those that did not (e.g. size of school). However, it is not possible to examine or quantify all potential sources of non-response bias. For example, it may be that schools that place a higher priority on substance use education may be more likely to take part. Schools that place a higher priority on substance use education may do so because it is more of problem among their pupils. In this case the survey results may be biased by over-representing pupils who use substances. Alternatively, if the education is effective, the survey results may be biased by under-representing pupils who use substances.

Table 2: Comparison of sample profile with pupil census information^[10],[11]

Pupil Non-Response

Pupil non-response within classes resulted from illness on the day of the survey, other absence (this could be authorised or unauthorised) or refusal (either from the pupil or the parent). In order to maximise the response from pupils and to limit any bias, teachers were asked to administer questionnaires for absent pupils at a later date. This led to a very high pupil response rate of 91%.

Item Non-Response

Item non-response is where respondents do not answer some questions. If the item non-response is systematic in any way, i.e. if there is a reason why some groups of respondents are less likely than others to answer a particular question, there is the potential for bias in the results.

The level of item non-response in the survey was generally low. Of 90 questions in the survey, fourteen had levels of non-response of 10% or more (see Table 3). Therefore, item non-response is unlikely to have greatly affected the results. Questions that had higher levels of non-response tended to be those from later in the questionnaire relating to attitudes and lifestyles, rather than the core substance use questions. The highest level of non-response was at Q78 which asked pupils for their postcode – this is discussed further in the section on Data Processing.

Table 3 Item non-response where proportions were equal to or greater than 10%