Overall, 199 out of an invited 325 schools participated in the survey, giving a school response rate of 61%. This equated to 1,036 out of an invited 1,821 classes and 21,650 out of a possible 23,791 pupils participated in the survey. All pupils who completed the survey were included in the analysis of results. However, due to differences in sampling, the schools completing the RCS boost were not included in assessment of the overall response rate, discussed in more detail below.
The class response rate was 57%, 1,036 classes out of the original sample of 1,821. Overall 21,650 pupils completed the questionnaire. Based on the class response forms sent out to participating schools, this equated to a pupil response rate of 91%.
Prior to 2002, the survey was conducted across the whole of the UK and not just in Scotland. While in previous years the response rate was calculated as the product of the school response rate and the pupil response rate, this changed in 2002 when the Scottish survey became separated from the English and Welsh survey. From this point on, the overall response rate was calculated as the product of the class response rate and the pupil response rate, with the exception of 2006. The overall response rate in 2018 was 52% (Table 1 and Figure 1).
The SALSUS response rate has declined over time (Table 1 and Figure 1). This mirrors the trend seen with other school surveys and population surveys. For instance, the response rate for the Smoking, Drinking and Drug Use among Young People in England 2016 survey (the latest available at the time of writing) was 26%, down from 55% ten years prior. Similar, if less substantial, declines have also been seen in population surveys.
Table 1: Response rates for SALSUS and predecessors: 1982-2018
|Survey year||School response rate||Class response rate||Pupil response rate||Overall response rate|
Figure 1: Response rates for SALSUS: 2002-2018
Source SALSUS 2002-2018
N.B. The response rate in 2006 was calculated in a different way than other years of SALSUS.
It should be noted that Glasgow City Council originally agreed that their schools could be included in the SALSUS 2018 sample. However, as a result of their own health and wellbeing survey being delayed, they did not wish for their schools to be approached to participate in SALSUS whilst their survey was ongoing. We were not advised that they could not be recruited until after fieldwork in other areas had started. This had an impact on the response rate as Glasgow City Council schools were included in the sample, but ultimately did not take part.
Most surveys are subject to possible bias due to non-response. Within this survey there were several possible reasons for non-response to occur: school and class non-response; pupil non-response; and item (question) non-response. The impacts of non-response bias can be addressed through the use of weights which is discussed later in further detail.
School and Class Non-Response
The extent to which school non-response leads to bias in the survey results will depend on the extent to which this leads to a systematic under-representation of schools with particular features, where those features are linked with the variables the survey measures. For example, smoking prevalence can be higher at schools with a high proportion of pupils living in areas of greater deprivation.
The overall school response rate was 61%. Table 2 presents a comparison of the sample with pupil census information to allow assessment of the existence of non-response bias. This shows that the sample was representative in terms of school denomination and whether the school was independent or not.
There was some under-representation of S4 pupils (46% in the sample, compared with 49% of the school pupil population) and over-representation of 13 year olds (54% in the sample, compared with 51% of the school pupil population).
Pupils in the sample were also more likely to be in rural areas than the population profile (21% in the sample, compared with 18% of the population) which could be indicative of bias.
There did not seem to be any other obvious differences between the schools that participated and those that did not (e.g. size of school). However, it is not possible to examine or quantify all potential sources of non-response bias. For example, it may be that schools that place a higher priority on substance use education may be more likely to take part. Schools that place a higher priority on substance use education may do so because it is more of problem among their pupils. In this case the survey results may be biased by over-representing pupils who use substances. Alternatively, if the education is effective, the survey results may be biased by under-representing pupils who use substances.
|2018 Unweighted Sample %||2018 Pupil Census %s|
|In another way||3||-|
Pupil non-response within classes resulted from illness on the day of the survey, other absence (this could be authorised or unauthorised) or refusal (either from the pupil or the parent). In order to maximise the response from pupils and to limit any bias, teachers were asked to administer questionnaires for absent pupils at a later date. This led to a very high pupil response rate of 91%.
Item non-response is where respondents do not answer some questions. If the item non-response is systematic in any way, i.e. if there is a reason why some groups of respondents are less likely than others to answer a particular question, there is the potential for bias in the results.
The level of item non-response in the survey was generally low. Of 90 questions in the survey, fourteen had levels of non-response of 10% or more (see Table 3). Therefore, item non-response is unlikely to have greatly affected the results. Questions that had higher levels of non-response tended to be those from later in the questionnaire relating to attitudes and lifestyles, rather than the core substance use questions. The highest level of non-response was at Q78 which asked pupils for their postcode – this is discussed further in the section on Data Processing.
Table 3 Item non-response where proportions were equal to or greater than 10%
|Q46 – How much do you think your father/carer really knows about…?||Who your friends are||All pupils||10%|
|How you spend your money||All pupils||11%|
|Where you are after school||All pupils||11%|
|Where you go at night||All pupils||11%|
|What you do with your free time||All pupils||11%|
|Q48b||How well off would you say your family/the people you live with are?||All pupils||11%|
|Q73 – How old were you when you first?||Drank alcohol||All pupils||14%|
|Got drunk||All pupils||12%|
|Used drugs||All pupils||11%|
|Smoked a cigarette||All pupils||10%|
|Q77 - Here is a list of things that young people sometimes do in their free time, when they aren't at school. What about you?||Do nothing||All pupils||11%|
|Q78 - Do you know the postcode for your home address?||All pupils||65%|
|Q81 - Thinking about a typical week, how many evenings do you spend with friends?||All pupils||12%|
|Q83 - In school, how much have you learned about the following?||The risks to your health from cigarettes||All pupils||11%|
|The risks to your health from alcohol||All pupils||11%|
|The risks to your health from drugs||All pupils||12%|
|The effects that drinking alcohol can have on other areas of your life||All pupils||12%|
|The effects that taking drugs can have on other areas of your life||All pupils||12%|
|That your ability to make decisions can be affected by drinking alcohol||All pupils||12%|
|That your ability to make decisions can be affected by taking drugs||All pupils||12%|
|That people's views about smoking, drinking and drug use can be affected by the things their friends say or do||All pupils||12%|
|Q84 – To what extent do you agree or disagree with the following statements? 'My school provides me with enough advice and support about…?||Drinking alcohol||All pupils||13%|
|Taking drugs||All pupils||13%|
|Leading a healthy and active life||All pupils||14%|
|Q85 – Thinking about the future, how confident do you feel about…?||Having the information you need to make the right choices about your health and wellbeing||All pupils||13%|
|Saying no to Doing something that you don't want to do||All pupils||13%|
|Knowing where to go to for information and support about substance related issues||All pupils||13%|
|Avoiding getting into risky situations due to alcohol||All pupils||13%|
|Avoiding getting into risky situations due to drugs||All pupils||13%|
|Q86 - How much do you like school at the moment?||All pupils||11%|
|Q87 - How often do you feel strained or pressured by the schoolwork you have to do?||All pupils||11%|
|Q88 - In the past year, how many times did you skip or skive school?||All pupils||13%|
|Q89 - Since you started secondary school, have you been excluded?||All pupils||12%|
|Q90 Strengths and Difficulties Questionnaire||All items||All pupils||15-18%|