Publication - Statistics

Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS): technical report 2018

Published: 26 Nov 2019
Directorate:
Population Health Directorate
Part of:
Education, Health and social care, Law and order
ISBN:
9781839603327

Information on the fieldwork and data processing for the 2018 Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS).

Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS): technical report 2018
Response rates

Response rates

Overall, 199 out of an invited 325 schools participated in the survey, giving a school response rate of 61%. This equated to 1,036 out of an invited 1,821 classes and 21,650 out of a possible 23,791 pupils participated in the survey. All pupils who completed the survey were included in the analysis of results. However, due to differences in sampling, the schools completing the RCS boost were not included in assessment of the overall response rate, discussed in more detail below.

The class response rate was 57%, 1,036 classes out of the original sample of 1,821. Overall 21,650 pupils completed the questionnaire. Based on the class response forms[6] sent out to participating schools, this equated to a pupil response rate of 91%.

Prior to 2002, the survey was conducted across the whole of the UK and not just in Scotland. While in previous years the response rate was calculated as the product of the school response rate and the pupil response rate, this changed in 2002 when the Scottish survey became separated from the English and Welsh survey. From this point on, the overall response rate was calculated as the product of the class response rate and the pupil response rate, with the exception of 2006. The overall response rate in 2018 was 52% (Table 1 and Figure 1).

The SALSUS response rate has declined over time (Table 1 and Figure 1). This mirrors the trend seen with other school surveys and population surveys. For instance, the response rate for the Smoking, Drinking and Drug Use among Young People in England 2016 survey (the latest available at the time of writing) was 26%, down from 55% ten years prior.[7] Similar, if less substantial, declines have also been seen in population surveys.[8]

Table 1: Response rates for SALSUS and predecessors: 1982-2018

Survey year School response rate Class response rate Pupil response rate Overall response rate
2018 61% 57% 91% 52%
2015 67% 61% 87% 53%
2013 71% 67% 90% 60%
2010 75% 68% 91% 62%
2008 69% 65% 91% 59%
2006 69% - 82% 57%
2004 72% 71% 87% 62%
2002 75% 73% 89% 65%
2000 72% - 90% 64%
1998 81% - 89% 70%
1996 81% - 90% 71%
1994 92% - 90% 82%
1992 96% - 90% 86%
1990 97% - 90% 87%
1986 82% - 91% 78%
1984 89% - 90% 81%
1982 96% - 90% 84%

Figure 1: Response rates for SALSUS: 2002-2018

Figure 1: Response rates for SALSUS: 2002-2018

Source SALSUS 2002-2018

N.B. The response rate in 2006 was calculated in a different way than other years of SALSUS.

It should be noted that Glasgow City Council originally agreed that their schools could be included in the SALSUS 2018 sample. However, as a result of their own health and wellbeing survey being delayed, they did not wish for their schools to be approached to participate in SALSUS whilst their survey was ongoing. We were not advised that they could not be recruited until after fieldwork in other areas had started. This had an impact on the response rate as Glasgow City Council schools were included in the sample, but ultimately did not take part.

Non-response

Most surveys are subject to possible bias due to non-response. Within this survey there were several possible reasons for non-response to occur: school and class non-response; pupil non-response; and item (question) non-response. The impacts of non-response bias can be addressed through the use of weights which is discussed later in further detail.

School and Class Non-Response

The extent to which school non-response leads to bias in the survey results will depend on the extent to which this leads to a systematic under-representation of schools with particular features, where those features are linked with the variables the survey measures. For example, smoking prevalence can be higher at schools with a high proportion of pupils living in areas of greater deprivation.

The overall school response rate was 61%. Table 2 presents a comparison of the sample with pupil census information[9] to allow assessment of the existence of non-response bias. This shows that the sample was representative in terms of school denomination and whether the school was independent or not.

There was some under-representation of S4 pupils (46% in the sample, compared with 49% of the school pupil population) and over-representation of 13 year olds (54% in the sample, compared with 51% of the school pupil population).

Pupils in the sample were also more likely to be in rural areas than the population profile (21% in the sample, compared with 18% of the population) which could be indicative of bias.

There did not seem to be any other obvious differences between the schools that participated and those that did not (e.g. size of school). However, it is not possible to examine or quantify all potential sources of non-response bias. For example, it may be that schools that place a higher priority on substance use education may be more likely to take part. Schools that place a higher priority on substance use education may do so because it is more of problem among their pupils. In this case the survey results may be biased by over-representing pupils who use substances. Alternatively, if the education is effective, the survey results may be biased by under-representing pupils who use substances.

Table 2: Comparison of sample profile with pupil census information[10],[11]

  2018 Unweighted Sample % 2018 Pupil Census %s
Sex
Boys 49 50
Girls 49 50
In another way[12] 3 -
Year
S2 54 51
S4 46 49
School type
State 96 94
Independent 4 6
Denomination
Catholic 16 18
Non-denominational 84 82
Area Type
Urban 78 82
Rural 21 18

Pupil Non-Response

Pupil non-response within classes resulted from illness on the day of the survey, other absence (this could be authorised or unauthorised) or refusal (either from the pupil or the parent). In order to maximise the response from pupils and to limit any bias, teachers were asked to administer questionnaires for absent pupils at a later date. This led to a very high pupil response rate of 91%.

Item Non-Response

Item non-response is where respondents do not answer some questions. If the item non-response is systematic in any way, i.e. if there is a reason why some groups of respondents are less likely than others to answer a particular question, there is the potential for bias in the results.

The level of item non-response in the survey was generally low. Of 90 questions in the survey, fourteen had levels of non-response of 10% or more (see Table 3). Therefore, item non-response is unlikely to have greatly affected the results. Questions that had higher levels of non-response tended to be those from later in the questionnaire relating to attitudes and lifestyles, rather than the core substance use questions. The highest level of non-response was at Q78 which asked pupils for their postcode – this is discussed further in the section on Data Processing.

Table 3 Item non-response where proportions were equal to or greater than 10%

Question Subquestion Base Non-response
Q46 – How much do you think your father/carer really knows about…? Who your friends are All pupils 10%
How you spend your money All pupils 11%
Where you are after school All pupils 11%
Where you go at night All pupils 11%
What you do with your free time All pupils 11%
Q48b How well off would you say your family/the people you live with are? All pupils 11%
Q73 – How old were you when you first? Drank alcohol All pupils 14%
Got drunk All pupils 12%
Used drugs All pupils 11%
Smoked a cigarette All pupils 10%
Q77 - Here is a list of things that young people sometimes do in their free time, when they aren't at school. What about you? Do nothing All pupils 11%
Q78 - Do you know the postcode for your home address? All pupils 65%
Q81 - Thinking about a typical week, how many evenings do you spend with friends? All pupils 12%
Q83 - In school, how much have you learned about the following? The risks to your health from cigarettes All pupils 11%
The risks to your health from alcohol All pupils 11%
The risks to your health from drugs All pupils 12%
The effects that drinking alcohol can have on other areas of your life All pupils 12%
The effects that taking drugs can have on other areas of your life All pupils 12%
That your ability to make decisions can be affected by drinking alcohol All pupils 12%
That your ability to make decisions can be affected by taking drugs All pupils 12%
That people's views about smoking, drinking and drug use can be affected by the things their friends say or do All pupils 12%
Q84 – To what extent do you agree or disagree with the following statements? 'My school provides me with enough advice and support about…? Drinking alcohol All pupils 13%
Taking drugs All pupils 13%
Smoking All pupils 14%
Leading a healthy and active life All pupils 14%
Q85 – Thinking about the future, how confident do you feel about…? Having the information you need to make the right choices about your health and wellbeing All pupils 13%
Saying no to Doing something that you don't want to do All pupils 13%
Knowing where to go to for information and support about substance related issues All pupils 13%
Avoiding getting into risky situations due to alcohol All pupils 13%
Avoiding getting into risky situations due to drugs All pupils 13%
Q86 - How much do you like school at the moment? All pupils 11%
Q87 - How often do you feel strained or pressured by the schoolwork you have to do? All pupils 11%
Q88 - In the past year, how many times did you skip or skive school? All pupils 13%
Q89 - Since you started secondary school, have you been excluded? All pupils 12%
Q90 Strengths and Difficulties Questionnaire All items All pupils 15-18%[13]

Contact

Email: salsus@gov.scot