Publication - Statistics publication

# Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS): Technical Report 2015

Published: 25 Oct 2016

Detailed information on the fieldwork and data processing for the 2015 Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS).

96 page PDF

1.5 MB

96 page PDF

1.5 MB

Contents
Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS): Technical Report 2015
Precision of results and measurement of change

96 page PDF

1.5 MB

### Precision of results and measurement of change

Survey respondents represent only a sample of the total population of 13 and 15 year old pupils in Scotland, and results are therefore subject to sampling error. The sampling error is the amount by which the value of a sample estimate for a particular parameter is expected to differ from its true value in the population sampled from. This means that observed differences between sub-groups may have occurred by chance. Throughout the report only differences that are statistically significant, where we can be 95% confident that such a difference has not occurred by chance (p<0.05), are commented upon.

The formula for calculating the sampling error ( SE) of a simple random sample is:

where p = the estimate of the parameter and n=sample size.

The formula for calculating the sampling error for the differences between two estimators (p1 and p2) derived from two independent samples (assuming a simple random sample) is:

Rather than using a simple random sample, whole classes were sampled within the schools that agree to participate. Therefore, classes were clustered within those schools, and pupils are clustered within those classes. Pupils within the same class and within the same school were more likely to be similar to each other, and therefore values cannot be assumed to be independent of one another. Further details on the calculation of standard errors and design effects are provided below.

It is important to recognise that sampling error is only one of the sources of error that affect the accuracy of survey results. Other sources of error include response bias (previously discussed) and over/under reporting, both of which are difficult to quantify.

Calculating standard errors and design effects

The sample design of SALSUS is complex, involving stratification by local authority and school type (state or independent), as well as clustering within schools. In addition, weights were applied when obtaining survey estimates.

Complex design and weighting affects standard errors for survey estimates, which are generally higher than the standard errors that would be derived from an unweighted simple random sample ( SRS) of the same size. For example, clustering reduces the precision of estimates, whereas stratification can increase precision. Weighting can also reduce the precision of estimates.

The ratio of the standard error of the complex sample to that of a simple random sample of the same size is known as the design factor. Put another way, the design factor (or 'Deft') is the factor by which the standard error of an estimate from a simple random sample has to be multiplied to give the true standard error of the complex design.

A Design Factor greater than 1.0 indicates a confidence interval wider than it would be with simple random sampling, meaning that the precision of estimates is reduced. A Design Factor of less than 1.0 indicates a narrower confidence interval and greater precision.

The true standard errors and Defts for SALSUS 2015 have been calculated in STATA using Linearization (Taylor series method), also known as the propagation of variance by Kish, and was the method used from 2006 to 2013. Thirty-six strata were included for the calculation of standard errors and Defts, one for each combination of local authority and school type (state or independent sector). For five local authorities: Dundee City, Fife, Inverclyde, Moray and Stirling, the variance between clusters could not be calculated for independent schools because there was only one independent school sampled in each of these local authorities. The independent schools in these ten local authorities were combined into one stratum to calculate sampling errors and Defts. There were 851 clusters used in the calculations, one for each class that participated in the survey [4] .

The Deft values applied and adjusted true standard errors (which are themselves estimates subject to random sampling error) are shown in Tables 5 to 10 for six key variables from the survey along with 95% confidence intervals.

When comparing the 2015 key variables with the 2013 key variables, significance tests were applied based on the 'pooled standard error' for each variable (a weighted sum of the true standard errors [5] for each year). Table 11 shows the six key variables for 2013 and 2015, with the true standard errors for each year and whether or not the difference is significant at the 5% level.

Table 5: Linearised standard errors and 95% confidence intervals for the proportion of pupils who are regular smokers, by age and gender: Scotland 2015

Sample Size Proportion Linearised Standard error Binomial Wald 95% CI Defts Lower CI 6439 1.51% 0.19% 1.15% 1.88% 1.19 6712 1.65% 0.21% 1.24% 2.07% 1.29 5657 7.39% 0.48% 6.45% 8.33% 1.45 5595 7.21% 0.48% 6.27% 8.14% 1.44

Table 6: Linearised standard errors and 95% confidence intervals for the proportion of pupils who drink alcohol at least once a week, by age and gender: Scotland 2015

Sample Size Proportion Linearised Standard error Binomial Wald 95% CI Defts Lower CI 6522 2.25% 0.24% 1.78% 2.71% 1.24 6743 2.38% 0.24% 1.91% 2.85% 1.22 5714 11.96% 0.59% 10.80% 13.12% 1.45 5607 12.98% 0.68% 11.65% 14.30% 1.58

Table 7: Linearised standard errors and 95% confidence intervals for the proportion of pupils who drank alcohol in the last week, by age and gender: Scotland 2015

Sample Size Proportion Linearised Standard error Binomial Wald 95% CI Defts Lower CI 6166 3.83% 0.34% 3.16% 4.51% 1.35 6431 4.22% 0.33% 3.56% 4.87% 1.27 5485 15.89% 0.67% 14.57% 17.21% 1.44 5479 18.91% 0.78% 17.38% 20.44% 1.54

Table 8: Linearised standard errors and 95% confidence intervals for the proportion of pupils who have ever used drugs, by age and gender: Scotland 2015

Sample Size Proportion Linearised Standard error Binomial Wald 95% CI Defts Lower CI 6193 5.80% 0.39% 5.03% 6.57% 1.28 6531 4.92% 0.37% 4.20% 5.65% 1.31 5476 20.89% 0.80% 19.33% 22.45% 1.53 5490 15.94% 0.69% 14.58% 17.30% 1.47

Table 9: Linearised standard errors and 95% confidence intervals for the proportion of pupils who have used drugs in the last year, by age and gender: Scotland 2015

Sample Size Proportion Linearised Standard error Binomial Wald 95% CI Defts Lower CI 6193 4.92% 0.35% 4.23% 5.62% 1.24 6531 4.15% 0.34% 3.50% 4.81% 1.29 5476 18.78% 0.76% 17.28% 20.27% 1.52 5490 14.20% 0.64% 12.93% 15.46% 1.43

Table 10: Linearised standard errors and 95% confidence intervals for the proportion of pupils who have used drugs in the last month, by age and gender: Scotland 2015

Sample Size Proportion Linearised Standard error Binomial Wald 95% CI Defts Lower CI 6193 3.14% 0.29% 2.58% 3.70% 1.24 6531 2.88% 0.27% 2.35% 3.41% 1.23 5476 13.44% 0.67% 12.13% 14.75% 1.52 5490 8.65% 0.51% 7.65% 9.65% 1.41

Table 11: Statistical significance of comparisons between 2013 and 2015 results for key variables

2013 SALSUS 2015 SALSUS T test (2 sided) P - Value Significant at the 5% level? % True standard error Sample size % 1.82 0.199 8518 1.52 0.188 6439 1.111 0.267 No 1.71 0.166 8546 1.65 0.211 6712 0.208 0.835 No 8.27 0.48 8080 7.39 0.478 5657 1.296 0.195 No 9.00 0.467 7969 7.21 0.478 5595 2.648 0.008 Yes 1.90 0.212 8664 2.25 0.237 6522 -1.097 0.273 No 1.82 0.177 8607 2.38 0.239 6743 -1.932 0.053 No 11.86 0.513 8156 11.96 0.593 5714 -0.128 0.899 No 11.25 0.512 8020 12.98 0.676 5607 -2.061 0.039 Yes 4.08 0.255 8671 3.83 0.342 6166 0.571 0.568 No 4.22 0.294 8617 4.22 0.334 6431 0.005 0.996 No 18.37 0.634 8178 15.89 0.674 5485 2.661 0.008 Yes 19.06 0.629 8027 18.91 0.780 5479 0.146 0.884 No 2.34 0.198 8293 3.14 0.286 6193 -2.376 0.018 Yes 1.75 0.166 8425 2.88 0.269 6531 -3.778 0.000 Yes 10.92 0.494 7925 13.44 0.667 5476 -3.092 0.002 Yes 7.83 0.405 7897 8.65 0.510 5490 -1.276 0.202 No 3.81 0.289 8293 4.92 0.354 6193 -2.460 0.014 Yes 2.82 0.222 8425 4.15 0.335 6531 -3.455 0.001 Yes 16.68 0.682 7925 18.78 0.762 5476 -2.056 0.040 Yes 14.30 0.559 7897 14.20 0.644 5490 0.122 0.903 No 4.79 0.322 8293 5.80 0.393 4 -1.998 0.046 Yes 3.52 0.247 8425 4.92 0.369 6531 -3.281 0.001 Yes 19.24 0.763 7925 20.89 0.797 5476 -1.496 0.135 No 16.34 0.651 7897 15.94 0.695 5490 0.418 0.676 No