Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS) 2015: Online Pilot

Executive Summary

Background and aims

The Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS) is a continuation of a long established series of national surveys on smoking, drinking and drug use.

In the past the survey has always been administered on paper. However, as technology has advanced, the transition from paper to online administration is being considered for the 2015 wave of SALSUS. The move to a web-based survey is in line with other national surveys and reflects greater engagement with IT, particularly among young people.

In spring 2015, Ipsos MORI conducted an online pilot ahead of the potential move to conduct some of the SALSUS 2015 fieldwork online rather than on paper.

The overall purpose of the pilot was to:

• pilot the instructions given to both liaison teachers and class teachers
• ensure that the survey works as it is intended to (e.g. that the links work and that the data is submitted successfully)
• identify any problems and potential solutions
• identify any ways in which the survey processes can be improved and the burden for schools can be minimised.

The pilot was not intended to identify or measure any difference in response between the paper and online modes. This would be the purpose of the mode experiment. However, if there were major concerns (e.g. clear evidence of poor quality data), that would be highlighted.

Methods

Twelve schools, each in a different local authority, took part. The survey was piloted with four classes in each school. Eight schools administered the survey on PCs and four on tablets.

Researchers visited six schools to: observe a class completing the survey online; undertake a focus group with pupils; and undertake depth interviews with liaison teachers and class teachers who organised and administered the survey.

Researchers conducted telephone depth interviews with liaison teachers in the other six schools.

Key findings

Overall, pilot schools found that administering the survey online was relatively straightforward. Pupil reactions were positive and they would prefer to complete the survey online rather than on paper. There was only one major problem encountered: in two of the twelve schools the Local Authority (LA) firewall blocked the names of specific drugs so the survey was stopped at the drugs section. Now that this issue has been identified, we can liaise with LA IT officers to address this in advance of the main fieldwork.

Pupils were generally positive about their experience of completing the survey online and the dominant view was that an online survey was ‘easier’, ‘more fun’, ‘less dull’ and ‘more modern’ than a paper survey. In addition, they frequently mentioned the environmental benefits. Teachers also thought that pupils reacted better to an online survey because they found it easier and more engaging.

On balance, pupils tended to feel that an online survey would lead to more honest answers. However, they perceived advantages and disadvantages to each mode.

Overall, both liaison and class teachers were positive about administering the survey online and found it straightforward. The need to book ICT suites (or laptops or tablets) meant that aspect was more burdensome than administering a paper survey and required more advance planning – but it was do-able. However, they also reported that once that aspect was arranged, the actual administration of the survey with the class was much easier.

Topline analysis of the data suggests that there are no major problems with data quality. The responses relating to some key measures (regular smoking, drinking in the last week and ever taken drugs) are broadly in line with what might be expected given the 2013 SALSUS results.

On the basis that there were no unresolvable problems identified, our recommendation is to proceed past Break Point Two and undertake the mode experiment (i.e. undertake half the main fieldwork online and half on paper).