Scottish Household Survey 2020: methodology and impact of change in mode

Chapter 6: Impact of change of mode of interview

The previous chapter examined the impact of changes to the mode of approach – the way potential respondents were invited to take part. This chapter focuses on the impact on how interviews were undertaken – the mode of interview. With no interviewer travel allowed, surveys had to be undertaken remotely, either by telephone or video. Video interviewing used one-way Microsoft Teams, so that the respondent could see the interviewer, but the interviewer could not see them.

At around an hour in length and often involving relatively complex questions and showcards, the SHS questionnaire has relied on interviewer facilitation to maximise participant engagement. The design of the questionnaire was not optimal for interviewing by telephone or video, and was reviewed and revised prior to the pilot. However, changes to the questionnaire also had to be kept to a minimum, so that the data could be compared with the data collected face-to-face prior to lockdown.

The main challenge was how to adapt questions that relied on showcards^[35]. If interviewing using video, the interviewer could use showcards via screenshare, but an alternative strategy was needed for the telephone interviews. Two main approaches were devised. First, in instances where the question was factual (e.g. ethnicity and educational qualifications), interviewers were instructed to read the question, wait for the respondent to answer and then select the corresponding code. Interviewers then confirmed the code they had selected with the respondent before continuing. Second, for questions where the range of response options are not obvious from the question, the interviewer was directed to read out all the response options along with the question.

Additionally, where there might be ambiguity – for example when a question related to the participants' "usual" pre-COVID circumstances, or their current circumstances – this was clarified in interviewer instructions and/or through tweaks to the question wording. There were also a limited number of questions that were temporarily paused from the survey because of their complexity and heavy reliance on detailed showcards.

As discussed in Chapter 3, the impact of mode of interview on how people respond is more complicated and harder to estimate than the impact of mode of approach on response patterns. While the impact of non-response biases on response patterns are binary – people either take part or do not – the impact of mode on how people respond to questions, and whether their measured responses accurately capture this information, is more complex. We look for evidence of this in a number of ways:

• Levels of missing information from refusals or don't know information in selected variables (missing information on earnings and benefits).
• Use of agree/disagree scales, mid-points and don't know responses (views on council services and material deprivation)
• Number of responses to multi-code questions (such as educational qualifications, number of health conditions, and cultural activities undertaken)
• Impact on rarer response categories (sexuality, religion and whether smoke)
• Other indicators of interview quality (consent to be recontacted and the random adult conversion rate).

Before we examine the impact of mode of interview on a selection of survey estimates, we briefly compare the preferences for undertaking the interview by telephone or video, and the resulting profiles of respondents by mode of interview.

Variation in mode preference and impact on sample composition

Generally, there was reluctance from both interviewers and respondents to undertake interviews by video. Overall, 16% of household respondents undertook the SHS interview by one-way video link, and 84% by telephone. (Table 6.1). A similar split was seen in the random adult interviews, with 18% undertaken by video and 82% by telephone.

Table 6.1 Mode of interview by sample type: Household interview (unweighted)

	Telephone-matched	Opt-in only	Push-to-TV all
Household respondents
Video	8.1%	22.3%	16.1%
Telephone	91.9%	77.7%	83.9%
Total	100%	100%	100%
N	1,718	1,313	3,031
Random Adult respondents
Video	8.1%	20.8%	17.9%
Telephone	91.9%	79.2%	82.1%
Total	100%	100%	100%
N	1,612	1,176	2,788

Mode of interview differed considerably by mode of approach. For the opt-in only sample, 22% undertook the household interview by video. In contrast, only 8% of those in the telephone-matched sample did likewise.

This difference is likely to be because of when respondents agreed to take part. For the telephone-matched sample, most respondents will have agreed to take part when telephoned by an interviewer. At this point, changing mode from telephone to video would have required additional effort for both interviewers and respondents, and would involve scheduling a video interview. Additionally, the possibility of reluctant respondents braking video appointments may have dissuaded interviews from switching mode. In contrast, almost all respondents from the opt-in only sample agreed to take part through the respondent portal before any contact with the interviewer. Respondents were asked to state a preference for a telephone or video interview in advance of speaking to an interviewer.

Table 6.2 shows the mode of interview by a number of household characteristics. Overall, younger householders, those working, and those in privately rented accommodation were more likely to undertake the interview by video.

Table 6.2 HIH characteristics and tenure by mode of household interview (unweighted, row percentages).

	Video	Telephone	Total	N
Age of HIH
16-24	41%	59%	100%	27
25-44	27%	73%	100%	547
45-59	20%	80%	100%	880
60+	10%	90%	100%	1,577
Total	16%	84%	100%	3,031
Gender of HIH
Male	16%	84%	100%	1,760
Female	16%	84%	100%	1,269
In another way	*	*	*	2
Total	16%	84%	100%	3,031
Economic status of HIH
Working	22%	78%	100%	1,599
Retired	9%	91%	100%	1,222
Other	13%	87%	100%	210
Total	16%	84%	100%	3,031
Tenure
Owner-occupied	17%	83%	100%	2,373
Social Rented	6%	94%	100%	394
Private Rented	22%	78%	100%	228
Other	*	*	*	36
Total	16%	84%	100%	3,031
N	489	2,542	3,031

A similar pattern is seen among the random adult sample (Table 6.3). While 25% of 16-24 year olds completed the interview by video, this proportion dropped to 9% among those aged 60 and over. Similarly, those with higher educatonal qualifications were more likely to take part via video, with 24% of those with degree level qualifications taking part by video compared to only 3% of those with no qualifcations.

Table 6.3 Mode of household interview by Random Adult characteristics (unweighted)

	Video link	Telephone	Total	N
Age of random adult
16-24	25%	75%	100%	111
25-44	23%	77%	100%	556
45-59	21%	79%	100%	715
60+	9%	91%	100%	1406
Total	15%	85%	100%	2788
Gender of random adult
Man/Boy	16%	84%	100%	1253
Woman/Girl	15%	85%	100%	1533
In another way	*	*	*	2
Total	15%	85%	100%	2788
Highest Educational qualification
No qualifications	3%	97%	100%	377
Level 1 - O grade or equivalent	9%	91%	100%	368
Level 2 - Higher, A Level or equivalent	13%	88%	100%	423
Level 3 - HNC/HND or equivalent	12%	88%	100%	331
Degree or professional qualification	24%	76%	100%	1171
Other qualification	7%	93%	100%	104
Total	15%	85%	100%	2788

Different preferences to mode of interview among different gropus meant that the sample profile of those who undertook interviews by telephone is different from those who undertook interviews by video. Details are provided in Tables A4.2 and A4.3 in Appendix 4. In summary.

• Age: Household respondents who took part by telephone were more likely to be older than video respondents, with 41% aged 60 or over compared to 17%.
• Gender: There was no difference in the gender profile by mode of interview.
• Economic status: Linked with age, telephone respondents were more likely to be retired than video respondents (33% compared to 13%) and less likely to be in employment (56% compared to 78%).
• Educational qualifications: Telephone respondents were more likely to have no qualification than video respondents (13% compared to 2%) and less likely to have a degree or professional qualification (36% compared to 56%).
• Tenure: Telephone respondents were more likely than video respondents to be in social rented accommodation (18% compared to 6%) and less likely to be owner-occupiers or privately renters.
• SIMD: Telephone respondents were more likely than video respondents to be in the most deprived quintile and less likely to be in the least deprived areas.
• Urban/rural: Telephone respondents were less likely to be in large urban areas than video respondents (33% compared to 41%).
• Property type: Telephone respondents were more likely to live in houses than video respondents (70% compared to 60%) and less likely to live in flats (30% compared to 40%).

The remainder of this chapter examines a selection of survey findings and the possible impact of change in the mode of interview on how people respond. The different profiles of those who respond by video and those who respond by telephone, as described above, should be bourne in mind.

Impact of mode of interview on selected survey findings.

Educational qualifications

Respondents in the adult interview were asked to indicate all educational qualifications they hold. This question used a showcard when administered face-to-face. For the push-to-telephone/video approach, interviewers were asked not to read out each category, but to code based on respondents' answers. Therefore, unlike face-to-face and video respondents, telephone respondents did not have a visual prompt to assist them in responding to the question.

Table 6.4 shows highest educational qualification by mode of interview. Overall, the pattern is similar between the 2020 telephone respondents and the earlier face-to-face respondents. However, those who were interviewed by video are much less likely to hold no qualifications (2% compared to 15% in 2019) and much more likely to hold a degree or professional qualification (56% compared to 32% in 2019).

Table 6.4 Highest educational qualification by mode of interview

	2019	2020 f2f	2020 video	2020 telephone
Highest educational qualification
No qualifications	15%	15%	2%	13%
Level 1 - O grade or equivalent	17%	17%	8%	15%
Level 2 - Higher, A Level or equivalent	17%	15%	20%	19%
Level 3 - HNC/HND or equivalent	13%	15%	13%	13%
Degree or professional qualification	32%	33%	56%	36%
Other qualification	5%	3%	1%	4%
Unknown	1%	1%		1%
Total	100%	100%	100%	100%
N	9,776	1427	431	2,357

This is likely to be driven by the profile of those who reponded by each mode rather than by how the question was answered – in other words, that those with lower educational qualifications are more likely to prefer telephone over video.

However, Table 6.5 shows the number of different types of qualification held among those who have a degree or professional qualification. As this group are likely to hold more than one type of qualification, it is a useful measure of the likelihood of capturing all qualifications held.

Table 6.5 Number of types of qualification held among those who have a degree or professional qualification by mode of interview

	2019	2020 f2f	2020 video	2020 telephone
Number of qualification categories held among those who have a degree or professional qualification
One	21%	30%	11%	17%
Two	16%	14%	9%	13%
Three	29%	26%	34%	38%
Four or more	35%	30%	45%	32%
Total	100%	100%	100%	100%
N	2,957	425	285	886

Overall, more types of qualification were recorded for graduates who undertook the interview by video than those who undertook it by telephone or face-to-face. While only 11% of video respondents who were recorded as holding a degree or professional qualification did not have any other types of qualification, the corresponding figure for telephone respondents with a degree was 17%, and for 2019 respondents was 21%. This suggests that video interviews measure educational qualifications held better than the other modes of interviewing.

It is interesting to note that telephone interviews – which did not have a visual cue to help respondents answer the question – appear to be no less accurate than the face-to-face interviews in capturing all qualfications. This suggests that the difference by mode is likely to be shaped for more factors than just these visual cues.

Household income – under-reporting of receipt of income components and level of missing data on amount received.

Mode of interview has the potential to affect the data quality of the income data in two ways – in correctly recording receipt of all components of income that the respondent is receiving, and in minimising the level of missing information on the amount received. Within the SHS, total net annual household income is the main indicator of household income.This is derived from a large number of different components:

• 4 earnings components. Income for the main job and other jobs of the HIH and their spouse.
• 42 different benefits (such as child benefit, state retirement benefit, means-tested benefits, disability benefits, various sources of crisis support etc.)
• 10 different sources of miscellaneous income (such as private pensions, income from investments, income from property etc.)

The questionnaire approach for each component is broadly the same. The respondent is asked if they received each, and if so how much they receive and over what period. Amounts are adjusted to give an annual figure, gross earnings are converted to net earnings, and amounts are imputed for any components were receipt is confirmed but where the respondent does not know or does not want to give the amount. Note that only missing amounts are imputed and receipt of any components of income are not imputed^[36]. Incomes are set to missing if the total received after all imputation and summing is less than £25 a week.

Receipt of the benefits is split into four questions that have traditionally used showcards. These were adapted from the push-to-telephone-video approach so that the interviewer would read out the list of benefits if the respondent could not see the list. Figure 6.2 shows these revisions.

There will be a considerable difference in how each mode transmits visually the information to respondents here:

• For face-to-face interviews, respondents should see the list of benefits on a showcard.
• For telephone interviews, respondents will get no visual clue but should have each list of benefits read out by the interviewer.
• For the video interviews, respondents will see the list of benefits and the screen will display what the interviewer is seeing. The interviewer may also read out the list of categories.

As well as the difference in visual cues, any difference in the engagement of respondents (and interviewers) is likely to shape patterns of response of receipt of different components of income. In contrast, the following questions on the amount received for each component – how much did you receive and over what period – do not have any visual cues and are asked in a relatively consistent way across modes.

It is also difficult to completely disentangle the impact of the mode of interview from the change in the respondent profile, particularly with regard to any under-reporting of receipt of different components. Video respondents are younger, more likely to be working, and less likely to be living in the most deprived areas than telephone respondents. This is reflected in their recorded sources of income. Table 6.6 shows a summary of all information by waves.

Table 6.6 Summary of income information by wave, mode of approach and mode of interview.

	2019 f2f	2020 f2f	2020 Video	2020 Telephone
Summary of earnings
No earnings	38%	38%	18%	38%
Received - no imputation	39%	41%	64%	38%
Received - some imputation	23%	21%	18%	24%
Total	100%	100%	100%	100%
HIH Income for main job
Not in paid employment	42%	42%	21%	44%
Amount given	44%	44%	70%	44%
Amount imputed	15%	14%	9%	12%
Total	100%	100%	100%	100%
Spouse income for main jobs
No spouse/not in employment	68%	71%	47%	68%
Amount given	23%	23%	45%	23%
Amount imputed	9%	7%	8%	9%
Total	100%	100%	100%	100%
Summary of income from benefits
No income from benefits	37%	37%	47%	36%
Received - no imputation	35%	34%	39%	40%
Received - some imputation	28%	29%	15%	24%
Total	100%	100%	100%	100%
Summary of miscellaneous income
No miscellaneous income	69%	68%	69%	63%
Received - no imputation	22%	23%	26%	28%
Received - some imputation	8%	9%	5%	9%
Total	100%	100%	100%	100%
Summary of Total income
Income - none imputed	50%	52%	71%	57%
Income - some imputed	45%	44%	27%	39%
Missing	4%	4%	1%	3%
Total	100%	100%	100%	100%
N	10,577	1,545	489	2,542

The following patterns emerge from Table 6.6:

• Earnings. Those interviewed by video were more likely to say they received income from earnings than those interviewed by telephone (82% compared to 62%). Additionally, less than a quarter of those interviewed by video, who received income from earnings, needed the amount imputed. In comparison, over a third of those interviewed by telephone had the amount of earnings imputed. The same pattern is seen in HIH income from their main job and their spouse's income. Overall, the pattern of receipt of earnings income, and the amount of imputation necessary, was very similar between the telephone interviews and the 2019 data across all three measures.
• Benefits. A higher proportion of telephone respondents than video respondents said that received benefit income (53% compared to 64%). However, among respondents in receipt of income from benefit, levels of missing data were lower for video respondents compared to telephone respondents.
• Miscellaneous sources of income. Levels of receipt of income from miscellaneous sources among telephone and video respondents are similar to those from the 2019 wave. However, the level of imputation required for video respondents is lower again indicating lower levels of missing data.

To try to disentangle the impact of mode of interview from the change in the respondent profile, two benefits where receipt should be very high among key sub-groups were examined: child benefits among household types with children, and state retirement pension among households where the HIH and/or their spouse is aged 65 or over. These are shown in Table 6.7:

For both these measures, the profile of responses from the revised approach is broadly in line with estimates from the earlier face-to-face data. Overall, there is no clear difference in level of receipt. Non-receipt of child benefits is higher among telephone respondents than video respondnets (28% compared to 24%) while non-receipt of state-pension shows the reverse pattern (9% among telephone respondents and 12% among video respondents).

Table 6.7 Receipt of Child Benefit among single parent, small family and large family households and Receipt of State Retirement Pension (asked where HiH or their spouse is >64) by mode of interview

	2019	2020 f2f	2020 video	2020 telephone
Receipt of Child Benefits among single parent, small family, and large family households
Receipt not known or refused	4%	3%	2%	2%
Not received	21%	25%	24%	28%
Received, amount given by respondent	66%	61%	70%	63%
Received, amount imputed	9%	11%	6%	7%
Total	100%	100%	100%	100%
N	2,435	331	127	365
Receipt of State Retirement Pension (asked where HiH or their spouse is >64)
Receipt not known or refused	5%	5%	1%	3%
Not received	7%	8%	12%	9%
Received, amount given by respondent	64%	64%	70%	67%
Received, amount imputed	24%	24%	16%	21%
Total	100%	100%	100%	100%
N	3,408	481	103	1,097

Overall, there is no clear pattern that the change in mode impacts on changes in measurement error in relation to what components of income are received. While interviewing by video appears to reduce the level of missing data on the amount of income received, this is less likely to be because of any change to the mode of interview but the profile of this group being different, higher education levels possibly meaning they are more willing and able to give this information.

Long-term health and disability

Household respondents are asked whether any of the people in the household has any physical or mental health condition or illness lasting or expected to last 12 months or more. Details of the conditions affecting each household member who has a long-term illness of disability are also captured. This question used a showcard when administered face-to-face. For the push-to-telephone/video approach, interviewers were not asked to read out each category but to code based on respondents' answers (Figure 6.3).

Table 6.8 shows shows the proportion of household respondents and second person in the household with long-term health conditions and the number of health conditions recorded.

Table 6.8 Long-term health and disability of respondent and second person in the household by mode of interview

	2019	2020 f2f	2020 video	2020 telephone
Physical or mental health condition or illness > 12 months for second person in the household
Yes	22%	23%	22%	25%
No	78%	76%	78%	75%
Don't know/refused	0%	1%		0%
Total	100%	100%	100%	100%
N	6,999	975	377	1,723
Number of different conditions if second person in household has long-term illness/disability.
Refused	1%	2%	-	1%
One	61%	60%	56%	68%
Two	20%	22%	26%	18%
Three plus	18%	17%	18%	13%
Total	100%	100%	100%	100%
N	1,560	232	89	460
Physical or mental health condition or illness > 12 months for the household respondent
Yes	32%	34%	24%	35%
No	67%	65%	76%	65%
Don't know/refused	0%	1%	-	0%
Total	100%	100%	100%	100%
N	10,577	1,545	489	2542
Number of different conditions if household respondent had a long-term illness/disability.
Refused	0%	1%	-	0%
One	52%	51%	66%	60%
Two	22%	25%	18%	21%
Three plus	26%	23%	16%	19%
Total	100%	100%	100%	100%
N	3511	552	122	945

For the second person in the household where applicable, the prevelance of long-term illness or disability does not differ by mode of interview. In relation to the number of conditions recorded, telephone respondents are more likely to record a single condition (68%) than video respondents (56%) and 2019 respondents (61%) if the second person in the household has a long-term illness or disability.

For the household respondent, the prevalence of long-term ilness or disability does differ by mode of inteview with fewer video respondents (24%) than telephone respondents (35%) or 2019 respondents (32%) being recorded as such. Fewer conditions were reported by video respondents with a long-term illness or disability (34% with two or more) compared to telephone respondents (40% with two or more) or 2019 respondents (48% with two or more).

Those who respond by video are more likely to younger than average. However, this effect appears to be to independent of age as a confounding variable. It may also be that long-term illness or disability impacts on their propensity to use a particular mode of interview, directly or indirectly. Regardless, householders with a long term physical or mental condition make up a disproportionately smaller proportion of the survey sample who completed the video link survey.

It is difficult to interpret the differences by mode. However, there does not appear to be clear evidence of the impact of mode on the number of conditions recorded.

Cultural attendance and cultural activity

Adult respondents are asked about attendence at cultural events or places and about partcipation in cultural activity. Both these questions involved long showcards and interviewers were instructed to read out the full list. Figure 6.4 shows the list of cultural attendance categories.

Table 6.9 shows the extent of cultural attendance and activity by mode of interview. Given the impact of the pandemic, it is unsuprising that cultural attendence has decreased among video respondents and telephone responderts, 43% and 59% reporting that they had done none of the categories listed compared to 19% in the 2019 data. Neither is it a surprise that the reverse pattern is seen with regard to cultural activity (such as reading, playing a musical instrument, crafts, viewing cultural content online) where participation rates have increased.

Table 6.9 Extent of cultural attendance and cultural activity done in the last year by mode of interview

	2019	2020 f2f	2020 video	2020 telephone
Number of different types of cultural attendance (cinema/library/etc.) in last year
None	19%	19%	43%	59%
1 or 2	29%	26%	34%	29%
3 or 4	23%	20%	15%	7%
5 to 8	22%	24%	6%	4%
9+	7%	10%	1%	1%
Total	100%	100%	100%	100%
N	9,776	1,427	431	2,357
Number of different types of cultural activity done in last year
None	25%	24%	10%	18%
1 or 2	50%	45%	40%	52%
3 or 4	16%	22%	35%	20%
5 to 8	8%	9%	14%	9%
9+	1%	0%	1%	0%
Total	100%	100%	100%	100%
N	9,776	1,427	431	2,357

Overall, estimates for cultural attendance and cultural participation were higher for video respondents than telephone respondents. In order to disentangle the potential impact of the profile of the different samples with the effect of the different modes, regression models were fitted to each.^[37] After controlling for key characteristics, mode of interview was a significant factor. This means that, unless there is some confounding variable like technophobia that is causing the same people who use the telephone to also be less likely to engage culturally, or some other difference in the samples that have not been included in the models, mode of interview is having an effect on the measurement of these two estimates.

In other words, video interviewing may be slightly better than telephone interviewing at recording cultural attendance and activity. This would be consistent with the lack of a showcard leading to a slight undermeasurement of cultural participation and attendance.

Sports participation

Adult respondents were also asked about sports participation in the last four weeks. The question used a similar format to those on cultural attendence and participation with a showcard listing different sports and exercise activities being read out to telephone respondents. Compared to 2019, fewer video respondents or telephone respondents reported having not done any sporting activity in the past four weeks (6% and 16% compared to 20%).

A regression model was fitted to whether they had participated in sports in the last four weeks. Again, after controlling for key characteristics, mode of interview was significant. This again suggests that video interviewing could be more accurate at capturing sports participation than telephone interviewing.

Table 6.10 Number of types of sporting activity undertaken in past four weeks and number of days undertaking at least one sporting activity by mode of interview

	2019	2020 f2f	2020 video	2020 telephone
Number of sports done in past four weeks
None	20%	19%	6%	16%
1	32%	31%	39%	45%
2	21%	22%	27%	24%
3+	26%	27%	28%	15%
Total	100%	100%	100%	100%
Number of days doing sport in last four weeks
None	20%	19%	6%	16%
1-13 days in last 4 weeks	37%	37%	27%	31%
14-20 days in last 4 weeks	15%	17%	25%	19%
21+ days in last four weeks	26%	26%	42%	33%
Don't know	1%	1%	-	1%
Total	100%	100%	100%	100%
N	9,776	1,427	431	2,357

Views on council services

A number of questions in the SHS use five point disagree scales with an additional 'no opinion' category. This is a standard formulation in questionnaire design. One important example is to gather views on council services. Respondents are asked to say whether they agree or disagree with 7 different statements about their local council and two statements about decisions in their local area.

When conducted face-to-face, respondents were given a showcard with the five agree/disagree categories on it. A response of don't know/no opinion could be recorded although this was not prompted on the showcard. For the telephone respondents interviewers were asked to read out the response categories after the first statement and then again as needed. Video respondents were able to see all the options including the no opinion option. (See Figure 6.5)

Previous research has suggested that with telephone interviews with no visual cues to the response codes tend to result in fewer neutral responses, 'neither agree or disagree' and 'no opinion', than face-to-face interviews. Table 6.11 shows the results for the statement "My Council provides high quality services". Overall, 27% of video respondents said 'neither agree nor disagree' while 20% of telephone respondents did likewise. This suggests that having no showcard results in fewer neutral responses.

Table 6.11 Agreement with the statement, 'My council provides high quality services' by mode of interview.

	2019	2020 f2f	2020 video	2020 telephone
Strongly agree	7%	8%	7%	10%
Tend to agree	38%	36%	44%	45%
Neither agree nor disagree	24%	24%	27%	20%
Tend to disagree	16%	16%	16%	13%
Strongly disagree	10%	10%	4%	8%
No opinion	5%	6%	2%	4%
Total	100%	100%	100%	100%
N	9,776	1,427	431	2,357

Table 6.12 shows the total number of neutral responses to the seven statements about their local council by mode of interview. Overall, while only 17% video respondents do not use a neutral response to any of the seven statements, the corresponding figure for telephone respondents is 34%^[38]. This again suggests that visual cues do have an impact on the estimates and that the lack of cues for the neutral options mean that these are lower among the telephone respondents. It is notable that the 17% of video respondents who do not use a neutral response to any of the seven statements is lower than the corresponding figure of 25% for the respondents in the 2019 fieldwork. This difference is likely to be driven by the visibility of the 'no opinion' option to video respondents.

Table 6.12 Number of neither agree nor disagree or don't know response to the seven statements about their local council by mode of interview

	2019	2020 f2f	2020 video	2020 telephone
0	25%	25%	17%	34%
1-2	34%	36%	43%	36%
3-4	22%	22%	28%	19%
5-7	18%	18%	12%	10%
Total	100%	100%	100%	100%
N	9,776	1,427	431	2,357

Table 6.13 shows the correlation coefficients of the initial statement, "My local council provides high quality services" with the other six statements with each mode. The questions on local council services show a relatively high correlation. Moreover, there is a high degree of consistency in the correlations across mode. This is reaasuring that mode is not having a more substantive impact on measuring views on local council services.

Table 6.13 Correlation of "My local council provide high quality services" with other statements on the local council by survey mode

	2019	2020 f2f	2020 video	2020 telephone
Correlation with "My council does the best it can with the money available".	0.58	0.57	0.50	0.54
Correlation with "Local council is addressing the key issues affecting the quality of life in my local neighbourhood"	0.60	0.58	0.52	0.54
Correlation with "My council is good at listening to local people's views before it takes decisions"	0.51	0.44	0.40	0.45
Correlation with "My local council designs its services around the needs of the people who use them"	0.55	0.51	0.56	0.51
Correlation with "My council is good at letting local people know how well it is performing"	0.33	0.32	0.22	0.31
Correlation with "My local council is good at letting people know about what services it provides"	0.40	0.37	0.35	0.40

Material deprivation.

The questions that were used to measure material deprivation have a different set of response options. Respondents are read out a list of different items and asked whether they have it, whether they don't have it but don't want it, or whether they don't have it and can't afford. They were also given a don't know response option. These response options were shown in the video interviewing but not the telephone interview. Seven items (such as enough money to take part in a sport or exercise and enough money for home contents insurance) were asked to the full sample.

Table 6.14 shows the frequency of don't know responses to these questions. There does not appear to be a difference by mode. Unlike council services, the lack of a showcard does not appear to affect these questions.

Table 6.14 Number of don't know responses in seven material deprivation categories asked to the full sample by mode of interview

	2019	2020 f2f	2020 video	2020 telephone
None	95%	94%	96%	95%
1	4%	5%	4%	4%
2+	1%	1%	1%	1%
Total	100%	100%	100%	100%
None	10,577	1,545	489	2,542

Religion, sexual orientation, and smoking.

Table 6.15 shows the religion and sexual orientation of adults. While the differences by mode were relatively small, it is noticable that the estimates for 'other religion' and 'gay/lesbian/bisexual/other' were higher among video respondents than telephone respondents. The size of the diference is not large and the difference between the estimates could be due to sampling error or differences in the composition of the profiles. However, it is also possible that these differences are related to the mode of interview and people are more comfortable giving these responses when interviewed by video.

Table 6.15 Selected characteristics of adults aged 16-59 by mode of interview

	2019	2020 f2f	2020 video	2020 telephone
Religion
None	63%	63%	64%	60%
Church of Scotland	12%	13%	13%	14%
Roman Catholic	13%	12%	9%	13%
Other Christian	7%	8%	8%	9%
Other Religion	4%	4%	6%	4%
Total	100%	100%	100%	100%
Sexual orientation
Heterosexual/straight	97%	96%	92%	96%
Gay/Lesbian/Bisexual/Other	3%	3%	7%	4%
Refused	0%	1%	1%	1%
Total	100%	100%	100%	100%
N	5763	841	305	1077

Table 6.16 shows smoking rates by age band and mode of interview. Overall, the estimate for smoking among video respondents (5%) and telephone respondents (11%) is smaller than it was among 2019 respondents. The pattern is seen in all age groups.

Table 6.16 Whether smokes nowadays by age band and mode of interview

	2019	2020 f2f	2020 video	2020 telephone
16-24
Yes	14%	15%	*	7%
No	86%	85%	*	93%
Total	100%	100%	*	100%
N	617	105	28	83
25-44
Yes	21%	19%	4%	14%
No	79%	81%	96%	86%
Total	100%	100%	100%	100%
N	2,740	393	130	426
45-59
Yes	20%	19%	8%	15%
No	80%	81%	92%	85%
Total	100%	100%	100%	100%
N	2,406	343	147	568
60+
Yes	12%	11%	4%	8%
No	88%	89%	96%	92%
Total	100%	100%	100%	100%
N	4,013	586	126	1,280
All
Yes	17%	16%	5%	11%
No	83%	84%	95%	89%
Total	100%	100%	100%	100%
N	9,776	1,427	431	2,357

Modelling was undertaken in order to control for the impact of different respondent profiles. Two regression models were fitted, the first to compare video respondents with telephone respondents and the second to compare telephone respondents with 2019 respondents^[39]. After controlling for key characteristics, the mode of interview was significant in both models. This means that regardless of age, educational qualifications, SIMD and the other factors controlled for, face-to-face respondents were more likely than telephone respondents to say they smoked, and telephone respondents were more likely than video respondents to say likewise.

These results are difficult to fully explain. The question is simple and short, "Do you smoke cigarattes nowadays?" and was asked in a consistent way across modes. The response options are obvious. It could be that there are factors missed from the model that would explain the differences, or that smoking patterns may have changed over time.

However, there is also the potential that the differences are due to mode effects, but not driven by how the question is asked. In particular, under-reporting smoking might be more likely when interviews are being undertaken remotely and the interviewer is not in the respondent's home. However, while this would explain the higher rate of smoking among face-to-face respondents, it would not explain the difference between telephone respondents and video respondents.

Permission to recontact and random adult conversion rate

Both household and adult respondents are asked for consent to be recontacted for future research. Overall, video and telephone respondents were more likely than 2019 face-to-face respondents to give this permission (89% and 79% compared to 66%). This is likely to reflect that lower response rates will tend to lead to higher agreement to recontact rates as fewer 'reluctant' respondents are interviewed.

Table 6.17 Permission to be recontacted for household and random adult respondent by mode of interview.

	2019	2020 f2f	2020 video	2020 telephone
Household respondent permission to be reinterviewed
Permission given	66%	71%	89%	79%
Permission refused	34%	29%	11%	21%
Total	100%	100%	100%	100%
N	10384	1521	489	2520
Random adult permission to be reinterviewed
Permission given	63%	71%	87%	76%
Permission refused	37%	29%	13%	24%
Total	100%	100%	100%	100%
N	9776	1427	440	2348

There is no difference in random adult conversion rate – the proportion of household interviews where a random adult interview is also acheived – between telephone respondents and 2019 respondents. The conversion rate is slightly lower among the video sample than the telephone sample (89% compared to 92%). This suggests that if the adult respondent is different from the household respondent, the transition is more difficult using video interviewing than telephone or face-to-face interviewing.

Table 6.18 Random Adult conversion rate by mode of interview

	2019	2020 f2f	2020 video	2020 telephone
Random adult interview achieved	92%	92%	89%	92%
N	10,577	1,545	489	2,542

Summary

Two modes were available for remote interviewing, video and telephone. Overall, 16% of household respondents undertook the SHS interview by one-way video link, and 84% by telephone.This difference was likely to be partly because of the relevative effort on the part of both respondents and interviewers to set-up the interview. Younger householders, those working, and those in privately rented accommodation were more likely to undertake the interview by video.

Mode of interview also differed considerably by mode of approach. For the opt-in only sample, 22% undertook the household interview by video. In contrast, only 8% of those in the telephone-matched sample did likewise.

Mode effects are complex and interact with response patterns. As well as differences in visual cues given through the showcards, there are a number of ways in which patterns of measurement error are shaped by differences in the way interviewers and respondents interact.

On a variety of other measures examined, there did not appear to be any differences by mode of interview. However, evidence of mode effects was found in a number of estimates:

• Educational qualifications. Video interviews appear to measure the full list of educational measures held better than other modes. This was probably due to differences in visual cues given.
• Components of income. Interviews conducted by video had less missing data compared to interviews conducted by telephone.
• Cultural attendance, cultural engagement and sports participation. Estimates for these measures were higher among those interviewed by video than among those interviewed by telephone. This appeared to be independent of any impact of the different sample profiles.
• Use of agree/disagree scales on questions on council services. There were fewer neutral responses (neither agree nor disagree and don't know) in telephone interviews than in video interviews. This is likely to be due to differences caused by showcards.

Despite efforts to minimise measurement error, the analysis suggests that the mode of interivew is likely to have had some effect on some estimates.