Scottish Surveys Core Questions 2013

6 Technical Notes

This chapter provides additional background on the methodology and reporting conventions of the SSCQ and its constituent surveys:

• source surveys and core questions (section 6.1)
• weighting (section 6.2)
• confidence interval calculations (section 6.3)
• statistical disclosure control (section 6.4)
• presentation of data on country of birth (section 6.5), ethnic group (6.6) and religion (6.7)

6.1 Source surveys and core questions

Three large-scale Scottish Government population surveys are published separately as National Statistics:

• Scottish Crime and Justice Survey (SCJS)
  www.gov.scot/Topics/Statistics/Browse/Crime-Justice/crime-and-justice-survey
• Scottish Health Survey (SHeS)
  www.gov.scot/Topics/Statistics/Browse/Health/scottish-health-survey
• Scottish Household Survey (SHS)
  www.gov.scot/Topics/Statistics/16002

Further information on Population Surveys in Scotland can be found here: www.gov.scot/Topics/Statistics/About/Surveys

Since the beginning of 2012 each of the surveys has included a set of 20 core questions that provide information on the composition, characteristics and attitudes of Scottish households and adults across a number of topic areas including equality characteristics, housing, employment and perceptions of health and crime. Responses on these questions from all three surveys have been pooled to provide the Scottish Surveys Core Questions dataset with a sample size in excess of 20,000 responses.

Full details of the questions are available on the Scottish Government website.^[41]

The first set of pooled response tables for the year 2012 were published as data under development here: www.gov.scot/Topics/Statistics/About/Surveys/PooledSample2012

Following further consultation and methodological development, the 2013 dataset is now published as Official Statistics as the Scottish Surveys Core Questions 2013. The website contains further information and supplementary tables to this main report.

www.gov.scot/Topics/Statistics/About/Surveys/SSCQ

The SSCQ 2013 publication was pooled from the third and fourth quarter year^[42] of the Scottish Crime and Justice Survey 2012/13 and all four quarters each of the Scottish Health Survey 2013 and the Scottish Household Survey 2013. Responses from adults aged 16 and over were included. Due to the different sampling nature to suit the primary aims of each survey, the number of respondents vary between different SSCQ questions. The questions were hence batched into three groups: household questions, individual questions and crime questions, and three different sets of weights calculated to ensure representative results. Sampling, weighting and pooled sample numbers are described separetely for each survey below.

6.1.1 Scottish Crime and Justice Survey (SCJS) technical notes

Sampling, survey response and weighting are described in full in the SCJS2012/13 technical report: www.gov.scot/Topics/Statistics/Browse/Crime-Justice/crime-and-justice-survey/publications/SCJS2012-13-TR

Briefly, the survey consists of a simple random sample, designed to achieve a minimum effective sample size of 750 interviews in the eight Police Force Areas (PFAs), enabling robust analysis at this level. One random adult per household was interviewed and asked all SSCQ and SCJS questions.

The response rate was 67.7%, resulting in 12,048 interviews during the 2012/13 financial year field work. Of these, 6,226 interviews (completed in the second two quarters of the financial year) were pooled into the SSCQ 2013 dataset. The subset was re-weighted to be in itself representative of the adult Scottish population distribution, as described for the SCJS publication.

6.1.2 Scottish Health Survey (SHeS) technical notes

Sampling, survey response and weighting are described in full in the SHeS 2013 technical report: www.gov.scot/Publications/2014/12/6634/downloads

The SHeS sample is clustered in each calendar year and unclustered over four years. All adults and up to two children in each household are eligible for interview. Only one adult in each household was asked the crime and household questions, to remain in line with the SCJS sampling procedure. The SHeS sample is boosted by participating health boards (Ayrshire & Arran, Fife, Grampian and Eilean Siar in 2013). It is further boosted to interview children in further households. These households were excluded from the SSCQ dataset as equality questions were not asked.

The response rate was 66.3%, and 3,259 households were interviewed in the main and health board boost samples. 4,894 resulting adult interviews were pooled into the SSCQ 2013 dataset. Of these, 3,251 were asked the crime questions. The subset of households (excluding the child boost), and adult respondents were re-weighted to be representative of the Scottish private household and population distribution, as described for the SHeS publication.

6.1.3 Scottish Household Survey (SHS) technical notes

Sampling, survey response and weighting are described in full in the SHS 2013 technical report: www.gov.scot/Topics/Statistics/16002/PublicationMethodology/Methodology13

The SHS consists of a simple random sample with a target minimum effective sample size of 250 per local authority. The SSCQ household questions are asked of the highest income householder or their spouse/partner, and one adult is randomly selected to answer the individual and crime questions, in line with the other two surveys.

67.5% of eligible households responded, leading to 10,652 household interviews. The response for the random adult interview was 62.8%, yielding 9,918 interviews. Weighting is fully described in the SHS technical report and no further weighting required for the SSCQ.

6.2 Weighting

Datasets from the three source surveys were combined into three new SSCQ datasets: SSCQ household variables (20,743 responses), SSCQ individual variables (21,038 responses) and SSCQ crime variables (19,395 responses), see Table 6.1.

Each variable response category in each of the surveys carries a different design effect. If we were solely seeking the most efficient estimate for each variable separately then separate scale factors could be derived for each one. However, this would restrict the use and understanding of the dataset. Rather, for each constituent survey dataset the design effects were estimated for each response category and then the median design effect over all response categories for all variables was used as the representative design effect of that survey. These design effects were then used along with the sample sizes to calculate the effective sample sizes (neff) and scaling factors for combining the three datasets.

Table 6.1: Numbers of sample and effective sample pooled from the source surveys

	SCJS		SHeS		SHS		SSCQ
	sample	neff	sample	neff	sample	neff	sample	neff
Household responses[43]	6,226	5,666	3,259	2,365	10,652	8,848	20,743	16,741
Individual responses[44]	6,226	4,769	4,894	2,365	9,918	6,752	21,038	13,164
Crime responses[45]	6,226	4,647	3,251	1,615	9,918	6,600	19,395	12,416

To combine the data the scale factors were applied to the grossing weights for the individual surveys (described in section 6.1). The neff of each survey contribution formed the basis for the scaling factors:

survey A weight scaling factor = neff (surveyA) / (sum of three survey neffs).

The weights were then re-scaled to be proportionate to effective sample size contribution of each survey and used as pre-weights.

The three pooled SSCQ datasets were then weighted again to be representative of National Records of Scotland population estimates^[46].

6.3 Confidence Interval Calculations

All three of the source surveys are stratified to ensure sufficient sample sizes in the smaller local authority areas. In addition, SHeS is clustered in each annual fieldwork period and, while this effect cancels out over each four-year period, it must be accounted for in producing annual results.

Confidence intervals have therefore been calculated using a method to account for stratification and clustering and the resulting design effects (surveyfreq in SAS). This method is to compare of estimates of all quantities estimated by SSCQ. Confidence intervals across all subgroup estimates are provided in the accompanying supplementary tables.^[47]

Confidence intervals are plotted on all charts and figures in this report. If the intervals do not overlap then there is a significant difference between two points, but if they do overlap it does not necessarily mean there is no significant difference.^[48] In the report text the term "significant" refers to "statistically significant" differences.

A comparison of estimates of key variables across the three constituent surveys and the SSCQ are provided in Annex B.

6.4 Statistical Disclosure Control

All estimates based on a single respondent and displayed in main and supplementary tables have been denoted with '*' to safeguard the confidentiality of respondents with rare characteristics.

For individual variables crossed with individual variables (e.g. Ethnicity by Religion), further cells with zero or low respondent numbers in the same row and column as the single response have also been suppressed with '*' to ensure confidentiality. For household and geographic variables, only one further cell in the same row was suppressed, as these cross-tabulations are not transposed.

Cells with true zero counts are denoted with '.' throughout, unless denoted '*' as part of disclosure control.

6.5 Presentation of Data on Country of Birth

The country of birth of respondents is collected in minute detail by the pooled surveys. Reporting on the data in this way would uniquely identify, and therefore potentially disclose the identity of, participants from a range of countries. For this reason, country of birth has been presented as follows:

• Scotland: Respondents who specifically list "Scotland" as their country of birth
• Rest of UK: England, Northern Ireland, Wales, Great Britain/United Kingdom (Not Otherwise Specified)
• Rest of EU: Austria, Belgium, Bulgaria, Croatia, Cyprus (European Union), Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Latvia, , Lithuania, Luxembourg, Malta, Netherlands, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden
• Rest of World: All other responses (excluding refusals)

6.6 Presentation of Data on Ethnic Group

Ethnicity of respondents is collected in considerable detail by the contributing surveys to the SSCQ.

We have tried to present the data on ethnic group in a way that would be most helpful to users, with consideration to producing analysis to reveal inequalities that highlight the need for action. However, in Scotland, many ethnic groups are small in number and this can often lead to statistical unreliability when analysing and presenting data drawn from a sample survey. This can hinder publication of figures because of the need to avoid identification of individuals.

In some instances we have tried to overcome this problem by combining categories. This is not an ideal solution as it can hide inequalities that occur between each of the separate categories, but it is often the option which provides the best balance between data utility, sensitivity and the protection of the individual. Our schema for doing so is provided in Table 6.2.

Where it has been necessary to combine categories, we have tried to make the headings reflective of the individual categories that have been combined. We have tried to take account of the sensitivities around differing stakeholders' views of the ethnic group categories used in Scottish surveys. In the accompanying commentary we have used current terminology on ethnicity.

Where the numbers are too small to publish and we haven't grouped categories, Statistical Disclosure Control has been applied.

Table 6.2: Grouping of ethnicity in the SSCQ

Base Collection Categories	Sample	Super Groups	Sample
A - WHITE - White Scottish	16,949	White: Scottish	16,949
A - WHITE - Other British	2,631	White: Other British	2,631
A - WHITE - Polish	211	White: Polish	211
A - WHITE - Irish	172	White: Other	661
A - WHITE - Gypsy/Traveller	3
A - WHITE - Any other white ethnic group	486
C - ASIAN, ASIAN SCOTTISH OR ASIAN BRITISH - Pakistani, Pakistani Scottish or Pakistani British	97	Asian	359
C - ASIAN, ASIAN SCOTTISH OR ASIAN BRITISH - Indian, Indian Scottish or Indian British	120
C - ASIAN, ASIAN SCOTTISH OR ASIAN BRITISH - Bangladeshi, Bangladeshi Scottish or Bangladeshi British	5
C - ASIAN, ASIAN SCOTTISH OR ASIAN BRITISH - Chinese, Chinese Scottish or Chinese British	70
C - ASIAN, ASIAN SCOTTISH OR ASIAN BRITISH - Other Asian, Asian Scottish or Asian British	67
B - MIXED OR MULTIPLE ETHNIC GROUP - Any mixed or multiple ethnic groups	33	All other ethnic groups	209
D - AFRICAN - African, African Scottish or African British	58
D - AFRICAN - Other African background	22
E - CARIBBEAN OR BLACK - Caribbean, Caribbean Scottish or Caribbean British	8
E - CARIBBEAN OR BLACK - Black, Black Scottish or Black British	4
E - CARIBBEAN OR BLACK - Other Caribbean or Black background	1
F - OTHER ETHNIC GROUP - Arab, Arab Scottish or Arab British	12
F - OTHER ETHNIC GROUP - Other	71

6.7 Presentation of Data on Religion

The religion of respondents is collected in considerable detail by the contributing surveys to the SSCQ. However, some religious group sample sizes are too small to present with reliable statistical accuracy. For this reason, religions have been grouped as shown in Table 6.3.

Table 6.3: Grouping of religion in the SSCQ

Base Collection Categories	Sample	Super Groups	Sample
None	8,521	None	8,521
Church of Scotland	7,108	Church of Scotland	7,108
Roman Catholic	3,001	Roman Catholic	3,001
Other Christian	1,779	Other Christian	1,779
Muslim	176	Muslim	176
Hindu	72	Other	394
Buddhist	62
Pagan	29
Jewish	24
Sikh	19
Another religion	188