Scottish Social Attitudes Survey 2025: Technical Report

10 Analysis techniques

10.1 Attitudes towards discrimination report analysis

Significance testing

Where reports authored by ScotCen discuss differences between two percentages (such as two different groups of people within a single year), this difference is significant at the 95% level or above, unless otherwise stated. Differences between groups within a given year are tested using logistic regression analysis, which shows the factors and categories that are significantly (and independently) related to the dependent variable. Analysis is carried out in IBM SPSS Statistics v.25, using the CS logistic function to take account of the sample design in calculations.

Multiple logistic regression analysis

Additional logistic regression analysis was conducted across Chapters 3, 4 and 5 in the Discrimination module report. Logistic regression is a method of summarising the relationship between a binary ‘outcome’ variable and one or more ‘predictor’ variables. It allows us to estimate the odds of an individual having a score of ‘1’ on the outcome variable (as opposed to ‘0’) from their responses to the predictor variables (i.e. demographic and other key attitudinal variables).

Models were run using a range of demographic variables (age, gender, household income, education, religion, sexuality, ethnicity, SIMD, urban-rural, perceptions of ability to live on present income, and social class) in addition to several of the attitudinal ones described throughout the report. The variables considered for each particular model are listed on the ‘Notes’ page of the regression tables. The final models presented in the tables were achieved by adding and removing variables in a stepwise manner until a stable model was achieved, containing only analysis variables that demonstrated significant relationships with the outcome variables, independent of other variables included in the model.

The table below lists the regression models included in the supplementary tables. The table number in brackets indicate where this is summarised in the main report.

Table 6 - Supplementary tables key

Model no.: 1 (Table 2.2)

Title:

Whether believe that sometimes there is good reason for people to be prejudiced against certain groups

Model no.: 2 (Table 3.3)

Title:

Whether would be unhappy if a close relative married or formed a long-term relationship with someone with schizophrenia

Model no.: 3 (Table 3.4)

Title:

Whether would be unhappy if a close relative married or formed a long-term relationship with someone who is trans

Model no.: 4 (Table 3.5)

Title:

Whether would be unhappy if a close relative married or formed a long-term relationship with a Gypsy/Traveller

Model no.: 5 (Table 4.1)

Title:

Whether disagree that people from outside Britain who come to live in Scotland make the country a better place

Model no.: 6 (Table 4.2)

Title:

Whether believe number of immigrants should be reduced a lot

Model no.: 7 (Not included in main report)

Title:

Whether think someone aged 70 is unsuitable as a primary school teacher

The regression tables show how the odds of being in a particular outcome category (indicated by the title) vary for each category of each predictor variable compared with the odds for the reference category. For example, Model no. 6 shows the odds that someone would believe that the number of immigrants should be reduced a lot (as opposed to reduced a little, remain the same, or increased) for each category, compared with the reference category. An odds ratio of greater than 1 indicates that, holding all other factors constant, there is an increased likelihood of an individual in that category being in the category ‘1’ for the outcome variable (i.e. believing that the number of immigrants should be reduced a lot) compared with an individual in the reference category. For example, in Model 6, the odds ratio of 2.59 for the category ‘25-64’ means that people aged 25-64 were more likely than those aged 16-24 (reference category) to believe the number of immigrants should be reduced a lot (with the odds being 2.59 times as high for those aged 25-64 compared with those aged 16-24). Conversely, an odds ratio of below 1 means they have lower odds of holding this belief than respondents in the reference category.

Because data are taken from a sample, we recognise that the odds ratios are only estimates, so we also include confidence intervals around each estimate. If the survey were to be repeated, we would expect the true value to fall within these odds ratios 95 times out of 100.

Two measures of statistical significance are provided. The first, calculated from a t-test, is for the comparison between a particular category and the base category, while the second, calculated from a Wald F test, is for the variable as a whole. Where the independent variable has just two categories, these are the same. A significance level of 0.05 or less indicates that there is less than a 5% chance we would have found these differences between the categories just by chance if in fact no such difference exists, hence we can say that we are 95% sure there is a relationship between the predictor and outcome variables. A level of <0.001 indicates that there is a less than 0.1% chance, so we can say that we are 99.9% sure that the relationship exists. Wald F values are also shown. Higher values indicate a greater contribution of that variable to the model than variables with lower values.

The Nagelkerke R-square value provided at the bottom of each table is a rough indication of the proportion of variation in the outcome variable explained by the predictor variables in the model. Nagelkerke R-square is most often quoted in logistic regression as a measure of strength of association, ranging from 0 to 1. The closer the R-square value is to 1, the better the model is at accurately predicting the value of the outcome variable. A value closer to 0, suggests that there are important explanatory factors which are not included in the model. Models 3, 4 and 6 all demonstrate fairly high values of Nagelkerke R-square, suggesting the data is well-explained by the variables presented. Models 1, 2 and 5 have lower values, but still typical for this type of analysis, with a fair degree of explantory power. Model 7 has a low Nagelkerke R-square, suggesting it does not explain any variation in the data well. For this reason it is not discussed in the main report.

10.2 Core report analysis

Within the core module report, all percentages cited are based on the weighted data and are rounded to the nearest whole number. A percentage may be quoted in the text for a single category that aggregates two or more of the percentages shown in a table. The percentage for the single category may, because of rounding, differ by one percentage point from the sum of the percentages in the table.

Estimates from a survey may be affected by sampling errors, and so small apparent differences between percentages may not reflect real differences in the underlying population. It might be the case that the true values for the population are similar but the random selection of respondents has produced, by chance, a lower estimate in one year and a higher estimate in another.

A difference between two estimates is statistically significant if it is so large that a difference of that size or greater is unlikely to have occurred purely by chance. Conventionally, significance is tested at the five per cent level, which means a difference is considered significant if it would only have occurred once in 20 different samples.

Testing significance involves comparing the difference between the two estimates with the 95 per cent confidence limits for each of the two estimates. Statistical sampling theory suggests that the absolute value of the difference between the two estimates is significant if it is greater than the square root of the sum of the squares of the limits for the two estimates.

Within this report, year-on-year differences for each question were tested by comparing the latest year with the previous year^[11] in which the same question was asked with the same mode of collection (push-to-web). Differences between percentages from different survey modes (e.g. from years prior to 2023, where collection was through face-to-face interview) were not significance tested due to the uncertainty introduced by mode change on attitudinal questions.

Where questions have been significance tested, this is made clear in the text; apparent differences are described as either ‘the same’ or ‘in line’ to indicate no significant difference, or as a statistically significant increase or decrease. Where questions are not described in this way, significance testing was not done.

Broader trends over time, including those described in the relevant time series charts, were not tested for significance.

Differences between subgroups (for example by demographic subgroups) were tested for significance using chi-squared tests, with the Holm-Bonferroni method used to correct for false positives. Annex A provides a list of significant associations between questions and subgroup variables.