Poverty and Income Inequality in Scotland 2022-25

12   Measurement uncertainty

The poverty estimates in this report are based on the Family Resources Survey (FRS). This is a sample survey, and therefore there is some degree of uncertainty around the estimates produced. For example, when it is reported that 18% of individuals are living in relative poverty after housing costs, then this should be understood not as an exact figure but as a best estimate within a range.

12.1  Why estimates are uncertain

Different random samples from the same population will produce slightly different results, and these will also differ from the result you’d get by surveying the whole population. This uncertainty is called sampling error.

We estimate sampling error by considering how results would vary if many samples were taken for the same time period. This gives us a range around the estimate—called a confidence interval—that tells us how likely it is that the true population value lies within that range. A 95% confidence interval means we can be 95% sure the true value falls inside the limits. Confidence intervals provide a guide to how robust the estimates are.

Tables 1 to 4 below provide 95% confidence limits around the key poverty estimates. For example, the proportion of individuals in relative poverty after housing costs in 2022-25 was 17.5%, with a lower confidence limit of 15.1% and an upper confidence limit of 19.8%. This means that we can be 95% confident that the percentage of individuals in relative poverty lies between 15.1% and 19.8%. Similarly, the lower confidence limit for the number of people in relative poverty was 820,000, and the upper confidence limit was 1,070,000.

We measure uncertainty in two different ways. Bootstrapping is more accurate, but takes more time to calculate. We also calculate indicative confidence intervals, which are available to download as spreadsheets. These are less accurate, but still reflect how sample size and variation affect measurement uncertainty over time.

12.2  Bootstrapped intervals

The most accurate methodology used to calculate confidence intervals for poverty estimates is called bootstrapping.

In the bootstrap, multiple new samples (resamples) of the dataset are created, with some samples containing multiple copies of one case with no copies of other cases. Exploring how an estimate would change if we were to draw many survey samples for the same time period instead of just one sample allows us to generate confidence intervals around the estimate.

The bootstrapped confidence intervals below were produced by DWP. More information their methodology can be found in the  statistical notice (pdf).

Table 2: Relative poverty after housing costs - central estimates and 95% bootstrap confidence intervals, Scotland 2022-25

Table 3: Relative poverty before housing costs - central estimates and 95% bootstrap confidence intervals, Scotland 2022-25

Table 4: Absolute poverty after housing costs - central estimates and 95% bootstrap confidence intervals, Scotland 2022-25

Table 5: Absolute poverty before housing costs - central estimates and 95% bootstrap confidence intervals, Scotland 2022-25

12.3  Indicative intervals

We also produce simpler 95% confidence intervals to show measurement ranges over time. Using Scotland data across two single‑year periods and four three‑year periods, we calculated simplified standard errors for several poverty indicators and population groups. These standard errors account for the survey’s complex design but treat poverty rates as linear, effectively assuming the poverty line has no measurement uncertainty.

We compared these simplified standard errors with bootstrapped standard errors. Because the simplified method ignores uncertainty in the poverty line, it underestimates overall uncertainty and produces confidence intervals that are too narrow. To correct this, we apply an adjustment factor based on the average difference between simplified and bootstrapped standard errors. This adjustment was similar across all groups. The corrected values form our indicative confidence intervals.

For three-year averages, we pooled the data to calculate standard errors. (Note that this is a different approach from calculating poverty rates for a three-year period – for that, we would take a simple mean of the three single-year estimates.) The pooling leads to a larger sample size, resulting in smaller standard errors.

As an example, for 2017-20, the correction factor ranged from 1.2 to 1.8, with a mean of 1.5.  Note that our approach made the following assumptions:

• We assume that an average correction factor can sufficiently correct for the fact that we consider the poverty line not subject to measurement uncertainty.
• The average correction factor is based on large groups (all people, all children, all working-age adults, all pensioners in Scotland), but we assume that it applies to further disaggregations equally.
• In successive years, samples are not independent (half of the primary sampling units get reused), but we treat them as if they were, and we assume that the impact on standard errors is negligible.

To mitigate the difference between (accurate) bootstrapped and (less accurate) simplified confidence intervals, we used correction factors of 1.5 for two-, three- and five-year averages, and 1.7 for single-year estimates. These correction factors are a pragmatic compromise, taking into account mean correction factors.

Applying the correction factor widened the resulting confidence intervals, reflecting the fact that the poverty line itself is subject to measurement error as well. The widened confidence intervals can be obtained for any subgroup of the population, and they are indicative of measurement uncertainty around poverty estimates.

We think that the resulting indicative confidence intervals are useful to assess:

• uncertainty over time and provide an understanding of ranges around central estimates
• provide clues as to whether trends and differences are likely to be meaningful or noise
• support a conversation about measurement uncertainty and other sources of error