The Scottish Government’s Quality Assessment of the Office for National Statistics Labour Force Survey and Annual Population Survey data for Scotland

4 Quality Assessment Results

4.1 Overview

Of the 12 LFS estimates we assessed, 11 were considered “robust” under current data quality rules for the 2023/2024 data. One was considered “less robust” for the same time period. Of the 229 APS estimates we assessed, 181 were considered “robust”, 31 were considered “less robust”, and 17 were considered “not robust” for the 2023 data.

When assessing the estimates against our data confidence ratings for the 2023/2024 data, we have “high confidence” in 90 of the 241 estimates assessed (six from the LFS and 84 from the APS). In addition, we have “moderate confidence” in three LFS estimates and 49 APS estimates. These estimates indicate higher quality and less volatility in the data.

We have “limited confidence” in two estimates from the LFS and 50 estimates from the APS. Caution is required whilst using these estimates and these should be used alongside other available indicators.

Lastly, we have “low confidence” or “no confidence” in one estimate from the LFS and 45 estimates from the APS. We therefore recommend that these estimates are not used as principal measures of labour market information for Scotland.

The annexes and supporting tables provide details on which estimates are in each of these categories.

It should be noted that the current data quality rules do not map directly to the data confidence ratings as the data quality rules also consider sample size. As such, it is possible for a “robust” estimate to be of “low confidence” (for example, an estimate with a sample of 26 and a high CV of 15.00) or a “less robust” estimate to have “high confidence” (i.e. an estimate with a sample of 25 and a low CV of 4.99).

4.2 Labour Force Survey quality overview

4.2.1 Sample Size

All 12 estimates had a reduction in sample size between 2019 and 2023/2024.

• the average reduction in sample size across all estimates was -45.5%
• the smallest reduction in sample size was -26.8%
• the largest reduction in sample size was -70.5%

4.2.2 Coefficient of Variation

All the CVs for the 12 LFS estimates saw an increase between 2019 and 2023/2024.

• the average CV for all 2023/2024 LFS estimates considered was 6.36
• the smallest 2023/2024 CV for an LFS estimate was 1.14
• the largest 2023/2024 CV for an LFS estimate was 17.51

4.2.3 Data Confidence Ratings

When assessing the 12 LFS estimates against data confidence ratings, we have “low confidence” in one estimate from the LFS. We therefore recommend that this estimate is not used as a principal measure of labour market information for Scotland.

We have “limited confidence” in two estimates from the LFS indicating these provide a restricted view on reality and caution is required whilst using these estimates and these estimates should be viewed alongside the suite of other indicators.

In addition, we have “moderate confidence” in three LFS estimates, indicating the data is suitable, but incomplete and limitations should be understood before using them. We have “high confidence” in six estimates from the LFS, indicating we can draw accurate conclusions from these estimates as they provide a good reflection of reality.

Table 5: LFS estimates by data confidence ratings in 2023/2024 (non-seasonally adjusted)

4.3 Annual Population Survey Quality overview

4.3.1 Sample Size

Of the 229 estimates we assessed from the APS, 228 estimates had a reduction in sample size between 2019/2020 and 2023. Excluding the estimate with an increase in sample size:

• the average reduction in sample size across all estimates was -60.5%
• the smallest reduction in sample size was -30.0%
• the largest reduction in sample size was -90.7%

4.3.2 Coefficient of Variation

All the CVs for the 229 APS estimates saw an increase between 2019/2020 and 2023. In 2023:

• the average CV for all APS estimates considered was 10.40
• the minimum CV for an APS estimate in 2023 was 0.34
• the maximum CV for an APS estimate in 2023 was 65.03

4.3.3 Data Confidence Ratings

When assessing the 229 APS estimates in 2023 against the data confidence ratings, results show we should have “no confidence” in 27 estimates and “low confidence” in 18 estimates. We therefore recommend that these estimates are not used as principal measures of labour market information for Scotland.

We have “limited confidence” in 50 estimates from the APS indicating these provide a restricted view on reality and caution is required whilst using these estimates and these estimates should be viewed alongside the suite of other indicators.

In addition, we have “moderate confidence” in 49 APS estimates, indicating the data is suitable, but incomplete, and limitations should be understood before use. Finally, we have “high confidence” in 84 estimates from the APS indicating we can draw accurate conclusions from these estimates as they provide a good reflection of reality.

One estimate from the APS could not be assessed.

The annexes and supporting tables provide details on which estimates are in each of these categories.

Table 6: APS estimates with confidence ratings in 2023