The Scottish Government’s Quality Assessment of the Office for National Statistics Labour Force Survey and Annual Population Survey data for Scotland

Annex 4: Detailed Quality Assessment Results for National Performance Framework Indicators

The ONS Labour Force Survey and Annual Population Survey data provide a number of indicators for Scotland’s National Performance Framework (NPF). The following table provides all the indicators from these sources.

Table A27: Summary of National Performance Framework Indicators and their source data

We will not discuss the economic participation indicator as the estimate is constructed from LFS estimates by UK country which has been discussed earlier in this paper (see Detailed Quality Assessment Results for Labour Market Trends). We also do not discuss gender balance in organisations as this is a composite of headline APS estimates by sex (see Estimates by Sex in Detailed Quality Assessment Results for Scotland's Labour Market: People, Places, and Regions).

The table above notes that work-related ill health is sourced from Health and Safety Executive (HSE) publications and we do not include their methodology here.

In this section, we will look in more detail at the arts, culture, and creative industries; contractually secure employment; workplace learning; skills profile of the population; and employee voice indicators.

The arts, culture, and creative industries; contractually secure employment; workplace learning; and skills profile of the population are sourced from the APS and are compared over the Jan-Dec 2019 and Jan-Dec 2023 period. Employee voice is sourced from the LFS and is compared over the Oct-Dec 2019 and Oct-Dec 2023 period.

Sample sizes

When considering the change in sample size in Scotland for National Performance Indicators between 2019 and 2023:

• the achieved sample size for all estimates has dropped considerably
• the average decrease was -57.2%
• the largest decrease was for the “skills profile of the population” indicator which fell by -69.2%
• the smallest decrease was for “the arts, culture, and creative industries” indicator which fell by -50.2%

Figure A25: Since 2019, sample sizes for have decreased for all LFS and APS sourced National Performance Indicators

Estimates of percentage change in sample size between Jan-Dec 2019 and Jan-Dec 2023 for APS sourced National Performance Framework indicators and Oct-Dec 2019 to Oct-Dec 2023 for LFS sourced NPF indicators

Note: Employee Voice is sourced from the Labour Force Survey. All other estimates are sourced from the Annual Population Survey

Coefficient of Variation

When considering the change in CVs between 2019 and 2023 for the estimates in Scotland for National Performance Indicators:

• all CVs have increased
• the average increase was 0.98
• the largest increase was for the “skills profile of the population” indicator which increased by 1.58 from 1.98 to 3.56
• the smallest increase was for the “contractually secure employment” indicator which increased by 0.14 from 0.20 to 0.34

Figure A26: Since 2019, coefficients of variation for all National Performance Framework estimates have increased

Annual Population Survey estimates of coefficient of variation for National Performance Indicators, Jan-Dec 2019 and Jan-Dec 2023 and Labour Force Survey estimates of coefficient of variation for National Performance Indicators, Oct-Dec 2019 and Oct-Dec 2023

Note: Employee Voice is sourced from the ONS Labour Force Survey. All other estimates are sourced from the ONS Annual Population Survey

Current Data Quality Rules

In 2019, all five of the NPF indicators were considered “robust” under current data quality rules and this was still the case in 2023. However, it should be noted that NPF indicators are required by users at increasingly detailed breakdowns which would result in “less robust” and “not robust” estimates. Already the coefficient of variation for the arts, culture, and creative Industries estimate had increased from 3.40 in 2019 to 4.78 in 2023 which can be considered an early indication that the estimate is approaching a point where we may not be able to provide robust breakdowns.

Table A28: Summary of robustness for headline APS estimates by local authority in Jan-Dec 2019 and Jan-Dec 2023

Data Confidence Ratings

From the Scottish data reported in the NPF Indicators, there was no change in the confidence rating between 2019 and 2023 for any of the five estimates assessed.

Based on 2023 data for Scotland, we have “high confidence” in all five of the NPF estimates assessed in this section indicating we can draw accurate conclusions from these estimates as they provide a good reflection of reality.

Table A29: Summary of confidence rating for headline National Performance Framework Indicators in 2019 and 2023