The Scottish Government’s Quality Assessment of the Office for National Statistics Labour Force Survey and Annual Population Survey data for Scotland

 Annex 6: Detailed Quality Assessment Results for Job Quality Indicators

The Fair Work oversight group set an action for the Scottish Government to develop and implement a Fair Work evidence plan. From this, the Scottish Government committed to publishing Fair Work estimates available from the ONS Annual Population Survey (APS) as a stand-alone output by the end of 2024. In light of the uncertainty, volatility and quality concerns surrounding the APS, this publication has been delayed to allow for the data quality assessment to take place.

The estimates under consideration are as follows: “respondents’ opinion of their job and career”, “respondents’ opinion of their job representation”, “support from immediate boss”, “bullied or harassed in past 12 months”, and “how well skills correspond to duties”.

At present due to the availability of historic variables, only estimates for “respondents’ opinion of their job and career” and “respondents’ opinion of their job representation” can be compared to earlier time periods. In addition to this, the variables were only added to APS datasets in 2020 which means comparisons are made with this time period. Due to the inability to compare the remaining three variables to previous time points, only the first two variables will be discussed in this section.

Sample sizes

When considering the change in sample size in Scotland for Job Quality Indicators between Jan-Dec 2020 and Jan-Dec 2023 from the APS:

• the achieved sample size for all estimates has dropped considerably
• the average decrease was -45.5%
• the largest decrease was for “job representation for women” which fell by -47.6%
• the smallest decrease was for “career progression for men” which fell by -41.5%

Figure A29: Since 2020, sample sizes for have decreased for all Job Quality Indicators and both sexes

Annual Population Survey estimates of percentage change in sample size between Jan-Dec 2020 and Jan-Dec 2023 for Job Quality Indicators by sex

Coefficient of Variation

When considering the change in CVs between Jan-Dec 2020 and Jan-Dec 2023 for the APS Job Quality estimates for Scotland:

• all CVs have increased
• the average increase was 0.51
• the largest increase was for “job representation for men” which increased by 0.77 from 1.85 to 2.63
• the smallest increase was for “career progression for people” which increased by 0.33 from 1.19 to 1.52

Figure A30: Since 2020, coefficients of variation for the Job Quality estimates assessed have increased

Annual Population Survey estimates of coefficient of variation for Job Quality Indicators by sex, Jan-Dec 2020 and Jan-Dec 2023

Current Data Quality Rules

Despite the drop in sample sizes, the current data quality rules indicate that all Scottish estimates for the job quality indicators assessed remained “robust” in 2023.

Table A37: Quality assessment results for Job Quality Indicators derived from the APS for Scotland

Data Confidence Ratings

From the Scottish APS data for the Job Quality estimates considered in this section, the confidence rating remained the same between Jan-Dec 2020 and Jan-Dec 2023 for the six estimates assessed.

Based on Jan-Dec 2023 APS data for Scotland, we have “high confidence” in all the indicators indicating we can draw accurate conclusions from these estimates as they provide a good reflection of reality.

Table A38: Quality assessment results for Job Quality Indicators derived from the APS for Scotland