Changes to Labour Market Statistics in Scotland: What You Need to Know

Annex1: Impact of discontinuing the LFS boost on Labour Market Estimates for Scotland

Annex 1.1 Introduction and methods

This annex explains what the potential impact of discontinuing the local LFS boost for Scotland will be on the labour market statistics published by the Scottish Government and what it means for the information we can continue to provide.

To understand the potential impact of this change, we analysed the most recent APS microdata (January to December 2024) and simulated[1] what the results would look like without the boost. This allowed us to compare the estimates currently possible with those that would have been feasible without the 2024 boost.

Our early findings suggest that only a small number of additional statistics were affected by the removal of the boost for the January to December 2024 data. This suggests that the overall impact of removing the boost is relatively limited when considered alongside existing data quality constraints and we are confident that this will continue to be the case for future APS releases. It is important to note that the ONS are actively working to improve achieved sample sizes in the LFS, which is expected to lead to corresponding increases in the APS sample sizes over time.

Annex 1.1.1 Quality Rules

Labour market estimates are assessed using three quality classifications: “robust”, “less robust”, and “not robust”. These categories reflect the minimum quality standards required for publication. “Robust” estimates meet the minimum quality threshold and are considered suitable for public release. “Less robust” estimates may be published with appropriate caveats, while “not robust” estimates typically fall below acceptable quality standards and are not published.

Whilst the Scottish Government and ONS apply the same rules for assessing if an estimate is “robust”[2], the Scottish Government apply more stringent criteria than the Office for National Statistics (ONS) when determining whether an estimate is “less robust” or “not robust.” These enhanced rules incorporate Coefficient of Variation (CV) calculations, which are derived using sample weights.

For this analysis, it was not possible to apply the Scottish Government’s quality criteria to estimates that excluded the local LFS boost cases, as we did not have the correct weighting to apply to the sample. As a result, the focus of our analysis is on estimates that are considered “robust”, which are calculated directly from the sample size only and uses the same rules applied by both the Scottish Government and ONS.

While this analysis focusses on estimates considered “robust”, we have also included estimates considered “less robust” [3] and “not robust” [4] using the ONS quality rules to provide a complete picture. It should be noted that, had the Scottish Government’s stricter criteria been applied, the number of estimates classified as “less robust” would likely be lower, and those deemed “not robust” and therefore require suppression, would likely be higher.

Annex 1.1.2 Summary of analysed variables

In our Quality Assessment of the Office for National Statistics Labour Force Survey and Annual Population Survey data for Scotland report, we looked at 229 APS estimates broken down into five categories:

• 47 estimates included in the Labour Market Statistics for 16 to 24 year olds publication
• 167 estimates included in the annual People, Places and Regions publication
• 4 estimates used as National Performance Framework (NPF) indicators
• 5 estimates used as Fair Work indicators
• 6 estimates considered for Job Quality indicators

It should be noted that there are nine estimates in the Labour Market Statistics for 16 to 24 year olds publication which also feature in the annual People, Places and Regions publication. This resulted in double-counting of these estimates in the original report. We have accounted for the double-counting in this analysis and will only report on the 220 unique estimates in this paper.

Annex 1.2 Impact on publishable estimates

Annex 1.2.1 Overall impact on publishable estimates

Applying the ONS data quality rules, which are based solely on the achieved sample sizes, our analysis found that once the Scottish boost is removed from the sample, there would be a reduction in the number of estimates classified as 'robust', alongside an increase in the number of estimates deemed 'less robust' or “not robust” i.e. subject to suppression due to insufficient quality.

Based on analysis of the January to December 2024 APS data, the removal of the local LFS boost is projected to result in a 17.3% reduction in the number of estimates classified as “robust” under ONS data quality rules (Table A1). While this headline figure may appear significant, it is largely driven by the loss of local authority level estimates, which are particularly sensitive to sample size reductions.

Table A1: Number of estimates by quality category (ONS rules), with and without the Scottish LFS boost for January to December 2024

When local authority estimates are excluded from the analysis (Table A2), the reduction in “robust” estimates falls to just 5.5%, with only six estimates reclassified from “robust” to “less robust.” This highlights that the overall impact of removing the boost is relatively modest for national level statistics, and that the most substantial effects are concentrated at the local level.

Table A2: Number of estimates by quality category (ONS rules), with and without the Scottish LFS boost excluding local authority estimates, January to December 2024

Annex 1.2.2 Impact on local authority level estimates

When focusing specifically on local authority level estimates, the impact of removing the local LFS boost is more pronounced. Analysis of the January to December 2024 APS data indicates a 35.7% reduction in the number of estimates classified as “robust” under ONS data quality rules (Table A3). This substantial decline reflects the sensitivity of local level statistics to changes in sample size.

In parallel, there is a corresponding increase in the number of estimates falling into the “less robust” category or being suppressed entirely due to insufficient data quality (“not robust”). These findings underscore the critical role the LFS boost has historically played in enhancing sample sizes and improving the robustness of statistics at more granular geographic levels. However, due to persistent data quality challenges in recent years, many local authority level statistics have already been suppressed or published with caution, meaning the removal of the boost is unlikely to significantly alter the current landscape of labour market data publication.

 Table A3: Number of estimates by quality category (ONS rules), with and without the Scottish LFS boost for local authority estimates, January to December 2024

The tables above demonstrate that the reduction in the number of estimates classified as “robust” is concentrated primarily within local authority level data. A more detailed breakdown by estimate type reveals that this decline is driven largely by reductions in the quality of estimates for employment among 16 to 24 year olds and headline economic inactivity.

These findings underscore the impact of removing the local LFS boost on specific subgroups within local areas, where sample sizes are inherently smaller and data quality is more sensitive to changes in survey coverage. In particular, estimates for younger age groups and economically inactive populations at the local authority level are most affected (Table A4).

However, it is important to recognise that the quality of APS data at sub-national levels has been poor for some time, and this has already constrained the Scottish Government’s ability to publish many local authority level estimates and detailed unemployment breakdowns. As such, the difference between having the boost and not having it is relatively limited, as many of the affected estimates have already been suppressed or classified as “less robust” in recent years due to longstanding data quality concerns.

Table A4: Change in number of estimates by quality category (ONS rules), with and without the Scottish LFS boost for local authority estimates by key breakdowns, January to December 2024

Annex 1.2.3 Impact on estimates excluding local authority samples

Excluding local authority level estimates, the remaining reduction in robustness is primarily concentrated within two key categories: unemployment and ethnicity (Table A5). When grouped accordingly, these areas account for the majority of the remaining estimates that have shifted from “robust” to “less robust” or have been suppressed due to data quality concerns.

This clustering highlights the particular sensitivity of unemployment and ethnicity related statistics to changes in sample size, even at broader geographic levels, and underscores the importance of maintaining sufficient coverage to support reliable analysis in these domains.

Table A5: Change in number of estimates by quality category (ONS rules), with and without the Scottish LFS boost excluding local authority estimates by key breakdowns, January to December 2024

For the purposes of this analysis, unemployment by ethnicity has been grouped under the broader 'ethnicity' category rather than 'unemployment', to ensure consistency in classification. This approach reflects the intersectional nature of the data and allows for clearer interpretation of quality impacts within demographic breakdowns.

It is also important to note that when using a binary ethnicity classification (e.g. 'White' and 'Minority ethnic group') rather than the more detailed five-category breakdown, there is no observed reduction in the quality of estimates across the categories. This suggests that broader groupings may help maintain statistical robustness where sample sizes are more limited.

Annex 1.3 Conclusion

Analysis of the January to December 2024 APS data suggests that the removal of the local LFS boost will potentially have a limited impact on the overall number of publishable estimates. While there is a noticeable reduction in the number of estimates classified as “robust,” this is largely concentrated within local authority level data and unemployment breakdowns.

Importantly, the quality of APS data, particularly below regional and national levels, has been a concern for several years, as highlighted by the ONS. These longstanding issues have already restricted our ability to publish certain estimates, including detailed unemployment statistics by characteristics such as disability and ethnicity, and many local authority level estimates.

As such, the difference between having the boost and not having it is relatively modest, given that data quality constraints have already limited the scope of publishable statistics. Assuming APS sample sizes without the boost remain consistent with those observed in this analysis, the overall impact on the number of estimates we can currently publish is expected to be minimal.

However, it is anticipated that estimates produced without the boost will be more volatile, with wider confidence intervals and higher Coefficient of Variation (CV) values. This may affect the precision and reliability of certain statistics over time.

To mitigate these challenges and enhance the quality of labour market data, the Scottish Government is actively exploring alternative data sources that could complement the existing LFS and APS datasets (See section 3).