The Scottish Government’s Quality Assessment of the Office for National Statistics Labour Force Survey and Annual Population Survey data for Scotland

Annex 2: Detailed Quality Assessment Results for Labour Market Statistics for 16 to 24 year olds

This publication contains headline estimates of employment, unemployment, and economic inactivity for 16 to 24 year olds in Scotland. It also contains these estimates split by full-time education, by local authority level, and by unemployment breakdowns. This data is sourced from the ONS Annual Population Survey (APS).

Annex 2.1 Headline 16 to 24 year old estimates

Sample sizes

When considering the change in sample size for 16 to 24 year olds in Scotland between Jan-Dec 2019 and Jan-Dec 2023 from the APS:

• the achieved sample size for all estimates has dropped considerably
• the average decrease was -66.8%
• the largest decrease was for “employed men” which fell by -69.4%
• the smallest decrease was for “unemployed women” which fell by -62.3%

Figure A3: Since 2019, sample sizes for people aged 16 to 24 in Scotland have decreased for all headline labour market indicators and for both sexes

Annual Population Survey estimates of percentage change in sample size between Jan-Dec 2019 and Jan-Dec 2023 for headline labour market indicators of 16 to 24 year olds by sex

Coefficient of Variation

When considering the change in CVs between Jan-Dec 2019 and Jan-Dec 2023 for the APS estimates for 16 to 24 year olds in Scotland:

• all CVs have increased
• the average increase was 3.54
• the largest increase was for “unemployed women” which rose by 7.70 from 13.18 to 20.88
• the smallest increase was for “employed people” which rose by 1.36 from 1.65 to 3.01

The CVs for the three unemployment estimates included in our publications (16 to 24 year old people, men and women) were above 10, indicating a level of volatility in the data that causes concern. The latest CV for the unemployed estimate for women was above 20 which means we would not publish this estimate in our publication.

Figure A4: Since 2019, coefficients of variation for the labour market estimates for people aged 16 to 24 in Scotland have increased for all indicators assessed

Annual Population Survey estimates of coefficients of variation for headline labour market indicators of 16 to 24 year olds by sex, Jan-Dec 2019 and Jan-Dec 2023

Current Data Quality Rules

Headline estimates for Scotland are commented on in the Labour Market Statistics for 16 to 24 year olds publication alongside estimates by sex. These headline estimates are compared with estimates for the UK.

In Jan-Dec 2019, all the estimates were considered “robust”. However, by Jan-Dec 2023, the estimates for 16 to 24 year old unemployed women was considered “less robust”, while all other estimates remained “robust”.

Table A3: Headline estimates for 16 to 24 year olds by sex for Scotland from the APS and their status as of Jan-Dec 2019 and Jan-Dec 2023

Economic Status	Sex	2019	2023
Employed	People	Robust	Robust
Employed	Men	Robust	Robust
Employed	Women	Robust	Robust
Unemployed	People	Robust	Robust
Unemployed	Men	Robust	Robust
Unemployed	Women	Robust	Less Robust
Inactive	People	Robust	Robust
Inactive	Men	Robust	Robust
Inactive	Women	Robust	Robust

Data Confidence Ratings

From the Scottish APS data reported in the Labour Market Statistics for 16 to 24 year olds, five of the nine estimates have decreased in confidence rating between Jan-Dec 2019 and Jan-Dec 2023.

Based on Jan-Dec 2023 APS data for Scotland, we have four estimates with “high confidence”, two estimates with “moderate confidence”, one estimate with “limited confidence”, one estimate with “low confidence”, and one estimate with “no confidence”.

Table A4: Headline estimates for 16 to 24 year olds by sex for Scotland from the APS with confidence rating as of Jan-Dec 2019 and Jan-Dec 2023

Economic Status	Sex	2019	2023
Employed	People	High Confidence	High Confidence
Employed	Men	High Confidence	High Confidence
Employed	Women	High Confidence	High Confidence
Unemployed	People	Moderate Confidence	Limited Confidence
Unemployed	Men	Limited Confidence	Low Confidence
Unemployed	Women	Limited Confidence	No Confidence
Inactive	People	High Confidence	High Confidence
Inactive	Men	High Confidence	Moderate Confidence
Inactive	Women	High Confidence	Moderate Confidence

Annex 2.2 APS 16 to 24 year olds estimates by Education Status

In addition to the commentary in the Labour Market Statistics for 16 to 24 year olds in Scotland publication, we release some supporting tables with additional breakdowns. Due to the age range we focus on in the publication, users are particularly interested in the employment status of 16 to 24 year olds who are not in full-time education.

Therefore we publish statistics on employed, unemployed and economically inactive 16 to 24 year olds split by whether they are “in full-time education” or “not in full-time education”.

Sample sizes

When considering the change in sample size for 16 to 24 year olds in Scotland split by education status between Jan-Dec 2019 and Jan-Dec 2023 from the APS:

• the achieved sample size for all estimates has dropped considerably
• the average decrease was -66.1%
• the largest decrease was for “employed people not in full-time education” which fell by -71.7%
• the smallest decrease was for “unemployed people in full-time education” which fell by -60.0%

Figure A5: Since 2019, sample sizes for people aged 16 to 24 in Scotland have decreased for all headline labour market indicators irrespective of education status

Annual Population Survey estimates of percentage change in sample size between Jan-Dec 2019 and Jan-Dec 2023 for headline labour market indicators of 16 to 24 year olds by education status

Coefficients of Variation

When considering the change in CVs between Jan-Dec 2019 and Jan-Dec 2023 for the APS estimates for 16 to 24 year olds in Scotland split by education status:

• all CVs have increased
• the average increase was 4.23
• the largest increase was for “unemployed 16 to 24 year olds in full-time education” which rose by 8.77 from 15.12 to 23.89
• the smallest increase was for “employed 16 to 24 year olds not in full-time education” which rose by 1.42 from 1.44 to 2.85

The CV for the “unemployed and in full time education” estimate for Jan-Dec 2023 was above 20, indicating a level of volatility in the data which would mean we would not publish this figure in our publications.

Although a CV of 20 is the threshold for the current data quality rules, the “unemployed 16 to 24 year olds not in full-time education” estimate for Jan-Dec 2023 had a CV of almost 16 which requires further quality considerations, including assessing the confidence intervals alongside this.

Figure A6: Since 2019, coefficients of variation for the labour market estimates for people aged 16 to 24 in Scotland have increased irrespective of education status

Annual Population Survey estimates of coefficients of variation for headline labour market indicators of 16 to 24 year olds by education status, Jan-Dec 2019 and Jan-Dec 2023

Current Data Quality Rules

In Jan-Dec 2019, the estimates for the APS 16 to 24 year old by Education Status were all assessed as “robust” based on current data quality rules. However, in Jan-Dec 2023, the estimate for unemployed 16 to 24 year olds in full-time education was considered “less robust”. We do not publish this estimate but it can be calculated by subtracting the unemployed people not in full-time education from the total unemployed. Both these estimates are provided in the published tables and so declining quality in the unpublished estimates also indirectly affect the published estimates.

In Jan-Dec 2023, the estimates for unemployed 16 to 24 year olds in full-time education were “less robust” and due to the impact these estimates have on other education status estimates we publish, breakdowns by education status will not be published while we conduct this quality review.

Table A5: Quality Assurance results of the headline estimates for 16 to 24 year olds by education status from the APS and their status in Jan-Dec 2019 and Jan-Dec 2023

Economic Status	Education Status	2019	2023
Employed	Full-Time Education	Robust	Robust
Employed	Not Full-Time Education	Robust	Robust
Unemployed	Full-Time Education	Robust	Less Robust
Unemployed	Not Full-Time Education	Robust	Robust
Inactive	Full-Time Education	Robust	Robust
Inactive	Not Full-Time Education	Robust	Robust

Data Confidence Ratings

From the Scottish APS data reported in the Labour Market Statistics for 16 to 24 year olds by Education Status, four of the six estimates have decreased in confidence rating between Jan-Dec 2019 and Jan-Dec 2023.

Based on Jan-Dec 2023 APS data for Scotland, we have two estimates with “high confidence”, one estimate with “moderate confidence”, one estimate with “limited confidence”, one estimate with “low confidence”, and one estimate with “no confidence”.

Table A6: Confidence Ratings of the headline estimates for 16 to 24 year olds by education status from the APS in Jan-Dec 2019 and Jan-Dec 2023

Economic Status	Education Status	2019	2023
Employed	Full-Time Education	High Confidence	Moderate Confidence
Employed	Not Full-Time Education	High Confidence	High Confidence
Unemployed	Full-Time Education	Low Confidence	No Confidence
Unemployed	Not Full-Time Education	Moderate Confidence	Low Confidence
Inactive	Full-Time Education	High Confidence	High Confidence
Inactive	Not Full-Time Education	Moderate Confidence	Limited Confidence

Annex 2.3 16 to 24 year olds by Local Authority Estimates

Another key output for users of the Labour Market Statistics for 16 to 24 year olds in Scotland publication are the local authority estimates of employment for 16 to 24 year olds.

Current Data Quality Rules

We applied the current data quality rules to each of the 32 Local Authorities in Scotland, meaning we assessed sample sizes and CVs for each of the headline employment estimates. In Jan-Dec 2019, there were 28 local authority areas considered “robust”, three considered “less robust”, and one considered “not robust”.

However, for Jan-Dec 2023, only three Local Authorities were considered “robust”, 20 were considered “less robust”, and nine Local Authorities were “not robust”.

Table A7: 16 to 24 year olds employment by local authority in Scotland and data quality rules applied

Robustness	2019	2023
Robust	28	3
Less Robust	3	20
Not Robust	1	9

In recent publications (since December 2023), we have not released commentary on these estimates due to their declining quality but have continued to release the estimates in supporting tables, with the relevant shading and suppression applied, along with supporting caveats advising caution when using the estimates.

Data Confidence Rating

From the estimates for 16 to 24 year old employment by local authority, 29 of the 32 Local Authority estimates have decreased in confidence rating between Jan-Dec 2019 and Jan-Dec 2023. Of the remaining three, one (Orkney Islands) could not be assessed, one (Shetland Islands) remained the same at “no confidence” and the third (Fife) remained the same at “moderate confidence”.

Table A8: 16 to 24 year old employment by local authority in Scotland with confidence rating applied

Confidence Rating	2019	2023
High Confidence	0	0
Moderate Confidence	23	1
Limited Confidence	8	10
Low Confidence	0	10
No Confidence	1	10
Not Assessed	0	1