Fairness and diversity impact of online assessments in recruitment queries: FOI release

Information requested

1. Assessment Design and Use

• The criteria and scoring methodology used in any online assessments that contribute to candidate shortlisting, including how "suitability" or other composite scores are derived.
• A list of all third-party providers (e.g., psychometric test publishers or platforms) currently used for online assessments.

2. Fairness Across Protected Characteristics

• Any internal analysis, audit, or evaluation conducted to determine whether candidates from different protected characteristic groups (including age, disability, race/ethnicity, sex, sexual orientation, gender identity, and neurodivergent status) performed differently on online assessments.
• A breakdown of candidate progression rates at the online assessment stage by demographic group for all roles using such assessments in the last 12 months.

3. Equality Impact Assessments (EqIAs)

• Copies of any Equality Impact Assessments or other fairness evaluations related to the use of online assessments in recruitment processes over the past 12 months.
• Any internal communications, reports, or meeting minutes discussing concerns or considerations regarding the accessibility or fairness of these tools for candidates from diverse backgrounds.

4. Complaints and Appeals

• The number of complaints, concerns, or formal appeals submitted in the last 12 months regarding the fairness, accessibility, or accuracy of online assessments.
• How many of these complaints were upheld, and what remedial actions (if any) were taken.

Response

The information you have requested has been provided, where possible, under each question you asked. This response refers to Scottish Government Core DGs and does not include Executive Agencies, Non-Ministerial Offices or Executive Non-Departmental Public Bodies.

Assessment Design and Use

• The criteria and scoring methodology used in any online assessments that contribute to candidate shortlisting, including how "suitability" or other composite scores are derived.

The Scottish Government does not hold the detailed scoring criteria or algorithms used in online assessments. This information is held by the two third-party providers who delivered such assessments over the last 12 months: The Keil Centre and SHL.

SHL provided assessments for the Student Planner vacancy and for the ongoing Graduate Development Programme recruitment. For Senior Civil Service (SCS) appointments, additional forms of assessment such as psychological testing are required under standards set by the UK Cabinet Office. These assessments have supplemented interviews for over a decade and are delivered on behalf of the Scottish Government by The Keil Centre.

This is a formal notice under section 17(1) of FOISA that the Scottish Government does not hold the information you have requested.

• A list of all third-party providers (e.g., psychometric test publishers or platforms) currently used for online assessments

The Scottish Government has used two providers for online assessments during the period specified: The Keil Centre and SHL.

Fairness Across Protected Characteristics

• Any internal analysis, audit, or evaluation conducted to determine whether candidates from different protected characteristic groups (including age, disability, race/ethnicity, sex, sexual orientation, gender identity, and neurodivergent status) performed differently on online assessments.

Adverse impact analysis was undertaken for both the Student Planner and Graduate Development Programme recruitment exercises. The analysis for the Graduate Development Programme is currently being completed, and the Scottish Government has not yet received the final report.

For the Student Planner recruitment, SHL carried out an adverse impact analysis across a range of protected characteristic groups. In addition, candidates who applied under the Guaranteed Interview Scheme progressed to the assessment stage using a lower pass threshold than other candidates, providing additional support for those with a disability.

• A breakdown of candidate progression rates at the online assessment stage by demographic group for all roles using such assessments in the last 12 months.

The Scottish Government does not hold this information. Progression rate data is not routinely recorded for Senior Civil Service recruitment due to the relatively low number of applicants for each role. Reporting this information could risk identifying individuals with protected characteristics.

The Keil Centre may undertake wider analysis across its client base, but this information is not held by the Scottish Government. Progression rate data will be collected for the ongoing Graduate Development Programme recruitment, but at the time of writing the assessment process has not yet concluded.

This is a formal notice under section 17(1) of FOISA that the Scottish Government does not hold the information you have requested.

Equality Impact Assessments (EQIAs)

• Copies of any Equality Impact Assessments or other fairness evaluations related to the use of online assessments in recruitment processes over the past 12 months.

Please find the attached EQIA for the Student Planner and Graduate Development Programme. This covers the impact which the SHL assessments will have on candidates.

Information regarding the ‘Raven’s Adaptive’ which is an assessment administered by The Keil Centre and used for SCS recruitment is also attached. This document provides information about potential adverse impacts to candidates. Minutes from a meeting on 23 January 2025 where this assessment were discussed are also attached.

• Any internal communications, reports, or meeting minutes discussing concerns or considerations regarding the accessibility or fairness of these tools for candidates from diverse backgrounds.

The attached Options Paper for the Graduate Development Programme discusses potential assessment options and considers their likely impact on accessibility and candidate experience.

Complaints and Appeals

• The number of complaints, concerns, or formal appeals submitted in the last 12 months regarding the fairness, accessibility, or accuracy of online assessments.

No complaints, concerns, or formal appeals have been received in the last 12 months regarding the fairness, accessibility, or accuracy of online assessments.

• How many of these complaints were upheld, and what remedial actions (if any) were taken.

As no complaints were received, no remedial actions were required.

About FOI

The Scottish Government is committed to publishing all information released in response to Freedom of Information requests. View all FOI responses at https://www.gov.scot/foi-responses.

FOI 202500485953 - Information released - Annex

File type

PDF

File size

969.2 kB