Attainment Scotland Fund 2022 to 2026 evaluation: analytical plan - year 1 (2022/23)

First annual evaluation plan for the Attainment Scotland Fund (ASF) 2022-26 which covers 2022-23, the first year of the refreshed Scottish Attainment Challenge. This is a companion document to the ASF Evaluation Strategy 2022-26, published in November 2022.


Executive Summary

Purpose of the document

This document outlines the first annual evaluation plan for the Attainment Scotland Fund (ASF) 2022-26 and covers 2022-23, the first year of the refreshed Scottish Attainment Challenge. This is a companion document to the ASF Evaluation Strategy 2022-26, published in November 2022.

Audience

The document is intended to be accessible to all those with an interest in education in Scotland and beyond; including children and young people, teachers, support workers, parents/carers, public bodies, and third sector organisations.

Our approach

This annual plan, and the Evaluation Strategy published in November 2022, follow a refreshed approach to evaluating progress in closing the poverty related attainment gap. The approach takes account of the refreshed Scottish Attainment Challenge and its' new mission, but also a revised set of evaluation principles. Whereas the evaluation to this point has taken an annual retrospective approach, the new multi-year evaluation strategy takes a more flexible, adaptive, and responsive approach to building an evidence base over time. The new approach will continue to ensure that the evaluation assesses progress towards closing the poverty related attainment gap within the context of the new Scottish Attainment Challenge Mission, and offers insights into the operation of the refreshed programme. This will be supported by shorter, more frequent research outputs in a range of formats to increase accessibility across a wide range of stakeholders.

Evaluation priorities for 2022/23 school year

Our priorities for 2022/23 are to undertake a programme of analytical activity in support of the new Evaluation Strategy, exploring different aspects of the programme to enable insight into the current year to support ongoing improvement. This will be supported by shorter, more frequent publications throughout the year. The early focus on process is deliberate, to enable insights into the refreshed Scottish Attainment Challenge in the first year of implementation.

All aspects of the evaluation – including the thematic strand and NIF reporting – will provide insight into the impact of the programme. However, the 2022/23 period also provides an opportunity for in-depth and collaborative consideration of the most promising potential approaches to assessing impact which will be put in place for Years 2 – 5. Whilst this does mean that evidence of and reporting on impact will be more limited in Year 1, it is anticipated that the development of a robust plan for gathering impact assessment will lead to a strengthened multi-year evaluation.

Developing thematic areas of focus also broadens the scope of the evaluation and Year 1 analytical activity will seek to explore 'what works, for whom and in what circumstances' on a number of priority themes including readiness to learn, families and communities, and engagement in decision-making/voice. This will also support emerging insights into impact in these thematic areas.

Our specific areas of activity are:

Evaluation strand

Objective

Activities

Process

To undertake a study in the first year of the refreshed SAC, to provide learning on what worked well/what could be improved in the process of implementing ASF overall and at the funding stream level (Strategic Equity Fund, Pupil Equity Fund and Care Experienced Children and Young People Fund)

  • Review of existing evidence
  • Survey of Local Authority Scottish Attainment Challenge Leads
  • Interviews and focus groups with key stakeholders and partners

Thematic

To provide learning and increase the evidence base on 'what works and what could be improved, for whom, and in what circumstances' at the thematic level:

  • Readiness to learn
  • Families and communities
  • Engagement in decision-making (children and young people, families and communities)

Thematic evaluation and development activity including:

  • Review of existing evidence on priority themes: readiness to learn; families and communities; engagement in decision-making;
  • Literature review on 'readiness to learn' to inform design of primary research in Year 2;
  • Scope, undertake and publish a small scale study on engaging and supporting families and communities in the context of the Scottish Attainment Challenge;
  • Work with evaluation partners to explore 'what works and what could be improved, for whom and in what circumstances' in engaging children and young people in decision-making in the context of the Scottish Attainment Challenge.

National Improvement Framework (NIF) measures reporting

To assess the impact of the overall fund in improving outcomes in educational attainment and achievement and health and wellbeing and closing the attainment gap between the most and least socio-economically disadvantaged children and young people.

  • Undertake analysis on the available data for NIF measures related to the 2021/22 academic year including 11 original key measures plus two new measures identified in consultation exercise
  • Output: Publish a standalone summary in April 2023

Impact

To take a collaborative approach to assessing the feasibility and efficacy of quantitative and qualitative methods of assessing impact, and to develop an impact evaluation plan for years 2 to 5.

  • Work with stakeholders and partners in existing groups to develop scope/approach;
  • Undertake activities to assess existing and new approaches to measuring impact;
  • Output: Publish an impact evaluation plan for Years 2 to 5.

Contact

Email: Fiona.Wager@gov.scot

Back to top