Independent Report - Evaluability Assessment - WPI Economics - Evaluation of Curriculum Reform Programme

Introduction

Purpose

This report presents the findings of the evaluability assessment conducted by WPI Economics in October 2025. This follows the creation of Theories of Change for the Education Reform Programme and Curriculum, Qualifications and Assessment Reform (CQAR) project, and a data mapping exercise, as detailed in previous reports.

This report considers the suitability of different evaluation approaches for the Education Reform Programme. It considers whether and how impact, process and economic evaluation approaches could be meaningfully used to evaluate the range of activities and outcomes within the reform programme. This is a key part of evaluation scoping, as set out in the Magenta Book, to understand evaluation approaches.

The findings detailed in this report provide a building block to inform the design of the Monitoring & Evaluation (M&E) Strategy, along with our existing work to ‘understand the intervention’ and assess data availability. The purpose of this stage is to show the range of options for evaluations as part of a strategy to evaluate the reforms, and to inform prioritisation and definition of the specific evaluations that can be commissioned as part of the final strategy.

How the evaluability assessment was conducted

As part of the evaluability assessment, we have considered relevant theory and best practice guidance for each type of evaluation, and how different evaluation approaches can be applied to reform programmes of this nature. Our desk-based review considered:

• the evaluation questions that each type of evaluation can be used to evaluate
• the appropriateness of each type (and sub-types) of evaluation to the Education Reform Programme and CQAR outcomes and context
• the types of data and primary research that would be required to conduct the evaluation
• any limitations or challenges that could hinder the use of each type of evaluation as part of the overall evaluation strategy

Alongside this, we have engaged with stakeholders who represent underrepresented groups to understand how the evaluation strategy can ensure the experiences and outcomes for these groups are captured and understood. This has included Scottish Qualifications Authority (SQA) and the Scottish Government’s Equality in Education team (Learning Directorate) and Gaelic and Scots team (Education Reform Directorate). We are also scheduled to speak with the Centre for Excellence for Children’s Care and Protection (CELCIS) and Association of Directors of Education in Scotland (ADES) Special Educational Needs (SEN) Network but were not able to hold these calls ahead of submitting this report.

Based on the above, we have provided an assessment of how each type of evaluation could form part of the overall evaluation strategy, and their key independencies.

Scope and limitations

This exercise is a conceptual and high-level review of the suitability of the evaluation approaches that could be adopted for the evaluation of the reform programme. It does not define in detail the methodology and design of future evaluations for the reforms. In line with the Magenta Book, here we consider the scope and high-level design of the evaluation approach available.

The Education Reform Programme is broad in scope and will be delivered in full over an extended time period. Further work is planned to establish how in practice evaluations should be undertaken as part of the strategy. Core to this will be identifying the primary evaluation needs.

As part of this, steers from Scottish Government on the questions below would allow for a more detailed evaluation strategy to be defined. This includes:

• the priority areas for evaluation and the range of outcomes each specific evaluation should consider
• how a package of evaluations can be commissioned that is informed by these priorities, the timeline of the reforms and the available budget
• when evaluations should take place, and whether early evaluations are desired to feed into policy design
• whether evaluations should align with the existing departmental evaluation strategy

Answering the questions above will ensure we ‘understand the purpose of evaluations and questions to be answered’ in line with the Magenta Book guidance for evaluation scoping. This will then allow us to include more detailed recommendations in the strategy for:

• the research questions that should be considered, to inform the design of each evaluation
• the extent and scope of data collection that should take place for both monitoring and evaluation of the reforms

These aspects can also be developed further by future evaluators as part of the scoping stage for each evaluation.

Due to the evolving nature of the reform programme and its activities, a scoping stage at the outset of any evaluation(s) will be needed to check these recommendations remain relevant and feasible. In addition, there may be additional primary data collection needed once the evaluator selects the specific evaluation methods to be used.

Report structure

The report presents our findings for each type of evaluation in turn, with a separate section for impact, process, and economic evaluations. Each section details:

• an overview of the purpose of this type of evaluation
• the aims and evaluation questions that can be explored
• the different approaches that can be used, and the data and primary research each requires
• considerations for the inclusion of underrepresented groups
• a summary of the suitability of the approaches to the various Education Reform Programme and CQAR activities and outcomes

The report ends with a conclusion section summarising how each type of evaluation could be best incorporated into the M&E Strategy, and the key decisions that are needed to determine which should be taken forward as part of the strategy.