Education reform: evaluation strategy 2026 - 2034

3. Evaluation approach

The evaluation will be theory based and centred around the associated Theory of Change (ToC). The ToC (see supporting documents) has been adapted from the WPI work and developed collaboratively by Scottish Government policy and programme management colleagues and external research leads including the Association of Directors of Education in Scotland (ADES), Education Scotland (ES), Qualifications Scotland (QS) and His Majesty's Inspectorate of Education in Scotland (HMIE).

The ToC is a living document and will be subject to change. We will review it at a minimum on an annual basis and publish updates as they are made, recognising that the programme is evolving and will continue to do so as the years progress.

Scottish Government and the other partners will work together to ensure the evaluation is sufficiently comprehensive yet at the same time does not impose excessive bureaucracy and research activity. The evaluation will gather data from a broad range of sources and will work very closely with other stakeholders who are also involved in collecting data on the progress and implementation, to make best use of existing data. Only where there are clear gaps in evidence will new work be planned.

The evaluation approach will be based on the following guiding principles:

• Evaluating the implementation (process evaluation) and impact (outcome evaluation) of curriculum, qualifications and assessment reform across Scotland from 2026 to 2034; creating a body of evidence to ‘tell the story’
• Collaborating with all delivery partners to join up data collection and dissemination of evaluation insights
• Mindful of context, and bringing in learning from introduction of the Curriculum for Excellence (CfE) where appropriate. Recognising that a ‘transitional’ stage has already put the structural reforms in place which underpin the current ‘transformational’ stage
• Periodic learning for the system through regular reporting and dissemination, to ensure learning is available to support system changes
• Consideration of implementation readiness and variation across the system
• Inclusion of wide range of stakeholder voices: learners and educators, families, communities
• Informed by theory (agreed ToC), practice and evidence
• Makes best use of existing data including academic literature, routine monitoring, surveys already in place and evidence from attainment related measures. Only where there are clear gaps in evidence will new work be implemented
• Responds to relevant national reviews and takes into account broader policy context
• Mixed methods approaches drawing on wider expertise and with a mix of in-house, collaborative and potentially commissioned pieces

The following data sources will be considered to answer the evaluation questions. The weighting and priority of the various sources will be addressed in the first evaluation report:

• Routinely collected monitoring and administrative data: from all delivery partners, including management information (milestones, participation patterns and system readiness indicators, school reports, plans)
• Qualitative information and interviews: in the form of workshops, key stakeholder interviews, focus groups, case studies will be essential in corroborating or discounting the role of the reform activities and outputs in driving longer-term outcomes. We are engaging with initiatives such as YoungScot’s YSEquity panel and the Scottish Youth Parliament (SYP) to ensure we make the most of available data from learners and are best placed to conduct ethical research in the event that more data is needed
• Survey data: local authorities, headteachers, learners
• School inspection data: we will work with HMIE to ensure we build in emerging data from routine and thematic inspection activities
• National level outcomes data: Learner outcomes are key. There are 57 data sources related to NIF outcome 3. Scottish Government records data regarding learner attainment (though Achievement of Curriculum of Excellence (ACEL) levels data) and subsequent destinations, with snapshot data both at three and nine months after leaving education, and for all 16 to 19 year olds in Scotland through the Skills Development Scotland’s (SDS) Annual Participation Measure. SDS also record data regarding the planned destinations of senior phase learners and the difference between these plans and the actual destinations
• Data linkage: we are exploring the potential for data linkage between school and Longitudinal Educational Outcomes (LEO) data. This will allow tracking of learners through school, and determination of their longer term life outcomes. As well as future data, this will also facilitate obtaining baseline historical data prior to the introduction of the programme
• International assessments: the Programme for International Student Assessment (PISA), Trends in International Mathematics and Science Study (TIMMS) and Progress in International Reading Literacy Study (PIRLS) include teacher questionnaires including questions regarding recent uptake of training
• Academic evidence: a light touch evidence review will be conducted to determine which evidence is available, particularly around understanding how the CfE was introduced.