SG Centre of Expertise in Appraisal and Evaluation

High level theory of change

We have developed a Theory of Change for the Centre of Expertise in Appraisal and Evaluation. This sets out how activities and outputs within each of the four themes will lead to intermediate and longer-term outcomes and impact. There is a PDF version of this theory under 'supporting documents'.

1. Inputs

There are three main inputs:

• Centre of Expertise staff and supporting Analytical Services Division (ASD) analysts 
• Centre of Expertise staff, ASD analysts, senior policy leads and ASD heads 
• Centre of Expertise staff and supporting Analytical Services Division (ASD) analysts  

2. Activities

There are four groups of activities, which each lead to a set of outputs and initial outcomes.

2.1 Improving and monitoring the quality of Appraisal and Evaluation (A&E) and how it is done

Outputs

• Evaluation Academy and Appraisal and Evaluation (A&E) training for analysts, including in AI and Theory of Change
• guidance and tools in A&E for analysts, including peer support, and joining up A&E
• training and guidance for policy and other professions in A&E

Initial outcomes (2-3 years)

• increased knowledge and confidence of analysts to deliver a wider range and quality of A&E types and methods 
• improved use of appraisal evidence to inform evaluations
• more Theories of Change for new and existing spend (especially ‘major projects’)
• data on knowledge, skills and quality of A&E used to inform Centre’s work

2.2 Improving which SG projects are appraised and evaluated, and when 

Outputs

• when and what to evaluate guide and how much to spend; and separate lessons learned guidance
• work with Exchequer and Finance colleagues and the PPM CoE to build and strengthen A&E into SG policy, delivery and financial reporting and guidance systems, especially for 'major projects'
• data to monitor and report A&E rates to senior leaders (incl. establishment of baseline)

Initial outcomes (2-3 years)

• increased knowledge and confidence of policy makers to recognise the need for A&E and to commission the right type and quality at the best time
• proportionate inclusion of A&E in all relevant SG accountability systems & processes and use by SG officials
• senior policy and analytical colleagues are interested in data on A&E rates of major projects

2.3 Improving the use of Appraisal and Evaluation (A&E) evidence (how and when it is used) 

Outputs

• management and promotion of Evaluation Registry and Development of SG appraisal library
• liaise with Exchequer on use of A&E evidence in Spending Review and budget decisions
• ASD heads, senior policy leads and Ministers champion evaluation of all ‘major projects’ and use of this evidence

Initial outcomes (2-3 years)

• increased access to and use of A&E evidence 
• demand for A&E from senior leaders and DG - Exchequer needing evidence to inform decisions on spending/saving during annual budgets and Spending Reviews 

2.4 External engagement activities (openness and transparency)

Outputs

• increase openness and transparency on A&E via Evaluation Registry / other online Centre presence
• stakeholder engagement, including with UK Government and other administrations and organisations

Initial outcomes (2-3 years)

• greater openness and transparency about the SG’s approach to A&E
• increased awareness and access to A&E evidence by stakeholders and the public

3. Intermediate outcomes (5 years or sooner)

• improved quality of Scottish Government appraisals and evaluations (linked to Activity Group 2.1) 
• appraisal and evaluation is an integral and proportionate part of the policy making and budget setting and Spending Review processes, with Appraisal and Evaluation evidence better used for decision making (linked to Activity Groups 2.2 and 2.3) 
• all ‘major projects’ have high quality business cases/appraisal and evaluation as a minimum (linked to Activity Groups 2.2 and 2.3) 
• stakeholders and the public can find evidence on what policies work and do not work, and can see the role of appraisal and evaluation in decision making (linked to Activity Group 2.4) 

4. Long-term outcomes and impact (5 years plus)

The four groups of activities will each contribute to long-term outcomes and impacts:

Long-term outcome:

• increase in the amount of high quality, evidence informed decision making, especially on ‘major projects’ (interventions with the highest spend / scale / impact / profile / risk and uncertainty)

Long-term impacts:

• increased effectiveness and efficiency (Value for Money) of Scottish Government spending 
• through high quality appraisal and evaluation, Scottish Government policies and interventions deliver better outcomes for the people of Scotland