Prevention toolkit

Tool 8 – Evaluation

The Tool

• Evaluation is a tool for understanding the impact of an intervention on outcomes.

Use this to

• Systematically assess whether an intervention achieves intended outcomes, improves delivery, and generates evidence to inform future decisions.

You end up with

• Robust evidence on what works, for whom, and why, including insights into effectiveness, implementation, and value for money.
• With actionable learning that supports continuous improvement, enabling refinement of interventions in future.

Who can use this?

• Policy makers, analysts, and service commissioners at all levels of government can use evaluation to design, test, and improve interventions aimed at achieving better outcomes.

How does this tool support prevention thinking?

• Evaluation supports prevention by identifying the tangible impact of preventative interventions on longer term outcomes. This can help shift focus toward upstream interventions that reduce long‑term demand on services, improve wellbeing and building confidence in early intervention approaches and outcomes.
• Evaluation is an essential tool for measuring the success of the wider Public Service Reform (PSR) programme and drawing on learning for continuous improvement. A Monitoring, Evaluation and Learning Framework for the PSR Strategy, including the Prevention Pillar, is being developed, and a PSR Strategy evaluation is planned.

Using the tool

This Toolkit summarises information contained within the UK Magenta Book, which is the key resource for anyone looking to understand how evaluation is used in public policy.

The Magenta Book defines evaluation as the systematic assessment of an intervention’s design, implementation and outcomes, with a focus on understanding what works, for whom, and why.

It is linked to the ROAMEF cycle from the Green Book (see Tool 7). It emphasises that evaluation supports decision‑making across the entire policy cycle—before, during, and after implementation.

The Magenta Book walks through evaluation in the following topics, which we summarise as steps below:

• Step 1 - Why, how and when to evaluate
• Step 2 – Evaluation scoping
• Step 3 – Evaluation methods
• Step 4 – Data collection, access and data linking
• Step 5 – Managing an evaluation
• Step 6 – Use and dissemination of evaluation findings

This guidance provides an overview of these different stages and discusses evaluation in the context of preventative interventions. For more information, see the Magenta Book.

Steps in Magenta Book Evaluation

• Step 1 - Why, how and when to evaluate – The Magenta Book suggests you should plan evaluation early in the development of an intervention, ensuring it is proportionate and aligned to decision needs. This includes identifying the main type(s) of evaluation required: such as evaluating the process (how effective a process is in delivering an intervention, an evaluation of impact (how effectively to the intervention achieve outcomes), and value for money (did the benefits of the intervention outweigh the costs). You should look to integrate evaluation into intervention design where possible, so that delivery generates evidence and supports future policy development. For example, a programme to provide early support to households at risk of eviction should plan an evaluation to assess whether early advice and mediation lead to preventing homelessness. It should include a process evaluation (is the support delivered as intended?), impact evaluation (does it reduce homelessness?), and value for money (are savings to temporary accommodation greater than costs?).
• Step 2 – Scoping and early design - Next, define the purpose, scope, and key questions of the evaluation. Develop a theory of change that explains how the intervention is expected to produce outcomes, including assumptions and risks. Translate this into clear evaluation questions linked to outcomes and delivery. For the hypothetical homelessness prevention intervention above, the evaluation should aim to test whether early intervention reduces evictions. A theory of change would link advice and mediation to sustained tenancies and reduced homelessness.
• Chapter 3 – Evaluation methods – You should select evaluation methods that are fit for purpose for your evaluation and align this to evaluation questions. You could use different methods to understand different elements of the process, impact and value for money evaluation from Step 2 such as interviews, surveys or impact assessment methods (such as randomised control trials). For the homelessness advice, this could include mix of process evaluation via staff interviews and case reviews; impact evaluation by comparing outcomes to a matched comparison group of similar at-risk households; and value-for-money analysis comparing programme costs to avoided temporary accommodation use.
• Chapter 4 – Data collection, data access and data linking - Define what data is required and ensure it is adequate, reliable, and relevant to the evaluation questions. Identify existing data sources, including administrative and monitoring data, and plan for collection of new data where gaps exist, using both quantitative and qualitative approaches. This could include, for example, administrative data on homelessness applications, evictions, and temporary accommodation, complemented by participant surveys to capture housing stability and wellbeing outcomes.
• Chapter 5 – Managing an evaluation - Establish clear governance and management arrangements for the evaluation. This should define roles and responsibilities for delivery, oversight, and quality assurance. As part of this, develop a clear evaluation specification, including scope, methods, timelines, and outputs. Manage risks and ensure that evaluation remains aligned with objectives.
• Chapter 6 – Use and dissemination of evaluation findings - Once the evaluation is complete, you should plan how evaluation findings will be used and communicated. This includes Identifying key audiences and tailoring outputs to their needs. The use of evaluation evidence in decision making should be actively encouraged to maximise its value and impact.

In addition to the process above, an annex to the Magenta Book contains information about using Test and Learn techniques. Test and Learn methods are ways of making decisions by trying something in a controlled way, measuring the results, and then adjusting based on evidence.

Test and Learn is not a substitute for evaluation, but a complementary approach that strengthens design, builds adaptive capacity and helps ensure that policies and services are fit for purpose before they are rolled out at scale. They may be a more appropriate method for trialling, learning and adjusting smaller scale prevention projects but it is still recommended that a full evaluation is carried out at a later stage.

Where to find more information/ support

More information on evaluation can be found through the HM Treasury Magenta Book

Resources

• For examples of evaluation, see the evaluation register Home - Evaluation Registry - GOV.UK
• For the Magenta Book ‘Test and Learn’ annex see Test and Learn (HTML) - GOV.UK
• In Scottish Government, the Centre of Expertise for Evaluation and Appraisal is driving forward a culture of appraisal and evaluation in Scottish Government

Decision Support Tools

The methods so far have focussed on defining, mapping and quantifying the benefits of preventative interventions. These tools provide information that can be used in decision making.

However, when choices need to be made about priority areas of action, they use a range of information, including from these tools and from elsewhere.

This section looks at tools that can support decision making when there are multiple considerations, such as impact, feasibility and risk. The main tool focussed on is prioritisation frameworks.

We also include a tool used to support Community Planning Partnership development in Scotland, which has relevant questions more broadly for making decisions around prevention.