Evaluation of the Adaptation Scotland Programme

1. Introduction

In 2025, the Scottish Government commissioned Ipsos to evaluate the Adaptation Scotland Programme. The aim of this evaluation framework is to provide an up-to-date guide on the evaluation process and the methods used to conduct the evaluation. As described below, this evaluation will apply a theory-based approach, utilising mixed-methods to data collection and analysis, to incorporate both qualitative data collection and analysis of secondary data and gather and assess evidence from the Adaptation Scotland Programme. The evaluation framework identifies key questions and indicators to assess the Programme’s performance (effectiveness and impact), strengthening understanding of what works and what does not work, for who, in what contexts and why.

Background

The Adaptation Scotland Programme lies at the heart of the Scottish Government's efforts to build a climate-resilient nation. It has been funded by the Scottish Government since 2005, formerly as the Scottish Climate Change Impacts Partnership, and has evolved in line with Scotland's statutory commitments under the Climate Change (Scotland) Act 2009. The Adaptation Scotland Programme delivers activities across seven delivery clusters, focusing on strengthening the skills and capacity of businesses, communities, third sector and public sector organisations to adapt to climate change through capacity building, advice and collaborative action.

The Adaptation Scotland Programme plays a key role in supporting delivery of the third Scottish National Adaptation Plan (SNAP3). The Scottish National Adaptation Plan 2024-2029 sets out a comprehensive set of actions to build Scotland's resilience to the impacts of climate change over the next five years, fulfilling the legal duty established under the Climate Change (Scotland) Act 2009. The plan commits to delivering over 200 policies and actions across 23 national objectives. The current plan follows on from the first Scottish Climate Change Adaptation Programme (SCCAP) published in 2014 and the second iteration (SCCAP2) published in 2019.

The current iteration of the Programme, which began in April 2023, focuses on supporting leadership across sectors, while emphasising place-based approaches. This aligns closely with SNAP3's commitment to driving adaptation action across all of Scotland's regions through both national and local initiatives. The Programme directly supports 16 out of the 23 objectives set out by SNAP3 and is a key vehicle for building capacity and catalysing adaptation action. Adaptation Scotland leads the Public Sector Climate Adaptation Network which provides opportunities for learning and knowledge sharing across the public sector. Alongside its public sector-facing work, the Programme aims to increase engagement and support adaptation action across private sector, community and voluntary sector leadership, delivering training, advice and learning resources.

Purpose of the evaluation

The overall aim of this evaluation is to provide an independent assessment of the current iteration of the Adaptation Scotland Programme, with the evaluation intending to generate evidence-based insights and recommendations that can feed into any re-procurement and re-design of the Programme for beyond 2027, as well as any shorter term changes that can support improvements in the final part of the current programme. Understanding the Programme’s progress, successes, and challenges to date will enable informed decision-making around its future direction and shape and apply learnings to optimise the design and delivery of future support.

The evaluation will therefore explore how and how well the Programme’s activities are contributing to adaptation outcomes, focusing on the perceptions, experiences and insights of those involved in or supported by the Programme. More specifically, the evaluation will:

• Assess the overall impact of the Adaptation Scotland Programme and its progress towards its five intended outcomes.
• Engage with both recipients of Adaptation Scotland support and Programme delivery staff to identify what aspects of the Programme have been effective at contributing to adaptation outcomes, what could be improved and what might be missing from the current Programme.
• Identify best practice examples as well as any gaps in delivery or barriers to accessing support across the seven delivery clusters.
• Develop actionable recommendations for changes in Programme design and delivery for enhancing the Programme's effectiveness in supporting Scotland's adaptation efforts.
• Develop a set of updated KPIs to guide future monitoring for the next round of the Programme.

Evaluation components

The evaluation will comprise both process evaluation – exploring how Programme funding has been used and how it has been delivered, to facilitate understanding of the mechanisms through which outcomes have been achieved – and impact evaluation, to explore whether the Programme is achieving its intended outcomes, as well as any long term outcomes where evidence is available. Qualitative fieldwork will facilitate in-depth evaluation across different areas within Scotland. The overall approach to delivering the evaluation has six stages, detailed in chapter 2. Stages one and two have led to the development of this evaluation framework.

This report is structured into the following sections:

Chapter 2 - Evaluation approach: sets out the overall approach for the process and impact evaluations, the contribution claims. It details the evaluation stages summarised above, including how the evaluation framework has been produced.

Chapter 3 - Theory of Change: provides a description of a Theory of Change for the Adaptation Scotland Programme, developed in workshops during the scoping phase of the evaluation.

Chapter 4 – Contribution Analysis: details the approach to Contribution Analysis and sets out the contribution claims to be tested for the evaluation.

Chapter 5 - Evaluation questions: introduces the process and impact evaluation themes, questions and sub-questions to be explored and answered by this evaluation.

Chapter 6 - Evaluation framework: lists the relevant indicators for each impact, process sub-question, and suggests whether data from interviews and documents provided by Verture and Scottish Government cover each sub-evaluation question.

Chapter 7 - Data collection methods and sources: This section outlines how the data collected will be collected and synthesised for this evaluation.