Evaluation of the Adaptation Scotland Programme

Methodology

The approach involved a process and impact evaluation, applying theory-based methods to understand how the Adaptation Scotland Programme has achieved its intended outcomes as well as identifying any barriers to this. A contribution analysis was undertaken to account for other contributing factors and build a plausible, evidence-based narrative about the Programme’s contribution.

Process and impact evaluation

A process evaluation aimed to examine the implementation of activities and delivery pathways of the Programme, and to answer questions such as what worked well and what did not, how delivery could be improved, and how contextual factors influenced implementation. The impact evaluation explored whether, and to what extent, the Programme was achieving its intended outcomes.

The process and impact evaluation questions were designed to focus on key aspects of the Adaptation Scotland Programme (see the accompanying appendices document for details):

• For the process evaluation, this included exploring the design, implementation, delivery and governance of the Programme.
• For the impact evaluation, this included exploring the key outcomes in relation to leadership, effectiveness, place-based initiatives, engagement and cross-sector collaboration.

Theory of Change

Ipsos developed a Theory of Change (ToC) for the Programme. An initial draft was developed through review of Programme documentation and scoping interviews with key stakeholders involved in the design and strategic oversight of the Programme. The ToC was subsequently refined through two stakeholder workshops involving representatives from Scottish Government, Verture and the Sustainable Scotland Network (SSN).

The ToC provided the analytical framework for the evaluation and informed the development of the contribution claims and evaluation question indicators. A summary version of the ToC diagram is provided in Figure 1 below and the full ToC narrative can be found in the accompanying appendices document.

Sampling and recruitment

A purposive sampling approach was adopted to ensure representation across the Programme's seven delivery clusters, stakeholder groups, sectors, national and regional organisations, and geographic areas of Scotland. The achieved sample included stakeholders from 11 local authority areas, spanning the Highlands and Islands, North East Scotland, Central Scotland, the Lothians, the West Coast and South Scotland. Stakeholders from island, coastal, urban, rural and upland areas were also represented in the sample, providing broad coverage of the climate vulnerabilities and geographic contexts across Scotland.

Sampling sought to capture perspectives from organisations involved in Programme governance, management, delivery and receipt of support, as well as organisations with more limited engagement with the Programme. Table 1 presents the achieved interview sample by stakeholder group and delivery cluster.

Participants were identified through Programme records, stakeholder mapping and recommendations from Scottish Government and Verture. They were recruited via email invitation by Ipsos and provided with information about the evaluation (see appendices).

Interviews explored views on Programme design, governance, implementation and delivery, as well as perceived outcomes and impacts, barriers and enablers, and priorities for future adaptation support.

This approach enabled the evaluation to capture perspectives from organisations involved in different aspects of Programme delivery, while ensuring coverage of a broad range of experiences and contexts.

Qualitative fieldwork

Qualitative fieldwork was undertaken between March and June 2026. In total, 63 semi-structured interviews were conducted. Depending on the nature and size of the organisation, some interviews were undertaken with one individual, while others involved multiple individuals from the same organisation. In total, 71 individuals participated in an interview. These were conducted online via Microsoft Teams, typically lasting around one hour. Interviews were structured using a discussion guide designed by Ipsos and approved by Scottish Government (see appendices).

Document and monitoring information review

The evaluation included a review of Programme documentation and monitoring information (MI). Documents reviewed included Programme workplans, annual and quarterly progress reports, Scottish Government policy documents and documentation relating to individual Programme activities and clusters. The review was undertaken to understand the design and implementation of the Programme, assess progress against planned activities and outputs, and identify evidence relating to outcomes and impacts.

Analysis

Qualitative interview data were analysed using a thematic framework approach. Data were systematically coded against the process and impact evaluation questions, Theory of Change outcomes and contribution claims.

A structured codeframe was used for extracting key data and insights against each evaluation question to feed into the contribution analysis. The codeframe consisted of rows representing each data source and columns representing key themes or variables of interest. This approach allowed the evaluation to identify themes, patterns and outliers within and across the data sources. MI was analysed using a structured analytical framework linked to the evaluation questions and contribution claims.

Qualitative data and MI were triangulated to validate and strengthen findings against each contribution claim. An evidence rating was used to assess the data collection strands and strength of conclusions (see Appendix A).

Limitations

This evaluation assessed the Adaptation Scotland Programme's contribution to observed outcomes rather than attributing outcomes solely to the Programme. Given the complex policy and delivery environment in which climate adaptation occurs, it was not possible to isolate the impact of the Programme from wider influences.

Data analysed as part of the evaluation was predominantly qualitative in nature. Qualitative research is intended to explore the depth and diversity of stakeholder experiences rather than provide statistically representative findings. References to stakeholder views throughout the report should therefore be interpreted as reflecting the experiences and perspectives of participants rather than the prevalence of views across all Programme stakeholders. Where possible, qualitative findings are complemented with MI.

The evaluation was also limited by the availability and nature of MI. This data was generally stronger for outputs and participation than for longer-term outcomes and impacts, which constrained the extent to which some outcomes could be evidenced through quantitative data. However, the qualitative evidence gathered from stakeholder interviews enabled exploration of short- and medium-term outcomes.

The achieved sample also reflected the nature of Programme engagement and delivery. A greater proportion of participants were drawn from public sector organisations than from private or third sector organisations. While efforts were made to ensure representation across stakeholder groups, some perspectives are represented more extensively than others.