Publication - Research and analysis

Evaluation of the Adaptation Scotland Programme

Published: 29 October 2026
From: Cabinet Secretary for Climate Action and Rural Affairs
Directorate: Energy and Climate Change Directorate
Topic: Environment and climate change
ISBN: 9781807757106

Findings from a process and impact evaluation of the Scottish Government's Adaptation Scotland Programme. The Programme supports the delivery of Scotland’s National Adaptation Plan.

Supporting documents

7. Data collection methods and sources

This section summarises the data that will be collected and analysed to answer the evaluation questions.

Monitoring information

Table 7.1 below summarises the monitoring information collected by Verture in delivery of the Programme, and relevance to the evaluation questions.

Table 7.1 - Monitoring information and relevant evaluation questions
Cluster	KPI	Outputs	Mapping to Evaluation questions
Cluster 1. Public Sector Leadership	Number of organisations actively benchmarking via the Adaptation Scotland Adaptation Capability Framework.	Number of organisations actively benchmarking via the Adaptation Scotland Adaptation Capability Framework.Number and theme of meetings/events/webinars held and attendees.Number of requests to join events and the network and conversations with prospective members.Input on consultations/guidance creation.Input into partner organisations’ board/steering groups.Benchmarking resources created and promoted.	To what extent has the Adaptation Scotland Programme been implemented as intended?Who is involved in the delivery of the support and how are Adaptation Scotland collaborating with other groups across their work clusters to deliver this support?What is working well and less well in the delivery of the Programme and where could improvements be made? Does this differ between different types of stakeholders and places? Is there anything missing from the current Programme?To what extent does the Programme delivery model support progress towards outcomes?Who is receiving support from the Adaptation Scotland Programme? Is its intended audience being reached, and if not why is this?Are there any types of stakeholders or places not being sufficiently reached by the Programme? If not, why is this and how could they be better supported and reached by the Programme?To what extent is the Programme supporting the public sector and its leadership to better plan and take action to adapt to climate change? (cluster 1)To what extent has the Adaptation Scotland Programme supported the development of a growing network of leaders across Scotland who are driving effective adaptation action?
	Number of communities supported to participate in place-based approaches to climate adaptation.	Number of community members and organisations supported.Number and theme of meetings/events/webinars held and attendees.Number of communities and third sector organisations using the Community Adaptation Routemap and other AS resources for local action.	To what extent has the Adaptation Scotland Programme been implemented as intended?Who is involved in the delivery of the support and how are Adaptation Scotland collaborating with other groups across their work clusters to deliver this support?What is working well and less well in the delivery of the Programme and where could improvements be made? Does this differ between different types of stakeholders and places? Is there anything missing from the current Programme?To what extent does the Programme delivery model support progress towards outcomes?Who is receiving support from the Adaptation Scotland Programme? Is its intended audience being reached, and if not why is this?Are there any types of stakeholders or places not being sufficiently reached by the Programme? If not, why is this and how could they be better supported and reached by the Programme?To what extent is the Adaptation Programme supporting capacity building in communities and organisations working with communities? (cluster 2)To what extent have adaptation tools, resources and processes led by Adaptation Scotland supported more effective adaptation action across organisations in Scotland?
Cluster 3: Private Sector Leadership	Number of businesses [and advisors] engaged and supported	Number of businesses [and advisors] engaged and supported.Number of training sessions delivered and attendance.Tools created and promoted.Downloads of business support tools and resources.	To what extent has the Adaptation Scotland Programme been implemented as intended?Who is involved in the delivery of the support and how are Adaptation Scotland collaborating with other groups across their work clusters to deliver this support?What is working well and less well in the delivery of the Programme and where could improvements be made? Does this differ between different types of stakeholders and places? Is there anything missing from the current Programme?To what extent does the Programme delivery model support progress towards outcomes?Who is receiving support from the Adaptation Scotland Programme? Is its intended audience being reached, and if not why is this?Are there any types of stakeholders or places not being sufficiently reached by the Programme? If not, why is this and how could they be better supported and reached by the Programme?To what extent is Adaptation Scotland engaging with the private sector? Are there enablers and barriers in private sector engagement, and how can this type of engagement be improved? (cluster 3)To what extent has the Programme effectively driven place-based adaptation initiatives across cities, regions and localities in Scotland in a way that is aligned with the Place Principle?
Cluster 4: Knowledge, Innovation, and Research	Range and number of institutions engaged with, and policy areas contributed to	Number of individuals engaged.Number of policy areas contributed to.Climate Ready Infrastructure Scotland (CRIS) secretariat and meetings.Number of regions developing adaptation investment plans.Training delivered with partners.Research projects and tools informed by AS Programme.	To what extent has the Adaptation Scotland Programme been implemented as intended?Who is involved in the delivery of the support and how are Adaptation Scotland collaborating with other groups across their work clusters to deliver this support?What is working well and less well in the delivery of the Programme and where could improvements be made? Does this differ between different types of stakeholders and places? Is there anything missing from the current Programme?To what extent does the Programme delivery model support progress towards outcomes?To what extent is the Programme embedding knowledge exchange, innovation and research? How is this supporting adaptation outcomes at different levels and with different groups? (cluster 4)To what extent has the Programme contributed to an increase in knowledge, engagement and capacity in shaping and delivering adaptation policy and action in Scotland?To what extent has the Programme contributed to bettering the value of and acting upon the views of the most vulnerable to climate change at both national and local level?
Cluster 5: Communication	Increased website performance (e.g. traffic, session duration, bounce rate)	Number of website visits and duration; bounce rate; event count; page views per user.Programme comms strategy and plan; new approaches tested.Promotion of the Programme.Case studies published.	To what extent has the Adaptation Scotland Programme been implemented as intended?Who is involved in the delivery of the support and how are Adaptation Scotland collaborating with other groups across their work clusters to deliver this support?What is working well and less well in the delivery of the Programme and where could improvements be made? Does this differ between different types of stakeholders and places? Is there anything missing from the current Programme?To what extent does the Programme delivery model support progress towards outcomes?How effectively does the Adaptation Scotland Programme communicate with and engage stakeholders to increase public awareness and engagement on climate risk and adaptation? (Cluster 5).To what extent has the Adaptation Scotland Programme contributed to the development and uptake of effective approaches by cross-sector collaborations to support fair, just and sustainable adaptation?
Cluster 6: Place-based and regional adaptation	Increased geographical coverage of regional adaptation collaborationValue of funding secured by regional adaptation collaborationsProgression of partnerships through the red, amber and green stages of the routemap	New and emerging partnerships supported.Public and private financing/investment secured.Resources created.Events held.	To what extent has the Adaptation Scotland Programme been implemented as intended?Who is involved in the delivery of the support and how are Adaptation Scotland collaborating with other groups across their work clusters to deliver this support?What is working well and less well in the delivery of the Programme and where could improvements be made? Does this differ between different types of stakeholders and places? Is there anything missing from the current Programme?To what extent does the Programme delivery model support progress towards outcomes?Who is receiving support from the Adaptation Scotland Programme? Is its intended audience being reached, and if not why is this?Are there any types of stakeholders or places not being sufficiently reached by the Programme? If not, why is this and how could they be better supported and reached by the Programme?To what extent is the Programme supporting the development of collaboration and partnerships, including Regional Adaptation Partnerships? To what extent is the Programme supporting the development of these types of partnerships and what works well and less well when providing this support? How could support to Regional Adaptation Partnerships be improved to achieve national coverage by 2029? (cluster 6).
Cluster 7: Policy support	Scottish Government policy priorities supported by Adaptation Scotland services	Number and type of policy areas supported.Contributions to policy documents, events, guidance, consultations.	To what extent has the Adaptation Scotland Programme been implemented as intended?Who is involved in the delivery of the support and how are Adaptation Scotland collaborating with other groups across their work clusters to deliver this support?What is working well and less well in the delivery of the Programme and where could improvements be made? Does this differ between different types of stakeholders and places? Is there anything missing from the current Programme?To what extent does the Programme delivery model support progress towards outcomes?In what ways does the Adaptation Scotland Programme engage across national policy making? (cluster 7)To what extent has the Programme supported a policy change in ambition on adaptation policy and helped mainstream adaptation across policy areas?How has this been done effectively over the course of the current Programme and how might the policy support provided to Scottish Government be improved in the future?
Cluster 8: Programme management	Timely reporting in line with contract requirementsQuarterly Programme Board meetings organised, with circulation of agenda and papers in advanceCustomer service satisfaction levels	Reporting requirements met.Board meetings organised and papers circulated in advance.Suppliers paid with 30 days of invoicing.Feedback from events.PSCAN member self-reflection on whether participation has supported the development of adaptation capabilities in their organisation.	To what extent has the Adaptation Scotland Programme been implemented as intended?Who is involved in the delivery of the support and how are Adaptation Scotland collaborating with other groups across their work clusters to deliver this support?Is the Programme sufficiently resourced and supported to deliver against the Programme aims?What is working well and less well in the delivery of the Programme and where could improvements be made? Does this differ between different types of stakeholders and places? Is there anything missing from the current Programme?To what extent does the Programme delivery model support progress towards outcomes?How does the Programme governance work and what role does this play in supporting delivery of the Programme and its deliverables?

The quarterly and year-end Adaptation Scotland Programme reports set out progress against the KPIs. Ipsos will review the reports from the current Adaptation Scotland contract period. Ipsos will also ask Verture to share anything other monitoring information collected or produced (e.g. Adaptation Scotland Board papers).

Secondary data

The review of secondary data will be guided by the key evaluation questions (see Table 7.2). We anticipate the principal focus for the review will be on the quarterly and year-end Adaptation Scotland Programme reports. We would agree criteria for the review of these reports with you, but at this initial stage have assumed the review would cover the past three years, and that the requirement will therefore be to review c.12 quarterly reports and three year-end reports. We also anticipate there may be further relevant secondary documentation to review, and have allocated an additional three days’ researcher time in our budget to cover this.

Table 7.2 – Secondary data sources
Source type	Description of Source	Relevance for the evaluation	Mapping to Evaluation questions
Public/private/community sector publications	Selected documents relating to planned adaptation activities for organisations taking part in an in-depth interview.	Impact: evidence for impact of Adaptation Scotland support.	All impact evaluation questions.
Regional adaptation partnership documentation	Documents from regional adaptation partnerships e.g. governance papers, workplans, progress notes.	Process: how Adaptation Scotland support has supported regional partnerships.Impact: maturity/depth of place-based collaboration; contribution to SNAP3 coverage indicator; local outcome signals.	To what extent is the Programme supporting the development of collaboration and partnerships, including Regional Adaptation Partnerships? (cluster 6)To what extent has the Programme effectively driven place-based adaptation initiatives across cities, regions and localities in Scotland in a way that is aligned with the Place Principle?
SNAP3 Monitoring & Evaluation Framework (Scottish Government)	Indicators relevant to adaptation action (e.g. Scottish Climate Survey data).	Impact: evidence for impact of Adaptation Scotland support.	To what extent has the Programme effectively driven place-based adaptation initiatives across cities, regions and localities in Scotland in a way that is aligned with the Place Principle?To what extent has the Programme contributed to an increase in knowledge, engagement and capacity in shaping and delivering adaptation policy and action in Scotland?
Scottish Government publications referencing Adaptation Scotland	Relevant Scottish Government news articles; consultation analysis; researched published that refers to the Programme.	Process: visibility of Programme; contribution to policy.	To what extent has the Programme supported a policy change in ambition on adaptation policy and helped mainstream adaptation across policy areas?How has this been done effectively over the course of the current Programme and how might the policy support provided to Scottish Government be improved in the future?
Climate Action Hubs	Quarterly reporting on Climate Action Hubs have completed to support the development of Regional Adaptation Partnerships.	Contribution analysis.	NA.

Qualitative primary data collection

Primary qualitative evidence will help illuminate how and why the Programme works across different contexts. We will conduct in-depth interviews with stakeholders who have received Adaptation Scotland support and those eligible but not reached, as well as interviews with delivery and governance stakeholders.

Table 7.3 - Interview sample
Cluster	No. Interviews	Targets
Cluster 1 - public sector leadership	22	c.9 Local Authorities (mix of geographies across Scotland) c. 4 other public bodies (e.g. colleges, NHS Board, national parks) c.6 National bodies (mix of sectors: health, infrastructure, justice, land, finance, tourism) c.3 public sector / other public bodies not receiving AS support directly
Cluster 2 - community and third sector leadership	11	c.11 across group
Cluster 3 - private sector leadership	8-10	3 Enterprise Agencies or other business support organisations c. 5 businesses or industry representative organisations receiving AS support: mix of sectors c. 2 umbrella organisations not receiving AS support directly: mix of sectors (including SNAP3 priority industries - agriculture, forestry, fishing, aquaculture)
Cluster 4 - knowledge, innovation and research	3-5	3 Academics and research organisations engaging with AS (inc. Met Office) 2 Climate Ready Infrastructure Scotland Forum (co-chairs, National Centre for Resilience delivery partners)
Cluster 5 - communication and engagement		To be explored across clusters 1-4 and 6-7 interviews (and monitoring information) rather than a specific sampling group
Cluster 6 - place-based adaptation/ regional partnerships	8	Mix of stakeholders representing Argyll & Bute, Dumfries & Galloway, Forth Valley, Outer Hebrides, Tayside (as partnerships being developed under current contract)
Cluster 7 - policy support	5	SG policy teams who have engaged with AS (e.g. land, islands, biodiversity, health, housing, planning) 1 SG official responsible for delivery of the SNAP3 public consultation
Cluster 8 - programme management	11	7 Verture staff (cluster leads) 4 SG AS board members (programme management – chair of AS board, contract manager, adaptation team leader, SG’s Cluster 3 lead)
Total	70	70 across all groups

We will work with Verture to identify and contact organisations. A data sharing agreement between Ipsos and Scottish Government will be in place to facilitate sharing contact details. We will attempt private sector recruitment without incentives; a small reserve is available if required. We will monitor participation by sector, size, geography and vulnerability profile and adjust outreach to address any gaps.

Interviews will last around 60 minutes. Interviews will be carried out via video call or phone call. It may be possible to carry out some interviews in person. Fieldwork will run between late March and late May 2026.

Interview materials

All interviews will be conducted in accordance with the Market Research Society (MRS) Code of Conduct. Each participant will receive an information sheet and privacy notice, clearly explaining what participation involves and their rights.

Discussion guides will be tailored by audience (public sector, private sector, community/ voluntary sector; Programme governance/delivery) and mapped explicitly to evaluation questions, ToC pathways and contribution claims. The first few interviews will act as a form of pilot, with the research team reflecting on how the discussion guides are working and suggesting any necessary. amendments. All interview materials with be submitted to Scottish Government for approval.

Synthesis

Synthesis of the various data sources outlined above will support a holistic and multi-dimensional evaluation. To synthesise the data sources, we will develop a structured codeframe in Excel for extracting key data and insights against each evaluation question to feed into the contribution analysis. Data will be extracted from sources manually by the research team. The codeframe will consist of rows representing each data source and columns representing key themes or variables of interest. This approach will allow us to identify themes, patterns and outliers within and across the data sources. This will be particularly important to allow us to identify what works well or less will in different contexts.

This approach will also allow us to triangulate the data (i.e. identify how many sources support a particular theme) to validate and strengthen findings. The evidence will likely vary in strength across the different data collection strands, and an explicit assessment will be needed of how strong a conclusion is (e.g. considering whether it relies primarily on third party assessments of impacts, or whether it also includes stakeholders’ own assessments). Our suggested assessment approach is summarised in the rubric below:

“Strength of Evidence” assessment for each evidence collected

3. Verifiable evidence

Refers to data that are both plausible and possible to verify. Such evidence generally describes quantifiable measures that can be physically counted.

2. Plausible evidence

This includes evidence which may make a plausible claim but may draw heavily on assumptions from secondary data, for example Scottish Climate survey on public adaptation actions in particular areas/regions.

1. Minimal evidence

Some documents may simply claim an outcome but there may be no information about the data or methodology used to evidence this claim.

Contact

Email: Rachael.Punton@gov.scot

Thanks for your feedback

Information