Climate Action Hubs programme: evaluation

2. Methodology

Key points:

• The evaluation used a mixed‑methods approach, combining qualitative and quantitative approaches aligned with the programme’s Theory of Change.
• A four-phase approach was adopted: inception and planning, secondary data review, fieldwork across all 24 Hubs, and quantitative and qualitative analysis.
• The secondary data review involved analysis of over 300 documents from all Hubs, mapping data against outcomes and research questions using a structured analysis framework.
• Fieldwork involved qualitative interviews, focus groups, an online questionnaire and event observation, capturing insights from 110 interview participants and 175 survey respondents across diverse stakeholder groups.
• Analysis followed structured qualitative coding aligned to the Theory of Change and contribution analysis principles, alongside quantitative analysis of survey responses and monitoring data.

Several data limitations were identified, including purposive sampling biases, inconsistent Hub monitoring data, staff turnover, and challenges attributing impact within complex systems.

This chapter describes the evaluation methodology. A mixed‑methods approach was taken in this evaluation, combining qualitative and quantitative methodologies to gather insight about programme delivery and impact. The methodology was developed in line with the programme’s ToC and the research questions set by the Scottish Government.

2.1: Evaluation design

The evaluation followed a four‑phase structure:

• Inception and planning
• Secondary data review
• Fieldwork with Hubs and wider stakeholders
• Analysis and reporting

Each phase is discussed in more detail below.

2.2: Inception and planning

The project began with an inception meeting followed by scoping calls with key stakeholders involved in the delivery of the programme at a national level. These calls were designed to enhance the research team’s understanding of the programme and refine priorities for the evaluation. A total of seven scoping calls were conducted with stakeholders from the Scottish Government’s Climate Change Directorate and the Scottish Rural Network. Scoping calls primarily explored the structure and governance of the Climate Action Hubs programme, key research questions and priorities for the evaluation.

Following these calls, an evaluation framework reflecting the programme’s ToC was developed. This mapped outcomes, indicators, research questions and data sources.

A suite of research tools was then designed, including:

• An information sheet and privacy notice for participants
• A discussion guide for interviews with those directly involved in the delivery of Hub activity and governance (e.g. Hub managers, Hub staff, Board members, volunteers)
• A discussion guide for interviews with those involved with the Hub on a less formal or non-professional basis (e.g. network members, representatives from community groups, individuals who have received support, seed funding or training from Hubs)
• A discussion guide for interviews with stakeholders involved in the programme on a national level
• An online questionnaire, with routing applied to ensure relevant information was collected from each respondent group

The evaluation tools are included in Annex 2.

2.3: Secondary data review

Each Hub submits quarterly and annual monitoring reports to the Scottish Government. These documents contain quantitative and qualitative information collated by Hubs, including a summary of activity delivered during the reporting period, the number of events, workshops or training undertaken in the reporting period and a description of progress against each of the Hub’s agreed objectives.

Throughout the evaluation, the research team reviewed project documentation provided by all 24 Hubs. The documents included in this analysis spanned from the first year of the programme (2021-2022) through to the current year (2025-2026). Over 300 documents were reviewed, including end‑of‑year reports, quarterly monitoring data, application forms, business cases, case studies and project reports.

A structured analysis framework was used to map all available data against the programme’s outcomes and research questions. Quantitative indicators were collated into a master dataset where possible, while qualitative content underwent thematic coding, to enable analysis. Findings from this analysis exercise are incorporated throughout this report, supplementing evaluation data collected via fieldwork.

2.4: Fieldwork

The fieldwork phase involved identification and recruitment of participants, the delivery of qualitative interviews and focus groups, distribution of an online questionnaire, and observation of Hub events.

Sampling and recruitment of participants

A core requirement of the evaluation was to gather insight from a broad spectrum of stakeholders and to ensure all Hubs were included in the data collection stage. However, due to project timescales and resource constraints, it was not possible to undertake in-depth fieldwork with all 24 Hubs. Instead, a tiered approach was used, meaning that, while all Hubs were included, the extent of fieldwork varied across Hubs (Table 2.1).

A sampling plan was designed to ensure representation of different Hub characteristics across each tier, including:

• Geography; coverage of Scotland, balance of urban and rural areas, island representation
• Hubs covering one/multiple local authorities
• Hubs delivered by multiple partners/one lead organisation
• Hub activity and climate risks faced e.g. coastal and inland challenges
• Hub maturity/length of operation

Once each Hub was allocated a tier, a purposive sampling approach was adopted to identify evaluation participants, with Hub managers acting as ‘gatekeepers’ to other contacts. To ensure that the evaluation captured data from a diverse range of stakeholders (i.e. not just Hub staff), Hub managers were instructed to identify individuals with a broad range of relationships to the Hubs, including:

• Hub staff
• Board members and trustees
• Delivery partners and host organisations
• Volunteers
• Network members
• Seed funding recipients
• Members of the community who had taken part in training or events hosted by the Hub
• Other professional contacts, e.g. local authority staff or councillors

A £20 shopping voucher was offered to community members, seed funding recipients and Hub network members to encourage participation in the evaluation from those with non-professional relationships to Hubs.

Scottish Government officials also identified national stakeholders to take part in interviews.

An online questionnaire was widely distributed by Hub managers to individuals with insight into Hub activities. Hub managers did this in different ways; for example, by sharing the link directly with relevant contacts via email, including a link to the survey in online newsletters or using mailing lists, websites or social media to promote the survey.

Quotas were set for each participant group and the evaluation team closely monitored the sample throughout the data collection stage to ensure that a broad range of stakeholders were represented in the evaluation.

Data collection

The data collection phase involved qualitative interviews, focus groups, an online survey and observation of Hub events.

Interview and focus group participants received an information sheet and privacy notice in advance of participation. Discussions followed a semi‑structured topic guide aligned to the research questions, but remained flexible to reflect local context and participant roles. Interviews and focus groups were recorded and transcribed^[2].

In total, 110 participants took part in qualitative interviews or focus groups, and the online questionnaire received 175 responses, providing a broad cross‑section of views across the Hub network. Table 2.2 provides further detail on the number of stakeholders interviewed and survey responses received from each stakeholder group.

Evaluators also observed Hub events, including one Hub’s online monthly network meeting and an in-person gathering of all 24 Hubs. At these events, evaluators recorded descriptive data on the nature of the activities taking place, the ways participants engaged with one another, and the approaches used to facilitate discussion, learning and collaboration. No identifiable information was included in the descriptive data, and this was captured at a general level rather than including specific examples or references to individual discussion points.

2.5: Analysis

Qualitative analysis

All qualitative data gathered through interviews, focus groups, events^[4] and open‑ended survey responses was analysed using a structured coding framework aligned to the programme’s ToC. This approach ensured that emerging themes were examined in relation to the programme’s anticipated outcomes.

Analysis was informed by the core principles of contribution analysis, recognising that the Hubs operate within complex systems where multiple factors influence outcomes. Rather than seeking to attribute impact solely to Hub activity, the evaluation examined the extent to which the data supports plausible contribution to the programme’s intended outcomes. Analysts tested the key causal assumptions embedded in the ToC (for example, that Hub engagement and capacity-building would increase knowledge, confidence, connections and enable communities to initiate or scale climate action). In practice, this involved triangulating independent accounts of causality across data sources, such as interviews, focus groups, observational data, survey findings and monitoring documentation, while also exploring alternative explanations and other influencing factors.

Through this approach, the evaluation has produced credible findings on the extent to which the Hub programme is contributing to early outcomes, while acknowledging areas of uncertainty and the wider contextual factors at play.

Quantitative analysis

Survey data was analysed using Microsoft Excel. This involved producing descriptive statistics (e.g. frequency tables) and cross-tabulations to identify patterns by stakeholder group.

Where Hubs provided quantitative monitoring information, this was compiled and analysed to identify trends in participation, engagement, grants and project delivery. This involved cleaning and consolidating monitoring datasets and producing summary statistics such as totals, averages and ranges.

2.6: Data and research limitations

When reviewing this report, it is important to recognise the data and research limitations which may impact the comprehensiveness and reliability of the findings. These include:

• Purposive sampling: While purposive sampling enables engagement with stakeholders with the most in-depth and relevant insight, it can also pose a risk that some groups are underrepresented, particularly those less engaged with Hub activity or those holding more critical views. As a result, the findings may reflect a degree of positive bias or may not fully capture the breadth of experiences across all perspectives.
• Inconsistencies in monitoring data: The level of detail, completeness and quality of data provided in monitoring reports varied across Hubs. While many provided rich and well‑structured data, others submitted information that was more limited or presented in different formats, reducing comparability and making it difficult to build a fully consistent dataset for the programme. These inconsistencies limited the extent to which quantitative data could be collated and analysed.
• Staff turnover: Staff turnover in some Hubs limited the ability to capture longer‑term perspectives on programme delivery or change over time. Newer staff members often provided valuable insight into current activity but were less able to reflect on earlier decisions, relationships or legacy impacts. This introduced some gaps in recall and continuity, particularly for Hubs that had experienced multiple staffing changes.
• Challenges in attributing impact: Given the complex systems in which Hubs operate and the wide range of external factors influencing community‑level climate action, it is not possible to attribute impacts observed solely to Hub activity. While the evaluation applied contribution analysis to assess plausible influence, the nature of the programme means that impact cannot be isolated with certainty. As a result, findings focus on contribution rather than attribution.

These limitations have been mitigated as far as possible through methodological triangulation, strategic sampling and careful interpretation of evidence.