Arctic Connections: policy framework evaluation

Introduction

Overview

GC Insight was commissioned by the Scottish Government to undertake the evaluation of the Arctic Connections Policy Framework (ACPF), including the Arctic Connections Fund (ACF).

The ACPF was launched in 2019 to strengthen collaboration between Scotland and Arctic / sub-Arctic nations. The Framework reflects Scotland’s commitment to international cooperation and knowledge exchange across shared policy challenges, including climate change, community resilience, and sustainable economic development. At the heart of the Framework was the ACF, a small grants programme that has supported 31 projects with a total value of £280,000 from 2021 to 2024. Grants were given to projects through three rounds of funding, and have fostered partnerships and exchanges across academic, cultural, environmental, and policy fields between Scotland and the Arctic.

Objectives

The main objective of this evaluation is to assess the effectiveness of the ACPF and the ACF in delivering Scotland’s international objectives in the Arctic region. The study aimed to examine three key areas:

• How ACF-funded activities contributed to achieving the outcomes of the Framework;
• How the ACF operated and the experience of applying for and using funding; and
• To identify key lessons from the ACF and ACPF that can be applied to future Scottish Government international policy.

To deliver against these objectives, the evaluation sought to address the following evaluation questions:

• To what extent has the Arctic Connections Policy Framework been delivered as intended, with specific attention to implementation through the ACF and the activities of the international network in Arctic and Sub-Arctic regions?
• Based on the experiences of ACF recipients and SG’s offices in Arctic and sub-Arctic nations, what impact did the Arctic Connections Policy Framework have on:
  • Scotland’s relationship with Arctic stakeholders;
  • understanding of Arctic issues in Scotland;
  • and of Scottish issues in the Arctic?
• To what extent was the Arctic Connections Fund delivered as intended, and what were the main impacts of Arctic Connections Fund projects?

Approach and methods

GC Insight adopted a multi-stage, mixed-methods approach to assess the development, implementation, and impacts of the ACPF and the ACF. The methodology was designed to ensure a comprehensive understanding of both the strategic objectives of the Framework and Fund, and the practical experiences of organisations engaged with these initiatives.

The approach comprised desk-based analysis of existing evidence, including 31 end-of-project reports from ACF recipients and responses to an Arctic Connections Fund recipient survey. A thematic analysis framework was applied to review the data in a structured and consistent way. This enabled for the identification of key themes relating to project activities, achievements, challenges, and lessons learned.

Qualitative research was also undertaken and involved interviews with ten ACF funding recipients, and six Scottish Government officials, including two representatives from international offices. These discussions explored experiences of applying for and managing ACF funding, delivery processes, perceived outcomes, and the overall effectiveness of the ACPF in supporting international collaboration

Desk-based analysis

ACF end of project reports

The evaluation team conducted a rapid evidence assessment of each report, these reports were a condition of ACF funding and were submitted to the Scottish Government at the end of the project. They included detailed information on:

• Activities.
• Outputs and deliverables
• Key outcomes.
• Completion of planned outputs.
• Barriers and challenges
• Project changes.
• Project impact.
• Lessons learned.
• Contributions to Scottish-Arctic knowledge exchange.
• Awareness of Scotland’s Arctic Policy Framework
• Longer-term collaboration opportunities.
• Sustainability.

A thematic analysis framework was developed to align the content of the reports with the evaluation research questions. Key themes included:

• Achievement of key outcomes, success factors, and inhibitors.
• Project changes over time and underlying reasons.
• Barriers, challenges, and implications.
• Contribution to knowledge exchange and collaboration.
• Financial performance and variance from planned budgets.

This structured approach ensured consistency, transparency, and the identification of any gaps or limitations in the available data.

ACF recipient survey

A number (13) of grant recipients participated in an online survey issued by the Scottish Government which focussed on the perceived impacts and legacy of funded projects, and the experience of applying for and utilising funding. The survey responses were reviewed using a parallel analysis framework to assess the weight and balance of findings across respondents. Themes included:

• Contribution of projects to ACF aims and objectives;
• Barriers to achieving outcomes;
• Dissemination activities;
• Policy impacts; and
• Sustainability and legacy of projects.

Qualitative research

Qualitative research was undertaken to provide greater depth and context to the desk-based analysis. This stage involved interviews with nine recipients of ACF grants, and six Scottish Government officials involved in the development and operation of the fund, including two Scottish Government officials in international offices. The quotations included in this report are illustrative of the interview findings. These quotes reflect the opinions of interviewees and are therefore subjective, and should not be considered to represent the position of the Scottish Government or Scottish Ministers. A list of stakeholders and recipients consulted with is available in Appendix A.

Discussions with funding recipients focused on their understanding of the ACPF, how their activities aligned with the Fund’s objectives, their experiences of applying for and managing funding, and the processes involved in delivering and monitoring their projects. Participants also reflected on the outcomes achieved and the wider benefits generated through their work.

Interviews with Scottish Government officials explored the development and implementation of the Framework, the effectiveness of collaboration with Arctic partners, and lessons learned to inform future policy design and delivery. The resulting data was systematically analysed in line with the evaluation research questions to identify recurring themes, key insights, challenges, and examples of good practice.

Limitations and Caveats

A key limitation of this evaluation is the absence of a counterfactual comparison. This evaluation does not compare the impact of the ACPF to an alternative scenario in which the ACPF was not delivered. It is therefore not possible to determine with certainty what outcomes would have occurred in the absence of the policy framework or fund. We therefore cannot conclusively say that any impacts described in this report are due exclusively to the ACPF and not to other external factors.There are also inherent limitations and caveats to qualiative research. While it is valuable for understanding people’s experiences and adding necessary context to quantitative data, qualiative findings are shaped by both the perspective of the participant as well as the analytical lense of the researcher. This makes qualitative data subjective in nature. The qualitative findings in this report cannot be generalised to a wider population and should therefore be understood as illustrative of the experiences and opinions of the participants, rather than statistically representative or objectively factual.

For both the interviews and survey included in this evaluation, the sample sizes are small due to the small number of individuals with the necessary experience of the ACPF. It is also important to recognise that the survey respondents represent only organisations who were successful in their applications for funding. Organisations that were unsuccessful in their applications were not able to participate in the survey due to data protection and privacy regulations. This introduces potential for positive bias in the survey findings, as organisations who received funding may have had more positive experiences of the process and have an interest in the continuation of such funding in the future compared to those who did not receive funding.

Report structure

The report is structured as follows:

• Chapter 2 presents an overview of the delivery of the ACPF and the ACF.
• Chapter 3 outlines the implementation and achievements of the ACF.
• Chapter 4 analyses the implementation and impacts of the ACF.
• Chapter 5 examines the issues and challenges of both ACPF and the ACF
• Chapter 6 comprises the individual three case studies of ACF projects.
• Chapter 7 discusses conclusions and recommendations gathered from the evaluation.