Evaluation of the Adaptation Scotland Programme

D.5: Text from research and innovation discussion guide

Aims of this interview

• For academics and organisations involved in research and innovation (cluster 4): The primary aim of this interview is to gather information on how the Programme has been delivered in practice, to gather insights for the process element of the evaluation.
• We also want to understand how well the Adaptation Scotland Programme has been delivered / contributed to its intended outcomes, so the interview gathers some insights for the impact element of the evaluation.
• Timings for sections are indicative and will be adjusted by the researcher depending on the individual’s involvement in the Adaptation Scotland Programme.

Contribution claims

Contribution claim 2 (CC2): The Adaptation Scotland Programme has contributed to the development of effective approaches to support fair, just and sustainable adaptation.

The Programme engages with specialists (i.e. practitioners, researchers, institutions and policy teams) to increase understanding of what effective adaptation looks like, connecting specialists and networks to encourage cross-sector collaboration and (co-) develop tools and resources. These tools and resources are used by public sector bodies, communities and businesses to develop effective approaches for tackling adaptation challenges which support a fair, just and sustainable adaptation.

Contribution claim 4 (CC4): The Adaptation Scotland Programme has contributed to an increase in knowledge, engagement and capacity in shaping and delivering adaptation policy and action in Scotland.

Adaptation Scotland regularly shares the work and findings of the Programme, and insights from wider relevant adaptation work, to its stakeholders. More stakeholders are engaging with the Programme to shape and deliver adaptation policy and action in Scotland. Stakeholders have a greater capacity and feel more empowered to shape and deliver policy and action in Scotland. 

Introduction and consent

• Introduce self and Ipsos (independent research organisation).
• Thank participant for taking part.
• Check that they received and if they have read the information sheet (if they haven’t, explain that you will go through the key points now and can share another copy with them if needed).
• Remind them of the purpose of this interview: As a reminder, the purpose of this interview is for us to understand your views on how the Adaptation Scotland Programme has been working in order to inform both a process and impact evaluation. This means I’ll be asking about your experiences of engaging with the Programme, to understand what you found worked well and what worked less well. I will also be asking about whether engaging with the Programme has led to any changes within your organisation, and why.
• Explain anonymity and confidentiality: As a reminder - your name will not be disclosed in the reports. However, quotes and elements of reporting may be linked to individual areas. If there is anything that you do not want us to quote you on as part of our reporting, do let us know. I will check in on this at the end of the discussion today.
• The only time that we would disclose your identity is if you told us something that indicated that you or someone else was at risk of harm, in which case we would pass your contact details over to our safeguarding lead.
• Remind participant that there are no right or wrong answers. The purpose of the evaluation is learning, to inform ongoing design and delivery and to understand impacts of the Adaptation Scotland Programme as a whole, and not to evaluate the success of particular organisations or projects.
• Emphasise that taking part is completely voluntary – they don’t have to answer any questions they don’t want to answer and can decide to stop the interview at any point.
• Remind that they are free to change their mind and decide not to take part at any time before or during the interview, or after the interview until the findings have been written up.
• This interview will last approximately 60 minutes.
• Check if participant has any questions (offer to go through information sheet if necessary).
• Request permission to record interview. Explain that this is for transcription and analysis purposes and that recordings will not be shared outside the research team at Ipsos.
• If participant does not consent, please take detailed notes.
• If participant provides consent, please start recording.
• Start recording if consent given

Their role and involvement in the Adaptation Scotland Programme

Firstly, I’d like to ask you a few questions about your organisation, your role, and your involvement in the Adaptation Scotland Programme. Could you start by briefly describing your role and your organisation?

• Job title and main responsibilities
• How long you’ve been in this role

Could you briefly describe your / your organisation’s involvement with the Adaptation Scotland Programme? Probe for tools/resources developed, events/networks attended

• How did this come about?
• Probe for whether involvement was initiated by the organisation or the Adaptation Scotland Programme.

Based on information gathered here, facilitator to establish the most prominent form of support and lead with that section (3-6). If other sections also apply, focus on gathering key info and relative impact (i.e. how it compares to other support received) using highlighted Qs for guidance and be mindful of previous answers given.

[If not covered already] When did your organisation first become involved with the Adaptation Scotland Programme?

• Is your involvement ongoing?
• [If not] For how long was your organisation involved? What are the reasons why you are no longer involved?

Has your organisation engaged with other climate adaptation initiatives outwith the Adaptation Scotland Programme? [If needed e.g. via other policy interventions or funding streams e.g. Climate Action Hubs, Sustainable Scotland Network, Climate Intelligence Service] [If needed: by climate adaptation I mean actions to address the actual or anticipated impacts of climate change.]

• What did this look like?
• How did you find out about this?
• How effective was this? What was effective about it?

Adaptation tools and resources

I’d now like to ask you some questions about the tools and resources that are produced through the Adaptation Scotland Programme.

If not already covered: Have you, or others in your organisation, been involved in creating any tools or resources developed by the Programme for the public sector, private sector and third sector? [if needed: e.g. Adaptation Capability Framework, Public Sector Handbook and Starter Pack, Investment training or finance resources, Climate change projections for Scotland] (CC2)

If yes, for each resource (prioritise most recent if there are many):

• When roughly were you involved in this?
• What did this look like/what did you do?
• Who else was involved?
• If relevant: did you see your contribution reflected in the resources developed?
• How satisfied or dissatisfied were you with this collaboration overall?
• Is there anything that could have been improved?

Are you aware of how Adaptation Scotland shares these resources with stakeholders who could benefit from them? (CC4)

• What works well?
• What works less well?

To what extent would you describe the tools or resources as ‘effective’? (CC2)

• What made it effective? Probe for examples (e.g. format, content)
• Do you know whether these resources or tools have been used by public sector bodies, communities and businesses?
• What impact is that having?
• What could make the tools/resources more effective?

Have these tools/resources helped you collaborate with other organisations or sectors on adaptation? (CC2)

• Probe: Have you shared these tools/resources with partners? (CC4)
• Can you give an example of cross-sector collaboration that was supported by Adaptation Scotland resources?

If not involved in creating tools or resources: Were you aware that the Adaptation Scotland Programme develops resources for public sector, private sector, third sector organisations? (CC2)

• [If not aware] Would contributing to tools/resources be something you would be interested in? Why/why not?
• [If not aware] How would you want to hear about opportunities to collaborate on tools/resources?
• [If aware] What are main the reasons why you have not? (CC2)
• Probe: Not relevant to your organisation, no capacity, something else?

Ask all: If not already doing so, what would be the most effective tools/resources the Programme could create for the public sector, private sector and communities? (CC2)

Other tools and resources

Moving on slightly, and how about Adaptation Scotland Programme input into resources or publications owned by you/your organisation? For example, has the Programme contributed to any of your research submissions, calls for evidence or development of any resources? (CC2)

If yes, for each adaptation scotland input (prioritise most recent if there are many):

• When roughly did this happen?
• How did this come about? Probe for whether involvement was initiated by the organisation or the Adaptation Scotland Programme.
• Who else was involved?
• How satisfied were you with this collaboration?
• Was there anything that could be improved about the process?

To what extent would you describe the input from Adaptation Scotland as ‘effective’? (CC2)

• What specifically did input from Adaptation Scotland provide that you couldn’t have received elsewhere?
• [If has contributed to tools/resources] Do you know whether these resources or tools have been used by public sector bodies, communities and businesses?
• How were these resources shared? (CC4)
• What impact is that having?
• What could make the resource more effective?

If not received adaptation scotland support with resources: Have you requested any support from Adaptation Scotland in research submissions, calls for evidence or development of any resources? (CC2)

• If yes: what support were you looking for? What happened? (Probe fully if interviewee mentions that Adaptation Scotland didn’t have capacity to support the request).

If haven’t engaged nor attempted to engage: what are the main reasons for not seeking support from the programme? Ask openly then probe:

• Capacity e.g. workload, budget constraints, competing priorities
• Accessibility e.g. format, location, timings
• Content e.g. relevance to their sector, their needs
• Organisational factors e.g. leadership buy-in
• What would have made it easier for you to engage with the programme?

Ask all: What would be the most effective way Adaptation Scotland could collaborate with your organisation? (CC2)

Training and events

I’m now going to ask some questions on any training sessions, workshops, conferences or other events.

As part of your organisation’s engagement with the Adaptation Scotland Programme, have you, or others in your organisation, delivered, attended or supported any training, workshops, conferences or events organised through the Programme? For example, presenting to the Public Sector Climate Adaptation Network (PSCAN) (CC2, CC4).

[If yes] probe on:

• How many different sessions have you/your organisation attended/delivered?
• What was the purpose?
• What did attending involve?
• How did you hear about the session/opportunity to deliver training?

To what extent would you describe the sessions as effective?

• [If attended event] Was there a session that you found particularly effective? What made it effective? Probe for examples.
• [If delivered event] To what extent did you find delivering the event was beneficial to you/your organisation?
• [If delivered event] And to what extent do you think that the event was beneficial to those attending? What was it about the event that makes you feel this way? E.g. did you receive feedback from attendees?
• Do you know whether training/information has been used by public sector bodies, communities and businesses?
• What impact is that having?

How did you find the experience of attending/delivering the sessions(s)?

• What worked well?
• Was there anything that could be improved?
• Was there anything missing that you would like to have seen?
• What was your main takeaway from the sessions(s)?

Did attending/delivering the session(s) help you to connect with other organisations working on adaptation? (CC2)

• Did the sessions(s) provide opportunities to work together or share experiences?
• What impact has this had? Probe for specific examples.
• Have you collaborated with any of these other organisations since the session(s)?

Is there anything that your organisation is doing differently as a result of attending/delivering the session(s)? (CC4)

Moving on slightly, has the Adaptation Scotland Programme delivered/ attended/ supported any training, workshops or events for your organisation? (CC2)

[If yes] probe on:

• How many different sessions?
• What was the purpose?
• What did attending involve?
• How did this come about? Probe whether initiated by the organisation or the Programme?

To what extent would you describe the sessions as effective?

• Was there a session that you found particularly effective? What made it effective? Probe for examples.
• What impact has this had on your organisation?

How did you find the experience of Adaptation Scotland delivering/ attending/ supporting the session?

• What worked well?
• Was there anything that could be improved?
• Was there anything missing that you would like to have seen?

If not covered already: has Adaptation Scotland involvement help you to connect with other organisations working on adaptation?

• What impact has this had?
• Have you collaborated with any of these other organisations since the session(s)?

If not aware/not attended any events via the programme: Were you aware that that Adaptation Scotland Programme hosts events/workshops/training sessions for public sector, private sector and third sector organisations? (CC2, CC4)

• [If not aware] Would contributing on /attending events be something you would be interested in? Why/why not?
• [If not aware] How would you want to hear about opportunities to collaborate on / attend events?
• [If aware] What are main the reasons why you have not? (CC2, CC4)
• Probe: Not relevant to your organisation, no capacity, something else?

Adaptation Scotland networks / cross sector collaboration

If not covered already: are you aware of any networks supported by the Programme? For example the Public Sector Adaptation Network (PSCAN) which brings public bodies together to benchmark their progress and identify and share learning on climate change adaptation.

If yes:

• What did you know about these networks?
• How did you hear about them?
• What involvement have you had with them? How have you contributed to the network?
• What impact has this had on your organisation?
• What is it about the network that works well?
• What could be more effective about it?

If no:

• Would a network like this be something you would be interested in being involved in?
• Why/why not?
• If not: what would make it useful to your organisation?

Ask all: If not covered already: are you part of any [other] networks focused on climate adaptation or sustainability?

If yes:

• How did this come about?
• Have you seen any examples of initiatives that drive effective adaptation action as a result of this collaboration?
• What difference, if any, has this made to your organisation?
• How have you contributed to the collaboration?

If no:

• Would that be something you would be interested in?
• Why/why not? What, if anything, would make cross sector collaborations useful to your organisation?
• Probe: Not relevant to your organisation, no capacity, something else?

Overall outcomes

So, to start summing up, overall, how satisfied are you with your engagement with the Adaptation Scotland Programme?

• What is working well and less well in the delivery of the Programme and where could improvements be made?
• If not covered already: Have you requested any support which the Programme has not been able to provide?
• If not covered already: How have you found the responsiveness of the Programme to requests for support, e.g. openness, willingness to respond to requests for information, response times, flexibility of the programme to account for specific needs of your organisation?
• How effectively would you say that the Programme communicates information on climate risk and adaptation and best practice adaptation action?

If not covered already: what impact has involvement in the Programme had on you / your organisation?

• Has your organisation done anything differently as a result?
• What specifically did the Adaptation Scotland programme provide that you couldn’t have received elsewhere?

To what extent would you say the Programme has contributed to the development of effective adaptation approaches? (CC2) Refer back to examples discussed and probe for specific types of support that have helped with this, and whether anything else has helped with this.

• To what extent, if at all, did involvement increase your understanding of what effective adaptation looks like? (CC2)
• To what extent, if at all, did involvement help you / your organisation consider fairness, justice or equity in adaptation approaches? (by this we mean approaches that considers the needs of different groups and addresses existing inequalities) (CC2)
• Probe: Have the tools/resources helped you consider the needs of groups or communities most vulnerable to climate change?

To what extent would you say the Programme is helping to embed knowledge exchange, innovation and research? (CC4) Refer back to examples given and probe for any other examples and specific types of support that have helped with this, and whether anything else has helped with this.

• To what extent if at all did involvement in the Programme increase your knowledge and understanding of what adaptation actions are needed

Wrap up and close

That’s everything I wanted to ask you today. Is there anything else you’d like to add before we finish?

Thanks very much for your time. Can I please check if there is anything you would prefer us not to quote or refer to directly in our reports for the Scottish Government?