Evaluation of the Adaptation Scotland Programme

D.2: Text from public sector discussion guide

Introduction

Aims of this interview:

• For public sector organisations who have received support from the Programme: The aim of this interview is to gather information on whether the organisation has made progress towards the intended outcomes of the Programme, as set out in the contribution claims (see below).
• We also want to understand how the Adaptation Scotland Programme has been delivered in practice, to gather insight into the activities and outputs of the Programme. Aspects of this guide will therefore focus on the interviewee’s experiences of engaging with the Programme.

Contribution claims

Contribution claim 1 (CC1): The Adaptation Scotland Programme has contributed to the development of a growing network of leaders across Scotland who are driving effective adaptation action.

The Programme engages more businesses, communities and public bodies to form and grow networks in place-based approaches to climate adaptation. These growing networks support leaders to increase their knowledge and understanding of what adaptation actions are needed and how to act. This enables leaders to take more ownership and responsibility to drive adaptation action and share knowledge on adaptation.

Contribution claim 2 (CC2): The Adaptation Scotland Programme has contributed to the development of effective approaches to support fair, just and sustainable adaptation.

The Programme engages with specialists (i.e. practitioners, researchers, institutions and policy teams) to increase understanding of what effective adaptation looks like, connecting specialists and networks to encourage cross-sector collaboration and (co-) develop tools and resources. These tools and resources are used by public sector bodies, communities and businesses to develop effective approaches for tackling adaptation challenges which support a fair, just and sustainable adaptation 

Contribution claim 3 (CC3): The Adaptation Scotland Programme has contributed to developing place-based initiatives that drive effective adaptation action across cities, regions and localities, aligned with the interests and needs of communities.

Adaptation Scotland has embedded the Place Principle into the foundation of the Programme. The Programme has helped to establish, expand and support regional adaptation partnerships to enable decisions on place-based approaches. This has led to the design and delivery of initiatives that drive effective adaptation action based on the interests and needs of communities.

Contribution claim 4 (CC4): The Adaptation Scotland Programme has contributed to an increase in knowledge, engagement and capacity in shaping and delivering adaptation policy and action in Scotland.

Adaptation Scotland regularly shares the work and findings of the Programme, and insights from wider relevant adaptation work, to its stakeholders. More stakeholders are engaging with the Programme to shape and deliver adaptation policy and action in Scotland. Stakeholders have a greater capacity and feel more empowered to shape and deliver policy and action in Scotland.

Contribution claim 5 (CC5): The Adaptation Scotland Programme has contributed to bettering the value of and acting upon the views of the most vulnerable to climate change at both national and local level.

The Programme has worked with community and third sector organisations to engage groups most vulnerable to climate change and connect them to the networks. Groups most vulnerable to climate change are empowered to share their views and input to effective adaptation resources. Their views are valued and acted upon in adaptation actions at the local and national level.

Introduction and consent

• Introduce self and Ipsos (independent research organisation).
• Thank participant for taking part (and for their involvement in the evaluation so far if applicable).
• Check that they received and if they have read the information sheet (if they haven’t, explain that you will go through the key points now and can share another copy with them if needed).
• Remind them of the purpose of this interview: As a reminder, the purpose of this interview is for us to understand your views on how the Adaptation Scotland Programme has been working in order to inform both a process and impact evaluation. This means I’ll be asking about your experiences of engaging with the Programme, to understand what you found worked well and what worked less well. I will also be asking about whether engaging with the Programme has led to any changes within your organisation, and why.
• Explain anonymity and confidentiality: As a reminder - your name will not be disclosed in the reports. However, quotes and elements of reporting may be linked to individual areas. If there is anything that you do not want us to quote you on as part of our reporting, do let us know. I will check in on this at the end of the discussion today.
• The only time that we would disclose your identity is if you told us something that indicated that you or someone else was at risk of harm, in which case we would pass your contact details over to our safeguarding lead.
• Remind participant that there are no right or wrong answers. The purpose of the evaluation is learning, to inform ongoing design and delivery and to understand impacts of the Adaptation Scotland Programme as a whole, and not to evaluate the success of particular organisations or projects.
• Emphasise that taking part is completely voluntary – they don’t have to answer any questions they don’t want to answer and can decide to stop the interview at any point.
• Remind that they are free to change their mind and decide not to take part at any time before or during the interview, or after the interview until the findings have been written up.
• This interview will last approximately 60 minutes.
• Check if participant has any questions (offer to go through information sheet if necessary).
• Request permission to record interview. Explain that this is for transcription and analysis purposes and that recordings will not be shared outside the research team at Ipsos.
• If participant does not consent, please take detailed notes.
• If participant provides consent, please start recording.

Their role and involvement in the Adaptation Scotland Programme

I’d now like to ask you a few questions about your organisation, your role, and your involvement in the Adaptation Scotland Programme.

• Could you start by briefly describing your role and your organisation?
• Job title and main responsibilities
• How long you’ve been in this role

What issues does your organisation face in terms of climate change/adaptation? Probe: National v. regional issues.

If needed: Adaptation is a process of on-going adjustments in response to observed and projected climate change impacts. This includes being prepared for increasing risks posed by climate change hazards, and identifying new opportunities our changing climate may bring, while considering how impacts may be felt differently across society.

What is your organisation’s involvement with the Adaptation Scotland Programme? When did this start and what did/does this look like?

• Probe: Type and extent of support (e.g. light touch or intensive, frequency, still ongoing or not, use of tools/resources, attendance at events, partnerships).
• Based on information gathered here, facilitator to establish the most prominent form of support and lead with that section (3-6). If other sections also apply, focus on gathering key info and relative impact (i.e. how it compares to other support received) using highlighted Qs for guidance and be mindful of previous answers given.

Has your organisation engaged with other support for climate adaptation, outwith the Adaptation Scotland Programme?

• What did this look like?

Had your organisation taken steps to adapt to climate change before engaging with the Programme? Probe for brief details.

Tools and resources

Moving on, I’d now like to ask you some questions about the resources that the Adaptation Scotland Programme produces. As part of your organisation’s engagement with the Adaptation Scotland Programme, have you, or others in your organisation, seen tools or resources developed by the Programme?

[If have seen Adaptation Scotland resources] probe on:

• How did you come across these tools/resources? (CC2) Probe: Adaptation Scotland website, social media, shared directly by individuals from the Programme, newsletter, shared by others.
• Did you seek out these resources, or were you made aware of them by the Adaptation Scotland Programme advertising/marketing?
• How often do you hear from the Programme about available resources? Is that too much, too little or about right?
• What kind of resources were they?
• If needed: E.g. Adaptation Capability Framework, Public Sector Handbook and Starter Pack, Climate Change projections for Scotland, guidance around Adaptation finance.

Did you use any of these resources?

[If yes] Which ones? What did you use these resources for?

• How did you find the experience of using these resources? (CC2)
• What were the positives?
• Was there anything that could be improved?
• To what extent would you describe the tools or resources as ‘effective’? (CC2)
• Was there a resource that you found particularly effective? What made it effective?

[If no] What are main the reasons why you have not utilised these resources? (CC2, CC4)

• Probe: Not relevant to your organisation, no capacity, something else?
• What kinds of resources would you find useful?

Were you involved in developing these resources in collaboration with the Programme? (CC2)

• What did this look like/what did you do?
• [If applicable] Were these resources developed in collaboration with regional adaptation partnerships / the local community? (CC3)
• Did you see [your contribution / the contribution of the local community] reflected in the resources developed?
• How satisfied or dissatisfied were you with this collaboration overall?

To what extent, if at all, did these resources increase …

• your knowledge and understanding of what adaptation actions are needed [in the local area]? (CC4, CC3)
• your confidence to take adaptation action?
• staff team skills and capacity to take adaptation action?
• To what extent, if at all, did these resources strengthen the case for action (e.g. in organisational business cases)?

Would you have gained this knowledge elsewhere without these resources? Where?

What specifically did the Adaptation Scotland resources provide that you couldn’t have received elsewhere?

To what extent have these tools/resources helped your organisation take effective adaptation action [in the local area]? (CC1, CC2, CC3)

• Probe: What does ‘effective adaptation’ look like for your organisation?
• Can you give a specific example of an adaptation action informed by these resources?
• What made it effective?
• How do you know whether your adaptation actions are effective? How is this measured?

To what extent, if at all, have these tools/resources helped your organisation consider fairness, justice or equity in your adaptation approaches? (by this we mean approaches that consider the needs of different groups and address existing inequalities) (CC2, CC5)

• Probe: Have the tools/resources helped you consider the needs of groups or communities who are most vulnerable to climate change? In what ways? [Researcher: if not clear, probe on who individual/organisation understands to be most vulnerable].
• As a result, have you changed any approaches to make them more inclusive or equitable? How effective have these new approaches been?
• Are there any vulnerable groups/communities whose needs resources haven’t addressed? What would be useful for the programme to do for you to be able to support these vulnerable groups/communities?

Have these tools/resources helped you collaborate with other organisations or sectors on adaptation? (CC2)

• Probe: Have you shared these tools/resources with partners?
• Can you give an example of cross-sector collaboration that was supported by Adaptation Scotland resources?

[If have not seen Adaptation Scotland resources] probe on: Were you aware that that Adaptation Scotland Programme develops resources for public sector organisations? (CC2, CC4)

• Would resources be something you would be interested in using? Why/why not? What would make them useful to your organisation?
• How would you want to hear about resources developed by the Programme?

Networks

And moving on again, I’m going to ask some questions specifically about any networks that your organisation is a member of. Is your organisation a member of the Public Sector Adaptation Network (PSCAN)? (CC1)

[If they are a member] probe on:

• How did you first hear about the network? (CC1, CC4)
• What do you understand as the purpose of the network?
• Who from your organisation attends the network?
• [If not interviewee] What is their role?

What does being a member of PSCAN involve for you/your organisation?

• Has your organisation been asked to present to the network?
• What did you present on?

How often do you hear from the network?

• Would you say the level of contact with the network is too much, too little or about right?

How do you find the experience of being a member of the network?

• What are the positives?
• Is there anything that could be improved?

From your perspective, is PSCAN a growing network? (CC1)

• Probe: Have you noticed more organisations joining? What would help the network grow?

Is there anything you/your organisation is doing as a result of being a member? Probe for specific examples.

• Do you feel more empowered to drive adaptation action [in the local area] as a result of being part of PSCAN? (CC1, CC3, CC4)
• Has being a member of PSCAN helped your organisation connect with other public sector organisations working on adaptation? (CC1)
• Probe: direct contact or collaboration with other PSCAN members?

To what extent, if at all, did these networks increase your knowledge and understanding of what adaptation actions are needed? (CC1)

• And how to act on this knowledge and understanding? What does this action look like?
• Can you describe any ways in which this action has helped to address existing inequalities? (CC2)
• And to what extent does it ensure climate adaptation over the longer term? (CC2)
• [If not] What would you say were the main barriers to action within your organisation?

And, aside from PSCAN, are you part of any other networks focused on climate adaptation or sustainability? (CC1)

• [If yes] How would you compare PSCAN to these other networks?
• Without PSCAN, do you think your organisation would have developed its adaptation leadership and capabilities anyway?

[If not a member] probe on: Were you aware that that the Programme hosts a network with public bodies to work together to benchmark their progress and identify and share learning on climate change adaptation? (CC1, CC4)

• [If not aware] Would a network like this be something you would be interested in attending? Why/why not? What would make it useful to your organisation?
• [If aware] What are main the reasons why you have not joined the network?
• Is there anything that would make you want to join?

Events

I’m now going to ask some questions on any [other] events that your organisation has attended as part of engaging with the Programme. As part of your organisation’s engagement with the Adaptation Scotland Programme, have you, or others in your organisation, attended any [other] events organised by the Programme?] (CC1)

[If yes] probe on: Who from your organisation attended?

• [If not interviewee] What is their role? Is this person a ‘senior decision maker’ within your organisation? (CC1)

How many different events have you/your organisation attended?

What was the purpose of the event(s)?

• What did attending involve?
• Probe: were any of these events focussed on how to support implementation of Scotland’s’ Climate Change Adaptation Plan? (CC5)

How did you hear about the event(s)? (CC1, CC4)

• Probe on whether this was before or after engaging with resources from the Programme?

Did attending the event(s) help you to connect with other organisations working on adaptation? (CC1)

• Did the event(s) provide opportunities to work together or share experiences?
• Have you collaborated with any of these other organisations since the event(s)?
• Has attending the event(s) helped anyone in your organisation to take on more of a leadership role in adaptation? What does this look like?

How did you find the experience of attending the event(s)?

• What worked well?
• Was there anything that could be improved?
• Was there anything missing from the events that you would like to have seen?
• What was your main takeaway from the event(s)?

Is there anything that your organisation is doing differently as a result of attending the event(s)? (CC1) [If they say their organisation is doing anything differently]:

• Can you give a specific example of an adaptation action your organisation has taken as a result of attending?
• [If applicable] Is this change solely within your organisation, or does this action drive adaptation action in the local area? (CC3)
• Would you describe this action as ‘effective’? Why/why not? How do you know whether it’s working?

To what extent have attending the event(s) helped your organisation consider fairness, justice or equity in your adaptation approaches? (by this we mean approaches that consider the needs of different groups and address existing inequalities) (CC2, CC5)

• Probe: Have the events helped you consider the needs of vulnerable groups or communities? In what ways?
• Have they influenced how you engage with or involve different communities in adaptation planning?
• Have you changed any approaches as a result to make them more inclusive or equitable? How effective have these approaches been?
• Are there any vulnerable groups/communities whose needs events haven’t addressed? What would be useful for the programme to do for you to be able to support these vulnerable groups/communities?

To what extent, if at all, did the event(s) increase your knowledge and understanding of what adaptation actions are needed? (CC1, CC2)

• Has attending these events increased your organisation’s capacity to plan and deliver adaptation action [in the local area]? (CC3)
• Probe: In terms of skills, confidence, understanding of how to access further support?
• [If not] What would you say were the main barriers to action within your organisation?

Have the event(s) led to leaders within your organisation taking on more responsibility to share knowledge and drive adaptation transformation [within your organisation/within the local area]? (CC1, CC3)

• What does adaptation leadership look like in your organisation? Who do you consider the leaders to be?

To what extent, if at all, did these events meet your expectations?

• Was there anything that you thought was missing from the events? What was missing?

Have you attended any other events on climate adaptation out with the Programme?

• Did you find these other events useful? What was it about these events that was useful to you?

[If haven’t attended any Adaptation Scotland events] probe: How does your organisation currently build knowledge and capacity on adaptation? (CC4)

• What sources do you use? Have you attended any other events outwith the Programme? (CC1, CC2)
• Do you feel like you have sufficient support and knowledge?
• What’s missing that Adaptation Scotland events could potentially provide?

Were you aware that that Adaptation Scotland Programme hosts events for public sector organisations? (CC1, CC2, CC4)

• [If not aware] Would events be something you would be interested in attending? Why/why not? What would make them useful to your organisation?
• How would you want to hear about events like these?
• [If aware] What are main the reasons why you have not attended events?
• What kind of event would you find useful? Is there anything that would make you want to attend?

Place-based partnerships

And lastly, I’d like to ask a few questions about regional adaptation partnerships.As part of your engagement with the Programme, have you been involved in any regional adaptation partnerships? (sometimes referred to as Climate Ready regions)? E.g. Highland Adapts, Climate Ready Clyde (facilitator to check/mention most regionally relevant RAP) (CC3)

[If yes] What did this involve? (CC3)

• Have you developed any resources with regional partners?

To what extent would you say that the partnership is collaborative?

• What does this look like?
• To what extent, and in what ways, did the Adaptation Scotland Programme help with this?

How would you describe the impact of your organisation’s involvement?

• Have you seen any examples of initiatives that drive effective adaptation action as a result of the partnership?
• To what extent were these initiatives based on the interests and needs of communities in this region? (CC5)
• As a result of your organisation’s involvement, have you seen the partnership support place-based adaptation action? Probe for examples

How did you find the experience of engaging with the regional adaptation partnership?

• What worked well about the partnership?
• Was there anything that could be improved?
• What was your main takeaway from your involvement?

To what extent has your involvement in the Regional Adaptation Partnership… (Probe for examples):

• Increased your knowledge and understanding of what adaptation actions are needed? (CC1)
• Increased your confidence to take adaptation action? (CC3)
• Enhanced team skills and capacity to take adaptation action? (CC3)
• Enabled your organisation to take effective adaptation action? (CC1)
• Enabled more collaboration between your organisation and others on adaptation? (CC4)
• Helped your organisation/hub consider fairness, justice or equity in your adaptation approaches? (CC2, CC5)

Would you have gained any of this without being involved in the RAP?

What specifically has the ASP provided that you couldn’t have got elsewhere?

[If no] Regional Adaptation Partnerships (sometimes referred to as Climate Ready Regions) are partnerships that bring together local authorities, public bodies, businesses, communities and the third sector from across a region to collaborate on shared climate risks and build climate resilient places, communities and economies.

• Is this something that would be of interest to your organisation? Why/why not?
• How easy or difficult would it be for your organisation to be part of a regional adaptation partnership? Probe on not relevant, no capacity, something else?

Overall outcomes

So, to start summing up, overall, how satisfied are you with the support that your organisation received from the Adaptation Scotland Programme? (Process)

• What is working well and less well in the delivery of the Programme and where could improvements be made?
• Have you requested any support which the Programme has not been able to provide?
• How have you found the responsiveness of the Programme to requests for support, e.g. response times?
• How effectively would you say that the Programme communicates information on climate risk and adaptation and best practice adaptation action?

Since engaging with the Programme, to what extent do you feel more knowledgeable about how to implement adaptation action within your organisation? (CC1)

To what extent would you say the Programme is helping to embed knowledge exchange, innovation and research [within your organisation / the local area]? (CC3, CC4)

To what extent, if at all, do you feel the actions your organisation has taken is supporting a fair, just and sustainable adaptation? (by this we mean adaptation that considers the impacts on different groups, addresses existing inequalities, and seeks solutions that are for the long term) (CC2)

To what extent, if at all, has your engagement with the Programme led to you/leaders within your organisation taking on more responsibility and ownership to share knowledge and drive adaptation transformation? (CC1) [If has] What does this look like in practice?

• How did this come about?
• Which aspects of the Programme were most important in this regard? Probe on aspects discussed, e.g.: Events, Resources, Networks
• Were there any other contributing factors that led you/leaders within your organisation to take on this responsibility and ownership? (CC1)
• Have you shared examples of effective adaptation/AS Programme resources with others? Who? (CC1)

[If has not] What barriers are there to you/leaders within your organisation taking on this responsibility and ownership to share knowledge and drive adaptation transformation? (CC1)

Wrap up and close

That’s everything I wanted to ask you today. Is there anything else you’d like to add before we finish?

Thanks very much for your time. Can I please check if there is anything you would prefer us not to quote or refer to directly in our reports for the Scottish Government?