Evaluation of the Adaptation Scotland Programme

D.3: Text from third sector discussion guide

Adaptation Scotland evaluation:

Aims for this intervew:

• For community and third sector organisations who have received support from the Programme: The aim of this interview is to gather information on whether the organisation has made progress towards the intended outcomes of the Programme, as set out in the contribution claims (see below).
• We also want to understand how the Adaptation Scotland Programme has been delivered in practice, to gather insight into the activities and outputs of the Programme. Aspects of this guide will therefore focus on the interviewee’s experiences of engaging with the Programme.

Timings for sections are indicative and will be adjusted by the researcher depending on what forms of support the organisation has received.

Contribution claims

Contribution claim 1 (CC1): The Adaptation Scotland Programme has contributed to the development of a growing network of leaders across Scotland who are driving effective adaptation action.

The Programme engages more businesses, communities and public bodies to form and grow networks in place-based approaches to climate adaptation. These growing networks support leaders to increase their knowledge and understanding of what adaptation actions are needed and how to act. This enables leaders to take more ownership and responsibility to drive adaptation action and share knowledge on adaptation.

Contribution claim 2 (CC2): The Adaptation Scotland Programme has contributed to the development of effective approaches to support fair, just and sustainable adaptation.

The Programme engages with specialists (i.e. practitioners, researchers, institutions and policy teams) to increase understanding of what effective adaptation looks like, connecting specialists and networks to encourage cross-sector collaboration and (co-) develop tools and resources. These tools and resources are used by public sector bodies, communities and businesses to develop effective approaches for tackling adaptation challenges which support a fair, just and sustainable adaptation.

Contribution claim 3 (CC3): The Adaptation Scotland Programme has contributed to developing place-based initiatives that drive effective adaptation action across cities, regions and localities, aligned with the interests and needs of communities.

Adaptation Scotland has embedded the Place Principle into the foundation of the Programme. The Programme has helped to establish, expand and support regional adaptation partnerships to enable decisions on place-based approaches. This has led to the design and delivery of initiatives that drive effective adaptation action based on the interests and needs of communities.

Contribution claim 4 (CC4): The Adaptation Scotland Programme has contributed to an increase in knowledge, engagement and capacity in shaping and delivering adaptation policy and action in Scotland.

Adaptation Scotland regularly shares the work and findings of the Programme, and insights from wider relevant adaptation work, to its stakeholders. More stakeholders are engaging with the Programme to shape and deliver adaptation policy and action in Scotland. Stakeholders have a greater capacity and feel more empowered to shape and deliver policy and action in Scotland.

Contribution claim 5 (CC5): The Adaptation Scotland Programme has contributed to bettering the value of and acting upon the views of the most vulnerable to climate change at both national and local level.

The Programme has worked with community and third sector organisations to engage groups most vulnerable to climate change and connect them to the networks. Groups most vulnerable to climate change are empowered to share their views and input to effective adaptation resources. Their views are valued and acted upon in adaptation actions at the local and national level.

Introduction and consent

• Introduce self and Ipsos (independent research organisation).
• Thank participant for taking part (and for their involvement in the evaluation so far if applicable).
• Check that they received and if they have read the information sheet (if they haven’t, explain that you will go through the key points now and can share another copy with them if needed).
• Remind them of the purpose of this interview: As a reminder, the purpose of this interview is for us to understand your views on how the Adaptation Scotland Programme has been working in order to inform both a process and impact evaluation
• This means I’ll be asking about your experiences of engaging with the Programme, to understand what you found worked well and what worked less well. I will also be asking about whether engaging with the Programme has led to any changes within your organisation, and why.
• Explain anonymity and confidentiality: As a reminder - your name will not be disclosed in the reports. However, quotes and elements of reporting may be linked to individual areas. If there is anything that you do not want us to quote you on as part of our reporting, do let us know. I will check in on this at the end of the discussion today.
• The only time that we would disclose your identity is if you told us something that indicated that you or someone else was at risk of harm, in which case we would pass your contact details over to our safeguarding lead.
• Remind participant that there are no right or wrong answers. The purpose of the evaluation is learning, to inform ongoing design and delivery and to understand impacts of the Adaptation Scotland Programme as a whole, and not to evaluate the success of particular organisations or projects.
• Emphasise that taking part is completely voluntary – they don’t have to answer any questions they don’t want to answer and can decide to stop the interview at any point.
• Remind that they are free to change their mind and decide not to take part at any time before or during the interview, or after the interview until the findings have been written up.
• This interview will last approximately 60 minutes.
• Check if participant has any questions (offer to go through information sheet if necessary).

Request permission to record interview. Explain that this is for transcription and analysis purposes and that recordings will not be shared outside the research team at Ipsos.

• If participant does not consent, please take detailed notes.
• If participant provides consent, please start recording.
• Start recording if consent given

Their role and involvement in the Adaptation Scotland Programme

I’d now like to ask you a few questions about your organisation, your role, and your involvement with the Adaptation Scotland Programme.

Could you start by briefly describing your organisation/community hub? What are its aims and objectives?

• What are your main responsibilities?
• How long have you been in this role/involved with the organisation/hub?
• How big is your organisation/hub? Where is it primarily located?

What issues does your organisation/hub face in terms of climate change/adaptation? Probe: on national vs. regional issues.

If needed: Adaptation is a process of on-going adjustments in response to observed and projected climate change impacts. This includes being prepared for increasing risks posed by climate change hazards, and identifying new opportunities our changing climate may bring, while considering how impacts may be felt differently across society.

What is your organisation/hub’s involvement with the Adaptation Scotland Programme? When did this start and what did/does this look like?

• Probe: Type and extent of support (e.g. light touch or intensive, frequency, ongoing or not, use of tools/resources, attendance at events, partnerships).
• Based on information gathered here, facilitator to establish the most prominent form of support and lead with that section (3-5). If other sections also apply, focus on gathering key info and relative impact (i.e. how it compares to other support received) using highlighted Qs for guidance and be mindful of previous answers given.

Has your organisation/hub engaged with other support for climate adaptation, outwith the Adaptation Scotland Programme?

• What did this look like?

Had your organisation/hub taken steps to adapt to climate change before engaging with the Programme? Probe for details.

Tools and resources

Moving on, I’d now like to ask you some questions about the resources that the Adaptation Scotland Programme produces.

As part of your engagement with the Adaptation Scotland Programme, have you, or others in your organisation/hub, seen tools or resources developed by the Programme?(If needed, e.g. Stakeholder and Power Mapping tool, briefing document on Climate Resilient, Healthy and Equitable Places, Insight report on Land Use and Climate Adaptation in Scotland, Local Authority Climate Contact Directory).

• What kind of resources were they?

[If have seen Adaptation Scotland resources] probe on:

How did you come across these tools/resources? (CC2, CC5)

• Probe: Adaptation Scotland website, social media, shared directly by individuals from the Programme, newsletter, shared by others.
• Did you seek out these resources, or were you made aware of them by the Adaptation Scotland Programme advertising/marketing?
• How often do you hear from the Programme about available resources? Is that too much, too little or about right?
• Were these resources specific to your sector? Probe on: examples of resources.

Did you use any of these resources? [If yes] Which ones? What did you use these resources for?

• How did you find the experience of using these resources? (CC2)
• What were the positives?
• Was there anything that could be improved?
• To what extent would you describe the tools or resources as ‘effective’? (CC2)
• Was there a resource that you found particularly effective? What made it effective?

[If no] What are main the reasons why you have not utilised these resources? (CC2)

• Probe: Not relevant to organisation/hub, no capacity, something else?
• What kinds of resources would you find useful?

Were you involved in developing these resources in collaboration with the Programme? (CC2, CC5) [If yes] What did this look like/what did you do?

• On how many occasions have you/your organisation/hub been involved in developing resources or tools?
• How satisfied or dissatisfied were you with this collaboration overall?
• What worked well / less well?

[If no] Is this something you think would be useful for your organisation/hub?

• Were there any particular reasons why you weren’t involved in co-developing these resources? Probe: Not invited to, no capacity, did not think it worthwhile, something else?

To what extent, if at all, did these resources increase …

• your knowledge and understanding of what adaptation actions are needed?
• your confidence to take adaptation action?
• team skills and capacity to take adaptation action?
• Would you have gained this knowledge elsewhere without these resources? Where?
• What specifically did the Adaptation Scotland resources provide that you couldn’t have received elsewhere?

To what extent have these tools/resources helped your organisation/hub to take effective adaptation action? (CC1, CC2)

• Probe: What does ‘effective adaptation’ look like for your organisation/hub?
• Can you give a specific example of an adaptation action informed by these resources? What made it effective?
• How do you know whether your adaptation actions are effective? How is this measured?

To what extent, if at all, have these tools/resources helped your organisation/hub consider fairness, justice or equity in your adaptation approaches? (by this we mean approaches that consider the needs of different groups and address existing inequalities)

• [If did help] In what ways? Probe for examples
• What was it about the tools or resources that helped you to considered these approaches?
• Did anything stand out as particularly useful? (CC2)
• [If did not help] What would have made them more useful?

Have the tools/resources helped you consider the needs of groups or communities most vulnerable to climate change? [Researcher: if not clear, probe on who individual/organisation understands to be most vulnerable].

[If did help] Which groups/communities?

• How did the resources help? What was most useful?
• As a result, have you changed any approaches to make them more inclusive or equitable? How effective have these approaches been?
• To what extent is this as a result of engagement with the programme?

[If did not help] Is this something you are working towards?

• What would have made them more useful?
• Were there any barriers to your organisation changing your adaptation approach to consider the needs of these groups?

[All] Are there any vulnerable groups/communities that the tools/resources aren’t helping you to support?

• Which groups/communities?
• What would be useful for the programme to do for you to be able to support these vulnerable groups/communities?

Have these tools/resources helped you collaborate with other organisations on adaptation? What kinds of organisation – businesses/third sector/public sector? (CC2)

• Probe: Have you shared the tools/resources you have used with others? Who?

[If not] What are the reasons why the tools/resources have not helped you to collaborate?

• What would be useful for the programme to do in order to help improve collaborations between organisations?

[If have not seen Adaptation Scotland resources] probe on: Were you aware that that Adaptation Scotland Programme develops resources for communities and the third sector? (CC2)

• Would resources be something you would be interested in using? Why/why not? What would make them useful to you?
• How would you want to hear about resources developed by the Programme?

Events

I’m now going to ask some questions on any events that your organisation has attended as part of engaging with the Programme.

As part of your engagement with the Adaptation Scotland Programme, have you, or others from your organisation/hub, attended any events organised by the Programme? (CC1)

[If yes] probe on:

• Who from your organisation/hub attended?
• How many different events have you/others from your organisation/hub attended?
• What was the purpose of the event(s)?
• Would you describe these events as community training or learning events?
• What did attending involve?
• Probe: were any of these events focussed on how to support implementation of Scotland’s’ Climate Change Adaptation Plan? (CC5)
• How did you hear about the event(s)? (CC1, CC4)
• Probe on whether this was before or after engaging with resources from the Programme?
• What other organisations were in attendance? (CC1)
• Were you already aware of these other organisations, before attending?

Did attending the event(s) help you to connect with other organisations who are taking adaptation action? Probe on types of organisation – private sector/public sector/third sector (CC1)

• Did the event(s) provide opportunities to work together or share experiences?
• Have you collaborated with any of these other organisations since the event(s)?
• Has attending the event(s) helped anyone in your organisation/hub to take on more of a leadership role in adaptation? What does this look like?

How did you find the experience of attending the event(s)? (Process)

• What worked well?
• Was there anything that could be improved?
• Was there anything missing from the events that you would like to have seen?
• What was your main takeaway from the event(s)?

Is there anything that your organisation/hub is doing differently as a result of attending the event(s)? (CC1)

• [If they say they are doing anything differently]:
• Can you give a specific example of an adaptation action your organisation/hub has taken as a result of attending?
• Would you describe this action as ‘effective’? Why/why not? How do you know whether it’s working?

To what extent have attending the event(s) helped your organisation/hub to consider fairness, justice or equity in your adaptation approaches? (by this we mean approaches that consider the needs of different groups and address existing inequalities) (CC2)

• Probe: Have the events helped you consider the needs of vulnerable hubs or communities?
• As a result. Have you changed any approaches to make them more inclusive or equitable? How effective have these approaches been?

To what extent, if at all, did the event(s) increase your knowledge and understanding of what adaptation actions are needed? (CC1, CC2)

• Probe: In terms of skills, confidence, understanding of how to access further support?
• [If not] What would have made the events more useful to you and your organisation/hub?

Have the event(s) led to people within your organisation/hub taking on more responsibility to share knowledge and drive adaptation transformation? (CC1)

• What does adaptation leadership look like in your organisation? Who do you consider the leaders to be?

[If has not] What would you say are the reasons why the events did not help people within your organisation/hub to take on this responsibility?

• What could be changed about the events to help people within your organisation/hub to take on this responsibility?

To what extent, if at all, did these events meet your expectations?

• Was there anything that you thought was missing from the events? What was missing?

Have you attended any other events on climate adaptation outwith the Programme?

• Did you find these other events useful? What was it about these events that was useful to you?

[If haven’t attended any adaptation scotland events] probe: How does your organisation/hub currently build knowledge and capacity on adaptation? (CC4)

• What sources do you use? Have you attended any other events outwith the Programme? (CC1, CC2)
• Do you feel like you have sufficient support and knowledge?
• What’s missing that Adaptation Scotland events could potentially provide?

Were you aware that that Adaptation Scotland Programme hosts events for community hubs and third sector organisations? (CC1, CC2, CC4)

[If not aware] Would events be something you would be interested in attending? Why/why not? What would make them useful to your organisation/hub?

• How would you want to hear about events like these?

[If aware] What are main the reasons why you have not attended events?

• What kind of event would you find useful? Is there anything that would make you want to attend?

Place-based partnership

And lastly, I’d like to ask a few questions about regional adaptation partnerships. As part of your engagement with the Programme, have you been involved in any regional adaptation partnerships (sometimes referred to as Climate Ready Regions)? E.g. Highland Adapts, Climate Ready Clyde (facilitator to check/mention most regionally relevant RAP) (CC3)

[If yes] What did this involve? (CC3)

• Have you developed any resources with regional partners?

To what extent would you say that the partnership is collaborative?

• What does this look like?
• To what extent, and in what ways, has the Adaptation Scotland Programme helped with this?

How would you describe the impact of your organisation/hub’s involvement?

• Have you seen any examples of initiatives that drive effective adaptation action as a result of the partnership? [If yes] What was it about your involvement with the partnership that helped to drive this action?
• As a result of your organisation/hub’s involvement, have you seen the partnership support place-based adaptation action? Probe for examples?

How did you find the experience of engaging with the regional adaptation partnership? (Process)

• What worked well about the partnership?
• Was there anything that could be improved?
• What was your main takeaway from your involvement?

To what extent has your involvement in the Regional Adaptation Partnership… (Probe for examples):

• Increased your knowledge and understanding of what adaptation actions are needed? (CC1)
• Increased your confidence to take adaptation action? (CC3)
• Enhanced team skills and capacity to take adaptation action? (CC3)
• Enabled your organisation to take effective adaptation action? (CC1)
• Enabled more collaboration between your organisation and others on adaptation? (CC4)
• Helped your organisation/hub consider fairness, justice or equity in your adaptation approaches? (CC2, CC5)

Would you have gained any of this without being involved in the RAP?

What specifically has the ASP provided that you couldn’t have got elsewhere?

[If no] Regional Adaptation Partnerships (sometimes referred to as Climate Ready Regions) are partnerships that bring together local authorities, public bodies, businesses, communities and the third sector from across a region to collaborate on shared climate risks and build climate resilient places, communities and economies.

• Is this something that would be of interest to your organisation? Why/why not?
• How easy or difficult would it be for your organisation to be part of a regional adaptation partnership? Probe on not relevant, no capacity, something else?

Overall Outcomes

So, to start summing up, overall, how satisfied are you with the support that your organisation/hub received from the Adaptation Scotland Programme? (Process)

• What is working well and less well in the delivery of the Programme and where could improvements be made?
• Have you requested any support which the Programme has not been able to provide?
• How have you found the responsiveness of the Programme to requests for support, e.g. response times?
• How effectively would you say that the Programme communicates information on climate risk and adaptation and best practice adaptation action?

Since engaging with the Programme, to what extent do you feel more knowledgeable about how to implement adaptation action within your organisation/hub, and beyond? (CC1)

As a result, have you, or others within your organisation taken on more responsibility of ownership to share this knowledge and to drive adaptation action? (CC1) [If has] What does this look like in practice?

• How did this come about?
• Which aspects of the Programme were most important in this regard? Probe on aspects discussed.
• Were there any other contributing factors that led you/others in your organisation/hub to take on this responsibility and ownership? (CC1)

[If has not] What barriers are there to you/others in your organisation/hub taking on this responsibility and ownership to share knowledge and drive adaptation transformation? (CC1)

To what extent, if at all, do you feel the actions your organisation has taken is supporting a fair, just and sustainable adaptation? (by this we mean adaptation that considers the impacts on different groups, addresses existing inequalities, and seeks solutions that are for the long term) (CC2)

[If yes] What are some examples of this? e.g. land, finance, climate justice.

• To what extent, if at all, is this action supporting a fair, just and sustainable adaptation? Why/why not? Is this an aim?

[If no] What have been the barriers to developing effective approaches?

• Probe on: financial issues/lack of funding, lack of knowledge/understanding, not sure how to act on the advice received, lack of support (within the organisation, or external to the organisation?),

Wrap up and close

That’s everything I wanted to ask you today. Is there anything else you’d like to add before we finish?

Thanks very much for your time. Can I please check if there is anything you would prefer us not to quote or refer to directly in our reports for the Scottish Government?