Regional Land Use Partnerships: phase 1 process evaluation - final report

The Regional Land Use Partnerships (RLUPs) evaluation report considers the lessons learned from the Year 1 (2021 to 2022) pilot process. The report was completed by SAC Consulting and outlines key early findings from RLUP pilot establishment.

3. Overview of evaluation methodology

a. Logic model and Theory of Change model

To understand the full context of the RLUP development process, we structured our evaluation method around a logic model (LM). The logic model (see Figure 3a) is a schematic which outlines how RLUPs as an intervention collate a series of inputs (resources) to enable outputs (activities), which then forward the delivery of a specific set of outcomes. We have refined an overarching LM for the RLUP pilot work with a primary focus on Phase 1, which has been used to structure our evaluation.

While policy interventions are designed to lead to certain outcomes, realising these on the ground can be uncertain due to influences from a range of factors. A Theory of Change model (ToC) is a way of predicting outcomes of policy actions by capturing baseline conditions and identifying the factors which will influence results.[1] In this evaluation, we revised the LM into a ToC diagram (see Appendix B: Detail on evaluation methodology) to provide a concise and interlinked ‘theory of change’ pathway from the inception of RLUPs towards long-term outcomes. These two linked schematics are cross-referenced within this report and provide a useful, visual guide to the written outputs.

b. Data collection

i. Desk-based research

Initial data collection involved reviewing documentation pertaining to RLUP development to date. This included:

  • Relevant Scottish Government policies
  • The body of work surrounding the previous 2013-2015 Land Use Strategy pilots
  • Advice from the Scottish Land Commission
  • Academic literature
  • The original project specifications for these pilots
  • Meeting agendas and minutes
  • Other reports and updates.

This information review enabled us to establish an understanding of local circumstances (baselines) before meeting with representatives from the pilot regions. This also ensured the evaluation captured the separate starting points of each region. This initial review enabled efficient use of interview time such that these sessions could focus on exploring the qualitative aspects and lessons learned, following confirmation of the initial baseline data for each region.

Figure 3a: Logic model for RLUPs. This schematic provides a conceptual map of the RLUP process for Phase 1 and beyond.


Legislative/Policy Drivers

  • Climate Change Act 2019; Climate Change Plan; Agricultural Policy; Just Transition / PfG commitments
  • NPF4 and Planning (Scotland) Act 2019 Regional Spatial Strategies
  • Land Use Strategy; Guidance on Engaging Communities in Decisions Relating to Land; National Standards for Community Engagement
  • RLUP Policy team and cross policy teams in Scottish Government

Enabling Factors

Existing organisational capacity
  • Local Authority/National Park Authority input and capacity lead partners
  • Existing landscape partnerships/ frameworks and expertise
Human capital
  • Existing and new staff capacity
  • Regional stakeholder capacity/input and expertise
  • Stakeholder knowledge and expertise in i) data; ii) natural capital; iii) participatory land use planning

Existing evidence base/research on regional natural capital assets

Office space, technology/equipment

Project funding to facilitate pilots

Outputs (Activities)

RLUP Function

  • RLUP pilots established/Project spec agreed
  • RLUPS Policy team support for pilots; encourage collaboration; administer funding; manage progress milestones and risk


  • Existing regional governance/partnership structures identified; how pilots will work alongside these
  • RLUP decision-making board established; Terms of Reference drafted

Stakeholder Engagement

  • Stakeholder mapping - stakeholders identified, roles, influence assessed; relations developed
  • Pilot stakeholder engagement strategy developed by RLUP pilots
  • RLUPs coordination network established by Policy team

Data, knowledge transfer and the natural capital approach

  • Relevant expertise identified/work undertaken to assess data needs
  • Pilots identify need for natural capital tools/potential for natural capital approach
  • Training/knowledge exchange undertaken for RLUPs staff and stakeholders
  • Knowledge exchange with relevant bodies including other pilots/national stakeholders
  • Collective progress of RLUPs evaluated - establish future evaluation framework

Short term (To March 2022 and ongoing)

  • Accountable and legitimate RLUP governance structure incorporating diverse perspectives and a locally democratic approach established
  • ongoing improvement of understanding of governance structures and strengthening of stakeholder relationships
  • Ongoing engagement is progressing understanding of RLUPs, and key stakeholder roles. Stakeholders and communities increasingly experiencing collaborative approaches
  • Knowledge sharing ongoing, facilitating increased awareness and understanding of natural capital and role of RLUPs; building ambition on delivering on climate change goals through land use
  • Acquisition of data/knowledge is underway to support piloting a natural capital approach and developing tools for demonstrating value for development of RLUF
  • Understanding of processes for evaluation of RLUPs improving (RLUPs Logic Model)

Phase 2 (to March 2024)

  • RLUF developed identifying land use opportunities to deliver on policy
  • RLUPs a focal point for collaboration; supporting delivery through stakeholders
  • Diverse stakeholders engaged with RLUPs; increased wider understanding of land use benefits and trade-offs
  • Stakeholder network established across stakeholders and pilot teams; maintained through engagement
  • Improved RLUP staff and stakeholder knowledge and skills inrelevant areas
  • Data tools established for natural capital assessment to support land use change decisions.
  • RLUP pilots have confidence/skills in applying a natural capital approach; findings informing decision making
  • Long term framework established for evaluating RLUPS progress against outcomes

Impacts (Strategic Outcomes)
Long term

  • Land use opportunities implemented to address climate/biodiversity priorities
  • Social learning - regional stakeholders invested in design and delivery of land use benefits - affecting decision making
  • RLUPS established as a model for strategic, collaborative land use decision making
  • Increased local economic and skills development opportunities
  • Enhanced RLUP data systems; for monitoring/evaluating outcomes
  • Natural capital approach more widely adopted for valuing and enhancing land to support ecosystem services delivery
  • Enhanced participatory involvement in land use decisions
  • Shared learning from long term evaluation of outcomes and cross comparison of RLUPs

ii. Interviews with pilots

Interview questions were developed by the evaluation team and refined in consultation with the SG policy team. For more detail on the questions and wider methodology, see Appendix B: Detail on evaluation methodology and Appendix D: Full list of interview questions. The pilots were contacted via emails provided by the SG RLUPs policy team. Interviews were scheduled with pilot representatives and all participants were provided with a project information sheet and each signed a data consent form. Interviews ran between 12 Jan – 2 Feb 2022 and lasted between 50 and 94 minutes. There were five interviews in total (one session with each pilot region); two interviews were one-on-one and three were in groups, with 2-4 pilot representatives attending. Eleven pilot representatives were interviewed in total. All interviews were conducted via MS Teams by the Principal Investigator from the SAC Consulting evaluation team.

c. Data analysis

Transcripts of the interviews and the background material were collated and analysed using a thematic analysis process. This involved reviewing the transcripts to develop a set of overarching codes to identify key strengths, weaknesses, opportunities and threats (SWOT) relating to the four main themes of the evaluation (governance, stakeholder engagement, natural capital, and data) developed from Ministerial expectations, outlined in the Bute House Agreement. In addition, results were considered under a ‘pilot process’ theme to capture the practical lessons around taking a pilot approach to this new RLUP policy mechanism. The thematic analysis identified emerging commonalities between pilots regarding strengths and weaknesses, and highlighted individual innovations and lessons learned. These are reported in Results and discussion, below. For more on the development of the methods used in this evaluation, see Appendix B: Detail on evaluation methodology.

d. Inception and Closing Workshops

Our evaluation was bookended by workshops led by the evaluation team and attended by the SG RLUP policy team and the entire cohort of pilots. The inception workshop of 8th December 2021 included a presentation by the evaluation team on the process, the inputs required from pilots, and the desired outcomes. It also included a discussion around the enabling factors and existing activities taking place within each pilot. Since this evaluation will signpost future opportunities and threats for RLUPs, a key value added was the creation of space to discuss participants’ reflections on the process. Pilots came forward in round-table fashion with updates regarding ongoing activities. This was an opportunity to introduce the themes of the evaluation and get pilots talking and thinking about the ideas the interviews would interrogate.

Our closing workshop on 14 March 2022 provided an opportunity for pilot regions to revisit the priority outcomes for the evaluation. The evaluation team presented the headline findings from the data collection and thematic analysis process. The following plenary discussion with the pilots ensured validation of the results. These workshops were useful in that they made the evaluation process transparent to pilots and allowed them to raise any concerns, which led to increased trust and buy-in to the process.



Back to top