Environment, Natural Resources and Agriculture (ENRA) research strategy 2027-2032: consultation

Annex B: ENRA Research Programme Impact Framework

1. Purpose

Annex B sets out a proposed Impact Framework that describes the approach to defining, monitoring and evaluating research impact in the forthcoming ENRA 2027- 2032 Research Programme.

2. Definition of Impact

In the context of the ENRA Research Programme 2027-2032 impact is defined as “the real-world difference research makes — changing lives, shaping policy, strengthening the economy, improving the environment, and enhancing health and wellbeing."

This refers to the positive changes or benefits that extend beyond academic circles. This includes improvements to the economy, society, culture, public policies, services, health, the environment, and overall quality of life.

3. Approach

The proposed Impact Framework will be embedded within the strategic approach to the next ENRA research programme 2027-2032. The research set out in the ENRA Research Strategy will be centered around a series of ‘Missions for Scotland’, each underpinned by a set of supporting Challenges aligned with Scottish Government priorities and key policy drivers (Figure 5):

• Delivering sustainable and regenerative agriculture and food systems
• Delivering climate-positive and resilient landscapes
• Restoring nature and protecting our environment
• Enhancing rural and island communities
• Building the circular economy

Missions

Challenges

Scottish Government Policy Areas

Figure 5: The relationship between the five Missions of; Building the circular economy; Delivering sustainable and regenerative agriculture and food systems; Delivering climate-positive and resilient landscapes; Restoring nature and protecting our environment; Enhancing rural and island communities, and the Challenges: Protecting our food system resilience; Maximising the circular economy and reducing waste; Reforming Scotland’s agricultural system; Promoting healthy and sustainable food in Scotland; Ensuring Scotland’s food safety; Protecting and improving animal health and animal welfare; Promoting crop & livestock improvement; Maintaining high plant health; Protecting and enhancing biodiversity; Optimising Scotland’s land use; Protecting and restoring soils & peatland; Investing in Scotland’s natural capital; Enhancing Scotland’s environment; adapting to climate change; Reforming Scotland’s land system; Supporting rural & island communities & economies, link into Scottish Government policies on: Food and Drink; Circular economy; Agriculture; Plant and animal health and welfare; Biodiversity; Land use and land reform; Climate change; Climate adaptation; Water quality; Environmental protection; Flood risk resilience; and Rural and island communities.

Impact

The five ‘Missions’ can also be defined as the ENRA research programme ‘impact areas’ (see Figure 5) i.e. they define the areas that benefit from the project outputs and outcomes.

Impact Beneficiaries

Figure 6: The five Mission: Delivering sustainable and regenerative agriculture and food systems; Delivering climate-positive and resilient landscapes; Restoring nature and protecting our environment; Enhancing Rural and Island communities; and Building the circular economy, are defined as impact areas. The main customers or stakeholders of: Government; Industry/Innovators; and Producers/Land Managers are the beneficiaries or customers of impact.

It is important to note however that while the outputs and outcomes of the ENRA research programme will deliver significant impacts from 2027 to 2032, it is anticipated that longer term benefits will also accumulate, from this and previous research programmes, into the future. These longer-term cumulative impacts will take time to develop, however the outputs and outcomes in the ENRA research programme 2027-2032 will lay the foundations for these to happen.

Impact beneficiaries or customers for the ENRA research programme 2027-2032 include government, industry and innovators and producers/land managers and impacts could be realised for each in one or more of the missions/impact areas (see Figure 6).

Outputs

The outputs of the ENRA research programme are the tangible research products i.e. knowledge, resources or decision support tools^[6] delivered or produced by projects within each Mission (see Figure 3). However, impact can be achieved throughout the lifetime of a project through research activities such as co-production processes, meetings/engagement with stakeholders, workshops, briefings and graphics. It could also result in the use of living labs/farm demonstrators to demonstrate and incentivise uptake of innovative best practise. The final project output such as a report, guidance or new model or system support tool should therefore not be considered the only impact delivered.

Outcomes

Outcomes are the ways in which research outputs are used to inform and influence decisions and behaviours by impact beneficiaries. ENRA research programme will look to deliver impact on its Missions through four main types of outcomes i) better targeted and designed government/policy interventions, ii), enhanced and wider uptake of better practice, iii) creation and adoption of new technologies and processes to enhance economic and environmental outcomes, e.g. for business and iv) increased rural development, job creation, and community resilience (see Figure 7). These outcomes will help achieve impacts that will deliver wider environmental, economic and societal impacts for the ENRA Missions.

The ENRA SRP Theory of Change Impact Pathway

Figure 7: The Theory of Change Impact Pathway. All projects undertake activities with the following examples: Workshop and outreach activities; collaboration and co-creation; field studies, experiments; analysis and modelling; and stakeholder meetings. The project activities generate a range of outputs listed as: Reports, guidance, briefings and policy papers; use of farm demonstrators/living labs to demonstrate and incentivise uptake of innovative, best practise; decision support tools; case studies; enhancing technology readiness levels; website/online resources. These outputs are co-created to ensure they deliver outcomes in the form of: Better targeted and designed government/policy interventions; enhanced and wider uptake of best practice; adoption of new technologies and processes to enhance economic and environmental outcomes; increased rural development, job creation, community resilience. The outcomes in turn support our Missions of: Restoring nature & protecting our environment; Delivering climate-positive & resilient landscapes; Enhancing Rural and Island communities; Building the circular economy; and Delivering sustainable and regenerative agriculture and food systems.

4. Plan for Impact - Commissioning

Start with the impact we want to have.

By strategically thinking about the impact from the beginning of the research programme we can increase the likelihood of realising it.

Researchers will be asked to use an impact planning tool when drafting research proposals, i.e. theory of change, to map out the anticipated impact pathway for a project, highlighting how the project outputs and outcomes will deliver impact on the Missions, linking to key SG priorities. The theory of change is a schematic presentation with a narrative or illustration such as a logic model that shows how a project can bring about the desired change or outcome (see Figure 8). This approach will provide information on the expected timescales between activities, outputs, outcomes and impacts, external enablers and barriers to achieving the intended impacts and underlying assumptions relevant to the pathway being achieved. In a theory of change, assumptions are the core beliefs or conditions that are considered true and necessary for the intervention to achieve its desired outcomes. They represent the underlying logic of the program and explain why certain activities are expected to lead to specific changes. Essentially, they are the conditions that need to hold for the theory of change to work effectively.

Figure 8: depicts an example of a Theory of Change. The figure highlights a linear chain detailing how the proposed inputs (such as the actions that will take place) leads to outputs (what is delivered or produced), to outcomes (the early to medium term results) and concludes at the final expected impact (long term results). It also highlights that along the linear pathway there will be supporting activities to help bring about the changes required (assumptions around: the individuals who take up the intervention; the early changes in behaviour brought about by the intervention; the benefits produced and possible unintended effects) and the contextual factors and influencers including external enablers and barriers that could affect the intended, outputs, outcomes and impacts being achieved.

Developing a theory of change typically involves considering the proposed inputs (what investment/regulation/actions will take place) and the causal chain that leads from these inputs through to the expected outputs and outcomes. It considers the causal mechanisms by which an intervention is expected to achieve its outcomes, basing this theory on the gathering and synthesis of evidence.

Researchers will also be expected to develop project proposals in a co- creative and collaborative way with research users/stakeholders i.e. policy teams and key partners. Co-production will also be expected throughout the lifespan of the project. To emphasise this approach, it is suggested that researchers are asked to identify/map their key stakeholders/policy customers as part of their project bid. Researchers should be invited to take a tailored approach and consider questions such as: Who needs to know what, and why? How can stakeholders use the research output? Where might outputs influence their policy/work/process?

5. Impact Monitoring

The annual reporting requirements for the 2022-2027 Research Programme require project Principal Investigators (PIs) to submit onto Researchfish a Narrative Summary, including a 250-word Impact Narrative. Feedback following the Researchfish 2023-24 Narrative Report, noted that while excellent examples of impact were demonstrated, there was a wide variance in the quality of the impact narratives and the evidence provided.

The Proposed Changes for the 2027-2032 SRP

The following section sets out how we intend to monitor and evaluate evidence of impact and record the pathway to help achieve impact of the ENRA Research Programme from 2027 (see Figure 9).

It is proposed that the additional annual Impact Narrative question continues to be part of the annual Narrative Summary with a small increase in word count from 250 to 500 words. The purpose of the Impact Narrative is to give a more detailed summary of the project impacts over the year, highlighting successes or changes to the impact pathway, and providing a chance to detail new or previously unforeseen impact opportunities. PIs will be encouraged to record project activities, which deliver impact such as workshops and outreach activities, policy briefings and interactions/engagements with policy customers and stakeholders. This will enable PIs to demonstrate the significance and breadth of impact. Testimonials from stakeholders who have used or benefited from the project could be used for example to help evidence/highlight impact.

It is proposed that an annual Mission Impact Summary (no more than 4 pages) is also produced by the Mission Impact Officer (see below) summarising impact across the Mission. Mission summaries will draw upon project Impact Narratives.

Figure 9: Summary of Impact will require an annual impact narrative highlighting project impact. The Misson Impact Officer will summarise Mission Impact using project impact narratives, annual mission impact summaries and case studies.

It is acknowledged that while there will be continuity for some work from the 2022- 2027 research programme, with a clearly defined impact pathway, the route to impact for some new projects is not always certain at the start of the project. The proposed approach is therefore intended to foster a flexible and agile approach to impact. PIs will be required to regularly review the project’s impact pathway and to work in a collaborative/co-production way with their policy customers/stakeholders to address issues and capitalise on new, unforeseen opportunities to maximise impact during the course of the project.

Impact Case Studies

It is proposed that a series of Impact Case Studies (2-3 per Mission) are developed annually over the course of the programme to demonstrate the impact of the ENRA SRP. These will be published on the SEFARI Gateway website following review by the CSA ENRA and Scientific Advisory Board members. Impact Case studies should describe and evidence how the SRP science and research has made a material contribution to a particular Mission and how the research funded through the SRP has strengthened the link between research and societal impact by promoting high-quality research that delivers measurable benefits to society, the economy and/or the environment.

The impact included in the case studies must have occurred within the 2027-2032 SRP programme but may reference research from previous programmes recognising the long-term nature of the research programme.

How will the Impact Narratives, Mission Impact Summaries and Case Studies be used?

The annual project Impact Narrative will be used to monitor and evaluate project funding. It will be uploaded onto Researchfish. The summary will inform Mission Impact Summaries and will be available on SEFARI Gateway webpages.

Mission Impact Summaries and Case studies will be used to monitor and evaluate the outcomes and impact of the SRP and will be available to Boards within the governance structure of the research programme 2027-2032. The process will mimic the Research Excellence Framework (REF) evaluation of UK Higher Education Institutes.

Clear guidance on how to complete impact reporting and what constitutes impact will be provided. Impact training will be made available to PIs for creating a theory of change impact pathway and impact reporting through the Centre for Knowledge Exchange and Innovation (CKEI) and Mission Impact Officers.

Roles and Responsibilities:

Mission Impact Officer

To support and champion a culture of impact across the programme it is proposed that two Impact Officers are appointed for each Mission from 1-2 of the SEFARI Institutes/CoEs. The role of the Impact Officer would be to lead and motivate a culture of impact across the ENRA SRP; Impact Officers will support researchers to deliver impact by building capacity and capability across the Institutes they will be key in; supporting and facilitating the generation of impact; increasing awareness and communication of impact; coordinating stakeholder identification and engagement; supporting and leading the monitoring, evaluating and recording impact; developing Mission Impact Summaries and Case Studies, and tracking progress.

Impact Officers would provide support and guidance to PIs annual impact reporting, would coordinate and produce annual Mission level Impact Summaries and coordinate the delivery of case studies across the Mission.

The Impact Officers would also have a role to look across the Missions to ensure the co-benefits of project outputs are realised and reported and to identify opportunities to increase impact i.e. where outputs from one Mission may be relevant to the outcomes of another Mission for example. This work would support efforts to break down silos and promote cross-Mission working. It is suggested that Impact Officers working with the CKEI could lead an annual impact meeting/workshop highlighting and championing positive examples of impact and sharing examples of best practise.

The Impact Officers would also be the first point of contact for Scottish Government on Impact.

Project Impact Champions

Each commissioned project within the SRP will have an identified Impact Champion who will support the activities of the Impact Officer to promote and report project impacts. The impact champion can change throughout the course of the project.

Topic Leads

Scottish Government Challenge Leads will continue to review quarterly reporting of milestones, deliverables on Researchfish. They will also review the annual project Impact Narratives. This activity, however, will be supported by Mission Impact Officers.

6. Evaluation

The impact of the SRP will be evaluated at project level by Scottish Government Challenge Leads. Challenge Leads will review annual impact narratives for their projects and feedback will be provided to Impact Officers and PIs as appropriate.

The annual Mission Impact Summaries will be reviewed and assessed by CSA ENRA and the ENRA Scientific Advisory Board. Feedback will be provided to Mission Impact Officers.

CSA ENRA and the ENRA Scientific Advisory Board (SAB) members will also review a number of impact case studies for each Mission.

7. Communicating for Impact

The benefits and importance of capturing and communicating the impact of the

ENRA research programme research outcomes, Underpinning National Capacity (UNC) and CoE, to government and other key stakeholders are clear. Clear and demonstratable impact will enable Scottish Government to show the value of embedding environmental and agricultural evidence in the policy decision-making process, the positive impact for business, innovation and wider society and will be able to use this evidence to support the case for continued investment.

The audience is defined as the impact beneficiaries or customers i.e. Government, Industry and Innovators and Producers/Land managers.

Scottish Government already supports several routes to raise awareness of the impact of the ENRA research programme; for example - there is a monthly newsletter, regular events including an online Seminar Series focusing on discrete elements of the programme and an annual ENRA Science, Evidence and Policy Conference, which covers the breadth of the research programme. It is suggested that further opportunities to build on this activity could include:

• Impact Case studies - Impact case studies are already published on the SEFARI Gateway website^[8] but these could be developed further and increased in volume. Case studies will be proactively shared with policy teams and stakeholders or post on social media.
• An impact prize (non-monetary) (including an Early Career impact prize/category) could be incorporated into the ENRA Conference. This could be judged by a panel including academic, policy, stakeholder/industry representation.

Policy Engagement and Co-production

For research to have an impact it must be used by stakeholders. Evidence suggests that co-productive forms of research offer increased potential for academic, economic and social impact. Its potential benefits include:

• facilitating more holistic research, including the representation of different knowledge systems
• bringing greater accountability of publicly funded research
• building trust between researchers, policy makers, and other stakeholders

Researchers should be encouraged to engage in activities that promote collaboration throughout the delivery of research programme on for example: co-development workshops, ongoing engagement meetings, impact events and visits to Research organisations and secondments/work shadowing opportunities (SEPAL).

Feedback on how research outputs are used by impact beneficiaries such as policy teams/stakeholders will be encouraged. A potential mechanism would be to introduce use of the Single-Product Evaluation for Immediate Delivery (SPEIDY^[9])

SPEIDY framework Academic Report – March 2020

Feedback Questionnaire on a Research output – which provides a picture of the likelihood of research use.

8. Conclusion

A summary of the key points:

In the context of the ENRA Research Programme 2027-2032 impact is defined as the real-world difference research makes — changing lives, shaping policy, strengthening the economy, improving the environment, and enhancing health and wellbeing."

This refers to the positive changes or benefits that extend beyond academic circles. This includes improvements to the economy, society, culture, public policies, services, health, the environment, and overall quality of life.

• Commissioning: Impact will form part of the Invitation to Tender for Grant Funding (ITGF) process for the ENRA SRP 2027-2032. Researchers bidding for funding will be asked to use an impact planning tool when drafting research proposals i.e. theory of change to map out the anticipated impact pathway for a project highlighting how the project outputs and outcomes will deliver impact on the Missions, linking to key SG priorities. This approach will provide information on the expected timescales between activities, outputs, outcomes and impacts, external enablers and barriers to achieving the intended impacts and underlying assumptions relevant to the pathway being achieved. Risk mitigation plans for mitigating against impact fail should also be included. Researchers will be encouraged to go through a Theory of Change workshop ahead of project initiation with the Mission Impact Officer.
• Co-Development: Researchers will be encouraged to co-develop project proposals in a co-creative and collaborative way with research users/stakeholders, i.e. policy teams and key partners. Co-production will also be expected throughout the lifespan of the project. To emphasise this approach researchers are asked to identify/map their key stakeholders/policy customers as part of their project bid, and their processes of engagement. Policy customers/stakeholders will be encouraged to provide feedback on research outputs.
• Training and Guidance: Clear guidance will be provided on how to complete impact reporting and what constitutes impact. Impact training will be made available to PIs for creating a theory of change impact pathway and impact reporting through the CKEI and Impact Officers.
• Monitoring: For the 2027-2032 SRP the following impact monitoring steps will be requested as part of the reporting and evaluation requirements: i) increased annual project Impact Narratives, ii) annual Mission Impact Summary, iii) annual Case Studies.
• ENRA SRP 2027-2032 will have Mission Impact Officers: to support and champion a culture of impact across the research programme it is proposed that 1-2 Impact Officers are appointed for each Mission from 1-2 of the

SEFARI Institutes/CoEs. Each project will also have an identified Impact Champion to support the activities of the Impact Officer.

• Evaluation: Scottish Government Challenge Leads will review quarterly impact updates and annual impact narratives for their projects. The annual Mission Impact Summaries and Case Studies will be reviewed and assessed by CSA ENRA and the ENRA Scientific Advisory Board annually.
• Communicating Impact: Scottish Government will promote Impact Case Studies to ensure that research programme achievements are recognised.

Scottish Government will introduce an Impact prize.