Evaluation of the Family Nurse Partnership Programme in NHS Lothian, Scotland: 3rd Report - Infancy

2 Summary of Evaluation Aims and methods

Evaluation aims and objectives

2.1 The overall aim of the evaluation of FNP in Scotland is 'to evaluate the implementation of the programme in Scotland (Lothian), focusing on process and understanding how the programme works in the Scottish context'. In particular, it is intended to assess:

• Whether the programme is being implemented as intended (and if not, why not)
• How the programme works in Scotland (Lothian), looking in particular at:
  • How Nurses, clients and wider services respond to the programme
  • What factors support or inhibit the delivery of the programme, and
  • Implications for future nursing practice
• What the potential is for FNP to impact on short, medium and long-term outcomes relevant to Scotland.

2.2 The evaluation focuses on the experience of delivering FNP in the first Scottish site in NHS Lothian, Edinburgh, with the expectation that the learning from this will help inform decisions and practice relating to further roll-out of FNP in Scotland.

2.3 This evaluation is not an experimental impact evaluation and cannot, therefore, conclusively establish causal links between FNP and particular outcomes. However, where possible, it reports on the evidence for the potential for FNP to impact on key outcomes for parents, children and services. The current 'Building Blocks' Randomised Controlled Trial in England (described in Sanders et al, 2011 and due to report final results in 2014) will be able to provide causal evidence, and is therefore likely to be of considerable importance for those with an interest in FNP in Scotland too. Further details about the remit for the evaluation are provided in Martin et al (2011).

Monitoring and evaluation framework

2.4 The evaluation of FNP in NHS Lothian, Edinburgh is informed by a monitoring and evaluation framework, developed by Jacki Gordon in discussion with key stakeholders from Scottish Government, NHS Lothian and City of Edinburgh Council. The key questions set out at the start of the findings chapters in this report are taken from this framework (see Martin et al, 2011 for full details).

Overview of methods and data included in this report

2.5 The evaluation is addressing the aims set out above using a range of quantitative and qualitative methods. These are described in full in Martin et al (2011). This third report draws on:

• Quantitative data collected and collated by the NHS Lothian, Edinburgh FNP team for all clients covering the infancy period (including data at 6 and 12 months). This data is routinely collected by Family Nurses and collated and provided to the external evaluation team as anonymised, aggregate figures.^[3]
• Qualitative data from:
  • A smaller sub-sample of FNP clients, interviewed around 12 months after their babies were born. This was the third occasion on which this longitudinal client 'panel' was interviewed for the evaluation. Of the original panel of 15 clients recruited to the evaluation, 13 were re-interviewed at 12 months.^[4]
  • The NHS Lothian, Edinburgh Family Nurse team (including the Nurse Supervisors), interviewed in spring/summer 2012.^[5] Again, this was the third round of interviews with the team.
  • 6 telephone interviews with key operational and strategic stakeholders from Midwifery, General Practice and Social Work in the NHS Lothian area
  • Ongoing interviews with the FNP National Lead for Scotland.

2.6 Qualitative data compliments the quantitative data collected for all clients by exploring experiences of the programme in more depth, including aspects that are difficult to quantify. Interviews with local stakeholders are included at this stage in order to explore external views of FNP and its links with wider services. Although the number of stakeholder interviews was small (one strategic and one operational stakeholder from each of Midwifery, General Practice and Social Work), they nonetheless give an indication of how FNP has been received in NHS Lothian and some of the potential challenges and facilitators in communicating with key stakeholders about the programme. Clients' 'significant others' - as nominated by clients themselves - were interviewed for the second evaluation report to explore perceptions of FNP among the wider family. Their views will be included again in the fourth report.

2.7 The evaluation team had also planned to conduct three focus groups over the course of the evaluation with NHS Lothian, Edinburgh FNP clients who were not selected for the longitudinal panel interviews. However, the first two of these groups were not successful, with a high level of non-attendance on the day. Given this, the evaluation team is currently reviewing its strategy for capturing the views of a wider client group beyond the panel.

Reporting conventions

2.8 As discussed above, detailed information from FNP clients, Family Nurses and key stakeholders were collected using a qualitative approach. Qualitative samples are generally small, and are designed to ensure a range of different views and experiences are captured. It is not appropriate given the number of interviews conducted to draw conclusions based solely on the qualitative data about the prevalence of particular views or experiences of FNP. Given this, where possible quantifying language, such as 'all', 'most' or 'a few', is avoided when discussing qualitative findings.

2.9 It is also worth noting that interviews with clients, Family Nurses and stakeholders focused on their perceptions of FNP. These perceptions may not necessarily always agree with each other, or with the views of others on how the programme works. However, they each provide valuable information about how the programme is experienced from the point of view of different stakeholders.

2.10 In order to protect the anonymity of clients and Family Nurses, participants are referred to by numbers only. Where participants were in unique or identifiable roles, they were given the opportunity to review their transcripts and/or any sections of the report that summarised their views in a way that might be identifiable or which quoted them directly. Any requests to remove a quote or potentially identifiable summary were always respected.

2.11 Finally, this report does not include any explicit comparisons with quantitative findings from the implementation evaluation of FNP in England (Barnes et al, 2008, 2009 and 2011). This is because the implementation of FNP in Scotland has been informed by the experiences of FNP in England. Any comparisons may not, therefore, be entirely comparing like with like.