Family Nurse Partnership evaluation: methods and process

This paper presents the methods of using routinely collected health, education and social care data to evaluate the Family Nurse Partnership (FNP) in Scotland using a natural experiment methodology.


5. Results

5.1. Application process 

Applications were made to PBPP and EAS and approved on 21st Dec 2016 and 28th October 2016 respectively. The EAS application was submitted, considered and approved within 1 month. The PBPP application was submitted 30th September 2016, considered at Tier 1 on 2nd November 2016 (comments responded to on 16th November 2016); Tier 2 approval granted 21st December 2017 (~2.5 months). Two additional amendments were then submitted. The first on 21st April 2017, following the addition of data fields and sources, with approval given 12th June 2017. The second amendment on 1st March 2018, extended the project duration and added data fields and sources, with final approval on 1st May 2018.

5.2. Population 

5.2.1. FNP Clients 

Identifiers for FNP Clients from the ten HB areas were sent from FNP SIS and pilot data were extracted separately from Lothian systems. FNP Clients who had transferred from one FNP area to another were kept in their originally assigned site. Where an FNP Client had enrolled into FNP more than once, the first episode was discarded and the more recent one included. These accounted for very small numbers.

Figure 3 shows the flow chart of the FNP Clients. A total of 3,277 FNP Clients were received by eDRIS from FNP SIS. Of these, six Clients were excluded (0.2%) and 3271 (99.8%) were eligible to be linked with the SMR02 maternity record. A total of 48 women could not be linked to the SMR02; 23 were missing an SMR02 record indicating that the CHI number was not present in the SMR02 dataset (possibly due to inaccuracies in CHI number) and 25 without a delivery record. 3,223 cases with a delivery record (live or stillbirth) remained and were imported alongside their maternal characteristics data into the eDRIS portal to be considered for inclusion in the evaluation. 

Further data cleaning found an additional 18 women who were not eligible for FNP based on the fields in either the FNP SIS or the SMR02 dataset. 11 women were recorded as greater than 19 years of age at enrolment into FNP (from FNP SIS). Seven women were found to be outside the gestation criteria of 28+6 weeks. 3,205 FNP Clients (97.8% of the initial cohort) remained for analysis. Comparison of age at LMP between the two data sources for FNP Clients showed some small discrepancies with a median age of 18.4 years (SMR02) vs 18.1 years (FNP SIS) but a high correlation between the two (r = 0.862, p<0.001) (Table 9). Age at booking/enrolment was also very similar. Gestation at enrolment in to FNP was on average nearly seven weeks after the gestation at booking with the FNP Clients and this is to be expected. 

All eligibility criteria based on SMR02 fields had no or very low missing data, demonstrating that the SMR02 fields are robust enough to identify Controls. 

Table 9: Comparison of SMR02 vs FNP SIS datasets for FNP Clients

  Data source
SMR02
N=3,205
FNP SIS
N=3,205
Age at LMP (years)
Mean (sd) 18.30 (1.41) 17.91 (1.28)
Median (25th to 75th centiles) 18.4 (17.3 to 19.3) 18.1 (17.0 to 19.0)
Gestation (weeks) at booking
N=3,200
at enrolment into FNP
N=3,205
Mean (sd) 10.82 (6.37) 17.84 (4.24)
Median (25th to 75th centiles) 10.0 (8.0 to 12.0) 16.7 (15.0 to 20.1)
Age (years) at booking
N= 3,205
at enrolment into FNP
N=3,205
Mean (sd) 18.14 (1.22) 18.23 (1.29)
Median (25th to 75th centiles) 18.3 (17.2 to 19.2) 18.4 (17.3 to 19.2)

5.2.2. Controls (eDRIS)

eDRIS identified a total of 31,906 potential women whose age at delivery was ≤20 years from the  period 1st January 2008 to 31st March 2016 based on SMR02 fields (Figure 3). From this figure 3,515 women were identified as being FNP Clients, 19 without a unique patient identifier, and the remainder of reasons were due to being outside the eligibility inclusion criteria. 

After deleting Controls found to be in the recruiting period (2,214) and those with a booking date more than one year pre or post the recruiting periods (n=13,175), 5,227 Controls remained. A further 211 were identified with a gestation at enrolment >28+6 week, leaving 5016 controls for the evaluation. 

Figure 3: Flow of eligible FNP Clients and potential Controls 

Figure 3: Flow of eligible FNP Clients and potential Controls 

5.2.2.1 Maternal characteristics

Table 10 describes the maternal characteristics of the FNP Clients and the Controls identified from the 1-year pre and post recruitment periods and within intervals where recruiting stopped and also the standardised differences between FNP Clients and the Controls. Of the data requested, the education and social care data were not available. Imbalance between the FNP Clients and Control group can be observed in some maternal characteristics such as ethnicity, dispensing of medication for asthma and depression, age and gestation at booking. FNP Clients were on average younger (recorded at booking) and had a lower gestation at booking when compared to the Control population. However, age at last menstrual period was comparable (18.30 vs 18.22 years respectively). FNP also appear to recruit a higher proportion of white women (87.6% vs 78.7% respectively), with a higher proportion of drug misuse, and dispensing for asthma and depression. 

Table 10: Maternal characteristics All results are n(%) unless otherwise stated

  FNP Clients N=3,205 Controls N=5,016
Health board of residence
NHS Ayrshire/ Arran 227 7.1 483 9.6
NHS Borders 24 0.8 42 0.8
NHS Fife 380 11.9 450 9.0
NHS Forth Valley 100 3.1 263 5.2
NHS Grampian 93 2.9 172 3.4
Greater Glasgow and Clyde 404 12.6 1391 27.7
NHS Highland 100 3.1 197 3.9
Lanarkshire 250 7.8 722 14.4
Edinburgh 895 27.9 862 17.2
NHS Tayside 732 22.8 434 8.7
Missing 0 0 0 0
Ethnicity
White  2,724 87.6 3,573 78.7
Other 384 12.4 969 21.3
Missing 97 3.0 474 9.4
Scottish Index of Multiple Deprivation Quintile
1 - most deprived 1,532 47.9 2,478 49.4
2 821 25.7 1,221 24.3
3 441 13.8 643 12.8
4 264 8.3 459 9.2
5 least deprived 140 4.4 215 4.3
Missing 7 0.02 0 0
Age at last menstrual period (LMP) (years) (SMR02)
Mean (SD) 18.30 (1.41) 18.22 (1.23)
Median (25th to 75th centiles) 18.4 (17.3 to 19.3) 18.4 (17.4 to 19.2)
Missing 0 0 0 0
Age at antenatal booking (years) (SMR02)
Mean (SD) 18.14 (1.22) 18.45 (1.22)
Median (25th to 75th centiles) 18.3 (17.2 to 19.2) 18.7 (17.6 to 19.5)
Missing 0 0 0 0
Gestation at antenatal booking (weeks) (SMR02)
Mean (sd)  10.86 (6.25) 11.53 (4.69)
Median (25th to 75th centiles) 10.0 (8.0 to 12.0) 11.0 (8 to 13)
Missing 9 0.3 0 0
Body mass index (BMI) (kg/m2) at antenatal booking 
Mean (SD) 23.94 (5.0) 24.3 (5.1)
Median (25th to 75th centiles) 22.9 (20.4 to 26.4) 23.2 (20.6 to 27.0)
Missing 51 1.6 483 9.6
BMI category at antenatal booking
Underweight 259 8.2 325 7.2
Healthy 1840 58.3 2585 57.0
Overweight 680 21.6 1005 22.2
Obese 375 11.9 618 13.6
History of smoking during pregnancy
No  1,891 61.4 3,036 64.3
Yes  1,191 38.6 1,684 35.7
Not known/missing 123 3.8 299 6.0
Smoking history at antenatal booking
Never smoked, non-smoker/former smoker 1,856 59.1 3,000 62.9
Never smoked, non-smoker 1,254 39.1 2,212 46.4
Former smoker 602 18.8 788 16.5
Current smoker 1,285 40.9 1,767 37.1
Not known/Missing 64 2.0 249 5.0
Drug misuse at any time during the current pregnancy
No 2,788 94.7 3,768 96.8
Yes 156 5.3 123 3.2
Not known/Missing 260 8.1 1,125 22.6
Ever injected illegal drugs
No 2,927 99.8 3,856 99.7
Yes (during /prior to current pregnancy / yes but not know when) 7 0.2 13 0.3
Not know/Missing  270 8.4 1,147 22.0
Number of units of alcohol consumed in the course of a typical week at antenatal booking
None 2897 94.9 4174 95.4
At least one unit 156 5.1 203 4.6
Missing 152 4.7 639 12.7
Patient has diabetes or not
Yes (pre-existing, gestational, time of diagnosis unknown) 43 1.4 48 1.0
No (no diabetes during this pregnancy) 3,119 98.6 4,836 99.0
Unknown/Missing 43 1.3 132 2.6
Ever dispensed medication for asthma
No 2,504 78.1 4,431 88.3
Yes 701 21.9 585 11.7
Missing1 0 0 0 0
Ever dispensed medication for depression
No 2,614 81.6 4,598 91.7
Yes 591 18.4 418 8.3
Missing 0 0 0 0
Previous pregnancy
No 2,350 74.0 3,716 74.1
Yes 827 26.0 1,296 25.9
Missing 28 0.9 4 0.1
Births
Singleton 3,184 99.3 4,989 99.5
Multiple (2+ births) 21 0.7 27 0.5
Missing 0 0 0 0
Total Babies n=3,227 n=5,043
Outcome
Live births 3,210 99.5 5,010 99.3
Still births 17 0.5 33 0.7

1. Women with no medication flag were not linked to medications data and thus are assumed not to have any medication

5.2.2.2 Missing data

Table 11 shows the completeness of the variables used to describe the maternal characteristics. The dataset provided had a higher degree of missing data than anticipated in 11 important maternal characteristics (possibly associated with outcomes in the evaluation e.g. smoking, drug use, BMI, alcohol use, ethnicity etc.) thus rendering the propensity score matching method as unusable.

Table 11: Completeness of maternal characteristic variables

Variable N Categorised as ‘Not known’  Missing (%)
Exposure (FNP/Control) 11,638 0 0
Health Board of residence 11,638 0 0
SIMD quintiles (least to most deprived) 11,631 0 7 (0.06%)
Ethnicity (White/Other) 10,469 0 1,169 (10.0%)
Age at booking (SMR02 only) (years) 11,638 0 0
Age at LMP (SMR02 only) (years) 11,638 0 0
Gestation at booking (SMR02 only) (weeks) 11,629 0 9 (0.09%)
BMI (kg/m2) at booking 10,580 0 1,058 (9.1%)
History of smoking during pregnancy  11,002 636 (5.5%) 0
History of smoking recorded at booking 11,135 496 (4.3%) 0
Drug misuse (yes/no) 9,664 1,872 (16.1%) 102 (1.0%)
Injected (yes/no) 9,625 1,910 (16.4%) 103 (1.0%)
Alcohol or not (yes/no)  10,276 0 1,362 (11.7%)
Diabetes (yes/no) 11,385 253 (2.2%) 0
Asthma meds (yes/no) 11,638 0 0
Antidepressant (yes/no) 11,638 0 0
Previous pregnancy (yes/no) 11,601 0 37 (0.3%)
Births (singleton/multiple)  11,638 0 0
Outcome of birth (live/still) 11,638 0 0

5.3. Controls offered FNP but not enrolled or not offered FNP

Table 12 shows selected characteristics for women who were eligible for FNP but did not enroll in the FNP programme compared to women who did enroll (FNP Clients) and Controls. Women who were offered FNP but not enrolled appeared to be more likely to be of ethnic background, a non-smoker and less likely to misuse drugs. All other characteristics such as deprivation profile, age at last menstrual period, BMI and rate of previous pregnancy were comparable.    

Table 12: Controls offered FNP but not enrolled or not offered FNP

  FNP Clients N=3,205 Controls N=5,016 Women offered FNP but not enrolled or not offered FNP N=2,214
Ethnicity            
White  2,724 87.6 3,573 78.7 1,733 78.3
Other 384 12.4 969 21.3 360 16.3
Missing 97 3.0 474 9.4 121 5.5
Scottish Index of Multiple Deprivation Quintile
1 - most deprived 1,532 47.9 2,478 49.4 992 44.8
2 821 25.7 1,221 24.3 588 26.6
3 441 13.8 643 12.8 316 14.3
4 264 8.3 459 9.2 214 9.7
5 least deprived 140 4.4 215 4.3 101 4.6
Missing 7 0.02 0 0 3 0.1
Age at last menstrual period (LMP) (years) (SMR02)
Mean (SD) 18.30 (1.41) 18.22 (1.23) 18.4 (2.20)
Median (25th to 75th centiles) 18.4 (17.3 to 19.3) 18.4 (17.4 to 19.2) 18.7 (1.7)
Missing 0 0 0 0 0 0
Body mass index (BMI) (kg/m2) at antenatal booking 
Mean (SD) 23.94 (5.0) 24.3 (5.1) 24.4 (5.1)
Median (25th to 75th centiles) 22.9 (20.4 to 26.4) 23.2 (20.6 to 27.0) 23.3 (6.3)
Missing 51 1.6 483 9.6 87 3.9
Smoking history at antenatal booking
Never smoked, former smoker 1,856 59.1 3,000 62.9 1,389 62.7
Never smoked 1,254 39.1 2,212 46.4 1,048 47.3
Former smoker 602 18.8 788 16.5 341 15.4
Current smoker 1,285 40.9 1,767 37.1 739 33.4
Not known/Missing 64 2.0 249 5.0 86 3.9
Drug misuse at any time during the current pregnancy
No 2,788 94.7 3,768 96.8 1,866 84.3
Yes 156 5.3 123 3.2 59 2.7
Not known/Missing 260 8.1 1,125 22.6 289 13.1
Previous pregnancy
No 2,350 74.0 3,716 74.1 1,656 74.8
Yes 827 26.0 1,296 25.9 555 25.1
Missing 28 0.9 4 0.1 3 0.1

5.4. Final cohort

Consistent with our commitment to minimum processing by using only those Controls that are required, we will only use the Controls identified within one year pre and post recruitment and in the interval period. This means that we would require outcome data on 3,205 FNP Clients and 5,016 Controls (approximately a 1:1.56 ratio) mothers for this evaluation and we should expect data on 3227 and 5043 children respectively. 

Contact

Email: socialresearch@gov.scot

Back to top