We are testing a new beta website for gov.scot go to new site

Evaluation of Intensive Family Support Projects in Scotland

Listen

1. Introduction

1.1 The research and its key objectives

1.1 This report evaluates five intensive family support projects ( IFSPs) operating in Scotland in the period 2006-2008. Commissioned by the (then) Scottish Executive in 2006, the research was initiated mainly to evaluate the three 'Breaking the Cycle' (BtC) pilots funded by the Scottish Government as a two-year programme running from late 2006/07.

1.2 The BtC schemes - in Falkirk, Perth (the P4 Perth project) and South Lanarkshire ( SLC) - draw on the ground-breaking model of help for households at risk of eviction developed by the Dundee Families Project ( DFP) since its inception in 1996. These new initiatives were seen as 'demonstration projects' aimed at illustrating the benefits of DFP techniques as well as testing the effectiveness of these techniques delivered through a purely 'outreach support' model rather than incorporating core residential accommodation for the families concerned.

1.3 As well as covering the schemes established by Falkirk, Perth and South Lanarkshire councils, the evaluation also encompassed the original Dundee Families Project ( DFP) as well as the Aberdeen Families Project ( AFP) set up in 2005. These longer-established projects were included in the study mainly to enhance the scope for analysing the impacts of project support - particularly in terms of the longer-term sustainability of any improvements in families' lifestyles and behaviour achieved with Project help. It should be noted at this stage that, unlike the BtC projects, AFP also operates a core block facility, although this was opened only in summer 2007.

1.4 As specified in the research brief, the key questions required to be addressed were as follows:

(a). Which agencies deliver the service?

(b). What is the capacity of each service?

(c). What are the referral eligibility conditions and procedures?

(d). How do the Projects work with other agencies and how well do other stakeholders understand Projects' roles?

(e). How are potential clients identified and how effective is the process?

(f). What is the profile of service user households and does this change over time?

(g). What types of intervention are offered by/through the Projects, and to what extent is Project support distinct from assistance previously offered/provided?

(h). Which types of clients engage most successfully?

(i). What is the duration of support, to what extent are support programmes terminated early, and where this occurs, what are the reasons?

(j). What is the nature of relationships between service users and Project staff?

(k). To what extent do Projects successfully meet clients' needs?

(l). To what extent do former Project service users continue to require support?

(m). What is the impact of Project support on service users' awareness of their problematic behaviours?

(n). What is the impact of Project support on service users in terms of (i) reducing their anti-social behaviour, (ii) improving their housing circumstances, (iii) improving their family functioning

(o). To what extent have Projects fulfilled stakeholder agency expectations?

(p). If 'positive outcomes' are achieved, how sustainable are these?

(q). What are the relative impacts of core block and outreach service provision?

(r). What are the local attitudes towards core blocks?

(s). Do the projects represent value for money?

(t). How do core block and outreach services compare in terms of value for money?

1.5 Drawing on a range of evidence, Chapter 8 revisits each of these questions and provides a summary response.

1.2 The policy context

1.6 Reducing anti-social behaviour ( ASB) and building a culture of respect have been key Central Government priorities both in Scotland and south of the border in recent years. Accordingly, measures to tackle ASB have been located within a wider policy agenda of revitalising disadvantaged neighbourhoods and stimulating a process of civic renewal.

1.7 In developing sustainable solutions to ASB it is increasingly recognised by practitioners and policy makers that interventions must address the underlying causes of misconduct. There is now a considerable body of research examining the characteristics associated with anti-social behaviour. These characteristics are often referred to as 'risk factors' - e.g. emotional and mental health problems; disorder in the local community; a lack of extra-curricular activities; school exclusion; having a parent who is an offender; poor relations with parents and/or not spending much time with parents (Wilson et al, 2006; Margo, 2008).

1.8 Equally important is the emerging evidence of characteristics, which can counteract risk factors. Such characteristics include enhanced self-esteem, greater enjoyment of school activities and reduced levels of family adversity. Research confirms the potential benefits of early intervention in tackling individual and family factors particularly those associated with cognitive and behavioural development (El Komy et al 2008). Further, there is a recognised need to develop targeted strategies to improve the capacity of social services, health services and specialist programmes to both reach and improve the behaviour of marginalised and/or vulnerable groups, such as those committing anti-social behaviour (Margo, 2008).

1.9 Informed in part by research evidence, official ASB strategies in both Scotland and England have been based on a twin track approach involving legal sanctions to discipline perpetrators and protect communities alongside support measures to address the underlying causes of problem behaviour. More specifically as the Central Government ASB agenda has developed over the past few years increasing attention has been paid to control measures involving 'whole family' approaches and parenting interventions.

1.10 Intensive family support projects (sometimes known as family intervention projects - FIPs) respond to official concerns about social exclusion. As these have developed in both Scotland and England they have been substantially inspired by the groundbreaking Dundee Families Project ( DFP) as established in 1996. Drawing on the positive findings of an independent evaluation of the Project (Dillane et al 2001) the ' DFP model' has been promoted as good practice by the Westminster Government's Social Exclusion Unit ( SEU 2001). By 2002/03 seven further family projects had been set up in the North of England all of which demonstrated similar positive outcomes in terms of breaking the cycle of poor behaviour, homelessness and social exclusion.

1.11 Commenting on the Westminster Government's ASB strategy, the Home Affairs Select Committee concluded in 2005 that the development of ' intensive family-based interventions are essential if the deepest-rooted ASB problems are not simply to be recycled from one area to another' (Home Affairs Select Committee, 2005).

1.12 Further support for the family-based intervention model was provided in the (England & Wales) Youth Justice Board report on tackling anti-social behaviour (Solanki et al., 2006) which found that where an anti-social behaviour intervention had prompted a positive change in the person's behaviour it had usually taken place in conjunction with other support services or with some external factor, like a change in family situation, which motivated the individual to change. It is now recognised that working with perpetrators and their families to address underlying causes of problem behaviour is an essential element of local strategies for combating ASB.

1.13 In Scotland a review of approaches to tackle ASB across government, local agencies and local communities was announced in October 2007, to be led by the Scottish Government's Community Safety Unit (in collaboration with an External Expert Advisory Group), reporting to the Minister for Community Safety. Recommendations for improving the national strategy and delivering on change are due to be reported to the Scottish Parliament in early 2009.

1.3 Existing evidence on impacts of 'whole family' approaches to anti-social behaviour

1.14 There is now a considerable body of evidence on the efficacy of family interventions and the official commitment to 'whole-family' approaches has been informed by findings from four main studies:

  • Evalution of the Dundee Families Project (Dillane et al, 2001)
  • Evaluation of Rochdale Shelter Inclusion Project (Jones et al 2006)
  • Evaluation of six intensive family support projects in England ( Nixon et al 2006; 2008)
  • Research on the design, set up and early outcomes of Family Intervention Projects 2008 (White et al, 2008)

1.15 There are high levels of consistency in results across the four studies, particularly with regard to the characteristics of families targeted for this type of intervention, the welfare support needs of those concerned and the perceived root causes or risk factors associated with ASB.

Circumstances and needs of families referred for Project support

1.16 The empirical evidence on families referred to Projects indicates that:

  • Lone parent women are disproportionately represented; Dillane et al found 64% of families working with the DFP were lone parents; Jones evaluating the Shelter Inclusion Project found 60% were lone parents while in the two more recent studies undertaken by Nixon et al (2006) and White et al (2008) found that 68% and 69% respectively, of families working with FIPs were headed by lone parent women.
  • Referred families tend to be relatively large. Nixon et al (2006) reported that 62% of families had three or more children with projects operating in large metropolitan areas tending to have the highest concentration of very large families (4+ children). Similar findings were reported by White et al (2008) where 56% of families contained three or more children under the age of 18.
  • Referred families usually have very low incomes and frequently debt problems; for example Nixon found that 59% of households owed on average £1,358 at the point at which they were referred to a Intensive Family Support Project while Jones et al (2006) reported that 63% of households had at least one debt at the point of referral. [This compares with 34% of all UK adults found to be carrying some kind of unsecured debt in 2003 (Tudela & Young, 2003)].

1.17 In addition the research evidence illustrates a clear link between high levels of ASB, socio-economic disadvantage and a range of risk factors including child protection concerns and family violence.

1.18 Across all four studies families referred for intensive support were characterised as having multiple and inter-related support needs which in many cases had not been adequately addressed by other agencies. A wide range of health- related difficulties were prevalent among both adults and children. For example, Nixon et al (2006) found that poor mental health or physical health and/or substance abuse affected 80% of adults in referred families. Depression was the single most commonly reported problem, affecting 59% of adults in the Nixon et al (2006) evaluation, while White et (2008) reported that 69% of adults working with family intervention projects experienced depression, with 43% suffering from stress.

1.19 Research evidence demonstrates that children in referred families face a number of risks and adversities with many having school related problems including irregular attendance, exclusions and truancy. ADHD has been found to be very prevalent with White et al (2008) reporting that as many as a third (34%) of children working with family intervention projects suffer from this condition.

1.20 Just over half (53%) of women working with DFP had been in an abusive, violent relationship (Dillane et al 2001) while Nixon et al ( 2006) found that just under half of al referred families (47%) contained at least one person subject to intimate partner violence or intergenerational violence (recently or historically). Lower levels of violence in the home (25%) were reported in the White et al study, perhaps reflecting the more restrictive definition of domestic violence employed. Nixon et al (2006) found that, although very debilitating, violence within the home was often referred to in passing by Project workers with no causality attributed to the impact of violence within the home on behaviour outside of the family home.

1.21 The evidence from all four evaluations confirms that children working with projects were amongst the most disadvantaged in the country. Nixon found that project workers assessed the risk of family breakdown as high in over a third of families with a minority of children already on the Child Protection Register at the point of referral. In both White et al (2008) and Nixon et al (2006) evaluations further concerns were expressed about the need to take children into care or arrange alternative living arrangements in relation to around one fifth of families.

The efficacy of interventions

1.22 Perhaps most importantly for the Breaking the Cycle pilots, existing research evidence on the efficacy of interventions employed by family intervention projects is largely positive. In particular, all four studies found that where families engaged with Project caseworkers there was likely to be a reduction in ASB and, consequently, the threat of eviction and possible homelessness was also reduced. At the same time, however, Jones et al (2006) and Nixon et al (2008) point out that success was not universal, nor was it always complete but the weight of evidence strongly indicates that projects employing 'whole family' approaches are effective in promoting improved lifestyles and reducing ASB. More mixed findings have been reported in relation to promoting social inclusion and community stability and, in particular, there was less evidence of positive impacts on young people already known to youth offending agencies.

1.23 White et al (2008) identified a number of features of FIP working practices seen as critical to the model's success. These included the ability of projects to recruit and retain high quality staff; the designation of a dedicated case worker for each family; strict limitation of caseloads to permit intensive work with individual family members, and the embedding of projects within existing multi-agency community safety and welfare partnerships. It was also considered important to avoid time limiting interventions so that families could continue to receive Project support for as long as necessary.

The role of sanctions

1.24 A more controversial set of findings emerging from recent research relates to the role of sanctions in the provision of Project support. The FIP model promoted by the Central Government in England since 2006 has emphasised the use of assertive interventions backed up by the use of sanctions (Home Office, 2006). The role of Project workers was described as being to 'grip' the family and their problems, co-ordinate the delivery of services and employ a combination of support and sanctions to motivate a change in behaviour. An initial evaluation of the national network of English FIPs established on this model has been undertaken by White et al (2008). White et al (2008) found that in 29% of cases FIP staff played a role in putting into place enforcement actions. Families interviewed in this study expressed mixed views on the effectiveness of the 'support and sanctions' approach in stimulating improved behaviour. For some, such contact facilitated beneficial changes but for others Project attention was seen as an unnecessary and unwelcome intrusion into their lives.

1.25 In terms of the emphasis on sanctions, some of the working practices identified by White et al (2008) bear little resemblance to the practices described by Jones et al (2006) and Nixon et al (2006) in relation to an earlier generation of FIPs. The work of these earlier generation projects was informed by a number of shared guiding principles which included treating family members with respect, listening, being non-judgemental and accessible while also ensuring that the approach was consistent and honest. Such practices were highly praised by service users and were reported as being critical in enabling them to change their behaviour. Not surprisingly, since the 'earlier generation' English Projects were descended from the Dundee model, the Projects under evaluation in this current study shared many of the features described above. Unlike the FIP programme, they were not conceived on the 'support and sanctions' model.

1.26 The evaluation of the BtC projects, AFP and DFP provides an opportunity to explore in greater detail the impact of Project practices on family members within the Scottish policy context. This evaluation of five projects which were set up to establish innovative and creative ways of addressing the underlying causes of problem behaviour makes a significant contribution towards improving knowledge and understanding in this important area of work.

1.4 Evaluation methodology

1.27 The main elements of the study were as follows:

(i). Initial interviews with IFSP project staff, referral agencies and other key stakeholders

These interviews explored project origins, operational and governance arrangements, as well as service user referral rules and procedures. Importantly, they also provided an opportunity to outline the ongoing assistance required by the research team over the course of the evaluation.

(ii). Collection and analysis of socio-economic and housing data about families referred to IFSPs - 'inward referrals'

For the duration of the evaluation each IFSP was required to complete a pro forma about every referral received for consideration of possible support provision. This facilitated collection of comprehensive data on the characteristics, circumstances and support needs of the (potential and actual) service users. Covering all inward referrals to the five projects in the period 1 January 2007-30 June 2008, this system collected data on 88 families (84 of which were subsequently accepted for Project support).

(iii). Collation and analysis of service user Support Plans

To inform an understanding of IFSP activity, Support Plans relating to 21 families receiving support from the five projects in summer 2007 were analysed according to a common format.

(iv). In-depth study of cohort of IFSP service users - interviews with families, referral agencies, project staff

In-depth interviews with members of families currently being supported by the Projects formed the main component of this work. In all, 43 such families were interviewed (48 interviews achieved, including with children) - see Tables 1.1 and 1.2. There might, of course, be a concern about possible selection bias in favour of 'easier' or 'more successful' cases. We cannot state categorically that the families interviewed were entirely representative of all families being assisted. What can be said is that interviewed families certainly included some with deeply embedded difficulties and whose support programmes had not proceeded according to plans. Secondly, it must be emphasised that attrition rates here were relatively low - of all families invited to participate in the research 78% agreed to do so. And, of these, more than 80% were actually interviewed (see Table 1.2). Thirdly, it should be stressed that the research avoided over-reliance on service user evidence. Critically, our establishment of a comprehensive case monitoring system will have counterbalanced any service user interviewee selection bias. For further details on the service user interviews methodology see Annex 2.

(v). Interviews with IFSP former service users, following case closure

Complementing the discussions with current service users, these interviews mainly involved families formerly receiving Project support. As shown in Table 1.1 a total of 15 such interviews were achieved. This cohort mainly involved former clients of the Aberdeen and Dundee projects. This reflects the fact that the very recent establishment of the other three schemes meant that there had, as yet, been relatively few 'closed cases' at the point when the fieldwork was undertaken in Summer 2008. Mainly designed to explore perceived outcomes of Project assistance, these interviews sought to investigate the sustainability of resulting improvements in lifestyles and behaviour.

(vi). Collection and analysis of monitoring data on support outcomes in relation to families having their cases closed

For the duration of the evaluation each IFSP was required to complete a pro forma about every service user family subject to 'case closure', irrespective of whether the reason was 'successful completion of Support Plan'. The form collected data on the support provided to the family and on perceived intervention outcomes. In all, this system collected data on all 67 cases closed in the period 1 January 2007-30 June 2008.

(vii). Follow-up interviews with IFSP project staff, referral agencies and other key stakeholders

In summer 2008, at the end of the evaluation period, a final round of interviews were undertaken to elicit reflections on the experience of the demonstration programme and lessons learned. In the course of this fieldwork IFSP caseworkers were interviewed about the families in their own caseload. To complement material collected via the various other research instruments, each interviewee provided a brief resume of each case in relation to the family's initial problems, their support programme, their engagement with support and the impacts of assistance provided.

(viii). Economic evaluation

The economic evaluation considers the cost-effectiveness of the Projects and the extent to which they represent value for money. Two unit costs are calculated from the activity and expenditure data - the average cost per family month and the average cost per closed case.

Traditional measures of cost-effectiveness, which formally combine costs and benefits into a single measure, were considered unsuitable for this evaluation, which instead adopts a 'cost consequences' approach. Many of the expected benefits are qualitative rather than quantitative, and are likely to arise over different time horizons. Although potential costs for some Exchequer-funded services (e.g. for child protection, homelessness and anti-social behaviour) are likely to be prevented by the Projects, families will also benefit from improved life chances. The study therefore draws upon material from a literature review identifying the likely values of the potential savings (using Scottish data where available) and material from the study on outcomes to identify the cost savings and other benefits that might be delivered by the Projects. Decision makers can then use this information (supplemented by local information where available) alongside that on the costs of delivery to draw their own conclusions about the local suitability of such an intervention.

Table 1.1 Service user interviews undertaken

Project

Current service users (2007 and 2008)

Former service users (2008)

Total interviews

First round interviews 2007

Second round interviews 2008

Initial interviews

Re-interviews

AFP

6

2

2

2

12

DFP

13

3

3

8

27

Falkirk

5

4

1

6

16

P4 Perth

4

5

0

4

13

SLC

5

1

3

1

10

All projects

33*

15

9**

21**

78

*included 5 interviews with children aged 12-15. **including 1 child. ***including 4 children

Table 1.2 - Families participating in service user interviews: breakdown by size and type

No of children

Single female headed

Two adult family

Total

1

7

-

7

2

10

4

14

3

7

3

10

4

2

4

6

5

2

1

3

6

3

2

5

Total

31

14

45*

* The number is smaller than the total number of families interviewed due to missing data

Table 1.3 - Consents for interviews requested and granted; families in which interviews achieved

Project

Consents requested*

Consents granted*

Interview(s) achieved*

AFP

12

9

8

DFP

28

21

18

Falkirk

11

11

9

P4 Perth

20

13

9

SLC

9

9

7

All projects

80

63

51

*All figures relate to households rather than individuals

1.5 Structure of the report

1.28 The remainder of the report is structured as follows. First, in Chapter 2, we outline the five projects in terms of their origins, organisational arrangements and approaches to service delivery as revealed through the initial scoping visits (late 2006) and follow up 'key stakeholder' and IFSP staff interviews in summer 2008.

1.29 Chapter 3 provides an analysis of the characteristics, background and difficulties faced by households referred to the five projects. This is based on data collected through 'inward referral' monitoring system outlined in the evaluation methodology section above (see point (ii) in that section).

1.30 Chapter 4 aims to provide a further insight into the challenges facing referred families and the nature of the help envisaged by Project staff as appropriate to meet these challenges. The chapter draws on three sources: the family Support Plans analysis (see point (iii) in evaluation methodology section), case closure monitoring data and the final round of IFSP staff and stakeholder interviews (evaluation methodology points (vi) and (vii))

1.31 Chapter 5 is an analysis of interviews undertaken by the research team with families being assisted by the five projects (see evaluation methodology section, point (iv)). In most cases, initial interviews were undertaken 1-3 months into support programmes, with follow-up meetings taking place approximately 12 months later.

1.32 In Chapter 6 we explore project support outcomes. The chapter draws on three elements of the fieldwork: analysis of testimony from former service user interviews, case closure pro forma data, and Project and stakeholder staff member follow-up interviews (see evaluation methodology points (v), (vi) and (vii)).

1.33 A key question addressed by the research concerns the costs and cost-effectiveness of the five IFSPs. Chapter 7 sets out findings on this topic.

1.34 Finally, in Chapter 8 we revisit the original objectives of the study to draw together key findings from the research.