8.1 Study limitations
8.1 In drawing together the findings of the evaluation, this chapter revisits the original research questions (as listed in Section 1.1). Before doing so, however, it must be stressed that the duration of the BtC pilot programme (and the evaluation itself) constrain the extent to which firm conclusions about Project efficiency and effectiveness can be drawn at this stage. Firstly, being established 'from scratch' entailed each of the new Projects had to devise procedures, recruit and train staff and establish working practices which inevitably took some considerable time to bed down. Consequently, these Projects are unlikely to have achieved an 'optimal' regime at least during their first year of operation. Secondly, the research confirmed that the 'short term' horizon of guaranteed Project funding hampered both recruitment and retention of Project staff.
8.2 Thirdly, it seems likely that the initial cohort of families referred to the new Projects will have included significant numbers of families long-known to the relevant statutory agencies and in this sense constituting 'backlog' demand for Project services. The experience of Dundee seems to suggest that, over time, as such backlogs are addressed, capacity is freed up to help families whose problems are not so long established and whose starting position is not (yet) so dire - see Table 3.2 and accompanying text. In the long term, therefore, it may be that Projects find it possible to address the problems of referred families more quickly and/or with less intensive staff time inputs. Hence, unit costs should fall and 'success rates' should improve as Projects mature.
8.2 Findings in relation to research questions
(a). What is the capacity of each service?
8.3 Including families being assessed as possible service users, snapshot caseloads of the five projects as at September 2008 ranged from 13-16 (see Table 2.3). However, throughputs of families also varied substantially, so that the numbers being assisted annually by each project might be more diverse. Because, by summer 2008, the initial cohort of BtC referrals had yet to fully work its way through the system, it is not possible to specify exact 'throughput' figures for the BtC projects. However, if (as a 'mature Project') Dundee's experience is typical, the Projects might expect to help around 15-20 families per year in the longer term (see Table 6.1).
(b). What are the referral eligibility conditions and procedures?
8.4 All of the Projects were targeted on households responsible for anti-social behaviour and at risk of eviction or having been excluded from social housing following eviction. All five were aimed, primarily, at family households (i.e. those including at least one child aged under 16), although P4 Perth was also willing to accept single people (see Section 2.5). With the exception of P4 Perth, only family households were accepted for support. Referral procedures varied, but usually involved some form of initial exploratory discussions between referring agency and Project as to the potential suitability of a prospective service user (e.g. in terms of their history, support needs and commitment to change).
(c). How do the Projects work with other agencies and how well do other stakeholders understand Projects' roles?
8.5 The Projects worked with other local agencies at a variety of levels. In most instances there were high level, multi-agency, oversight bodies to whom Project managers were partly accountable and which exercised influence on Project activity at a strategic level. Project managers and staff also related to colleagues - especially in local housing and social work departments - at an operational level in relation to specific families. Such contacts were both bi-lateral (officer to officer) and in the context of case conferences or panels often involving representatives of various agencies (see Sections 2.5-2.7, 4.3).
8.6 Among local stakeholder bodies, understanding of Projects' roles was somewhat patchy, although this generally improved as the Projects developed. Particularly for the BtC Projects, their recent arrival on the scene (and possibly their perceived insecure status) meant that not all potentially relevant local players had yet come to fully appreciate the contribution the Projects could make. At least in some of the local authorities it was perceived that inter-professional tensions had yet to be fully overcome (see Sections 2.4 and 2.7).
(d). How are potential clients identified and how effective is the process?
8.7 Potential service users were identified mainly by housing and social work departments although small numbers of referrals originated from other agencies. The methods used were seen as generally effective by the agencies concerned (see Section 2.5).
(e). What is the profile of service user households and does this change over time?
8.8 Service user families were typically lone parent families containing relatively large numbers of children. Typically, such families have multiple support needs. Comparing the profile of families referred in Dundee with equivalent cohorts in the other four authorities suggests that typical family size may decline over time (perhaps as initial 'backlog cases' pass through the system) - see Section 3.3).
(f). What types of intervention are offered by/through the Projects, and to what extent is Project support distinct from assistance previously offered/provided?
8.9 Project support typically addresses multiple issues and is delivered in a variety of ways. In most instances, support includes help with parenting skills and improving children's school attendance, building self-confidence, emotional support and welfare benefits advice. Family members are also usually challenged to recognize the negative impact of their previous misconduct, and parents encouraged to face up to deep-seated problems such as alcohol or drug abuse. Especially in its supportive component, such intervention is typically far more intensive than anything previously experienced by the service user families concerned (see Sections 4.3, 5.6).
(g). Which types of clients engage most successfully?
8.10 Project staff found it difficult to generalise on this. However, there was a consensus that parental neglect due to drug abuse constituted a particular challenge, partly because the priorities of a drug-addicted person could present a serious obstacle to engagement with Project support (see Sections 4.3 and 6.4).
8.11 It was also found that older teenage children (especially boys) could be more difficult to engage than their parent(s). The typical absence of male Project workers was thought by some staff to be a potential constraint in working with teenage boys. However, others contended that strong personalities among female workers and less 'macho' male workers could help to challenge the impressions that some children have developed due to exposure to parental behaviour.
(h). What is the duration of support, to what extent are support programmes terminated early, and where this occurs, what are the reasons?
8.12 In Dundee, families completing support programmes had typically worked with Project for around 15 months. As a rule, families withdrawing from their support programme had, nevertheless, been receiving help for seven months at this point (see Table 6.2). The median duration of initial assessments in the 18 months to June 2008 was 2.5 months (see Table 2.5). This represents the period elapsing from the date of the original referral to a Project to the date that Project staff reached a final decision on whether to accept the family for Project support. However, given that the monitoring period (2007-08) may have been affected by exceptional factors a more typical assessment duration may be 1-1.5 months.
8.13 Having been formally referred to a Project, most families complete their support programmes. Only a small proportion (around 5%) fail to be accepted for Project support following their initial assessment. Around 30% of those accepted for support have this support 'prematurely terminated' - either because they withdrew from the programme or for other reasons (e.g. all children taken into care). Nevertheless, even this latter group usually engage with Project support at least to some extent. Withdrawal from Project support was, in some instances, attributed mainly to ongoing drug abuse. See Sections 2.6 and 6.2.
(i). What is the nature of relationships between service users and Project staff?
8.14 Service user testimony confirms that Project staff usually succeeded in establishing relationships of trust with the families under their responsibility. Service users tended to view Project staff as being highly committed to their welfare and contrasted this with social workers and other officials with whom they had previously interacted. This may be attributed partly to the essence of the Projects which was to focus substantial staff (and other) resources on a small number of families. It is also probably associated with the fact that - unlike social work, housing or ASB staff - the Project caseworker function is not a 'policing' role (see Section 5.6).
(j). To what extent do Projects successfully meet clients' needs?
8.15 One response to this question is, in relation to families accepted by them for assistance, that Projects subsequently 'sign off' 70% of cases with the families having completed support programmes designed with their active involvement (see Section 6.2). Other relevant evidence is the tendency for service users to speak in very positive terms about their relationship with Project staff and the help provided by them (see Sections 5.6, 6.4 and A1.3). It is, nevertheless, fair to acknowledge that a significant proportion (30%) of service users withdrew from support programmes and that some others will have reverted to unhealthy lifestyles and behaviour following 'successful' case closure. Hence, there was an appreciable proportion of service users whose needs Projects were unable to fully address. As well as drug abuse, mental ill health tended to be one of the more intractable issues (see Table 6.6(c)).
(k). To what extent do former Project service users continue to require support?
8.16 Although it is difficult to quantify this in precise terms it seems safe to say that a majority of former service users are likely to require some continuing support following case closure, at least for a time. Such needs might be limited to the kind of help available via the Projects themselves. For some, however, ongoing support needs will relate to services properly provided by other agencies. Such help might appropriately include help of an even more intensive kind than the ' IFSP product'- e.g. residential drug treatment. A group singled out as likely to need significant post-Project support on a lifelong basis were families including individuals with severe learning disabilities (see Section 6.4).
(l). What is the impact of Project support on service users' awareness of their problematic behaviours?
8.17 Most interviewees accepted that the behaviour which had triggered their referral to the Project was problematic and 'antisocial'. There was evidence of family members' awareness of the impact of their behaviour developing as a result of caseworker action (see Section 5.5).
(m). What is the impact of Project support on service users in terms of (i) reducing their anti-social behaviour, (ii) improving their housing circumstances, (iii) improving their family functioning
8.18 At least for the period that families were receiving Project support, reports of antisocial behaviour were almost always reduced and often eliminated. Although not all families exiting Project support succeeded in sustaining improved lifestyles and behaviour, recurrence of ASB appeared to be unusual (see Table 6.4 and Section 6.4).
8.19 Again, the vast majority of families receiving Project support saw their housing situation improve in that risk of eviction receded (see Table 6.3). In some cases, Project assistance included help in making housemoves to more suitable homes or homes in more suitable areas. Project help could also contribute to better housing conditions through relieving overcrowding by helping children over 16 access tenancies of their own (see Section 4.3).
8.20 It was clear that, rather than focusing exclusively on suppressing antisocial behaviour in the immediate term, caseworkers concentrated on identifying and helping families to deal with underlying problems - especially problematic family dynamics. One measure of success here is that (as judged by caseworkers) Projects succeeded in reducing the risk of family breakup in 50% of cases (see Table 6.5). Nevertheless, 'fixing' such problems often presented major challenges and sometimes Project support actually triggered 'family breakup' where the additional scrutiny brought to bear on a family revealed child protection concerns resulting in children having to be accommodated by the local authority (see Section 4.3 in relation to core block provision).
(n). To what extent have Projects fulfilled stakeholder agency expectations?
8.21 In terms of the social work and housing managers who collaborated in setting them up, Projects' achievements were universally recognised and celebrated. In general other stakeholders were also positive about their contribution. In some instances, however, not all local agencies responsible for tackling anti-social behaviour had been closely involved in the establishment, governance or operation of the Projects (in the Dundee case, this simply reflects the fact that the ASB team, as it existed at the time of the research, had not been created at the time the Project was established in 1996. See also Section 2.7).
(o). If 'positive outcomes' are achieved, how sustainable are these?
8.22 It is evidently the case that a proportion of families assisted by the Projects find it difficult to sustain improvements in lifestyles and behaviour achieved with Project support. This is even true among those completing support programmes. However, it would appear that - at least in the short to medium term - the majority of families exiting from the Projects succeed in sustaining their gains (see Section 6.4).
(p). What are the relative impacts of core block and outreach service provision?
8.23 This is a difficult question to answer. As discussed in Section 2.5, South Lanarkshire was explicitly committed to an 'outreach only' model and would not have aspired to establish a core block, irrespective of the resource issues involved. Aberdeen and Dundee, by contrast, saw core block provision as essential and the other BtC Projects saw such provision as desirable, if sadly unaffordable.
8.24 However, it would be difficult to conceive of 'evidence' which could prove or disprove the contention that core block provision is necessary. At this stage all that can be said is that none of the BtC Projects reported having 'turned down' potential referrals because the gravity of their support needs was such that these could have been addressed only via core block provision. In the longer term, perhaps, it might become apparent that the average duration of cases needs to be greater under an 'outreach only' model, or the 'failure rate' associated with the latter might turn out to be higher.
(q). What are the local attitudes towards core blocks?
8.25 Although this question was not addressed in any depth, there was no suggestion from the Project staff in either Aberdeen or Dundee that local hostility to the siting of core blocks was a current issue. It is believed that, at the outset (circa 1995), such opposition had arisen in Dundee. Seeking to learn from this example, there had been extensive consultation with local people in the neighbourhood where the Aberdeen Families Project planned to site its own core block in 2005/06. This was seen as having been highly successful in addressing neighbourhood concerns, and therefore eliciting local compliance.
(r). Do the projects represent value for money?
8.26 The Projects are deemed to represent value for money if they are cost-effective. Analysis of the costs is relatively straightforward and shows that the 'steady state' average cost per family month is likely to be in the region of £1,300 - £1,900, with lower values for Projects with larger caseloads. If a Project works with a family for a period of 12 months, the average cost per closed case will be about £15,000 - £23,000. If the Project is working with an average caseload of 20 families at a time, it will cost about £360,000 per year to deliver.
8.27 The outcomes data show that many families have achieved positive outcomes that will have reduced their needs for other services and interventions and improved their life chances. Many interventions relating to looked after children and youth justice are very expensive and can cost £50,000 - £100,000 over a year. In relatively crude terms, if the value of the financial savings resulting from a Project exceeds its costs to Exchequer-funded services, it is cost-effective. However, many of the benefits are 'quality of life' gains which cannot be quantified in monetary terms (e.g. improved family functioning). Some of them will be enjoyed for many years (e.g. better employment prospects due to improved school attendance) and even by future generations (e.g. children's improved future parenting skills).
8.28 Some Projects will have resulting in additional expenditure; for example, where dependency problems or child protection issues become apparent within a family only when it is exposed to the scrutiny which is part and parcel of Project support. However, addressing these should reduce future expenditure by the Exchequer and improve the family's prospects.
Having considered the outcomes achieved to date and the costs of key services that might have otherwise been required, the conclusion is that the Projects may be cost-effective in the short run. The extent of their overall cost-effectiveness depends on the extent to which benefits are realised and the timescale under consideration. However, it may not require many positive outcomes for the Projects' benefits to outweigh their costs.
(s). How do core block and outreach services compare in terms of value for money?
8.29 The two Projects with core blocks (Dundee and Aberdeen) have lower unit costs than those without such accommodation (though this may in part reflect the relative maturity of the Dundee and Aberdeen Projects). Furthermore, although the evidence to date is limited due to the small number of closed cases, it suggests that the families placed into a core unit may not necessarily require a longer overall duration of contact with the project than is needed for some families receiving outreach services.
8.30 Because their core accommodation is seen as an integral part of their service, these Projects do not have separate accounts for core and outreach services. Therefore, as it has not been possible to identify either the costs or the outcomes associated with the core units, no definitive conclusions can be made about their cost-effectiveness. However, managers of Projects with core blocks are convinced that they can accept and work with particularly complex families who need more intensive support and supervision than could be provided by an outreach service. It should be noted though that core blocks need to be used effectively and to be large enough (e.g. with accommodation for three families) to enjoy economies of scale enabling them to provide value for money.
8.3 Concluding reflections
8.31 The intensive family support projects examined in this report had been charged with a highly challenging task. Similarly, as a research undertaking, arriving at definitive judgements about Projects' efficiency and effectiveness has not surprisingly proved a complex undertaking. It can, however, be stated with confidence that the Projects have engaged - and in most cases achieved immediate positive impacts - with some of the country's most vulnerable and troubled families. And, although the evidence as yet available is limited, it also appears that in the majority of cases, improved lifestyles and behaviour achieved with Project support have tended to be maintained at least in the months immediately following case closure. The extent to which such gains are sustained and built on over the longer term is a matter for further research.