Child poverty pathfinders in Dundee and Glasgow: phase two evaluation

6. Value for money: Dundee

Introduction

This final Dundee findings chapter focuses on the value for money of the pathfinder.

Data and limitations

Initially, it was hoped that questions on value for money could be addressed using Social Cost Benefit Analysis (SCBA), whereby the benefits of an intervention are monetised and assessed against their known costs to produce estimates of net financial benefits and cost-benefit ratios. However, the lack of comprehensive data on all relevant outcomes from before and after clients engaged with the pathfinder and the lack of a control group, discussed in earlier chapters, means that it is not possible to robustly quantify the benefits of the pathfinder in the manner required for a SCBA. The uncertain timeframe for outcomes (particularly those relating to employment) also meant that some key stakeholders considered it too soon for the evaluation to fully assess the costs and benefits. While a potential future assessment of impacts on employment and benefits using a Quasi-Experimental Design (discussed in chapter 1) may partially fill this gap, it would not cover all relevant outcomes, and findings would not be available before 2026.

Given these issues, it was agreed with the Scottish Government that the evaluation would instead take a two-pronged approach to considering value for money of the Dundee pathfinder: 1) using the ‘4Es’ as a conceptual framework to consider qualitative evidence on value for money, and 2) break-even modelling. The value for money consideration is focused on the key worker model in Linlathen rather than any wider system change.

The ‘4Es’ provides a conceptual framework covering:

• Economy – the extent to which an intervention (in this case, the Dundee pathfinder) was provided at the minimum appropriate cost to the public sector needed to achieve its objectives.
• Efficiency – how efficiently the pathfinder was delivered – could any elements have been delivered differently for the same or better impacts?
• Equity – did the pathfinder meet the needs of all relevant groups, and in particular did it lead to resources being better targeted on those who most need them (i.e. families at most risk of poverty)?
• Effectiveness – to what extent did the pathfinder achieve its aims?

In examining economy, efficiency and equity, this chapter draws primarily on qualitative evidence from stakeholder interviews and a cost workshop held by Ipsos with stakeholders from Dundee City Council, DWP, Scottish Government and Social Security Scotland in late 2024.

As it is not possible to fully assess effectiveness, due to limited data on outcomes, we have supplemented the qualitative assessment with a break-even analysis which looks at what outcomes would need to be achieved in order for it to break even.Break-even modelling estimates what level of outcomes would need to be achieved by the Dundee pathfinder, or by future similar projects, in order for it to ‘break-even’ over a particular timeframe given its costs. Break-even modelling involves identifying a number of key target outcomes that are anticipated from the initiative (usually no more than four) and then using existing data (generally from recognised data sources, such as large-scale UK social surveys) to estimate a monetary value for achieving each outcome for one person. It then combines these values with the known costs of the intervention, to produce estimates of how many people the intervention would need to achieve each outcome for, over the timeframe covered by those costs, in order for it to ‘break-even’ (i.e. for the value of the benefits to equal the costs) given the costs incurred.

Unlike SCBA, break-even analysis stops short of assessing whether or not these break-even ‘thresholds’ have actually been reached by the intervention for any given outcome. However, the chapter does draw on the monitoring and qualitative data reported in earlier chapters to reflect on whether it is plausible the Dundee pathfinder has the potential to achieve this level of outcomes for families, while acknowledging that any such reflections must be tentative given limitations on available data.

How much does the Dundee pathfinder cost to deliver?

The Dundee pathfinder has primarily been funded by the Scottish Government, through funding from the Whole Family Wellbeing Fund and Place-based Social Justice funding streams. In addition, direct staff costs for a DWP key worker working two days a week on the pathfinder and a Social Security Scotland key worker working full time are covered by those partner organisations.^[16] The main estimated direct costs of developing and delivering the Dundee pathfinder in Linlathen and Mid Craigie over the period October 2022 to September 2024 (the main period covered by the monitoring data included in this evaluation), based on figures provided to the evaluators by these three organisations, are shown below.

Table 6.1: Cost estimates for delivery of the Dundee Linlathen and Mid Craigie pathfinder, October 2022 to September 2024

Period covered	Total from Scottish Government grants (WFWF and PBSJ) and partner salary costs (Key workers)
2022/23 Financial year (delivery in Linlathen started October 2022 to March 2023)	£222,160
2023/24 financial year (covering delivery from April 2023-March 2024)	£232,076
2024/25 financial year (partial)[17] – covering estimated costs April 2024 to September	£189,881
Total	£644,117

The costs shown in this table include

• Programme management and staff costs – for 2022/23 and 2023/24, these account for around three quarters of the costs.
• Other costs covered by the Scottish Government grant for 2022/23 and 2023/24 include:
  • Around £117,000 of funding for developmental costs, including: for external organisations to develop a service design ‘blueprint’ to inform early planning and implementation; to deliver values-based leadership training; to facilitate a series of ‘sprints’ to support thinking about how the pathfinder should develop; and to develop and deliver the Parents’ Champion Place-based academy (a ‘return to work’ programme trialled with a small cohort of pathfinder clients in August 2024)
  • Around £4,000 in training costs (to an external provider), covering training for the key workers and other partners, and
  • Around £2,000 for purchase in early 2024 of a tool (‘Outcomes Star’) to help support collection of ‘softer’ indicators of progress (as discussed in chapter 2).^[18]

The costs of external evaluation, including both this phase 2 evaluation and earlier phase 1 evaluability and early evaluation work, were excluded, as these are considered to be costs that would not necessarily be incurred in future were the initiative replicated.

Are there any hidden or additional costs?

Pathfinder partners identified a number of additional direct and indirect costs which they felt it was important to recognise, but which it was not feasible to give an accurate monetary estimate for. These included:

• Management time across the partners: While DWP and Social Security Scotland were able to give estimates for their key workers’ time, they noted that other staff at various levels had also contributed time to the pathfinder, via the Working Group and Oversight Board. The same applies to DCC. The amount of time spent was so variable it was seen as impossible to provide a realistic estimate.

There was some debate among stakeholders over whether this management time should or should not be considered as a ‘direct’ cost of delivering the pathfinder; one view was that the types of input required on the Working Group or Oversight Board are part of the role of managers and should not be attributed specifically to the pathfinder. There was also a view that some of this time reflected the fact that it was a ‘pathfinder’, and as such might require more senior time to support development, innovation and testing. The same level of management and strategic time input from partners might not be needed if the pathfinder model became mainstreamed. However, other stakeholders felt it was important to recognise the significant additional time contributed by the four partners, over and above the costs of providing key workers/operational staff.

• Administrative support costs: It was noted that a significant amount of administrative support had been provided to the pathfinder by the Scottish Government as DCC had not yet been able to recruit someone in this role to support the pathfinder. This cost ‘in kind’ was not reflected in the grant costs shown in Table 6.1 but had involved significant resource. However, again, as some of this was attached to the Oversight Board and Working Group, it is possible the same level of administrative resource would not be required if the pathfinder approach became mainstreamed.
• Costs of other partners’ time for attending the drop-in: As discussed in chapter 2, a range of other Council, NHS and third sector services provide staff to attend the Tuesday drop-in sessions. The costs of these partners’ time in attending Brooksbank are not captured in Table 6.1.

However, again, it is not clear that this is a fully additional cost, since these organisations would otherwise be providing similar support and advice to other clients. One view might be that the drop-in is just a different vehicle for them to offer their services to those who need it. At the same time, it was noted that the frequency and level of staffing support offered to the Linlathen drop-in was higher than it would have been if the pathfinder had not been a priority across partners. As such, arguably there have been some additional costs to partners who committed staff time to pathfinder schemes (while still retaining their existing obligations) – even if this has also created opportunities to support clients in a different way or to reach clients they might not otherwise have reached (as discussed in Chapter 3).

• Costs of wider investment in Linlathen. Stakeholders reported that there had been a wider public and third sector focus on Linlathen in recent years, including through the Linlathen Fairness Initiative and the Early Adopter Community School-aged childcare pilot at the local primary school. Again, while these were not direct costs to the pathfinder, investment in wider local services and activities were felt to be enhancing its impact, by providing more support to link families to.
• Venue costs. The Brooksbank centre had initially hosted the pathfinder without charge, although latterly use of the centre had been paid for through the Scottish Government grant.

Stakeholder views on economy, efficiency and equity

As discussed above, the 4Es framework provides a structure for considering value for money in a rounded manner. This section uses this structure to explore professional stakeholders’ views on the first three of the 4Es: economy, efficiency, and equity.

It is important to note at the outset that stakeholders’ views on economy, efficiency and equity are extremely difficult to disentangle from their views on the overall aims and effectiveness of the pathfinder and on the quality of data available to measure this. A recurrent view among stakeholders was that it was very difficult to answer questions on these issues given the iterative approach of the pathfinder, its evolving aims, and the challenges around the collection of data to quantify outcomes, discussed in previous chapters. Arriving at a shared understanding of appropriate timelines for demonstrating value for money was also challenging. While some stakeholders wanted to see more quantifiable evidence of progress at this stage of the pathfinder, others felt it was too early to assess value for money in a reliable way, especially since they felt some impacts (e.g. on employment for those at an early stage of their journey) might not even materialise until after the lifetime of the pathfinder.

A final general point was that, as a ‘pathfinder’, the Dundee model was expected to adapt and evolve in response to new learning. It was suggested that when a programme is being implemented in this ‘test environment’, it is harder to say whether or not it could have been delivered with less resource or more efficiently, since the model itself is not yet fixed.

“As a learning exercise, it feels like it was sort of a good spend. And then in terms of learnings, there are still some questions around best use of staff time and resources and things to be further figured out, I suppose.”

(Dundee professional interview)

Differing views on timelines and on what should be expected in terms of value for money from a ‘pathfinder’ were also associated with disquiet among some professional stakeholders around the appropriateness of considering or modelling value for money at this point in the Dundee pathfinder’s development. At the same time, there were also interviewees who were actively looking for greater detail on value for money than they felt had been available to date.

Overall, interviews with professional stakeholders highlighted the challenges around achieving consensus across partners around what ‘value for money’ means for a project that has multiple aims, some of which may be much longer-term, particularly at this ‘pathfinder’ stage. These challenges have also been reflected in the difficulties the evaluation team have faced in operationalising and assessing value for money on the Dundee pathfinder. While this chapter aims to provide additional insight, it must be recognised that there is no single agreed definition of what ‘value for money’ means for a project like the Dundee pathfinder, and that people’s interpretation of the findings presented here will be shaped by their own perspectives on this.

Economy

The economy of a project or policy relates to whether its objectives were achieved at the minimum appropriate cost to the public purse. The first issue to note here is that while economy is not simply about total cost (it is about whether the cost is appropriate to the objectives achieved), for some pathfinder stakeholders affordability per se was a significant concern. Competing views on the overall cost and affordability of the pathfinder model were expressed by stakeholders interviewed for this evaluation. One view was that the pathfinder has been relatively low cost for what it has delivered:

"… we've got … a couple of key workers, but you'd be surprised how little actual additional funding went into that area (Linlathen). It was just about using what was in there, you know, like working with (the) third sector. We were lucky with Brooksbank and that, you know, but that was just all about partnership work".

(Dundee professional interview)

However, another was that place-based key worker models in general are too expensive to be affordable at scale, particularly in a context of significant pressures on public spending:

“I think there's been previous iterations of this for different organisations. And the cost has always been the reason why we've pulled back on working so closely in communities because of the number of people and amount of time. And that's why self-service, et cetera, was introduced, was to reduce costs."

(Dundee professional interview)

A related concern was that continuing to commit similar levels of resource to the pathfinder would be challenging for some partner organisations. In particular, if ‘scaling up’ involved core partners providing key workers for additional outreach and delivery teams in other areas, it was suggested that it would be very difficult to find the staff resources. Suggestions for reducing the resources required included making the intervention more timebound and/or adopting a ‘mobile’ key worker team model, where a team provide additional support in different geographic areas for a fixed period of time before moving on to other areas.

These general views on affordability were reflected in professional stakeholders’ views on the costs of each element of the pathfinder. In terms of the overall key worker team size (3.83 full-time equivalent), it was argued that the team would not have been able to deliver the level and type of support they had offered with a smaller team. In particular, it was noted that targeted door-knocking requires key workers to double-up for safety reasons, so would have been difficult to deliver with fewer staff. However, as noted there was concern about whether all partners would be able to afford to commit similar levels of key worker resource if the model is scaled up.

On targeted outreach specifically, different views were expressed on whether key workers should continue to adopt a flexible approach to the number of calls they make, or whether this ought to be capped from an economy or efficiency perspective. One view was that the pathfinder has demonstrated that a flexible approach is what is required to engage families that are not currently known to services, since the team have been able to support families who only started engaging after they knocked on their door for the sixth or seventh time. However, other stakeholders, while recognising the benefits of the outreach approach in terms of engaging families, again raised concerns about affordability.

There was a desire for greater evidence on both the optimal number of calls given outcomes and for other approaches to reaching people (e.g. writing to some, rotating locations at which drop-in support is offered, coordinating outreach with other local partners) to be more directly tested alongside door-knocking, so that the most economic approaches can be used for different groups. As discussed in Chapter 3, the monitoring data available to the evaluation team does not capture how many clients were engaged via targeted door-knocking or how many initial calls were made before they engaged.

The frequency and level of resource allocated to the drop-in was also discussed. Although not all interviewees felt able to comment on this, the operational team and partners who attended the Tuesday drop-in felt this was resourced at about the right level given ongoing demand, which although it fluctuated, had not obviously tailed off by late 2024 (see data on attendance in chapter 3).

“So, although there is a lot of resource in the room and it almost begs the question, could it get scaled back? Everyone's busy or everyone's always got clients in. … It's been used.”

(Dundee professional interview)

There was less certainty over whether the Thursday drop-in (which is only available to Linlathen and Mid Craigie clients) was adding sufficient value to justify allocating a key worker to this. Average attendance on a Thursday was lower (a mean of 3 attendees compared with 24 on a Tuesday). In terms of the future affordability and feasibility of delivering drop-ins, the potential closure of community centres was also identified as a risk. As of late 2024, Dundee City Council was consulting on relocating services from four community facilities to a single East End campus.

Other elements of the Dundee pathfinder that stakeholders felt could potentially have been delivered more economically included the service design and development workshops and evaluation and monitoring activities. There was a perception that partners’ internal resources might have been used more effectively to support some of the ongoing design and development thinking at a lower cost.

Evaluation and monitoring activities were felt to have taken up a lot of the operational team’s time, reducing their capacity for other activities. This was partly attributed to a lack of a clear, agreed specification for what should be captured (and how) at the outset and a lack of an appropriate case management or monitoring system to record this in (as discussed in Chapter 2). More broadly, it was suggested that an administrative support role might have helped free the key worker team to focus on direct support to clients.

Efficiency

Perceived barriers or challenges to delivering the pathfinder as efficiently as possible included challenges relating to data sharing, the range of partners involved in the planning stages, and aspects of the management and oversight of the pathfinder. These are discussed further below.

Data sharing has already been discussed in detail in chapter 5. In terms of its impact on value for money, the inability of key workers from different partner organisations to share data electronically (or at all) with each other was seen as creating inefficiencies and duplication in delivery, since they could not easily check what another agency had already done for a client:

“It feels like you're double working because two of you are having to do the same thing."

(Dundee professional interview)

The impact of data sharing limitations on the perceived efficiency and effectiveness of monitoring of outcomes has already been discussed.

Challenges around the speed of inception of the pathfinder have also been discussed in the previous chapter. In addition, there was a belief that had a wider range of professionals, including a wider range of council and the third sector employability staff, been involved in discussions about the pathfinder at the outset, it would have been better linked to existing support and plans, which would have saved time and resource.

Issues relating to administration and management have also been covered in chapter 5. In particular, a perceived lack of clarity at times around roles and leadership, and whether the governance structure had been quite right, were seen as having created some inefficiencies. Based on learning from these perceived challenges in Dundee, it was reported that for the new Fairer Future Partnerships, local authorities were taking a strong lead through Community Planning Partnerships. The intention was that the Scottish Government role would be more focused on convening stakeholders and pulling lessons from local projects into the national arena, rather than detailed operational or management support.

Another aspect of efficiency is whether the delivery of an intervention leads to greater efficiency in the wider system of services working with the target group. Elements of this were covered in Chapter 5. As discussed, there was a belief across stakeholders that working relationships between services attending the drop-in had become more efficient. One specific example cited was that Brooksbank, Welfare Rights services, and Citizens Advice Scotland all being based together at the drop-in had led to better linking on local advice strategy and how to work together. Co-location at the drop-in was also felt to improve the efficiency of referrals by facilitating ‘warm handovers’ of clients between services.

In terms of whether the pathfinder was providing a more efficient way of reaching clients who need support, there was a shared perception that the pathfinder was probably helping partner services, as well as the key worker team, to reach some clients who do not usually reach out for support. However, interviewees found it impossible to quantify this. It was also noted that reaching those who may not otherwise have contacted services at this point is not necessarily associated with cost savings to the public purse now, though it may save services money in the future (if it prevents problems from escalating to the point where they need more intensive intervention). Again, this highlights uncertainty around the appropriate time frame over which the value for money of the pathfinder should be considered.

More generally, it was suggested that if the drop-in element of the model is being replicated elsewhere, it will be important in terms of maximising efficiency to ensure that the IT available in the venue can support drop-in partners to be able to continue with other aspects of their day job (e.g. contacting other clients), since demand for each drop-in partner can vary.

Equity

Two issues are particularly relevant in considering the equity impacts of the Dundee pathfinder: the geographic focus on Linlathen and Mid Craigie and the demographic profile of those reached by the pathfinder.

As described in chapter 2, Linlathen was selected for the pathfinder on the basis of being one of the most deprived areas of Dundee. Of the seven data zones within Linlathen and Mid Craigie, five fall into the most deprived fifth of data zones in Scotland (SIMD 1, as measured by the Scottish Index of Multiple Deprivation 2020), with one appearing in the 10 most deprived data zones (out of 6,976 data zones in Scotland).

One view among professional stakeholders interviewed for this evaluation was that the intensive geographic focus of the pathfinder (complemented by other investments in the area, including LFI and EAC) meant the pathfinder was having a significant positive impact on equity:

"It's amazing when you look at the amount of money that's been put into that area … and when you see the families and they look healthier and they're eating healthier, and even if they're not working even if they're going to volunteer now, or they're even just taking their kids and doing nice fun things with them, I know it really does make a difference"

(Dundee professional interview)

This view was supported by parents’ accounts of the impact the pathfinder had for them, and the perception that it was the first time they had felt this type of support had been offered in Linlathen.

However, a counterpoint to this view was that much of the financial support the pathfinder was helping people in Linlathen and Mid Craigie to access (such as food bank vouchers, help with fuel bills, help with furniture or children’s clothes) came from a ‘fixed pot’ which had not increased, and which would arguably have been distributed anyway, even if the pathfinder had not existed. As such, it was argued that the pathfinder might simply be redistributing support from those who would qualify for this type of support across other parts of Dundee to Linlathen and Mid Craigie specifically (although no specific evidence that it had led to people in other areas being less able to access support in practice was provided).

At the same time, it was argued that by making support more accessible to people in a particularly deprived area who had not been aware of it, or who had faced barriers to taking up support, the pathfinder was having a positive impact on equity even if it was not necessarily providing ‘additional’ net financial benefit when considered at the level of Dundee as a whole. (Evaluating this potential equity impact would require analysis of data on who had received the funding previously and how this had changed, which was not available to the evaluation.)

In terms of the demographic profile of pathfinder clients, the ‘reach’ of the pathfinder has already been discussed in chapter 3. Professionals raised a number of different points in relation to whether they felt the pathfinder was reaching the ‘right’ people from an equity perspective (i.e. those who most need support). First, as discussed in chapter 3, the use of Council Tax Reduction data to identify families with children and no earned income in Linlathen and Mid Craigie was generally felt to be a novel and effective way of targeting this group. However, as discussed above, there was a desire for further evidence on which approaches to targeting offers best value for money, including from an equity perspective.

Views on whether the overall pathfinder approach had enabled it to focus on those most in need of support were more mixed. One view was that the level of demand from outside the ‘core’ target group (people with children in Linlathen and Mid Craigie) illustrated the level of need for this kind of support and that from an equity perspective, it was right to keep the pathfinder offer open to all, in line with ‘no wrong door’ principles. Indeed, it was suggested that it might have been easier and have enabled the pathfinder to reach more people in need of support simply to have called at all addresses in Linlathen and Mid Craigie, rather than targeting. However, as discussed previously, the alternative view was that the ‘no wrong door’ approach had diluted the capacity of the team to work more intensively with the core target group.

Several interviewees also noted that there had been a point where the drop-in was ‘inundated’ with people wanting fuel vouchers, including people they felt were not those who most needed them. This had led the pathfinder to introduce a new process to ensure better targeting of fuel vouchers. While this specific issue was felt to have been resolved, there remained some uncertainty about the value of maintaining the outreach and the drop-in at the same level over a longer period in terms of either reaching more of those most in need of support, or further impact on families most in need of support.

The monitoring data indicates that, of the 266 Linlathen and Mid Craigie families with dependent children who had engaged with the pathfinder at some point, 67 (25%) had some contact with the pathfinder in the three most recent months included in the monitoring data. While these families may, of course, have been continuing to receive valuable help, these figures arguably provide some support for the view that there may not be a sufficiently large pool of families in need of ongoing support to warrant continuing with the same level of investment going forward, particularly if the focus is on families with children. This has, in part, shaped the plans for the future development of the Pathfinder (described in chapter 2), which proposed a reduction in the level of key worker resource allocated to Linlathen and Mid Craigie.

What would ‘break-even’ look like for this kind of project?

The final section of this chapter presents the results of break-even analysis and modelling for the Dundee pathfinder. As described above, break-even analysis seeks to estimate the level (or threshold) at which the pathfinder (or a similar project) would ‘break-even’ (i.e. the value of its benefits equal its costs). This is based on combining its costs (as set out above) with the monetary value of key outcomes based on large national datasets. The steps involved are summarised in box 6.1.

Break-even analysis: the methodology

Full technical details on the break-even analysis conducted for this evaluation are included in Appendix D. This includes a detailed description of the outcome indicators and approach to monetising each outcome, including detailed assumptions. This section provides a simpler summary. It takes each outcome in turn, explains very briefly why it is included, summarises the data sources used to estimate the monetary value, presents the ‘break-even’ estimates, and reflects on whether there is any evidence that the pathfinder might have plausibly achieved these impacts based on monitoring data to date.

Box 6.1: steps in break-even analysis

1. Estimating the costs of delivery over a specified period – these costs are set out in Table 6.1, above. Depending on whether the costs of design and development workshops are included or not, this gives a lower total estimated cost of £527,117 or an upper total of £644,117. The lower and upper ‘break-even’ thresholds discussed in this section are based on these lower and upper costs.

2. Identifying how the programme is expected to impact and selecting appropriate outcomes – Break-even analysis benefits from being able to consider multiple outcomes, where projects have a range of goals. However, typically, no more than around three or four core outcomes are included in break-even modelling, since more than four risks greater duplication across outcomes. Outcomes also need to be selected that are feasible to monetise, using available national data sources. Drawing on the theory of change for the Dundee pathfinder (see Chapter 2), further discussions with the Scottish Government and the pathfinder partners, and analysis of suitable proxy measures in available national data sources, the outcomes identified for inclusion in the break-even modelling were:

New employment

Financial stability, and

Subjective wellbeing.

3. Mapping each of these outcomes to relevant economic and social outcome variables collected in national datasets. Information based on these datasets is then used to estimate a monetary value associated with experiencing each outcome.

4. Producing ‘break-even’ tables that combine all of the data above, to show how many people would need to achieve each outcome for the pathfinder to ‘break-even’.

Key points to note when interpreting this analysis include:

• The primary approach taken to monetising the value of potential outcomes involves estimating the impact of a change in that outcome on individual social wellbeing, as measured by a question on general life satisfaction. The modelling then applies the WELLBY method, as set out in the Treasury Green Book Supplementary Wellbeing Guidance (2021), to assign a monetary value to that impact. In other words, the break-even analysis presented here views value for money largely through the lens of potential impacts on the subjective wellbeing and quality of life of clients (and not, for example on actual income gains from employment). This is a recognised approach for comparing the costs with the social benefits of a project to its participants and avoiding over-estimating financial gains where (as with employment) these would arguably have gone to someone, with or without the intervention.
• Fiscal savings are not included in this analysis, as it is not clear how they could be quantified from the available data, but it should be noted that there may be potential future cost savings to the Exchequer from the pathfinder approach, for instance, from less use of crisis support.
• The estimates of how many people would need to have achieved a particular outcome in order for the intervention to break even are based on a combination of the estimated value (in £) of achieving this improvement and the estimated costs of delivering the programme over its first two years (aligning with the period covered by the monitoring data included in this report). This chapter has already noted that there are some differences of opinion across pathfinder stakeholders around what should be included in the costs of delivering the pathfinder and what constitutes a realistic timeframe for outcomes being achieved.
• If two years is considered too short a time period to see meaningful impacts for sufficient people in terms of the outcomes discussed, then the pathfinder may be unlikely to break-even over this period. However, it could still, potentially, ‘break-even’ in the longer term if these outcomes emerged for Linlathen and Mid Craigie clients in the future as a result of the pathfinder (although attribution of outcomes beyond the lifetime of an intervention becomes more challenging, and a control group becomes even more important).
• If the QED study does go ahead, and finds greater changes in, for example, job outcomes in 2025 compared with previous years, it may be possible to revise the break-even modelling using information about any costs of the pathfinder over the additional period covered by the QED. However, this would still be looking at short to medium-term impacts – for those who see the pathfinder’s impact as much longer-term than this, this may still be viewed as too short a time period.

Job starts

As has been noted repeatedly already, learning about how far many Linlathen pathfinder clients had to go to move into employment led to the pathfinder moving away from a main focus on employment to a broader range of outcomes. However, the key worker team are all employability workers and it is clear that employment remains a key goal for those clients for whom it is a realistic prospect. As such, it was felt that it was still relevant to consider job starts as one of the outcomes in the break-even analysis. The modelling conducted to estimate a monetary value for each new job start assumes:

• That you cannot, in value for money terms, simply combine the salaries obtained by those who have moved into work (assuming these are known), because in most cases when someone starts a new job, the job would have existed and someone else would have benefited even if the pathfinder client had not been successful. Rather, the added value of the pathfinder is assumed to be in helping channel jobs into an area of higher deprivation, such that there is an additional value attached to each job, over and above the actual salary value (derived through a process known as ‘welfare weighting’ described in Appendix D).
• That each person who starts a new job is moving from unemployment to full-time employment for at least one year, at the living wage. In practice, of course, projects like the pathfinder will help people into a mix of types of job, including some full-time and some part-time, at different wage rates, and some people may be supported to move from an insecure or low paid job into a better or more sustainable job. For modelling purposes, however, the monetary values included below are based on a new full-time, living wage job over one year. The number of jobs required to break-even could be adjusted if reliable information on the actual mix of full and part-time jobs were available.
• Two monetary values for job starts have been estimated. These can be added together to create a total value per new job start. The first part of the value relates to the economic value to the local economy. This is based on data from the What Works Centre and HMT Green Book estimates of ‘welfare weights’. The estimated additional value to the local economy in Linlathen and Mid Craigie of each new job start facilitated by the pathfinder is £3,332. Note that to avoid the potential criticism of over-attribution of value to the pathfinder, a 40% deadweight has been applied to the employment break-even value to account for the fact that comparable employment outcomes could have been achieved without the Pathfinder intervention.
• The second part of the value relates to the impact of starting a new job on social wellbeing / life satisfaction. This comes from the HMT Green Book Supplementary Wellbeing Guidance (HM Treasury, 2021: 62), which is based on analysis of Understanding Society data^[19] to estimate the impact of having a full-time job on life satisfaction (as measured on an 11-point scale) in combination with the WELLBY method to attach a monetary value to this. The estimated WELLBY (social wellbeing) value of each new job start facilitated by the pathfinder is £3,391 (including a 40% deadweight, as above).

Based on total costs of between £527,117 and £644,117, and an estimated combined economy and WELLBY value of £6,723, the pathfinder would need to have supported between 78 and 96 people into full-time jobs, lasting at least one year, at living wage to break-even on this outcome alone.

Improved financial stability

As discussed in previous chapters, in addition to supporting clients around employability the pathfinder has offered wide ranging support and advice around finances, from benefits and debt advice and support, to grants for clothing and furniture, to fuel vouchers and discounts. Improved financial stability is one of the key aims of the pathfinder, as set out in the theory of change.

In order to attach a monetary value to improved financial stability, the break-even analysis takes the Understanding Society question “How well would you say you yourself are managing financially these days?” and compares the wellbeing (as measured by the 11-point life satisfaction question) reported by those who say they are ‘Living comfortably’ or ‘Doing alright’, compared to those who are less financially comfortable (who respond they are ‘Just about getting by’, ‘Finding it quite difficult’, or ‘Finding it very difficult’). It then uses the WELLBY method to estimate the financial value of this difference. The estimated WELLBY value of improving people’s financial stability is £4,372 per person, per year.

Based on total costs over two years of between £527,117 and £644,117, and an estimated WELLBY value of £4,372, the pathfinder would need to move between 121 and 147 people from feeling they are ‘just about getting by’ to feeling they are at least ‘doing alright’ to break-even on this outcome alone.

Improved subjective wellbeing

The holistic approach of the pathfinder has meant that in addition to working with families to improve their financial stability, it also aims to promote improved mental health and wider wellbeing.

For break-even analysis, the HMT Green Book WELLBY value of £11,952 per unit increase in life satisfaction (uprated to 2023-24 prices) can be used to monetise subjective wellbeing. A significant improvement (equivalent to moving one point up an 11-point life-satisfaction scale) has a value of £11,952 per person per year. A smaller increase (0.5 points on an 11-point scale) would have a value of £5,976 per person per year.

Based on total costs over two years of between £527,117 and £644,117, and an estimated social value of an improvement in subjective wellbeing of £5,976 (based on a 0.5 point increase in life satisfaction), the pathfinder would need to improve the subjective wellbeing of between 88 and 108 people at this level to break-even on this outcome alone.

Summary of break-even analysis

Table 6.2, below, summarises the break-even analysis presented above.

Table 6.2: Break-even analysis summary

Outcome indicator	Economic benefits (welfare weighted per person)	Social benefits (WELLBY value per person)	Break-even number: number of outcomes needed for the pathfinder to break even on this outcome alone (upper and lower thresholds)
New jobs facilitated by the pathfinder, assuming person moves from unemployment to full-time employment for at least one year, at living wage	£3,332 in Gross Value Added to the local economy, welfare weighted, annual value	£3,391 WELLBY annual value	78-96
Improved financial stability, assuming moves from ‘just about getting by’ to ‘doing alright’ or ‘living comfortably’	N/A	£4,372 per person	121-147
Improved subjective wellbeing (life satisfaction)	N/A	£5,976 per person per year for a 0.5-point improvement	88-108

Table 6.2 shows how many outcomes of each type would be required for the pathfinder to break even on that outcome alone. In practice, it could break even by achieving a combination of these outcomes – providing they are achieved for different clients to avoid the double counting which may occur because outcomes overlap (for example, improved life satisfaction may be a result of getting a job or improved financial stability).

The pathfinder could break-even over two years based on the lower cost estimate above (£527,117) by achieving either of the following combinations of outcomes for different (non-overlapping) groups of clients:

• 30 people gain new, full-time jobs (£201,690); another 40 report significant improvements in financial stability (£174,880); and a further 26 people report a 0.5-point increase in life satisfaction (£155,376)
• 15 people gain new, full-time jobs (£100,845); another 60 report significant improvements in financial stability (£262,320); and a further 28 people report a 0.5-point increase in life satisfaction (or 56 report a 0.25 point increase) (£167,328).

Break-even analysis: current estimates of pathfinder ‘performance’

Job starts

As discussed at the start of this chapter, unlike SCBA, break-even analysis stops short of assessing whether or not these break-even ‘thresholds’ have actually been reached for any given outcome. However, unless the figures recorded in the monitoring data are a very significant underestimate, it appears the pathfinder is unlikely to have ‘broken-even’ in the short-term on the basis of job outcomes alone. The maximum number of job starts recorded in the monitoring data covering the two year period from October 2022 to early October 2024 is 76 (of which 23 were Linlathen and Mid Craigie clients with dependent children). While the 76 job starts are close to the 78 required for break even, it is not clear how many of these 76 job starts would meet the required assumptions^[20] for it to break even.

As discussed in previous chapters, the target group of families in Dundee had included many more families on benefits with ‘no work-related requirements’ than expected, which created challenges supporting larger numbers into employment. If the pathfinder is conceived as primarily a pre-employability intervention, impacts on employment outcomes may also be seen further in the future for many of the families it has supported, so the pathfinder could potentially break-even on job starts over a longer period of time if more families moved into employment (who would not otherwise have done so, without the support they received from the pathfinder).

Improved financial stability

The monitoring data for the pathfinder shows that a high number of people had received advice and support around financial issues: 52% (n = 664) of all clients and 46% (n = 122) of the ‘core’ target group (Linlathen and Mid Craigie clients with children under 16) are recorded as having been referred for help with benefits, for example. The monitoring data does not include conclusive data on the outcomes of these referrals. If the QED goes ahead, it should be able to estimate the extent to which support from the pathfinder is associated with any increase in income from benefits. However, there will still be a gap in quantitative data around whether any financial gains lead to people feeling subjectively more financially stable.

Qualitative interviews with parents highlighted examples where the pathfinder had clearly impacted positively on families’ levels of debt and ability to manage financially (see chapter 4). However, they also highlighted external limitations on the ability of the pathfinder to help families arrive at a fully sustainable or comfortable point, given the wider context of high costs of living and the level of benefits. As one parent put it when asked how they felt their financial situation had changed since their contact with the pathfinder “I've not gotten any more from the system. I think I'm just managing better than I was two years ago.” There was also a lack of clarity on how far the pathfinder might be expected to increase people’s overall income from benefits specifically, since it was suggested that many of the families the pathfinder was working with in Linlathen and Mid Craige may be quite ‘benefit maximised’ already.

Overall, there is clear qualitative evidence of the pathfinder making a significant positive difference to how families are coping financially. However, is not possible to be sure whether this difference would be sufficient to move significant numbers of families in Linlathen and Mid Craigie to feeling they are ‘doing alright’ rather than ‘just about getting by’, particularly given the limited control the pathfinder has over the wider socio-economic and benefit context.

Improved subjective wellbeing

The monitoring data for the Dundee pathfinder does not include any measures of subjective mental wellbeing or life satisfaction. However, as discussed in chapter 4, qualitative interviews found that parents felt the practical, financial and emotional support they had received made a significant difference to their mental health and wider wellbeing. This tended to be associated with receiving more intensive support over a longer period of time. The key worker approach clearly has the potential to make a positive difference to people’s wellbeing, as well as their financial stability.

What is not possible to establish, given the data available, is how many people are likely to have achieved wellbeing impacts at a level that would register as a 0.5 point increase in life satisfaction on an 11 point scale, or what intensity of support would be required to achieve this. For example, moving from unemployment to employment (which is a very significant change in one’s life circumstances) would be associated with around a 0.5 point improvement^[21], so it might be argued that, for clients who do not move into employment, lower levels of change might be more likely. If before and after measures of life satisfaction were included in future monitoring data, this would enable better assessment of what level of improvement might be expected, and the WELLBY approach could be used to translate this into monetary values.

Other potential outcomes

Other potential outcomes included in initial break-even modelling but not included in this chapter were ‘health and wellbeing benefits from support with heating’, ‘social connectedness’ and ‘improvements in confidence’. Modelling for the former two are included in Appendix D, in case they are useful for considering the value for money of outcomes for future place-based initiatives, but we have not included them in this chapter, for the reasons summarised below.

While the pathfinder has provided a significant amount of support around fuel bills (30% of all clients and 28% of Linlathen / Mid Craigie clients with dependent children had been referred for fuel-related support), monetising the potential health impact from this support^[22] results in a relatively low value (around £43 per household per year), meaning that the number of households that would need to have accessed additional fuel support through the pathfinder to ‘break-even’ would have considerably exceeded the total number of Dundee Pathfinder clients.

‘Social connectedness’ was analysed using data from Understanding Society to look at the difference between agreeing that “I feel like I belong to this neighbourhood” and not agreeing with this statement and overall wellbeing / life satisfaction. As with other measures, this could then be combined with WELLBY model data to estimate a monetary benefit of moving from disagreeing to agreeing with the statement (around £1,657 per person per year). However, although there is some qualitative evidence of impacts from the pathfinder on families’ social connectedness, the strength of local identity and connection as an area was also identified as a pre-existing strength in Linlathen. As such, on reflection it was felt this outcome was less relevant to consider.

The third outcome, improved confidence, was dropped on the basis that the evaluation team were unable to find a relevant external data source that included a question that felt sufficiently relevant to the pathfinder and which could be used to monetise this outcome. Understanding Society includes a measure of whether people feel they have been ‘losing confidence’ recently. Although it could be argued that they are two sides of the same coin, the pathfinder is more focused on helping people to build confidence (sometimes from a very low base) rather than to prevent them ‘losing confidence’. Given this, the break-even modelling has focused on the outcome of improved overall subjective wellbeing instead of improved confidence specifically (including both would arguably have involved too much duplication).

Possible outcome measures for future value for money analysis

If there is a desire to assess the value for money of initiatives using some of the outcomes included in this break-even analysis in future, either in Dundee or for similar projects elsewhere, this would require the collection of more comprehensive before and after data to track clients progress towards these outcomes. An artificial control group could then potentially be created using data from external sources (like Understanding Society) to construct a counterfactual of how outcomes for comparable groups would have changed in the absence of the scheme. This would remove the need for the counterfactual deadweight adjustments used in the current analysis. If it were possible to ask clients about these outcomes, using the measures on which the break-even analysis is based, shortly after they engage with the pathfinder, and then at 6 to 12 month intervals, depending on how long it is expected to take for outcomes to be achieved, this would provide a more robust basis for assessing the initiative’s effectiveness and, by extension, its value for money. Possible questions that could be asked for this purpose are summarised below.

• Outcome: Job Starts
  • Possible question: The precise question asked of clients is less important than accurate recording of work status when someone first engages with an intervention, and then accurately tracking job starts (and length of employment). For those in, or moving into, employment, it is also important to capture hours worked (or at least whether they are full or part-time) and salary level (or at least whether they are paid at or above the living wage).
  • Notes: There are known challenges collecting accurate and complete employment data when a programme is voluntary and open ended. However, if additional administrative resource were available to support key workers in collecting this data, it may be possible to strengthen this. If data sharing issues with DWP and Social Security Scotland can be resolved, it may also be possible to use their data to identify whether people have newly moved into employment
• Outcome: Improved financial stability
  • Possible question: How well would you say you yourself are managing financially these days? Would you say you are...

    Living comfortably, Doing alright, Just about getting by, Finding it quite difficult, Finding it very difficult

  • Source: Understanding Society
• Outcome: Improved subjective wellbeing
  • Possible question: Overall, how satisfied are you with your life nowadays, on a scale of 0 to 10, where 0 is ‘not at all’ and 10 is ‘completely’.
  • Source: Office for National Statistics
  • Notes: This is the question that underpins the WELLBY method. Given this, it could, in principle, be used to estimate the ‘social value’ (in monetary terms) of the impacts of the Pathfinder independently of data on other contributors to this - if a robust quasi experimental design could be developed to isolate the impact of the scheme on life satisfaction, compared to a control group. (If this cannot be achieved, it would be recommended to continue to collect data on other outcomes, to allow for triangulation.)

Wider learning and reflections: Value for money

• Findings for Dundee highlight the challenges around achieving consensus across partners about what ‘value for money’ means for a project that has multiple aims, some of which may be very long-term. Future similar projects should agree what would constitute value for money at the outset, over what timescale, and how this should be measured (taking account of the outcome data that will be available).
• The break-even tool developed for this evaluation shows what scale of outcomes would be needed for these type of projects to break even. The Scottish Government and local partners could make use of this tool to inform the planning of future initiatives (particularly to inform the use of resource and scale of interventions). The outcome indicators identified also provide a guide for the types of data that would need to be collected in a future evaluation to assess value for money more robustly (provided that systematic data, including cost data, is collected in a way that will enable assessment of these outcomes).
• In considering value for money of similar initiatives in the future it is important to continue to reflect on the appropriate timeframes for evidencing value for money for this type of holistic support, particularly where projects are testing and iterating approaches. Including structured stakeholder reflections on an initiative’s economy, efficiency and equity can help ensure a more rounded picture of value for money. It is also important to note that break-even analysis is based on valuing individual client outcomes from an initiative. This makes it less suitable for assessing wider outcomes from system change.