Child poverty pathfinders in Dundee and Glasgow: phase two evaluation - summary report

Introduction

This report is a summary of key findings and learning from the phase 2 evaluation of the child poverty pathfinders in Dundee and Glasgow. These pathfinders were developed in line with the Scottish Government’s second Tackling Child Poverty Delivery Plan (March 2022). This set out an intention to invest and work with pathfinder partners to “refine, test, adapt and scale different approaches to provide person-centred solutions”, with the aim of generating evidence to “ensure we are making the most of our investments and assets to have the maximum impact on child poverty.” A separate full report includes more detail on all the findings. The evaluation was commissioned by the Scottish Government and conducted by Ipsos, an independent research company.

About the Dundee and Glasgow pathfinders

The two pathfinders differ significantly in scale, core partners and design.

Dundee pathfinder

The Dundee pathfinder was established in 2022 as a collaboration between Dundee City Council (DCC), the Department for Work and Pensions (DWP), the Scottish Government, Social Security Scotland, and various local partners. At the heart of the Dundee pathfinder's approach is a place-based, relational key worker model, emphasising the tailoring of support to the unique needs of each family. Other key features include:

• Targeted outreach via door-knocking in the Linlathen and Mid Craigie areas of the city, initially using Council Tax Reduction data to identify households with children and no earned income from employment, and
• Weekly multi-agency drop-in sessions at the Brooksbank Community Centre in Linlathen. Sessions on a Tuesday are open to anyone, whether or not they have children or live in the target areas. Thursday sessions are specifically for Linlathen and Mid Craigie households.

The pathfinder is overseen by a strategic Oversight Board, comprising senior staff from the four core partner organisations contributing resources to the pathfinder. Wider partners with key roles in supporting families in poverty locally include the Linlathen Fairness Initiative, the employability service, the Health and Social Care Partnership, Community Planning, the third sector and the Early Adopter Community childcare project. Close links are maintained across these services to jointly support and identify opportunities for families.

The evaluation focused on the key worker model in Linlathen, which began in October 2022, and which was the core element of the pathfinder during this period. Following consideration of early findings by the Oversight Board, the approach was subsequently expanded to different areas of the city, which are not covered by this evaluation.

Glasgow pathfinder

The Glasgow pathfinder also began in 2022 and has influencing and embedding “whole system change” as its main focus. “Systems change” has been described by New Philanthropy Capital as:

“[aiming] to bring about lasting change by altering underlying structures and supporting mechanisms which make the system operate in a particular way. These can include policies, routines, relationships, resources, power structures and values.”^[1]

A key feature of the Glasgow pathfinder has been the establishment of a multi-agency change team (MACT), comprising staff with a range of backgrounds and skills to drive forward system change. The embedding of a ‘no wrong door’ model in Glasgow, so that citizens (including families with children) can access person-centred, relational, holistic case management and support regardless of when, where and how they engage with services, is also central to the vision of the Glasgow pathfinder.

The Glasgow pathfinder is much bigger in scale and scope than the Dundee pathfinder – including the fact that it is open to anyone across Glasgow. This is reflected in the number of interlinked workstreams it includes, initially six and now nine, each with its own priority aims and activities. The pathfinder, now known as the ‘Glasgow Child Poverty Programme’, has agreed a new plan with the Scottish Government covering 2024-2027, which builds on learning from the first ‘phase’ of the pathfinder and introduces a number of new elements, including place-based ‘Demonstrations of Change’ in wards identified as having high numbers of children at risk of poverty.

Evaluation aims and methods

The main aims of this phase 2 evaluation^[2] of the pathfinders were to:

• Understand the short to medium term impacts of the child poverty pathfinders for families
• Examine how the pathfinders are working and how and whether they are leading to any systems change
• Understand the value for money of the pathfinders
• Collate information and distil learning to inform the development, further implementation and expansion of the pathfinders in the current sites as well as in other local authorities.

The phase 2 evaluation took a mixed method approach, combining data from primary data collection (interviews with parents and professionals, and workshops with professional stakeholders) and analysis of secondary data (documentation, cost data, and monitoring data). Theories of change, and hypotheses for how the pathfinders were expected to impact on families and systems, were developed for each pathfinder. These were used to structure data collection and analysis.

The evaluation also included a feasibility assessment for a Quasi-Experimental^[3] (QED) pilot study, in Dundee only. This would provide a more robust assessment of medium-term impacts on employment and income, by using administrative data to compare outcomes for pathfinder clients with those for a control group (i.e. a group who is as similar as possible to the beneficiaries, except for having not participated in the intervention being assessed.) A final decision on whether or not to go ahead with the QED pilot was still pending at the time of writing, depending on the outcome of negotiations over data access.

Scope and limitations

There were a number of significant challenges around conducting a comprehensive and robust impact evaluation of the two pathfinders. These included:

• The evolving nature of the pathfinders, which meant that individuals might receive different interventions if they engaged at different times.
• Expected timelines for impacts. There was uncertainty about timelines for impacts in both Dundee and Glasgow, and a clear view that the timing of this evaluation was too early to assess the full impact.
• Balancing the needs and timing of the evaluation with the impact on the pathfinder teams was difficult at times, particularly in Glasgow where the phase 2 evaluation began at a key transitional point when the team needed to focus on finalising 2024-27 plans.
• Limits to the availability of, and access to, quantitative data. The evaluation draws on monitoring data collected by the Dundee pathfinder team and, in Glasgow, by Glasgow Helps^[4]. While these data provided useful insights, there were some significant limitations in terms of coverage around client characteristics and outcomes. The evaluation also lacked data for a control group, which is required for confident attribution of outcomes (which is why the evaluation also included the feasibility assessment for a QED pilot study).
• The large scope and scale of the Glasgow pathfinder also meant that the evaluation could not explore every element of it in detail. As such, the evaluation of the Glasgow pathfinder has adopted more of a ‘case study’ approach, focusing on specific elements of the pathfinder and drawing out learning from these, while recognising that the pathfinder is expected to take longer to fully realise its aims. For Dundee, the aim remained to deliver, as far as possible, a theory-based impact and process evaluation of the place-based key worker intervention, while acknowledging the limits of the data (particularly quantitative data) available to support definitive conclusions.

The caveats listed above should be borne in mind throughout the report. It is also worth noting that the evaluation necessarily reflects a point in time for these evolving programmes (see Table 1.2 in the full report for details of fieldwork timings) and they have continued to evolve subsequently. The differences in the pathfinders themselves, and in the evaluation approach to each, means it is not appropriate to compare findings between the two directly. As such, findings are discussed first for Dundee, then for Glasgow, while the final section of the report draws together cross-cutting themes and learning from across the two.