Bringing Hope, Building Futures: Tackling child poverty delivery plan 2026-2031 – annex 3: Evaluation strategy (2026 update)

3. The evaluation approach in detail

The approach to assessing progress towards the Child Poverty (Scotland) Act 2017 continues to be anchored in a process, impact and economic assessment. This three layered approach allows us to understand:

1. What can be learned from how the strategy has been delivered? (Process)

2. What difference has the Act made and for whom? (Impact)

3. Is the Strategy a good use of resources? (Economic)

Process evaluation – what can be learned?

Process evaluations examine activities involved in an intervention’s implementation and the pathways by which the policy was delivered. In the context of child poverty, this is complex, as the different interventions are not distinct actions, but in many cases national cross-cutting policies. Many of these policies have their own strategies and outcomes in place, which may also be accompanied by their own tailored evaluations and governance protocols.

In order to understand whether the Strategy is being implemented as intended, we collate information from across Government to understand whether all of the commitments made in the Plan are on course to be delivered, and assess whether their scale and pace of delivery is on track to contribute towards the child poverty mission. This is reported annually, by June each year, to the Scottish Parliament.

A regular monitoring process takes place throughout the year on a smaller set of those commitments, considered to be most impactful. We collate key delivery data (and where relevant, on benefits and impact – further details in monitoring policy impact section) relevant to the child poverty context.

The type of data collated differs by policy. This is because the scope and nature of policies are vastly diverse and information needs to be relevant and specific to that particular context. Consistently across all policy actions, we collate information on status of the delivery, staffing and finance. It also includes updates on what has been achieved over the past quarter and activity planned over the coming one. This combines subjective issues (such as perceptions of how well a policy has operated) and objective issues (the factual details of how an intervention has operated, which is usually taken from administrative data where available, statistics, survey or research). It is then complemented with contextual information that could have influenced delivery.

What does this look like?

The quarterly reporting process looks like:

1. Quarterly commission for information on key Plan commitments.

2. Delivery and impact data reviewed and summarised.

3. Contextual information that impact on people’s income

4. Overall programme status for delivery assessed and discussed at Programme Board meetings and governance groups.

5. Learnings across the years summarised in annual progress reports.

This frequent monitoring allows an assessment and explanation of progress towards reducing child poverty. To allow corrective action to be taken, a strong governance structure is in place at official and Ministerial level.

How does this work in practice?

Ministers, Directors-General, and Directors are accountable for implementing the elements of the Plan within their remit. They are responsible for ensuring contributions to the Programme are delivered effectively, governed responsibly, and reported transparently, with any challenges escalated and addressed appropriately.

Following publication of the second Delivery Plan, Best Start, Bright Futures, a Programme was established to oversee the Scottish Government’s delivery of the Plan, based on Scottish Government Programme Management standards. The Programme includes appropriate governance arrangements and applies continuous learning and best practice. For Best Start Bright Futures, governance arrangements included a Programme Board and Ministerial oversight has been provided through a Child Poverty Ministerial Oversight Group, and then a Cabinet Sub-Committee chaired by the First Minister. Progress was tracked on a quarterly basis considering delivery status and data on impact which was scrutinised and considered by the relevant governance groups.

The performance of Best Start, Bright Futures is determined through a structured assessment that considers delivery, staffing, workforce and finance indicators into a single view of programme health. Each component is developed by the programme and analytical teams and validated with supporting evidence. The overall status is not an average but a holistic judgement that considers cumulative risks, relative weighting of critical factors, and forward-looking trends, ensuring alignment with strategic priorities and risk appetite. This agreed status, accompanied by a clear narrative on drivers, mitigations, and trajectory, provides governance groups with an evidence-based position to support informed decision-making and timely intervention.

A similar approach will be adopted for the third Delivery Plan. The Programme will operate an internal benefits management approach to monitor expected outcomes and delivery assumptions. This evaluation strategy is designed to work alongside that approach, drawing on its insights where appropriate to support robust and coherent assessment.

The Programme will also be supported by a cross-Government analytical group (see section on individual policy monitoring and evaluation).

The continuous measurement and performance of delivery is strongly linked to governance structures. This ensures that the evaluation is fit-for-purpose, can adapt to changing situations, can influence decision making and has the confidence of stakeholders.

Impact evaluation – what difference has the Strategy made and for whom?

Impact evaluations focus on the changes caused, in this instance by the introduction of the Child Poverty (Scotland) Act. The first stage of any impact evaluation is developing a theory of change that explains the anticipated changes to be seen. The summarised version of the theory of change can be found in Figure 2 below, with full details available in the first published version of the evaluation strategy.

The first and second Delivery Plans already set out three key elements of the impact assessment approach. These elements remain central to the evaluation.

Monitoring the targets tells us where we are now, what progress has been made, and how far we still have to go. Monitoring the drivers helps us to understand how they are changing over time, but not necessarily why they have – or have not - changed. To understand why, we assess the contribution of specific policies to poverty and its drivers.

This is represented in Figure 3, and explained in detailed in the sections that follow.

Monitoring the child poverty targets

The Act sets out specific targets to be reached by 2030. Statistics to monitor the four child poverty targets are published annually, usually in March of each year. As a brief reminder, the child poverty targets are:

• Less than 10% of children to live in relative poverty, that is living in households with an equivalised income below 60% of the contemporary UK median income.
• Less than 5% of children to live in absolute poverty. The absolute poverty line is fixed at 60% of the median UK income in 2010-11.
• Less than 5% of children to live in combined low income and material deprivation. This represents the proportion of children in families with incomes less than 70% of the contemporary median that lack certain basic necessities.
• Less than 5% of children to live in persistent poverty, that is living in households experiencing relative poverty for three out of the four last years.

The data for the relative poverty, absolute poverty, and low income and material deprivation indicators relate to After Housing Costs and come from the Family Resources Survey (FRS), which is administered by the UK Government’s Department for Work and Pensions (DWP). The Scottish Government provides funding to make sure that enough Scottish households are surveyed to give us robust data at the Scotland level. Detailed Scotland level analysis provides annual updates by various characteristics. The Department for Work and Pensions are carrying out a transformation programme for the FRS which involves a range of methodological improvements, including linking survey responses with administrative data to more accurately reflect the level of benefit receipt. Future plans include bringing FRS results into alignment with Census 2022 population figures and continuing research into the linkage of administrative data on earnings to provide a more accurate depiction of household incomes. .

The data on persistent poverty, comes from the longitudinal Understanding Society survey. Persistent poverty estimates tend to be routinely revised due to respondents dropping off and re-joining the survey in future years.

Both sets of statistics (from FRS and Understanding Society) are usually published together, with the date for publication set by DWP. Scottish Government analysts are actively involved in the quality assurance of the data.

We saw that annual revisions are made on persistent poverty rates due to the longitudinal nature of the survey. To mitigate the impact of respondent attrition over the years, the sample has included a boost of around 5,700 households across the whole UK, including around 500-600 in Scotland. Data including this increased sample will be available in the 2022-2026 publication, when four years’ worth of data can be included. The published output is anticipated for March 2028.

Monitoring the drivers of poverty

The drivers of poverty refer to direct improvements to households’ income. These are: increasing income from employment, reducing costs of living, and increasing income from social security and benefits in-kind.

Experience continues to show that the driver approach is appropriate. Particularly when assessing whether there is a balance of policies, and impacts, across the three drivers, for example between policies that aim to boost income through paid work and those that boost income through social security benefits. We will continue this approach during the third delivery plan, with some marked improvements:

• Updated indicators reflecting new data and approaches. As part of the analytical support for this third Delivery Plan, we have reviewed the measurement framework indicators. The process involved checking whether indicators were still relevant, appropriate and accurate; identifying any potential gaps; and selecting new improved indicators to better track and understand progress towards the targets. In addition, we have also included a new set of indicators that allow us to track the long-term consequences of poverty, such as wellbeing, attainment or housing security.
• A stronger focus on indirect drivers of poverty, such as family wellbeing and capabilities. We know that the three key drivers of income from employment, income from social security and cost of living impact directly on household incomes and, therefore, directly on the number of households in poverty. But ultimately, eradicating child poverty is about ensuring the whole family’s wellbeing.

As such, there is a need to support families holistically, through a whole family wellbeing approach. The whole family approach has a dual purpose. First, it can be a preventative tool, to reduce the risk of families falling into poverty or deeper into poverty in the future. Secondly, to improve incomes and wider circumstances of those families already experiencing poverty. This type of support is expected to have a long-term impact on the number of households in poverty.

As a first step we aimed to identify key health and wellbeing metrics that could be used to track progress on these outcomes. In that spirit, we published a first review of wellbeing indicators. The review confirmed that in the recent societal, political and economic context, focusing on supporting low income families in their health and wellbeing, as well as boosting their income, such as through the Scottish Child Payment, is crucial in reducing inequalities. But this was only the start of the analytical journey. Further work will be needed to examine what progress is being made on enhancing families’ wellbeing through a range of relevant policies and how this intersects with pathways out of poverty.

• Contextualisation. When looking at the progress across the drivers, it is important to contextualise any progress, or lack of, when interpreting the data. This is particularly relevant for Scotland, as many factors influencing poverty rates, can be out with the Scottish Government’s control. Over the past four years we have remained attuned to policy, economic, social and political changes and provided in-depth reports of evidence, for example, the deep dive into the cost of living crisis. Over the third Delivery Plan, we will continue to report on progress, ensuring that the wider socio-economic and political environment is considered. This contextualisation will also inform quarterly reporting and feed into internal governance processes.
• Focus on local implementation and continued support for local partners to access and interpret data. The national targets are set at a Scotland level. They are not set for individual local authorities. Progress will be different in different places and due to diverse contexts. As well as the wider socio-economic and political context, the local implementation context is important in understanding whether initiatives have their intended effect in supporting families out of poverty. Local partners will therefore need to use a range of data and evidence to consider levels of poverty within their areas, any progress seen and how this has been achieved. We will continue to closely work with the National Child Poverty Co-ordinators group and through our evaluation work to help local partners do this. There are a wide range of resources available and in development to support local actors, including Prioritise Child Poverty: a data and systems approach and Tackling Poverty Locally Directory.

Monitoring policy impacts

This element of the monitoring approach examines the complexity of child poverty and our specific efforts to address it.

In a cross-cutting policy space, it is necessary to clearly define the scope and regularity of what is being monitored and why.

To maximise consistency, impact monitoring has been aligned with quarterly governance delivery updates and benefit management realisation (see previous section on process evaluation). In this way, we are able to assess whether the policy delivery is at the pace and scale required (the process), and understand the impacts said policy is having on child poverty. A range of data sources are needed including qualitative or narrative data (collected from programme participants and stakeholders), quantitative monitoring data (collected by a programme or initiative), plus other quantitative data on outcomes (such as surveys or administrative data).

It is important to understand policy impacts at two different levels:

• Level 1: Impacts of individual policy efforts on child poverty
• Level 2: Whether the package of policies is working effectively together

Level 1. Individual policy monitoring and evaluation

The approach to assessing individual policies has evolved over the course of the first two plans. This has mainly been due to changes in the scope of the programme but also in the resources available.

During the first Plan, the evaluation strategy focused on seven key policies, shortlisted due to the likely impact they were anticipated to have. This included: Fair Start Scotland, the expanded Early Learning and Childcare, Best Start Grant, Scottish Child Payment, Connecting Scotland, Money Talk Team and Private Residential Tenancy.

During the second Plan, many of those policies continued in some form, which allowed for a longer term assessment of progress. In addition, considerable resource, time and energy was invested in expanding and improving our understanding of impact. For that purpose, we developed an evaluation framework, providing a shared understanding of how we measure the impact of individual policies on child poverty. The Framework has provided consistency in assessing impacts on child poverty across key policies (including those starting a new evaluation or those well under way). The Framework also formed the basis for the quarterly data collection that supports internal monitoring of progress, allowing for a closer and regular scrutiny of impact. Undoubtedly, the evidence base has been expanding thanks to the introduction of more nuanced assessment of evaluation of impact on child poverty across a wide range of policy areas.

The intention over this latest third Plan is to continue the close engagement between analysis and ongoing governance reviews, ensuring that regular updates on impact are being monitored and assessed. However, collating evidence of impact can be challenging. This is because policies operating in this space are wide ranging and strive for both short and longer-term outcomes. Some policies have clear quantifiable outcome measures (such as take-up rates of benefits, or number of parents increasing their income through paid work). Other policies, particularly those operating on longer-term outcomes (like systems change, place based approaches or family wellbeing) have a broader and often more fluid range of outcomes—especially in the early stages—which cannot all be easily quantified (e.g., measures of ‘systems change’). Instead we often rely on qualitative evidence. Availability of impact data will vary. While we are anticipating to monitor impact on a quarterly basis, it is unrealistic to expect new impact information across the wide range of actions to be available every quarter. Instead, we will update new evidence as it becomes available and assess impact on an ongoing, rolling basis.

Over the course of the third plan several improvements are being made to improve on the impact data collated and analysed.

• Closer alignment with governance programme ensuring that a new internal benefits management approach to monitor expected outcomes and delivery assumptions works and complements this evaluation strategy.
• A review of the policy evaluation framework guidance, supported by the SG Centre of Expertise in Appraisal and Evaluation, to ensure it covers latest learnings and expand guidance to include: techniques for assessing systems change, monitoring health and wellbeing impacts, and assessing value for money.
• Assurance from a newly formed cross-government analytical working group. The group has been established to strengthen the existing analytical basis and ensure we have the data and analysis required from across government to inform decisions, understand the impact of actions and provide assurance on progress towards the statutory 2030 targets.

Impact is not only understood as progress against the targets. For families experiencing multiple disadvantages or adversities, challenges are often deeply interconnected in complex ways. Although there is not a simple causal relationship between these issues and poverty, multiple disadvantages and adversities can make it harder to get out of poverty and poverty can also make it harder to overcome adversities.

Over the past two Plans, we have built a breadth of evidence on the challenges and barriers faced by those families at greater risk of poverty. We have a better understanding of what works to support families. We also know that gender equality is tightly linked to reducing child poverty, as women’s poverty and children’s poverty are intrinsically connected. Over the next Plan, we will aim to better understand how ongoing policies are supporting low income families, with a clear focus on improving our understanding of impact amongst families at greater risk of poverty and those with intersecting equality characteristics. It will be important to understand how, or if, inequalities are being reduced and how policies are supporting families with intersecting characteristics. This will be done via the priority family type lens, and will yield in-depth understanding of progress towards equality. For example, are policies ensuring that systems support women to increase their income from employment? Are childcare options aligning with family needs regardless of personal circumstances?

Level 2. Cumulative impact of policies

This level of assessment helps policy areas to minimise the risk of silo working and maximise collaboration and the sharing of learning. We assess the cumulative impact of policies in two ways:

1. At a systemic level through qualitative evidence

2. Through quantitative cumulative impact assessments

System level understanding

System change initiatives focus on making structural and procedural changes to the organisations that support families, with the aim of improving the services provided to them.

During the second Plan, we sought to better understand and assess the extent to which system change initiatives contribute to eradicating child poverty. We developed a theory of change which maps the attributes of system change initiatives towards eradicating child poverty. To date, we have published two analytical reports outlining our findings on system change initiatives. The first report outlines broad learnings from across place-based, system change initiatives and the second report explores successful engagement with families living in poverty.

We know that collaboration, flexibility and person-centred approaches are fundamental building blocks for longer term system change. We also know that system change initiatives take time to implement and embed. This means it can take time to see the benefits for families. These efforts face challenges related to funding constraints, effective monitoring and communication of progress, and the long-term sustainability and embedding of change.

The third Plan emphasises the need for policy coherence and alignment to maximise benefits for families, for example, ensuring that policies supporting people into employment or to increase their paid hours align with their childcare needs, transport options and other required social security support. It is important to continue assessing whether this alignment is occurring—and crucially—whether families are benefiting from it. To support this learning, we are updating the evaluation framework to include new guidance on measuring, assessing and understanding the impact of system change initiatives.

Quantitative cumulative impact assessment of a package of policies

Over the past four years, we have developed internal capacity to undertake cumulative impact assessments, enabling us to model the impact of a package of policies on the targets. Using UKMOD, an open-access microsimulation tax and benefit model based on the Family Resources Survey data, we can estimate the impact of various policies on child poverty.

Modelling can be particularly helpful in the design stage, when trying to anticipate the likely impact of a package of policies. This approach was first used as part of the second Plan, and now repeated for this third Plan. The latest cumulative impact assessment estimates that the package of Scottish policies will keep 100,000 children out of relative poverty and 80,000 children out of absolute poverty in 2026-27. The Scottish Child Payment policy alone is estimated to keep 50,000 children out of relative poverty in 2026-27.

Modelling can also be helpful on an ongoing basis. Statistics on poverty rates take time to be analysed and made publicly available. This means that there is a time lag between a policy being implemented, and its impact being seen in poverty rates. During this time lag period, we can use cumulative impact modelling to understand the likely direction of travel of our package of policies.

However, models are only a reflection of the data and assumptions made. Decreasing response rates in the Family Resources Survey introduce further uncertainty to modelled outputs and make analysis of the priority groups more challenging. Further, not all policy impacts can be modelled effectively, particularly due to the lack of certainty around impacts. Models are predictions, which in an ever changing socio-economic environment, are therefore not exact.

Despite this, they provide a helpful indication of progress or of the scale required. Going forward, we will continue to broader evidence, delivery updates, and evaluation findings both to improve the cumulative impact modelling as well as understand and assess progress. We will also work to include more policies into the modelling as evidence of impacts becomes clearer.

In a complex system like child poverty, it is imperative that impact information is understood both at a single intervention level, but also as a cumulative package. Complemented with a clear understanding of impacts for priority family types, this ensures that we maximise benefits for all regardless of individual circumstances.

Economic evaluation

Economic evaluations help us understand the benefits of a policy relative to its cost and allow for comparisons between policies, enabling better targeting of resources.

Indeed, over the second Plan, economic evaluation played a more prominent role. We have expanded our range of economic evaluation methods, and are currently developing guidance to support policy teams.

Economic evaluations will remain important for assessing value for money going forwards. However, in the child poverty space, an economic evaluation of the whole Plan is not recommended. This is because much of the activity is not solely targeted at eradicating child poverty and therefore it would not be feasible to assess the costs and benefits to the overarching Plan in isolation from other wider policy aims.

Instead, we have aimed to develop a better understanding of the economic assessment of individual policies. Tackling child poverty requires alignment of policies across government, and, therefore, requires the most optimal use of our limited resources to have the best possible impact on eradicating child poverty. This is particularly the case due to the increasingly challenging fiscal context, as well as the persistence of heightened global macroeconomic risks, which could directly and indirectly affect child poverty rates through the drivers of poverty and broader impacts on economic conditions, while being beyond the control of the Scottish Government.

For the Third Delivery Plan, we are expanding the evaluation framework to provide further guidance on how to evaluate value for money in policies related to child poverty, building on learnings from previous economic evaluations, such as the Child poverty pathfinders in Dundee and Glasgow, in which a 4Es approach – Economy (spending less), Efficiency (spending well), Effectiveness (spending wisely), Equity (spending fairly) – was used alongside Break-Even Analysis. The guidance will reflect the specific challenges and trade-offs associated with economic evaluations in the child poverty context. It will balance the need for robust evaluation methods against the increased burden of data collection and reporting, setting out approaches suitable for policies in different stages of evaluation and with varying levels of data availability.

Scottish Government analysts in the child poverty space will continue to work with officials across government, including the SG Centre of Expertise in Appraisal and Evaluation, to provide support in undertaking appraisals, and conducting economic evaluations, including Value for Money evaluations. This will help to ensure consistency in approaches across economic evaluations for individual policies, and further develop the evidence-base for how resources are directed towards supporting child poverty objectives.