Whole Family Wellbeing Funding (WFWF) - year 1 process evaluation: final report – annexes

These annexes relate to the full report which presents the final findings from a process evaluation of Elements 1 and 2 of the Scottish Government Whole Family Wellbeing Funding (WFWF) in its first year of operation.


Annex 1: Evaluation Framework

Key evaluation aim: 1. Provide an overview of types of activity the Element 1 and 2 funding is being used for

Key research questions: What activities, audiences and topics are CSPPs targeting (or not targeting) their funding towards?

Measures Source & timings Timings Element 1 and/or 2
Intended activities/topics CSPPs are targeting their funding towards Initial funding plan / EOI review Wave 1 1 and 2
Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Problem / challenge CSPPs are aiming to address and evidence for this (including needs assessment and system analysis) Initial funding plan / EOI review Wave 1 1 and 2
Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Ongoing insight provided by SG policy team
How intended individual CSPP activity/topics reflects that outlined in the WFWF logic model Logic model review Wave 1 1 and 2
Intended target beneficiaries, reasons for this, and plans to monitor that their targeting of their intended beneficiaries is working Initial funding plan / EOI review Wave 1 1 and 2
Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Differences / similarities in intended activities/topics depending on CSPP characteristics (including value of funding received) Initial funding plan / EOI review Wave 1 1 and 2
Intended programme team involved in delivering activity (including whether established, new, combination, whether there are vacancies to fill) Initial funding plan / EOI review Wave 1 1 and 2
Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Intended organisations involved in the programme (including their contributions, level of commitment) Initial funding plan / EOI review Wave 1 1 and 2
Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Any activity/topic not included in plan that was felt to be a priority and reasons for it not being included Strategic interviews Wave 1 1 and 2
Additional / alternative sources of funding the CSPPs intend to access to support activity Initial funding plan / EOI review Wave 1 1 and 2
Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
How Element 1 and 2 activity is intended to work together / support each other (Element 2 CSPPs only) Strategic interviews Wave 1 2 only
WFWF Lead interviews Wave 1 2 only
Anticipated obstacles/barriers to programme delivery Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Ongoing insight provided by SG policy team
Whether the CSPP developed any tools to improve children's services or map their context Strategic interviews Wave 2 2 and 2
WFWF Lead interviews Wave 2 3 and 2

Key research question: How does funding used compare with existing provision?

Measures Source & timings Timings Element 1 and/or 2
Extent to which WFWF activity is building on previous activity and where in the development journey CSPPs are Initial funding plan / EOI review Wave 1 1 and 2
Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Differences / similarities in the above depending on CSPP characteristics (including value of funding received) Initial funding plan / EOI review Wave 1 1 and 2
Initial impressions and expectations of the WFWF for CSPPs Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2

Key evaluation aim: 2. Understand local delivery partners’ views on how Element 1 and 2 funding has been used

Key research question: Funding allocation and SG support: To what extent and how did the approach taken by the SG in terms of allocation of funding, mechanisms of distribution and support, contribute to CSPPs’ ability to scale up family support services and drive system change?

Measures Source & timings Timings Element 1 and/or 2
How WFWF funding was allocated and distributed Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Experience of receiving funding (including ease/difficulty, challenges/barriers and best practice learning) Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Experience of support from the Scottish Government (including challenges/barriers and best practice learning) Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Views on how the funding allocation and SG support contributed to CSPPs ability to scale up family support services and drive system changes (and reasons why / why not) Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
CSP annual report analysis Wave 2 1 and 2
Suggestions for improving funding allocation and SG support Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
CSP annual report analysis Wave 2 1 and 2
Reasons why the intended activities were / were not delivered Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
Differences / similarities in views on funding allocation and SG support between Element 1 and 2 (Element 2 CSPPs only) Strategic interviews Wave 1 2 only

Key research question: Implementation and monitoring: How were funding decisions within CSPPs taken?

Measures Source & timings Timings Element 1 and/or 2
How funding decisions within CSPPs were taken Strategic interviews Wave 1 1 and 2
Perceptions of the effectiveness of how funding decisions were made within CSPPs (including barriers/challenges, best practice learnings and suggestions for improvement) Strategic interviews Wave 1 1 and 2

Key research question: Implementation and monitoring: What audiences were consulted and in what ways, including delivery staff, wider partners, families, young people?

Measures Source & timings Timings Element 1 and/or 2
Range of audiences consulted and in what ways, including delivery staff, wider partners, families, young people Initial funding plan / EOI review Wave 1 1 and 2
Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Family, CYP interviews (where appropriate) Wave 2 1 and 2
How the activity was designed / planned (including who was involved and time commitment) Initial funding plan / EOI review Wave 1 1 and 2
Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Family, CYP interviews (as appropriate) Wave 2 1 and 2
[Activity] CSPPs engage with wide range of children & families on design of services (e.g. those with experience of these services) Extent to which (and how) CSPPs engaged with children and families in the design of the programme (including those with experiences of services) Initial funding plan / EOI review Wave 1 1 and 2
Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Family, CYP interviews (as appropriate) Wave 2 1 and 2
[Activity] CSPPs engage with wide range of children & families on design of services (e.g. those with experience of these services) Challenges/barriers to engaging children and families in the design of services Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Family, CYP interviews Wave 2 1 and 2
[Activity] CSPPs engage with wide range of children & families on design of services (e.g. those with experience of these services) Best practice learnings for engaging children and families in the design of services Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Family, CYP interviews (as appropriate) Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Activity] CSPPs engage with wide range of children & families on design of services (e.g. those with experience of these services) Suggestions for improving engagement of children and families in the design of services Strategic interviews Wave 1 1 and 2
WFWF Lead interviews Wave 1 1 and 2
Frontline staff focus groups Wave 2 1 and 2
Family, CYP interviews Wave 1 1 and 2
[Activity] CSPPs engage with wide range of children & families on design of services (e.g. those with experience of these services) Involvement of children and families in service delivery (i.e. beyond the set-up stage) Frontline staff focus groups Wave 2 1 and 2
Family, CYP interviews Wave 2 1 and 2
Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2

Key research question: Implementation and monitoring: Explore experiences of: design, set-up, structure, practice, workforce, partnership, managing change?

Measures Source & timings Timings Element 1 and/or 2
How activity differs from that intended (and reasons for this) Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
How intended beneficiaries differs from that intended (and reasons for this) Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Project delivery barriers/challenges and lessons learnt Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
Project delivery successes/best practice learnings Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
Suggestions for improvement to project delivery Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
Experience of recruiting staff (where needed) to scale up and/or deliver plans Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Views on availability and effectiveness of support from SG to deliver WFWF activity Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Identified risks to project delivery and how these are being approached/mitigated Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
Initial funding plan / EOI review Wave 1 1 and 2
Differences/ similarities between Element 1 and 2 Follow-up WFWF Lead interviews Wave 2 2 only
CSP annual report analysis Wave 2 2 only

Key research question: Implementation and monitoring: How is performance monitored, by whom, when, targets, mitigations to demonstrate impact?

Measures Source & timings Timings Element 1 and/or 2
Monitoring / evaluation activity planned by CSPP (outside this IFF evaluation) and reasons for this Initial funding plan / EOI review Wave 1 1 and 2
Strategic interviews Wave 1 1 and 2
Monitoring / evaluation activity undertaken by CSPP (outside this IFF evaluation) and views on the effectiveness of this Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2

Key evaluation aim: 3. Understand families/young people experiences of family support services that have received Element 1 and 2 funding, and the extent to which this has achieved short-term outcomes (as far as possible)

Key research question: To what extent and how has the funded activity achieved intended short-term outcomes related to service delivery?

Measures Source & timings Timings Element 1 and/or 2
[Outcome] CSPPs begin embedding the key principles for holistic whole family support within their own systems and structures Perception that CSPPs have embedded the key principles of holistic whole family support within their own systems and structures (and how this has been achieved) Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
Family, CYP interviews Wave 2 1 and 2
[Outcome] CSPPs start to redesign/design delivery of new whole family support services, including removing barriers for children and families to accessing support Perception that barriers to children and families accessing support have been reduced/removed Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
Family, CYP interviews Wave 2 1 and 2
[Outcome] Early evidence that feedback on Children’s Services informing Adult and Related Services planning/delivery Volume and source of feedback gathered on children's services to inform adult and related services planning/delivery Follow-up WFWF Lead interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Outcome] Early evidence that feedback on Children’s Services informing Adult and Related Services planning/delivery Perceptions that feedback on children's services has informed adult and related services planning/delivery Follow-up WFWF Lead interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Outcome] Early evidence of non-siloed, aligned and proportionate FS funding that matches scale of need Views on the amount of WFWF funding received and extent to which this does (doesn't) meet the scale of demand Follow-up WFWF Lead interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Outcome] Early evidence of non-siloed, aligned and proportionate FS funding that matches scale of need Views on the integration of WFWF funding with other sources (meaning it is non-siloed) Follow-up WFWF Lead interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Outcome] Local investment in planning system change (recognise budgets already set for 2022-23) Breakdown of WFWF funding by CSPP activity Follow-up WFWF Lead interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Outcome] Local investment in planning system change (recognise budgets already set for 2022-23) Views on the ease/difficulty of local investment in planning system change Follow-up WFWF Lead interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Outcome] Early evidence of more collaborative work across CSPP partners and with adult services (share resources, data, feedback, and information) How CSPPs have collaborated with CSPP partners and adult services Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
[Outcome] Early evidence of more collaborative work across CSPP partners and with adult services (share resources, data, feedback, and information) Perception that WFWF activity/funding has increased/improved collaboration with partners/adult services Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
[Outcome] Delivery partners (including 3rd sector) are integral to service design & delivery of WFS How delivery partners have been involved in WFWF activity (including service design and delivery) Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
[Outcome] Delivery partners (including 3rd sector) are integral to service design & delivery of WFS Perception that delivery partners have been (more) integral to service design and delivery of WFS Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
[Outcome] Empowerment for innovation Perception that WFWF funding has allowed CSPPs to carry out innovative activity Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
[Outcome] Development of holistic workforce approach Evidence of changes to the CSPP workforce's way of working Follow-up WFWF Lead interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Outcome] Development of holistic workforce approach Perception that WFWF activity/funding has led to the development of a more holistic workforce Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
[Outcome] Build transformational capacity within CSPP for whole system change Views on whether there has been sufficient capacity within CSPPs to achieve outcomes and build whole system change Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
[Outcome] Build transformational capacity within CSPP for whole system change If/how transformational capacity for whole system changes has been built Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
[Outcome] Build transformational capacity within CSPP for whole system change Perception that (and how) WFWF activity/funding has led to transformational capacity for whole system change Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
If early evidence for all outcomes, mechanisms for achieving these, importance of outcome for service delivery, and reasons for working well within the local context Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
If no early evidence for all outcomes, suggestions for improving/strengthening achievement and potential timescales to evidence these Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
Differences in outcomes between Element 1 and 2 on outcomes for service delivery and reasons for this (Element 2 only) Follow-up WFWF Lead interviews Wave 2 2 only
CSP annual report analysis Wave 2 2 only
Frontline staff focus groups Wave 2 2 only
Triangulation of all evaluation evidence Wave 2 2 only
Views on early evidence for any unintended outcomes (either positive or negative) relating to WFWF activity Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
For engagement/participation, see earlier section of framework. n/a Wave 2 1 and 2
[Outcome] Early evidence of meaningful and ongoing participation by CYPF in service design which ensures choice and control Perception that children and families have more choice about service access Follow-up WFWF Lead interviews Wave 2 1 and 2
Family, CYP interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Outcome] Early evidence of meaningful and ongoing participation by CYPF in service design which ensures choice and control Perception that children and families have more control about the services they access Follow-up WFWF Lead interviews Wave 2 1 and 2
Family, CYP interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Outcome] Early evidence of meaningful and ongoing participation by CYPF in service design which ensures choice and control Experience of children and families of their ability to have control and choice about services accessed Follow-up WFWF Lead interviews Wave 2 1 and 2
Family, CYP interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Outcome] Early evidence of improved points of access to services in communities Perception that children and families have improved/increased points of access within communities to access support Follow-up WFWF Lead interviews Wave 2 1 and 2
Family, CYP interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Outcome] Early evidence of improved points of access to services in communities Experience of children and families of new points of access to services in their communities Follow-up WFWF Lead interviews Wave 2 1 and 2
Family, CYP interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Outcome] Increased whole family support service capacity – scaled and new services are integrated Perception that whole family support services have required capacity Follow-up WFWF Lead interviews Wave 2 1 and 2
Family, CYP interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Outcome] Increased whole family support service capacity – scaled and new services are integrated Experience of children and families in accessing services when they need to Follow-up WFWF Lead interviews Wave 2 1 and 2
Family, CYP interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
[Outcome] Increased whole family support service capacity – scaled and new services are integrated Perception that services are integrated Follow-up WFWF Lead interviews Wave 2 1 and 2
Family, CYP interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
If early evidence for all outcomes, mechanisms for achieving these, and reasons for working well within the local context Follow-up WFWF Lead interviews Wave 2 1 and 2
Family, CYP interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
If no early evidence for all outcomes, suggestions for improving/strengthening achievement and potential timescales to evidence these Follow-up WFWF Lead interviews Wave 2 1 and 2
Family, CYP interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Differences in outcomes between Element 1 and 2 for outcomes for children and families and reasons for this Follow-up WFWF Lead interviews Wave 2 2 only
Frontline staff focus groups Wave 2 2 only
CSP annual report analysis Wave 2 2 only
Triangulation of all evaluation evidence Wave 2 2 only
Views on early evidence for any unintended outcomes (either positive or negative) for children, young people and their families Follow-up WFWF Lead interviews Wave 2 1 and 2
Family, CYP interviews Wave 2 1 and 2
Frontline staff focus groups Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2

Key evaluation aim: 4. Provide evidence for policy and practice to inform future improvement

Key research question: What conditions are necessary for consolidating improvements in the funded CSPPs?

Measures Source & timings Timings Element 1 and/or 2
What worked well and why (including consideration of local context) Follow-up WFWF Lead interviews Wave 2 1 and 2
CSP annual report analysis Wave 2 1 and 2
Triangulation of all evaluation evidence Wave 2 1 and 2
What might have worked better and why (including consideration of local context) Follow-up WFWF Lead interviews Wave 2 1 and 2
Triangulation of all evaluation evidence Wave 2 1 and 2

Key research question: What are recommendations for 2023-2026 funding and practice?

Measures Source & timings Timings Element 1 and/or 2
Lessons learnt for future delivery Follow-up WFWF Lead interviews Wave 2 1 and 2
Triangulation of all evaluation evidence Wave 2 1 and 2
Best practice learning Follow-up WFWF Lead interviews Wave 2 1 and 2
Triangulation of all evaluation evidence Wave 2 1 and 2

Contact

Email: socialresearch@gov.scot

Back to top