Attainment Scotland Fund evaluation: headteacher survey 2018
Headteachers’ experience of the fund, covering themes such as governance, planning and evaluation, impact and sustainability.
6. Impact and sustainability
6.1. This section sets out findings on views and experiences on the impact of funded interventions. This includes views on the factors that contribute to or hinder success, whether positive impacts are likely to be sustainable, the extent to which ASF support has contributed to an increase in collaborative working, and whether schools have seen any unintended positive or negative consequences of ASF support.
Impact of interventions
6.2. Nearly 9 in 10 (88%) schools have seen an improvement in closing the poverty-related gap in attainment and/or health and wellbeing as a result of ASF supported interventions. A minority of these (12%) have seen 'a lot' of improvement to date, although more than half (56%) of respondents are expecting to see a lot of improvement over the next five years as a result of ASF supported interventions (see figure below). Survey responses indicate that those who have seen some improvement to date are significantly more likely to expect further improvement over the next five years.
6.3. There has been a 10 point increase from the 2017 survey in the proportion of schools seeing an improvement in closing the gap. The survey also shows some variation in views across funding streams, with Schools Programme respondents most likely and PEF-only least likely to have seen an improvement. This difference may reflect that PEF-only schools did not have access to ASF support prior to 2017/18.
Perceived improvement in closing the poverty-related gap in attainment or health/wellbeing
6.4. More detailed survey analysis has been undertaken to assess the strength of correlation between perceived improvements in wellbeing/attainment, and a range of other factors and respondent groups. This indicates that the following groups are significantly more likely to have seen improvement as a result of ASF support interventions: those who feel confident using evidence to inform development of interventions; those who feel they had sufficient support available to develop their school plan; those who have an evaluation plan in place; and those who have seen an increase in collaborative working.
6.5. The survey also asked for headteachers' views on the factors that contribute to or hinder the success of ASF supported interventions. The factors most commonly raised by respondents are summarised below.
Contributing to success (n=245)
|Teachers and staffing resource||70%|
|Teacher training and skills development||18%|
|Resources and funding||18%|
|Family engagement to support pupils||17%|
|A shared set of objectives, collective focus||13%|
|Use of data and evidence to inform approaches||13%|
|Effective targeting of support||13%|
Potential barriers to success (n=247)
|Staffing, staff time and workload||51%|
|Resources and wider funding||24%|
|Mental health, behavioural issues and support needs of pupils||14%|
|Wider community issues, pupils' experience at home and family wellbeing||14%|
|Organisational issues, reporting and paperwork requirements||12%|
|Difficulties ensuring effective targeting||10%|
|Teacher training and skills development||9%|
6.6. Around 2 in 5 (42%) survey respondents expect that improvement as a result of ASF supported interventions will be sustainable beyond the years of funding. This represents a 16 point reduction from the 2017 survey, where 58% of respondents felt improvements would be sustainable. However, it should be noted that a substantial proportion (42%) of respondents were unsure of the sustainability of impacts, and only 16% were of the view that impacts would not be sustainable.
6.7. Survey findings also suggest a correlation between views on sustainability and perception of improvement to date; those who have seen improvement in attainment or wellbeing to date are significant more likely than others to expect improvements to be sustainable beyond funding.
6.8. The survey asked headteachers to expand on why they do (or do not) expect improvements to be sustainable. Responses give an indication of the factors influencing views here.
Why feel improvement will be sustainable (n=115)
|Staffing training, skills development and capacity building||70%|
|Embedded practice, pedagogy development||26%|
|Raising awareness and change of ethos/culture||20%|
|Ongoing access to resources||13%|
|Developed a collective, shared focus||10%|
|Developed capacity to use data/evidence to inform approaches||9%|
|Collaboration within school, with partners and parents||7%|
|Longer-term health and wellbeing impacts||6%|
Why feel improvement will not be sustainable (n=47)
|Loss of staffing, resources and skills||81%|
|Loss of initiatives and interventions||28%|
|Loss of external support and access to services||12%|
|Too early to tell||12%|
|Pupils' health and wellbeing needs||9%|
|Reduction in local funding/resourcing||7%|
6.9. The majority of headteachers have seen an increase in collaborative working in their school as a result of ASF support. Around 7 in 10 (71%) indicate this, including around 3 in 10 (29%) who have seen a large increase in collaborative working.
6.10. Survey responses indicate significant variation across funding streams in the extent to which ASF support has led to an increase in collaborative working. Schools Programme respondents are more likely than others to have seen an increase in collaborative working as a result of funding (98% have done so), while PEF-only schools are least likely to have seen such a change (66%).
Whether seen increase in collaborative working as result of Fund
6.11. The survey asked headteachers to expand on why they feel there has been an increase in collaborative working as a result of ASF support. These are summarised below.
|Greater emphasis on collaboration from school leadership, empowerment of teaching staff||41%|
|More staff resourcing and staff time||23%|
|Collaboration embedded as a key part of CPD||16%|
|Change of culture/ethos across the school||12%|
|More opportunities for collaboration and partnership||12%|
6.12. A small number of respondents who feel the Fund has not led to greater collaboration within their school provided further comment on this. These respondents (around 2% of all survey respondents) most commonly referred to a culture of collaboration already being well-established prior to the school receiving ASF support. Some also referred to ASF support not having had sufficient impact on staff time or opportunities within their school for collaborative working.
6.13. In addition to views on the extent to which interventions had produced the anticipated impacts, the survey also asked headteachers about any unintended consequences of ASF support.
6.14. Nearly 2 in 5 (38%) respondents have seen unintended positive consequences as a result of their receipt of ASF funding, and nearly 1 in 5 (18%) have seen unintended negative consequences. These findings are broadly consistent across key respondent groups, although primary school respondents are more likely than secondary school respondents to have seen unintended negative consequences.
6.15. The survey asked headteachers to expand on the unintended consequences they had seen as a result of ASF support. Responses are summarised below.
Unintended positive consequences (n=108)
|More and better collaboration and partnership working||28%|
|Training and skills development for staff||21%|
|Improved pupil and parent engagement, attendance||19%|
|A change in school ethos/culture, a shared focus||16%|
|More leadership opportunities for staff||12%|
|Better awareness/understanding of the attainment and wellbeing gap, recognition of the value of targeting||7%|
|More and better use of data/evidence||7%|
Unintended negative consequences (n=66)
|Additional workload for leadership, management and administrative roles||39%|
|Reporting requirements, pressure to demonstrate improvement||20%|
|Reduction in other resourcing, loss of free access to 3rd sector support, perception that 'awash' with funding||9%|
|Potential for schools, pupils or parents to feel excluded where not the focus of ASF||9%|
|Difficulty balancing ASF with other priorities||3%|
There is a problem
Thanks for your feedback