Expansion of funded early learning and childcare to 1140 hours: 2018-2025 national outcomes evaluation

Appendix C: Further methodological detail

Testing for statistical significance

• Administrative or census data are collected on all (or most) individuals or cases of interest. This includes Care Inspectorate data on ELC services, SSSC data on the workforce and ELC Census data on children registered for funded ELC. Therefore, any changes seen in the data are expected to reflect real changes among the population of interest, and significance testing is not required.
• Probability samples are designed to capture data on a group of people (or places, services etc) that are broadly reflective of the whole population. However, figures in the data are only an estimate of the true figures within the population and so should not be interpreted as being precise for the population as a whole. When considering changes seen in the data, we therefore use a test for statistical significance to estimate how confident we can be that this reflects an actual change in the population, given the amount of uncertainty we are prepared to accept. Changes in figures derived from probability samples are only included in this report when they have been tested as statistically significant. This includes data from SSELC, SHS and other national surveys.
• The 2017, 2022 and 2025 parent surveys were self-selecting, non-probability samples, meaning that it is not appropriate to test figures for statistical significance or to claim that findings definitely reflect the experiences and views of all parents of ELC-age children. However, the large sample sizes and general demographic comparability with the whole population, combined with appropriate weighting of the data, means that we can have reasonable confidence in the findings as a general indication of the situation in the broader population.