11. Views On Designing Trials For Scalability
The Scottish Government wishes to maximise the value of the ELC programme by ensuring that the lessons learned from trials can be applied to similar ELC settings in Scotland. Consideration is required on what needs to be factored into the design of the trials to enable this.
Question 10: How can we design the trials in such a way as to ensure scalability?
11.1 44 respondents addressed this question. An overarching theme across a wide range of responses was that trials should be set up from the outset as learning mechanisms intended to produce lessons of more general applicability.
11.2 A few respondents recommended that thorough groundwork be undertaken prior to setting up trials including examining evaluations of trials in other jurisdictions, consulting with key stakeholders and ensuring common understandings of concepts such as "playful approaches to arts and creativity".
11.3 17 respondents from a wide range of sectors emphasised the need for trials to test a variety of settings and provider models; patterns of provision ( e.g. seasonal); scale of budget; size; rurality; "hard to reach" communities; outdoor space access. By doing so, they envisaged that aspects of the trials would be relevant to areas throughout Scotland.
11.4 11 respondents recommended that robust evaluative methods are used to assess the trials. Standardised approaches to evaluation with measurable outcomes were advocated, with appropriate indicators and data collection mechanisms in place from the start. One respondent called for "experts" to conduct the evaluations which some emphasised should be done with rigour. The challenge of time needed for outcomes to be realised was highlighted once again, with one respondent identifying the need for:
".....a reasonable timeframe for the trials so that approaches can be applied effectively without appearing to be a 'quick fix' and ending up being ineffective in the long term when rolled out" (Voluntary organisation).
11.5 A few respondents specified the importance of establishing baseline data; comparator areas where the same provision is trialled but in different settings; representative samples; and qualitative data collection during interviews with parents and other key stakeholders, in addition to quantitative measurement.
11.6 It was suggested that the Care Inspectorate Registration Team could usefully provide the Scottish Government with information on the pattern and location of variation to existing services and any new applications to register services in order to provide a database of provision as a backcloth to the trials.
11.7 28 respondents identified broad ways to set up the trials to make them mechanisms for shared learning. A few recommended establishing "cluster groups" which involve all providers in one area, supported by regional funding.
11.8 Two respondents referred to the "test of change" model established through the Early Years Collaborative as having potential for application to the trials. A voluntary organisation suggested action research as an appropriate approach to learning from the trials, or a "quality improvement model".
11.9 Two respondents emphasised their view that trials should be kept simple; a local authority recommended focusing trials on key themes and core concepts. One representative body suggested that standardisation across trials, for example, in childminder pay, would aid scalability.
11.10 A common view amongst five respondents from different sectors was that the trials should be underpinned by realistic budgets to provide a genuine picture of potential for expansion. One respondent, however, cautioned that limited funds should not be wasted:
"The government should ensure that what money is available for trials is not wasted on never to be employed initiatives - we know what the problems are with early learning and care provision. Hence a handful of excellently funded initiatives targeted in specific local authorities and issues (universality, rural provision, disability services, poverty mitigation childminding, etc.), because of their nature, would be better than a plethora of trials that didn't amount to anything" (Other body).
11.11 Eight respondents from a range of sectors identified robust dissemination of lessons, including sharing events run by the Scottish Government, as necessary to ensure issues of scalability can be addressed. One local authority suggested that there are currently effective local models of provision which could be usefully shared and scaled up and these could be the focus of early dissemination.
11.12 Several local authorities and others cautioned that it is unlikely that successful approaches can be transported wholesale to settings elsewhere in Scotland, but that aspects of approaches are more likely to be relevant to other areas. In learning from trials, they emphasised the need to identify pilots which could be tailored for other local circumstances, with aspects extrapolated for scaling up elsewhere.
11.13 Four respondents from different sectors emphasised the need for the supporting infrastructure for trials to be recognised as crucial and not simply the initiatives themselves. For example, appropriate funding and staffing were crucial for other areas looking to replicate successful aspects of trials, demonstrating the importance of transparency over the structural resources required for the trials.
Implications for proposed trials
11.14 In order for successful aspects of trials to be scaled up, respondents identified a number of design features to be incorporated from the start. These included ensuring they are set up as learning mechanisms; are simple and focused to enable lessons to be clearly identifiable; that thorough groundwork is undertaken to make sure they are evidence-based and testing clearly specific concepts and approaches; that they test a variety of settings which will produce relevant learning for different locations across Scotland; that they are underpinned with robust evaluative methods and have firm baseline data; that budgets for the trials are realistic and provide a genuine picture of funding and supporting infrastructure required; and that dissemination of lessons is robust and information widely shared post-trial.