Scottish education system: knowledge utilisation study

5. Strand 3: Validation Survey Findings

This section of the report documents the main findings from the validation survey. It begins with details of who responded to the survey, presents the main survey findings and concludes with a summary. The survey covered the following areas:

• Engagement in research activities
• Sources of evidence used 
• Reasons for engaging with research
• Skills required for engaging with research
• Other necessary supports for engaging with research. 

5.1 Who responded to the survey?

One thousand and thirty-six survey responses were returned. Responses were received from staff across all local authorities in Scotland although the responses from each authority were not necessarily proportionate to the size of their staff complement. For example, Glasgow city, the largest employer of teachers in Scotland, returned 2% (19 staff) of the survey total while staff in North Lanarkshire, Falkirk, and Dundee city returned 105 (10%), 97 (9%), and 94 (9%) questionnaires respectively^[4]. As this was a self-selecting sample, it is possible that those who are actively engaged with research are over represented and therefore, it should not be treated as representative of all educational practitioners.

Almost all responses were from primary and secondary school staff (see Table 5.1) with relatively few returns from early years centres.

Table 5.1: Survey responses by Establishment (N=1,036)

Establishment	Percentage response
Primary school	47
Secondary school	45
Early years centre	3
Other	5
Total	100

Other included a diverse range of respondents such as: staff in additional support units, psychological services, college staff and other local authority-based staff.

Just over half of responses were from class teachers (53%). Only eight responses were received from early years practitioners. Table 5.2 summarises responses by staff designation.

Table 5.2: Survey responses by designation (N=1,036)

Role	Percentage response
Class teacher	53
Principal teacher	16
Head teacher/OIC*	14
DHT/Deputy OIC	6
Early Learning and Childcare practitioner	1
Other	10
Total	100

*- Officer in Charge

Other included: technicians, quality improvement officers, transition teachers, Attainment Advisors and other additional support staff.

The vast majority of staff (87%) worked full-time while the remainder were part-time. Just over half of the respondents (55%) indicated holding a post graduate diploma and 19% had a Masters degree or PhD. Just over half (51%) had worked in education for 16 years or more. Table 5.3 summarises respondents’ length of experience in education. 

Table 5.3: Survey responses by experience (N=1,036)

Experience	Percentage response
Probationer (1st year of employment)	4
Fully qualified and working in education for up to 5 years	13
Working in education for between 6 and 15 years	33
Working in education for between 16 and 25 years	29
Working in education for between 26 and 35 years	18
Working in education for 36 years or longer	4
Total	100

5.2 Engagement in research activities

Over half of respondents (59%) indicated that they were currently involved in one or more of the research activities listed in Table 5.4. These activities were generated from themes arising from the qualitative findings, the literature and insights from the Research Advisory Group. The most common activities reported were ‘wider school-based research’ (25%) or individual research on a classroom intervention (24%). Interestingly, one in five respondents reported that they were collaborating with colleagues in a classroom intervention. Just under one in ten respondents indicated being involved in a postgraduate qualification involving research activity. The majority of ‘other’ responses were elaborations of categories listed in the question. However, using research findings as part of leadership courses did feature prominently in these responses. 

Table 5.4: Staff engagement in research activities (N=1,036)

Research activity	Percentage response
Involved in wider school-based research	25
Individual research on a classroom intervention	24
Collaborative research on a classroom intervention	20
Involved in a professional reading group	19
Postgraduate qualification involving research activity	9
Other research activity (not listed in question)	9
Not currently engaged in any of the above	41

Interestingly, 41% respondents reported not being currently engaged in the listed research activities. A number of reasons for this are possible. The question asked about participants’ current engagement and it may be that they have been engaged with these activities previously. Equally, they may be involved in research activities that do not fit easily into the listed categories. It is also possible that these respondents have had limited engagement with research. 

5.3 Support for informing, planning and developing practice

Participants were asked to indicate which sources of support they used when they were informing, planning and developing practice within their establishment. Table 5.5 lists these sources and the relevant percentages.

Table 5.5: Sources of support 

Source/ Activity	Percentage				Number responding
	Extremely or somewhat helpful	Neither helpful or unhelpful	Unhelpful or extremely unhelpful	Never engaged with
Googling/ web searches	94	4	1	1	880
Career Long Professional Learning (CLPL) courses/ opportunities including national, local and independent providers	91	4	2	4	884
Taking part in structured collegiate discussions	89	5	2	3	882
Working with colleagues in other schools/centres	89	6	1	4	876
Accessing academic literature e.g. journal articles	77	12	3	8	870
Accessing Education Scotland webpages	76	13	6	5	882
Reading professional periodicals/newsletters e.g. TES, University newsletters	75	14	3	8	882
Analysing data available within your local authority or school e.g. attainment data; data from evaluations of interventions	75	13	6	6	880
Accessing data available at a national level e.g. government statistical reports	61	19	7	13	866
Accessing local authority material	59	23	6	10	878
Accessing teacher groups on Facebook/social media	57	11	3	29	856
Accessing GTCS webpages	54	28	6	12	876
Following recognised educationalists on Twitter	52	10	3	35	867
Working with colleagues in further and higher education	51	13	3	33	859
Accessing library (e.g. university library, reference library, educational resource centre)	50	20	3	27	857
Working with local authority officers	46	22	6	26	864
Accessing Education Endowment Foundation webpages (Inc. via the Education Scotland site)	40	13	2	45	861
Working with Education Scotland officers, including attainment advisers	36	16	6	42	853

The following four categories were rated by more than 90% of those who had accessed them (excluding those who had never engaged with) as either somewhat or extremely helpful in informing, planning and developing practice within their establishment.

• Googling/web searches
• Taking part in structured collegiate discussions
• CLPL courses/opportunities including national, local and independent providers
• Working with colleagues in other schools/centres. 

Findings from the survey correspond with that from our interviews where Google/web searches and the value of professional dialogue were frequently cited by teachers as common sources of information. There was tendency for those sources that were rated most highly to have also been the ones most frequently accessed. Overall, the helpfulness of the different sources was rated fairly highly by those who accessed them. Accessing academic articles was also rated as helpful by 77% of respondents, which may further suggest that the survey was completed by those who are more likely to be engaged with traditional sources of research. 

The Education Endowment Foundation (EEF) webpages and Working with Education Scotland officers were the two sources least likely to have been engaged with. Whilst 45% of respondents indicated they had never engaged with the EEF webpages, 40% felt they were helpful. Similarly, whilst 42% indicated they had never engaged with Education Scotland officers, 36% found this resource to be helpful.  

5.4 Use of research evidence

Respondents were asked to indicate how research evidence (including their own and/or existing data) was used. A total of 890 respondents answered this question. Table 5.6 summarises responses.

Table 5.6: Staff use of research evidence (N=890)

Activity	Percentage
To inform teaching and learning	92
To understand the impact of teaching and learning	78
To understand school or pupil characteristics	68
To develop their establishment’s improvement plan	59

In addition, 8% reported it being used in other ways. The majority of those responding with ‘other’ used the category to elaborate on the existing response categories, especially ‘to understand the impact of teaching and learning’. Others suggested that they had used research findings to promote their own or colleagues’ professional development and leadership.

5.5 Skills for engaging more fully with research

The survey sought information on the research skills that staff felt were required for them to engage more fully with research and with research evidence. Table 5.6 summarises findings from this question, indicating which skills staff believed they possessed, those they felt they needed support to develop, and those they felt were not a required skill.

Table 5.6: Skills for engaging more fully with research

Research skill	Percentage			Number responding
	Required – already have this skill	Required – need support to develop this skill	Not a required skill
Statistical analysis	42	47	11	864
Qualitative analysis	42	49	9	864
Report writing	65	23	12	858
Developing research questions	34	49	17	865
Identifying appropriate methodologies	44	47	9	872
Research design	24	54	22	857
Critically evaluating research findings	50	42	8	869
Identifying key literature	57	35	8	869

It is evident from Table 5.6 that substantial numbers of staff already feel they possess relevant research skills. This is particularly the case in relation to; Report writing (65%), Identifying key literature (57%) and Critically evaluating research findings (50%). These three skills are probably the most closely aligned areas with the training and development that staff will have experienced during their teacher training. Interestingly, virtually identical numbers of teachers (42%) identified statistical and qualitative analysis as skills they either already possess or skills they would need support in developing. 

Research design was the skill staff were least likely to identify as possessing (24%) and the one most likely to be identified as requiring support to develop (54%). Interestingly, this was also the area staff were most likely to reject as a required skill (22%).

While substantial numbers of staff identify themselves as possessing relevant research skills it is also the case that sizeable numbers felt they required support to develop these skills.

5.6 Engaging more fully with research evidence

Respondents were presented with a list of potential supports for engaging more fully with research. They were asked to indicate whether the support was required for engaging with research evidence and secondly, whether the support already existed or if it needed developing. Results from this exercise are presented in Table 5.7.

Table 5.7: Support for engaging more fully with research evidence

Nature of support	Percentage			Number responding
	Required – already present	Required – need developing	Not required
A culture in your establishment which recognises the value of engaging in research	38	57	5	879
Dedicated time to engage with research evidence	13	84	3	886
Opportunities to work with colleagues on research activities	21	74	5	882
A culture in the local authority which recognises the value of engaging in research	28	68	4	876
National advice and support on engaging with research evidence to inform, plan and develop practice	21	79	-	784
Partnerships with research specialists e.g. university researchers, research organisations	13	74	13	870

On all specified supports, a clear majority of staff indicated that each was both required and needed developing. This was particularly the case with; Dedicated time to engage with research evidence (84%), National advice and support on engaging with research evidence (79%), Opportunities to work with colleagues on research activities (74%) and Partnerships with research specialists (74%). More positively, over a third of staff (38%) indicated that their establishment had a culture that recognised the value of engaging in research. 

5.7 Respondents’ own comments on engaging with data and research

The final survey question was open-ended and allowed respondents to make additional relevant comments. In total, 215 responses were received. Table 5.8 summarises the main themes emerging from these responses. It is clear from the table a lack of time for engaging with research and data was the most common response by far.

Table 5.8: Thematic summary of respondents’ open ended comments (N=215) 

Nature of Comments	Percentage
Lack of time is the most significant barrier to engaging with research and data	57
Research is being used to support professional development and to inform practice	16
Support is needed to ensure the effective use of data and research. e.g. CLPL and improving school culture regarding valuing research	13
Barriers to accessing data and research, e.g.: on-line access to libraries and journals only available when enrolled in University course.	5
Available research is perceived as too broad. In particular, there is a lack of material on effective classroom based practices.	4
Research skills that teachers possess are undervalued in career development terms and teachers’ professional profile.	4
Research should have practical implications for practice.	4
Research is important but is a lower priority compared with other issues including; class size, workload, salary and support.	2

5.8 Summary of survey findings

• Over half of respondents (59%) indicated that they were currently involved in one or more research activities. 
• The supports rated as most helpful in planning and developing practice were: taking part in structured collegiate discussions, CLPL courses/opportunities or working with colleagues in other schools/centres. These seemed to be those that offered both the dedicated time and opportunity to collaborate with colleagues. Practitioners also regularly used web searches to find relevant evidence. 
• Almost four out of five respondents reported using data/research material to inform teaching and learning while just over two thirds indicated its use in understanding the impact of teaching and learning.
• Substantial numbers of staff already felt in possession of relevant research skills. This was particularly the case in relation to; report writing (65%), identifying key literature (57%) and critically evaluating research findings (50%). 
• There was also evidence that respondents felt they needed support to develop their research particularly in relation to, for example, analysis of quantitative and qualitative data. 
• A large majority of respondents indicated a need for the following; dedicated time to engage with research evidence (84%), national advice and support on engaging with research evidence (79%), opportunities to work with colleagues on research activities (74%) and partnerships with research specialists (74%).
• The most frequently cited open comment by staff highlighted a lack of dedicated time to engage with research and data.