Coronavirus (COVID-19) - replacement of National Qualifications exams in 2021 session: equalities impact and Fairer Scotland Duty assessment

5. Evidence and Key Findings

This section sets out the evidence the Scottish Government has considered in the development of this policy. Key findings from this evidence are noted:

It includes:

• The Scottish Government’s EQIA on ^[11]Teacher and Lecturer Estimates in 2020.
• Professor Priestley’s independent review of national qualifications experience 2020.
• SQA consultation exercises 2020.
• School attendance figures.
• Stakeholder views (including the views of the teaching profession, young people, and parents).

The evidence compiled in respect of the Scottish Government’s EQIA on Teacher and Lecturer Estimates in 2020 using teacher and lecturer grade estimates has been reconsidered, and the conclusions noted.

The Scottish Government has analysed the available data and evidence on the impact for young people who share protected characteristics, where this has been available.

5.1 Professor Priestley independent review of qualifications experience

Evidence

This review, which reported in early October, drew on a range of evidence, including stakeholder testimony (generated in panel and individual interviews) and analysis of relevant documentation. Young people, parents/carers, teachers, senior school leaders, local authorities, SQA and government officials were involved in the panel discussions. The report sets out who was engaged but for the purposes of this assessment, representatives from Children in Scotland, Scottish Youth Parliament, Children & Young People’s Commissioner Scotland, SQA: Where’s our say?, were amongst those engaged in panel discussions.

All the recommendations have been carefully considered to mitigate against any adverse impacts of any policy decisions on the approach to be taken in 2021. The report contained 9 recommendations, of which two are of particular relevance:

1. Recommendation 1 - Suspension of the National 5 examinations diet in 2021, with qualifications awarded on the basis of centre estimation based upon validated assessments.

2. Recommendation 2 - The development of a nationally recognised, fully transparent and proportionate system for moderation of centre-based assessment.

Key Findings

This independent review of the 2020 experience identified a number of factors related to equity and wellbeing which are pertinent when considering any policy decisions that will impact on assessment and certification in 2021:

• The extent to which young people reported that extended periods of illness or extenuating circumstances were not considered.
• The extent to which estimates of learner grades varied and were moderated (i.e. changed), especially between different subjects, year groups or individuals.
• The equity implications of an over-reliance on statistical approaches which are based on historical cohort data.
• The impact of poverty, especially lack of access to digital technology and/or Wi-Fi, on young people’s ability to complete work digitally.
• The negative impact of young people’s health and wellbeing, particularly those with additional support needs.

In respect of Recommendation 1 of the Priestley Review – Suspension of the National 5 exams in 2021 – this was based on panel discussions statements that schools lacked the capacity, in an already disrupted school year, to prepare learners for exams and develop robust evidence for centre estimates of grades. The report cites it is based on the following rationale:

• National 5 is not a leaving qualification for the majority of candidates^[24], and therefore less high-stakes for most.
• National 5 involves large numbers of candidates – cancelling the examinations diet would enable considerable space to be freed for the arguably more important Higher and Advanced Higher examinations (both of which involve smaller numbers of students, and can be dispersed more readily across school building).
• The cohorts currently entering school year S3 and S4 have already experienced considerable disruption to teaching time; cancelling the National 5 examinations would allow for additional teaching time in the summer of 2021 for both cohorts.

The Priestley Review identified groups of learners who may have been disproportionally affected by the results process in session 19/20. These groups include learners who experienced extended periods of illness, experienced extenuating circumstances such as bereavement, learners who were care experienced and learners with little or no access to IT (including broadband connections). It is not possible to identify whether the learners consulted shared any protected characteristics. However, there would be a likely link between these groups and the ability to fully prepare for National Qualifications in session 2020/21. Removal of examinations and the reduction in course content would reduce the inequity faced by these groups by providing greater opportunity for learning.

The groups of learners more disproportionately affected by the results processes in session 19/20 overlap those who may be more affected by Covid-19. Learners who fall ill from Covid-19, or are forced to shield due to family members or those they have been in close contact with Covid-19, are more likely to come from areas of deprivation. Likewise pupils from more deprived deciles may lack the technology required to fully engage in learning.

We also know that people with disabilities and minority ethnic groups are disproportionately affected by Covid-19, which would suggest that school children of these protected characteristics would also feel the effect.

We are not aware of any other evidence from the Priestley Review that indicates whether the cancellation of exams could impact positively or negatively on any of the protected characteristics.

The Priestley Review’s findings and recommendations were considered and discussed by the stakeholders listed in section 4 above, and have informed policy decisions made by the Scottish Government in respect of the approach to be taken for 2021, and in particular led to the decision around the cancellation of National 5 exams.

5.2 SQA Consultation exercise

Evidence

At the start of the 2020-21 term, SQA undertook two consultation exercises^[12]. The first was a technical consultation on proposed modifications to National 5, Higher and Advanced Higher course assessments. Practitioners (teachers and lecturers) were the priority group for this consultation but to ensure transparency it was open to the public. It should be noted that it is not possible to breakdown the socio-economic backgrounds of consultees, or whether the respondents shared any protected characteristics. These were broadly recorded as teachers/practitioners, parents/carers, learners or ‘other’. As part of their consultation, they also invited comments on issues of equality and accessibility (see Annex D).

Around 22,000 people accessed the SQA consultation on modifications to course assessments; and just under 17,000 individual responses were received (60% from practitioners; 22% from candidates and 16% from parents and carers). The proposals that were strongly supported in the consultation document included one or more of the following: the removal of coursework or some elements of exam papers; shortened exam papers; or removal of practical assessments for National 5, Higher and Advanced Higher.

SQA’s EQIA considered the potential impact of course assessment changes on candidates who share protected characteristics along with how any potential negative impacts identified could be mitigated. In addition, other groups of learners, such as those who are socio-economically disadvantaged on the basis of Scottish Index of Multiple Deprivation (SIMD), or who experience other circumstances that present barriers to accessing qualifications, were considered. The SQA’s CRWIA^[13] set out to assess the impact of the proposed modifications to National 5, Higher and Advanced Higher course assessments for session 2020–21 on young people.

The SQA consultation on the timetable for the exam diet was a private consultation with its Timetable Advisory Group, Appointees/Markers, Young Scot, the Scottish Youth Parliament and the Children and Young Peoples’ Commissioner. Around 7,000 people were consulted, with 1,000 responses received. It examined options to make changes to the exam timetable to allow more time to teaching and learning lost in the 2019/20 and 2020/21 academic years. The options were to delay the existing timetable by 3 weeks or to change the order of the exams (having N5 exams first followed by Highers and Advanced Highers).

Key Findings

SQA consultation exercise into potential modifications for National 5, Higher and Advanced Higher courses found there was broad support (over 65.6%) for making changes to course assessment in light of continued Covid-related disruption; increasing opportunities for learning and teaching (56.8%); more flexible approaches to assessment for learners (67.1%)

Their consultation also included questions on equalities and the potential impact of any changes and mitigating actions. See Annex E. SQA have reported that the responses received to their consultation focussed on general issues of equity rather than protected characteristics. For example, learners may face difficulties accessing ICT (including issues relating to the digital divide and correlation with more deprived areas); the disparity between what support learners of different backgrounds may get from home (e.g. those from more deprived areas perhaps accessing less support from parents/carers); differential access to blended learning experiences; learners with ASN or those with underlying health issued who will need to stringently social distance could be disadvantaged by exams.

SQA’s EQIA^[14] have considered the potential impacts to learners of: the removal of question paper (i.e. the exam); changes to question papers; removal of and/or modification of coursework. They have undertaken this exercise for each course at National 5, Higher and Advanced Higher. This includes consideration of the impact of these changes to all protected characteristics groups. SQA’s work examined the effect that continuing with exams would pose to protected characteristics groupings. For example:

• Disability – learners who have difficulty concentrating for extended period and maintaining focus may find exams under stringent social distancing measures stressful and have difficulties demonstrating their attainment.
• Religion and belief – Ramadan is due to fall during the exam diet 2021 and those fasting because of religious beliefs may experiences a higher level of fatigue.
• Pregnancy and maternity and gender re-assignment – learners who are due to give birth during the exam period be adversely affected by the reliance on an exam, and their ability to prepare for it.

SQA are continuing to provide guidance, arrangements and services to mitigate negative impacts of the ACM.

The replacement of exams with the ACM intends to impact positively on all groups of learners, including those with protected characteristics. The quality assurances processes to be put in place will help teachers to make fair, accurate and consistent judgements about learner awards. Resources and guidance will be provided by SQA to support this, including guidance on the fair and equitable treatment of all learners, information on bias, and advice on assessment arrangements for learners who have additional support needs or a disability. The model will have flexibility, fairness and equity at the forefront, acknowledging the potentially lengthy or detrimental impacts of disruption to learning, particular for learners with a disability, suffering from illness, poverty and deprivation, being a carer or care experienced. Flexibility in gathering evidence will support learners with protected characteristics (e.g. pregnancy/maternity, mental health needs) who may have normally found exams problematic. Furthermore, SQA are committed to working with the schools/colleges to consider individual requests for reasonable adjustments.

SQA have now concluded their consultation exercise on their appeals process for 2021.

5.3 School attendance figures

Evidence

Scottish Government has monitored school absence levels since schools returned in August 2020. Data is published here.

This data allows consideration of, amongst other factors, time series data showing how attendance rates vary over time and by stage, SIMD and Local Authority. Annex A and B contains fuller versions of the tables used below.

The data (see table “Percentage of openings showing pupils were not in school because of any Covid-19 related reason”) shows that since early October, the prevalence of Covid-19 amongst secondary school aged learners has generally worsened. Specifically, whilst attendance levels were initially above 90% for the first few weeks of session 2020/2021, the attendance levels of Senior Phase pupils since September have been more variable and lower.

Attendance and Covid-related absences by stage on last Monday of each month:

	Overall attendance rate			Covid-related absence rate
Date	28 Sept	26 Oct	30 Nov	28 Sept	26 Oct	30 Nov
Secondary schools	90.9%	90.3%	87.1%	1.4%	2.9%	4.1%
S4	89%	88.9%	85.6%	1.6%	2.9%	4.2%
S5	90.6%	89.2%	87.5%	1.4%	3.6%	3.4%
S6	91.5%	90.2%	88.1%	1.5%	3.5%	3.7%

Absence rates vary within SIMD quartiles. Whilst it is ordinarily the case that pupils from more deprived areas have lower attendance rates, Covid-19 absences have added a further dimension. The table below shows the percentage of times in which pupils had at least 90% attendance, organised by stage and showing attendance rates of SIMD Q1 and Q5 (full data in Annex C).

Covid-related absences by SIMD and stage from 17 August to 8 December:

Stage	Stage	Percentage of time pupils were in at least 90% attendance
All Secondary	62.6%
		Most deprived 20%	Least deprived 20%
S4	All S4 - 61.9%	52%	67.3%
S5	All S5 - 62.6%	55.5%	67.3%
S6	All S6 - 63.1%	53.9%	68.2%

Absence rates also vary across Scotland and across Local Authorities. Within the data tableau, a Local Authority map - “CYP Attendance LA Map” – is available and shows this variability in attendance rates, and Covid-19 related absences across Local Authorities. For example on 10 November 2020 8% of total school absences within West Dunbartonshire were Covid-19 related, compared with 0.2% of absences on the same day within Na h-Eileanan Siar.

When considering protected characteristics (Annex C), it is noted that whilst attendance rates across Ethnic Background do vary across ethnic categories, they do not show any distinct pattern. For example, whilst pupils from an Asian background tended to have an absence rate “due to any Covid-19 related reason” than was greater than the “all backgrounds” figure, pupils from an African/Black/Caribbean background tended to be below “all backgrounds” average.

Key Findings

The Scottish Government’s online dashboard ^[15]contains and presents school attendance and absence information. Attendance and absence figures by stage (i.e. school year group) by SIMD, by Local Authority and by Ethnic Background have been analysed (Annex A to C).

This data shows the following for Senior Phase pupils:

• The attendance rate continued to decline through August to November.
• The percentage of pupil absences due to Covid-19 rises through the year.
• That the overall absence rate in the most deprived areas is greater than that of young people in the least deprived areas.
• That there is variation in Covid-19 related absences across Local Authorities
• There is variation in Covid-19 related absences across Ethnic Background

National 5, Higher and Advanced Higher courses have a very clear structure based around a time allocation for each course. National 5 and Higher courses are allocated 240 hours of study, Advanced Higher are allocated 320 hours. Included in these time allocations is time for assessment and/or examination preparation.

The variability in attendance rates across Local Authorities, across deprivation deciles and ethnic backgrounds, will mean that across Scotland there would be an inequity in the time available to learners to successfully complete courses. By removing examinations, some course content and amending assessment requirements, sufficient time should now be available for the completion of course, even by those most affected by reduced attendance. This could have a positive impact on some learners who are either shielding or in a shielding household, who would not be able to physical attend an exam.

We are not aware of any other specific evidence from this data that indicates whether the cancellation of exams could impact positively or negatively on any of the protected characteristics.

5.4 Stakeholder dialogue

Evidence

There has been ongoing discussion with key stakeholders including but not limited to young people, parents and teachers since August 2020 to gauge opinions on learning and teaching experience for all learners, and assessment approaches for 2021, in light of ongoing public health concerns.

This includes dedicated sessions with key stakeholder groupings prior to the policy decision to replace National 5 exams with an alternative certification model in October, and the decision that then followed in respect of Higher and Advanced Highers in December 2020.

To summarise, the stakeholders engaged include:

• CERG
• QCG
• CAB
• Local authorities (via Education Scotland)
• College principals
• National Qualifications 2021 Group
• Young people
• Parents/carers

The views of each stakeholder grouping are outlined in Section 6 below.

Key Findings

• Practitioners

The proposal to postpone National 5 exams was reflected in the BOSCH position paper submitted in response to SQA’s consultation exercise. This, alongside the views expressed by EIS and other teacher representative bodies, have expressed that the differing and often significant impact of Covid-19 on many young people, particularly those from less advantaged backgrounds, mean that an exam diet in 2021 would not be in the best interest of young people and potentially unfair to those who have faced greater disruption.

We are not aware of any evidence from the Bosch Position Paper that indicates whether the cancellation of exams could impact positively or negatively on any of the protected characteristics.

The anecdotal data from local authorities (via Education Scotland) reported that staff and pupil absences were having a variable impact on their ability to deliver Higher and Advanced Higher courses. They also reported equity-related concerns about a potential disproportionate impact on particular groups of learners, including those living in poverty.

The EIS welcomed the decision to cancel N5 exams and prioritise Higher exams. However, during the National Qualifications 2021 (NQ21) Group meeting on 13 November 2020 where contingency planning in relation to Higher and Advanced Higher exams was considered, the EIS made the case for cancelling these exams and for doing so as soon as possible. The following reasons were cited by the EIS for a full cancellation of the 2021 exam diet before February:

• the growing confidence that exists in relation to delivering an alternative assessment model for National 5s.
• the additional teaching time (c.4-8 weeks) that could be made available for Higher and Advanced Higher pupils and students, though that would largely be after the estimation and awarding processes are complete.
• the limited time available to teachers and lecturers to gather alternative evidence should a decision to cancel Higher and Advanced Higher exams be made at the February break.
• the significant logistical issues that exist in relation to delivering exams for Higher classes which in some cases are only marginally smaller than National 5 classes.

We are not aware of any further evidence from the teaching profession that indicates whether the cancellation of exams could impact positively or negatively on any of the protected characteristics.

• Colleges and Universities

Dialogue took place with Colleges Scotland and Universities Scotland who confirmed their understanding of the challenges and confirmed their willingness to adapt and work with the Scottish Government and others on the outcome of any policy change in relation to holding exams in 2021.

We are not aware of any evidence from the FE/HE sector that indicates whether the cancellation of exams could impact positively or negatively on any of the protected characteristics.

• Young people

As detailed above, the Ministerial led panel discussions with young people illustrated the breadth of views on whether or not the exam diet for 2021 should proceed as normal. These young people represented different geographic areas and socio-economic backgrounds, and had a range of experiences of qualifications and assessment across the Senior Phase of school.

In these discussions, there were mixed opinions on the need for exams to go ahead, particularly those “higher stake” exams (e.g. Higher and Advanced Higher) given their importance in securing further or higher education places and concerns regarding the perceived value of a qualification based solely on continuous assessment. However, there was a high degree of anxiety being reported by young people in respect of the current proposals and the impact on those learners with additional support needs or with protected characteristics or from certain socio-economic backgrounds, but also on the loss of learning and teaching for those affected by Covid-19.

Extra evidence via the “lockdown lowdown” reports from Young Scot, Youthlink and the Scottish Youth Parliament were also considered. These were undertaken at key points throughout 2020 to gauge young people’s views on the impact of Covid-19 on their lives, including but not limited to their educational experiences. Their reports broke down their data according to rural/urban experiences, SIMD, age, gender, and ethnicity.

Their first report ^[16](published in May 2020) found that 49% of their respondents were moderately or extremely concerned about exams and coursework. It also showed that respondents in areas of higher deprivation were more concerned about this issue than those in lower Quintiles. High levels of concern over exams or coursework were reported amongst learners who identified themselves as Asian/Asian British or Black, African, Caribbean or Black British.

Their second report^[17] published in November 2020 focussed on the concerns of young people as lockdown restrictions changed. It also ran alongside focus groups help by the Scottish Youth Parliament with young people whose voices are seldom heard^[18]. A pressing concern from respondents was around the impact of Covid-19 on their exams and calls for clarity on whether or not they would go ahead; anxiety amongst those whose exams had already been cancelled (National 5 candidates) and on the place of exams more generally. It is not possible to extrapolate the concern amongst learners with particular protected characteristics.

• Race/religion

The report on the voices of seldom heard groups considered the views of young people from black, Asian and minority ethnic communities; disabilities and additional support needs; care experience; experience of the criminal justice system; and young carers. As above it touched on a multitude of issues, including education. Specific questions were posed in relation to the impact of not sitting exams. The report summarises that the (qualitative) responses expressed little concern with not having exams.

• Parents/carers

NPFS ^[19] carried out a survey in early October which showed 58% of parents being opposed to a full cancellation of exams; 74% being in favour of an approach that saw Higher and Advanced Higher exams being prioritised; and 70% agreeing that work should continue towards the exams as normal but that exams should be cancelled later in the academic year if the public health situation worsens.

A further NPFS survey was carried out in November 2020 to gather the views of parents and carers of pupils sitting Higher and Advanced Highers to better understand how Covid-19 has impacted on their learning and preparations for exams in 2021.

The November survey had responses from 4,196 parents from across all local authority areas in Scotland. There were notably higher participation levels across the central belt local authorities and particularly from those currently in Covid-19 local protection level 4 (the highest proportion of responses were from South Lanarkshire - 11.3% of respondents, South Ayrshire - 10.8%, North Lanarkshire – 9.1% and Glasgow – 7.7%). Key pertinent points were:

• Almost all respondents (98%) felt that their child’s learning in the Senior Phase had been disrupted at least a little due to Covid-19 with almost three-quarters (74%) indicating it had been disrupted either a great deal or a lot.
• Responses on the level of support received during lockdown or when self-isolating, varied. Around 21% said they had received either ‘a great deal/a lot’ of support, compared with 43% saying the support was ‘moderate’ and 35% rating it as ‘little/none at all’.
• Over half (around 57%) of respondents were either ‘a little/not at all’ confident that their child would have an equal chance of attaining the grades they deserve by sitting their Higher/Advanced Higher exams as normal in 2021; compared with around 15% who had either a ‘great/a lot of’ confidence.
• Half (51%) of respondents favoured cancellation of 2021 Exam Diet due to their regions current health situation, with the remaining respondents either undecided (23%) or content for it to proceed (26%).
• A clear consensus emerged on the timing of any potential announcement regarding the 2021 exam diet, with over 75% stating this should happen before Christmas, compared with 22% who favoured an announcement before the February 2021 half term break.
• Just over half (55%) of respondents had either a great deal or a lot of confidence in teacher judgement being used in the absence of an exam.

We are not aware of any further evidence from parents or carers that indicates whether the cancellation of exams could impact positively or negatively on any of the protected characteristics.

• Governance groups

The views of stakeholders were shared with and discussed with the CERG, QCG and CAB.

We are not aware of any evidence from these discussions with stakeholders that indicates whether the cancellation of exams could impact positively or negatively on any of the protected characteristics.