Rapid Review of National Qualifications experience 2020

Overview

Remit

The review will include considering evidence, providing commentary and recommendations around the following themes. A focus on those issues which are most pertinent to consideration of awarding methodology in 2021 if there is further significant disruption to learning and teaching and/or the cancellation of exams is key:

  1. Events following the cancellation of the 2020 examination diet.
  2. Advice and support given by SQA and Local Authorities to awarding centres on determining and quality assuring of estimates
  3. Approaches to the gathering and quality assurance of teacher/lecturer estimates, including where possible feedback from teachers/lecturers/Directors of Education, prior to submission to SQA about the perceived rigour in the evidence base for making estimates, e.g. prelim marks, classwork, summative and formative assessment until the schools closed in March.  This will include consideration of local quality assurance approaches taken by centres and Local Authorities to aid estimation; the conclusions reached by centres about estimated grades;  and decisions about whether or not to share estimates with learners at that time.   
  4. Exploration of alternative approaches to grading and moderating national qualifications in the context of the disruption caused by Covid-19, that would maintain standards and the credibility of qualifications in Scotland and deliver public confidence.
  5. Impact on young people (and their families) who did not receive what they believed their estimated grade submitted to be.
  6. Feedback from teachers/lecturers on the estimation process and the moderated grades which were awarded on 4th August
  7. Consideration of the post certification review process as a means to address the issues in 5 and 6 above.
  8. Confidential draft report to ministers on findings by 15th September 2020.
  9. Final report published by 30th September 2020.

Methodology

The review will comprise a range of methods to generate evidence and draw conclusions:

  1. Review of documentation.
  2. Panel discussions with key stakeholders (young people, parents, teachers, school leaders, local authorities).
  3. Interviews with key individuals (e.g. SQA, government, academics with specialism in assessment/statistics).
  4. Analysis, where applicable, of short position papers from stakeholder organisations.
  5. Analysis of how moderation was applied to centres’ estimated grades.

The primary source for recruitment of panel members will be national stakeholder groups, for example:

  1. Young people: Young Scot, Children’s Commissioner, Children in Scotland, Refugee Council, Colleges Scotland.
  2. Parents: via both national parent organisations.
  3. Teachers and Lecturers: Teaching unions, subject associations, Scottish Council of Independent Schools, Colleges Scotland, etc. We will also constitute a separate teacher panel, representative of geographical location, school type, subject spread etc., to provide an additional channel for teachers’ voices that is independent of formal associations.
  4. School leaders: via SLS and BOCSH.
  5. Local authorities: via ADES and COSLA.

Individuals to be interviewed will be selected via a combination of recommendations from the government and national agencies, nomination by SCDE (Scottish academics) and contacts of the Review Team.

All panels and interviews will be conducted via Microsoft Teams, and will be recorded with the permission of participants, who will undergo a process of informed consent. The research will be conducted in accordance with the BERA Ethics guidelines, following approval by the General University Ethics Panel at the University of Stirling.  Where possible, the identity of participants will remain confidential, and testimony will be anonymised in the report.

Timeline

  • Weeks 1 and 2: Initial discussions with the government and SQA, ethical approval, set up panels, read documentation, initial interviews
  • Week 3: Panel discussions and interviews.
  • Week 4: data analysis and draft report
  • Week 5: preliminary report to the government (by 15 September)
  • 30 September: final report

Members

  • Professor Mark Priestley – Principal Investigator
  • Dr Marina Shapira – Co-Investigator (with responsibility for the statistical aspects of the review)
  • Dr Andrea Priestley – Co-Investigator (leader of the strand investigating the experiences and perspectives of young people)
  • Michelle Ritchie – Research Assistant (Grade 6)
  • Dr Camilla Barnett – Research Assistant (Grade 6)

Additionally, the Review will employ two independent external reviewers, to provide advice on process and preliminary findings and to review the final report.

  • Professor Robert Davis – Professor of Religious and Cultural Education, and Director of the Robert Owen Centre for Educational Change at the University of Glasgow
  • Associate Professor Gill Wyness – Associate Professor of Economics, and Deputy Director of the Centre for Education Policy and Equalising Opportunities (CEPEO) at the UCL Institute of Education.

Documents