It's Our Future - Independent Review of Qualifications and Assessment: report

3. Reform in Scotland and International Comparisons

3.1 Overview

In Chapter One, we explained why this Review was commissioned. In the previous Chapter we described how we undertook the Review and what we learned through the three phases of engagement.

In the first part of this Chapter, we look at what we might learn from our experience of CfE reform. In the second part of this Chapter, we discuss some of the approaches to qualifications and assessment in other jurisdictions.

Reform in Scotland

3.2 Learning from education reform: Curriculum for Excellence and the Senior Phase in Schools and Colleges

The mirror that the OECD Independent Reviews (OECD, 2021, Stobart, 2021) held up to the Senior Phase experience in Scotland was familiar to many who work in education. Qualifications and assessment in the Senior Phase had become very different from what had been envisaged in the early days of CfE. Building the Curriculum 5a framework for assessment (2011), provided the original national guidance on assessment and qualifications.

“The curriculum in the Senior Phase should be designed to meet the Principles of curriculum design and the entitlements set out for all learners in Building the Curriculum 3 and should include the four aspects of the curriculum (the ethos and life of the school as a community, curriculum areas and subjects, interdisciplinary learning and opportunities for personal achievement).” (p16).

CfE, at its outset, was regarded as a leader in the field for countries seeking to design curricula fit for the 21^st century (OECD, 2021). However, it was not without critics (Priestley & Humes, 2010), and it began to be enacted in practice, gaps began to emerge between the original intentions and what was happening in schools and classrooms. Many of the ideas that have emerged about the future of qualifications and assessment during this Review are very similar to those that were proposed in the early days of CfE. As part of the Review, we asked a number of people who had been involved in the original development of CfE why they believed this had happened. Several reasons were suggested:

• Insufficient attention had been paid in CfE planning to what would happen after the publication of the documentation.
• The original plan for CfE had been to continue to involve schools and teachers in developing ideas over time but the financial climate changed and the kinds of collaborative development that had been intended were no longer possible.
• The intention to leave the curriculum as open as possible to allow teachers to design programmes to meet the needs of their own students led to requests for clarity. Responding to requests led to the generation of vast quantities of guidance (over 20,000 pages) Too much guidance is as bad as too little.
• Teachers felt that all of the responsibility for putting CfE into practice was theirs. When they asked for support, more guidance was developed. What teachers wanted was practical support, for example, examples of classroom materials for them to adapt to meet the needs of their learners, time to develop ideas and practices working with colleagues in school, opportunities to explore practice across schools.
• The introduction of the qualifications in the Senior Phase later in the implementation process was well intentioned. This decision was intended to provide the space for schools to think about the curriculum before assessment and qualifications. In reality it had resulted in many secondary schools paying little attention to CfE, arguing that they had to wait to see what the qualifications looked like before thinking about teaching and learning. This, it was argued, had led to the position where CfE was seen by some as the Early Years and primary curriculum.
• The task of building shared national standards was underestimated and moderation processes became too bureaucratic.
• The opportunities for those beyond schools and colleges to understand CfE were variable. Parents and employers, in particular, were unsure of what to expect of CfE and what really mattered in qualifications.
• The decision to have an external component to N5 but not to N4 was well intentioned. However, in the Scottish context where examinations have been such a strong part of the education system over time, the decision had led to a perception amongst many parents that N5 was the qualification they wanted their children to achieve. This led to many young people taking courses that did not meet their needs.

There are lessons to be learned from that experience for this Review. These include:

• There has to be a plan for moving beyond the ideas in this document.
• Careful consideration has to be given to the nature of professional learning.
• Resources developed collaboratively with teachers should be available for teachers to adapt to reflect the needs of their learners.
• Resource implications have to be acknowledged.
• Roles and responsibilities, nationally and locally, have to be agreed and linked clearly to supporting teachers and learners.
• The importance of teachers developing and sharing standards should be recognised as an issue of equity - teachers must have a shared understanding to be able to give learners accurate feedback.
• Practices should focus on learning and actions should be agreed to ensure that they do not become bureaucratic. (Priestley et al, 2023)
• The CfE aim of moving to two-year delivery of courses – and bypassing of lower levels of courses and moving to exit point assessment. Whilst a small number of schools did attempt to implement this approach – it was not universally accepted.

CfE recognised the central role that subjects play in education but argued for more than knowledge of subjects. The intentions of CfE state that the curriculum as experienced by every learner should include opportunities to study curriculum areas and subjects in depth but should also include engagement with the school as a community, opportunities to discover how learning connects across different areas of the curriculum and opportunities to personalise learning through personal achievement. What were described in the document as ‘next generation qualifications’ as offered by SQA and other providers were intended to reflect the values and aspirations of CfE.

The nature of CfE as outlined above is such that formal qualifications were unlikely to be able to meet all of the CfE requirements. However, the original design of the new qualifications provided a closer alignment between CfE and qualifications than the current qualifications. The original National Qualifications were designed to gather evidence on what learners knew and understood and their skills and competences. Qualifications included a broad range of approaches to assessment. Ways of gathering evidence differed subject by subject, for example, art and design included a folio of work and physics a scientific project. Decisions were taken by panels of subject specialists, based on what they perceived to be the best way to demonstrate learners’ achievements in the subject. Skills and competences were built into the programmes of study and recognised in the awards.

However, the Priestley report (2020) following the COVID-19 experience of 2019, reported a very different learner experience in the Senior Phase from that envisaged by the developers of CfE qualifications. These findings have subsequently been supported by evidence emerging in this Independent Review from learners, parents and carers and employers. Section 1.11 above describes some of Priestley’s key findings and concerns referred to be Morgan (2018) and Muir (2022), including the prevalence of “two term dashes”, repetitive and formulaic learning experiences, lack of challenge for some learners, limited development of broader skills, little or no cross-subject learning and stress and negative impact on wellbeing caused by the intense focus on examinations. These findings have subsequently been supported by evidence emerging in this Review from learners, parents and carers and employers.

Many learners spoken to as part of this Review reported high levels of stress caused by the overt focus on examinations.

“Instead of one exam, the SQA should give smaller assessments throughout the year which is less stress. Assess on units instead of the whole topics together.” – CCG member – (CCG member, Intercultural Youth Scotland)

“I find exams really hard mentally so I think a good balance between coursework and exams would definitely benefit more people as I know there are people who prefer exams too!” – (CCG member, Scottish Youth Parliament)

Similar findings were also reported in the Muir Review (2022) and in the National Discussion (2023). Angela Morgan (2018) in her review of Support for Learning: All our Children and All their Potential also expressed concern about “the narrative around achievement and attainment.”

Angela Morgan further reported that learners with additional support needs often found accessing alternative assessment arrangements difficult and that these factors combined to have a negative impact on the wellbeing and educational experiences of young people.

“Many MSYPs shared their experiences of technology as learners with additional support needs and felt that the use of laptops was extremely helpful.” – (CCG member, Scottish Youth Parliament)

“As a parent of a child with dyslexia, using digital technology would put her more on an equal ground.” – (Individual, Phase Two consultation)

During the Review many Headteachers reported that the evidence gathered to judge the quality of education in schools, accountability metrics, was driving behaviours in schools. Schools felt judged not by how well they were serving the needs of every learner but by how many learners achieved National 5 and Higher qualifications. Learners, it was suggested, were at times advised to take qualifications that would be more valuable for school metrics than qualifications best suited to their strengths. Learners constantly referred to being over-examined. Teachers reported a sense of constant pressure to “get through the curriculum” in a series of “two term dashes”. There was little satisfaction amongst teachers or learners with qualifications as taught and learned in the Senior Phase, little sense of deep learning in schools and classrooms, only a sense of people in a system driven to find ever more efficient ways to predict and practise for examinations. All learning involves practice and making sure that learners understand how examinations work and how to perform well in them is important, but, taken to extreme, it will dominate learning and teaching, narrow the curriculum and risked demotivating a high percentage of learners.

3.3 Learning from International and UK Contexts

In addition to reflecting on our own history and previous attempts at reform in CfE, the Review looked beyond Scotland. Our research involved a rapid desk-based literature review of a small number of existing comparative studies. This was followed by a series of informal meetings with individuals with experience of education systems in other jurisdictions. Our objective in this was to expand our understanding of what qualifications and assessment practices might be possible, i.e., approaches that were already in practice in other high achieving countries and might, therefore be possible in Scotland, if the circumstances are conducive.

We were aware that assessment systems are products of a country’s culture, history, traditions and economic system and for these reasons what works in one country may not in another. As Stobart (2021) states, “student assessment systems are essentially a social, rather than scientific, process which reflects the history and culture within which they occur.” We were, therefore, not seeking to identify a system that could be imported into Scotland.

The Review looked at several existing international comparative reviews of education systems, including Gordon Stobart’s Upper-secondary education student assessment in Scotland (2021), SQA’s ‘Comparative study of organisational structures in high performing jurisdictions and how they support successful, assessment, curriculum, and qualifications’ (2023). Sharon O’Donnell’s report for the National Council for Curriculum Ireland ‘Upper Secondary Education in Nine Jurisdictions (2018) and Geoff Masters’ Review of High Performing Countries (2022).

Stobart’s report looked at nine systems which he indicated shared resemblances with Scotland’s. These included: England, Hong Kong, China, Ireland, Wales. Stobart also looked at France, New Zealand, Norway, Ontario and Queensland, which he noted had “instructive differences” to Scotland. The SQA report looked at practice in 11 jurisdictions: Norway, Finland, British Columbia, Wales, Ireland, England, New South Wales, New Zealand, France, Iceland and Singapore. (SQA, 2022). O’Donnell’s (2018) looked at the following nine jurisdictions: England, Finland, France, Ireland, the Netherlands, New Zealand, Ontario, Queensland and Sweden.

Informal meetings were also held with individuals with experience in the following jurisdictions: Canada, Denmark, Ireland, New Zealand, Norway, Queensland, Singapore and Wales.

It is important to note the limitations of this aspect of the Review. The purpose was to provide a flavour of practices taking place beyond Scotland. Some of the key messages arising are summarised below.

3.4 The role of external assessment in upper secondary

“In comparative terms, Scottish upper-secondary school students are more frequently examined than those in other jurisdictions. This is a consequence of the tradition of offering three suites of examinations (National 5; Highers; Advanced Highers) during secondary years S4, S5, and S6.” (Stobart, 2021) He notes further that “This is examination loading not found in other jurisdictions.” (Stobart, 2021)

In this quotation, Stobart points to the frequency of external examinations in Scotland but also to the reliance on external examinations rather than other forms of assessment such as teacher led and internal assessment methods.

At present, the role of teachers in National Qualifications in Scotland is largely confined to assessment of SQA specified coursework, which is assessed by them, submitted to SQA and then subject to moderation. Some coursework is assessed by teachers for example, performances in PE, and subject to verification. Other coursework is carried out in centres by learner and then submitted to SQA for assessment, for example, the folio of writing in English, folios in art and design, projects in the technical subjects, project in music technology. Other learner assessments are carried out in centres and subject to visiting assessment, for example, performance in Music and drama, speaking in modern languages. Further, although coursework is a feature of most NQ courses in the Senior Phase, in some subjects its weighting is relatively small, compared to the weighting of the external examination (Stobart, 2021).Please refer to the attached link that provides a highlevel summary of current course assessment arrangements Assessment of National Courses from 2023-24- SQA

The role of teachers in upper secondary in Scotland can be contrasted with the role of teachers in lower secondary and in primary schools, where they are entrusted to make on-going assessments of their pupils’ progression and achievements. Scotland’s upper secondary school approach to teacher assessment can also be contrasted with that in many other jurisdictions including Norway, Sweden, Ontario, New Zealand and Queensland where teacher assessment in its different forms, plays a much more central role in qualifications and assessment.

In Norway for example 80% of a student’s award is based on in-school assessment (Stobart, 2021) and “teachers are seen as the key experts not only in instructing but also in assessing their students” (OECD, 2011). Students only undertake external examination in Norwegian, other subjects are subject to examination in an examination lottery, whereby approximately 20% of learners sit each subject examination. No student takes more than two examinations.

Sweden also has a system of teacher assessment, which can be used as a basis for entry into university (O’Donnell 2018) Stobart calls these systems “high-trust” systems, where teachers have a great deal of autonomy in deciding the ways in which evidence can be collected. Evidence from assessment approaches such as portfolios, coursework, teacher observation is used to form a composite grade.

3.5 Leaving certificates and subject based qualifications

“Scotland, unlike many jurisdictions, does not provide a school leaving certificate but issues subject-based qualifications. Students who have had limited success in their examinations may therefore have little to show for their school achievements.” (Stobart, 2021)

As Stobart indicates, many assessment systems internationally have a senior secondary curriculum which brings together a number of components into a final diploma. These systems sometimes also have a certification process for students who fail to meet the requirements of the national qualification.

These awards are packaged differently, and decisions are taken as to the number of components, their focus and the relative weightings (for example, examinations, social contributions, practical projects, orals). Often, they will allow learners to combine flexibly traditional academic subjects and technical, vocational and professional subjects.

For example, in New Zealand the National Certificate of Educational Achievement (NCEA) is the main secondary school qualification. The NCEA is a credit-based system, where learners earn credits towards the achievement of their certificate. Credits can be earned through traditional academic as well technical, vocational and professional subjects. Learners can also continue to accumulate credits after they have left school through structured workplace learning” (O’Donnell, 2018).

In Ontario, students must obtain 30 credits to achieve a Secondary School Diploma. The Ontario Secondary School Certificate is available to students who earn a minimum of 14 credits and students who leave school before fulfilling the requirements for a Diploma or Certificate may be granted a Certificate of Accomplishment. Students have the flexibility to choose from vocational and academic study and can gain credits in skills training centres (O’Donnell, 2018).

3.6 The importance of skills in upper secondary assessment

Masters (2022) from the Australian Council of Educational Research, undertook a three year study of five learning systems in locations identified in international surveys as ‘high performing’. The study identified a number of common themes across the five jurisdictions one of which was an increased emphasis on skills and the need for a skills framework whether that be the OECD’s (21st century skills), the UNESCO’s (transversal skills) or the EU’s (general competences). Although the terms of these frameworks may differ, the underlying concepts are very similar. Each puts greater emphasis on supporting students’ use of knowledge and skills, for example, applying knowledge in different contexts, creative thinking, problem solving, team building, being entrepreneurial. These are similar to the types of skills referred to in the Withers’ Review (2023). Withers also proposes a single skills framework (Recommendation nine).

In Norway, changes to their curriculum have included building competences and attributes such as critical thinking and creativity. In Wales, Curriculum for Wales which launched in 2020 set out key goals for every learner with a focus on developing skills and experiences (Stobart, 2021).

Alongside each of these curriculum reforms, there have been changes in assessment approaches. Stobart reflects that translating these ambitions for greater focus on skills and attributes has been challenging in practice.

“Many jurisdictions around the world found challenges in how to translate these aspirations into their upper-secondary school assessment policies. The new curriculum intentions are often hard to align with historic assessment practices that are embedded in that society.” (Stobart, 2021).

Project work in upper secondary assessment

In a number of jurisdictions including Finland and Sweden and in Baccalaureate systems, upper secondary assessment includes an element of project work. There are different rationales for the inclusion of a project element, but common reasons included a desire to give learners an opportunity to apply learning in real life contexts and to develop skills necessary for the ‘real world’ as well as showing preparedness for work or further study. The interest in Project Learning is also evident in Scotland where the Review’s proposals (see Chapter Five) have been broadly welcomed.

In Finland, what they refer to as project-based learning’ has been mandatory for schools since 2016. (Symeonidis, & Fohanna, 2016) Learners from the age of seven are required each year to undertake a “phenomenon-based” project on a holistic topic such as energy, climate change, migration. Learners explore the topics through a range of practical ‘real-life’ scenarios and through a range of disciplines. This approach can be contrasted with traditional approaches to learning and teaching, centred on discrete subjects like geography, history and mathematics.

A second example, with a different focus, is the Extended Essay in the International Baccalaureate (IB). The IB states that the aim of the Extended Essay is for learners to gain skills in:

• formulating an appropriate research question;
• engaging in a personal exploration of the topic;
• communicating ideas and;
• developing an argument.

(International Baccalaureate, 2022)

An example of an optional project approach can be found in England, where the Level 3 Extended Project Qualification offers learners the opportunity to undertake a stand-alone project on a topic of their choice. Learners have flexibility in terms of the final output: some learners choose to write a dissertation, others create an artefact (for example, a poster, blog post or video) or performance. The Extended Project Qualification is taken by around 40,000 learners in England each year and is teacher assessed. (AQA, 2023)

Each of these examples offers extensive learner choice within a common framework where criteria are defined but can be met in a wide range of ways.

3.7 Flexible pathways in upper secondary

Many countries offer flexible pathways in terms of combining academic and vocational programmes. O’Donnell states that examples of this practice can be seen in Finland, France, Ireland, the Netherlands, New Zealand, Ontario and Queensland. (O’Donnell, 2018)

“The credit-based systems in Finland, New Zealand, Ontario and Queensland allow students the flexibility to combine academic and vocational study and so successfully complete upper secondary education. In England, students can combine the study of academic and vocational qualifications in the same upper secondary study programme. In Ireland, students following a Leaving Certificate programme, can combine vocational/ technical and academic subjects; some complete the Leaving Certificate Vocational Programme as a result.” (O’Donnell, 2018)

3.8 Learning through public engagement

A number of jurisdictions also include an element of student service. For example, in Ontario, since 1999 learners wishing to obtain a high school diploma must complete at least 40 hours of community service. The Ottawa-Carleton District School Board states that the purpose of the community involvement is to promote community values by:

• helping students understand how they can make a positive difference in their environment;
• having students contribute to their community;
• increasing student awareness of community needs;
• discovering the role students can play in making their communities better places in which to live and work;
• developing a positive self-image and a greater sense of identity in the community and;
• providing a possibility for exploring career opportunities.

(Ottawa-Carleton District School Board, 2023)

The history of ‘service learning’ is well established in the United States (Kraft, 1996). It is also a feature of the International Baccalaureate. The Creativity, Activity, Service (CAS) is a mandatory element of the IB diploma where students must complete a project related to these three concepts.

3.9 Lessons in reform

Scotland is far from alone in considering changes to its qualification and assessment system. Masters (2022) argues that ‘nations around the world recognise the urgency of transforming school education’. Like Scotland, other nations realise that successful societies in future will be those which have actively prepared for change.

“To ensure that a much larger proportion of the population achieves the levels of knowledge, skill and competence required for effective engagement in modern societies and workplaces. In today’s knowledge-based economies, the levels once attained by a relatively small percentage of the population must now be attained by almost all” – (Masters 2022, p2)

Masters (2022) sought to understand what actions were being taken by countries explicitly seeking to build world class education system. The five jurisdictions he studied were: British Columbia, Estonia, Finland, Hong Kong and South Korea. It was interesting to note that even although these countries are identified currently as high performing, they are all in the process of re-designing their school learning systems. Their rationale for change is very similar to the rationale for change that underpins this Review. They identify two main challenges;

• the need to prepare learners more effectively for future life and work and;
• to ensure that every learner “learns successfully, achieves high standards and leaves school well prepared for the future” (p4)

Masters (2022) identified a number of common themes across the five jurisdictions. First, he states that each recognised that their curricula, including pedagogy and assessment have to change. They recognise that learners will need to approach knowledge, skills and attributes differently if they are to thrive in the uncertainties of the future. For example, learners will need to be able to judge the quality of information online; they will have to understand how to differentiate between more or less dependable evidence. More broadly, they will need the skills to tackle major challenges to our existing ways of life, for example; climate change and the protection of the environment, the role of science in society and how to protect democratic institutions and processes.

Masters (2022) also notes that, although all five jurisdictions are sometimes described as high equity, the picture is more complicated. They may be more effective than most in reducing the number of learners who perform at low levels of achievement, but the achievements of 10-15% of their learners remain below the minimally acceptable OECD level with an additional 20% achieving only that minimal level. In all five nations or states, there remains a strong relationship between learner achievements and socioeconomic circumstances. The relationship between poverty and achievement is an international challenge. In these high achieving contexts, the gap between the most advanced 10% of 15-year-olds and the least advanced is four to five years, one third of their lifetime.

Changes in the curricula of these countries are being paralleled by changes in approaches to qualifications and assessment. According to Masters (2022), they recognise that traditional ways of assessing students’ learning may be appropriate if the aim is to have learners demonstrate what they know and are able to do in individual subject areas. However, competencies and attributes such as critical thinking, problem solving, or resilience will require different approaches to assessment. Competences and attributes are progressive, and learners will make progress during school and beyond. Assessing competences and attributes has to take a progressive, developmental approach. In all five jurisdictions, changes to assessment are being undertaken in parallel with the jurisdictions’ more traditional assessment approaches.

Masters (2022) states that each of the five jurisdictions is also aware of the potential washback effect of assessment processes and how these can undermine curricular reform, particularly when assessment is used for purposes of selection. He cites South Korea as an example of a country where the impact of washback is particularly strong. In South Korea, entry to university is highly selective. This has a washback effect on what happens in classrooms, as the examination becomes the driver for what is taught and learned rather than the broad curriculum. To avoid this effect, some jurisdictions have removed end of school examinations and use only teacher assessment, others have integrated other assessment approaches. For example, Estonia has introduced a creative cross-curricular project and Hong Kong has developed a student learning profile, where students summarise their achievements beyond formal courses. All five jurisdictions have introduced more student-centred approaches to learning and teaching.

Whilst recognising the importance of subjects as the building blocks of most curricula internationally, countries also acknowledge the importance of having learners transfer and apply their knowledge into other areas or disciplines. Some identify the motivational impact on learners of having them use and apply the knowledge, skills, capacities and values they have been developing in individual subject areas to tackle a global challenge.

The five countries examined by Masters (2022) are changing qualifications and assessment in different ways, but they are all changing.

3.10 Summary

Many of the issues being explored in a Scottish context are also reflected in developments underway in a number of other jurisdictions. The importance of having a qualifications and assessment system that provides learners with the maximum opportunity to demonstrate their achievements is a common theme across a number of jurisdictions, including Scotland. A further theme being considered is the importance of learner choice, both in learner pathways and choice within tasks. The awareness of the increasing importance of skills and using knowledge in context also resonates with thinking in Scotland. Finally, the CAS example from the IB and the Ontario service model are consistent with the desire in Scotland to reflect the wider community-orientated aspirations of CfE.

Recommendation 3: Work in partnership with countries with similar aspirations to Scotland to develop Qualifications and Assessment.

Learn from experience within Scotland but also be outward looking. Seek to learn with other nations with similar educational aspirations to build a Qualifications and Assessment system that will remain fit for the future.