Scottish National Standardised Assessments: national report for academic year 2017-2018

4 Writing

4.1 The scope of the writing assessment

In the 2017 to 2018 academic year, Scottish National Standardised Assessments in writing were available for P4, P7 and S3. At P1, children were presented with a single assessment combining elements of reading and writing. This is referred to as the P1 literacy assessment. For more information on the P1 literacy assessment, please see Section 3: Reading/Literacy.

The assessments of writing were based on elements of Curriculum for Excellence (CfE), as articulated in the Benchmarks: Literacy and English, published as a draft in August 2016. It should be noted that for the 2018 to 2019 academic year, the final version of the Benchmarks (published in June 2017) is used as the reference point for the assessments.

4.2 Coverage of the Curriculum for Excellence: benchmarks and organisers

SNSA is just one part of the range of assessments that teachers use in making their evaluations of children’s and young people’s learning. As a standardised assessment to be completed within a limited time, using questions capable of being scored automatically, only some parts of the writing benchmarks could be addressed. It was agreed that the writing assessments should be based on the assessment organisers Spelling, Grammar and Punctuation, which all fall under the curriculum organiser Tools for writing. Each of the questions selected for inclusion in SNSA writing assessments for the 2017 to 2018 academic year was aligned with a benchmark statement from Tools for writing.

Spelling, Grammar and Punctuation were all substantially represented in the writing assessments for P4, P7 and S3, with Spelling having a slightly higher weighting in comparison with Grammar and Punctuation. This reflected the importance placed on this area of Tools for writing by the Scottish education experts involved in defining the basis of the assessment. The weighting across the three organisers for each year group in SNSA was roughly the same, and all three assessment organisers were addressed by at least five questions in the year group’s full set, and in each learner’s assessment, regardless of which path they took through the adaptive system. As such, in the school-level reports, teachers received information about the relationship between the learner’s overall result and

organiser level result for each of these areas at an individual learner level. In the rest of this section, features of the assessments of Spelling, Grammar and Punctuation for the three year groups are described, with examples of questions for illustration.

4.2.1 Spelling

The writing assessment in the 2017 to 2018 academic year was designed to assess spelling words and strategies, covering the range of skills and the progression articulated in the Benchmarks. At the lower levels, learners were assessed on their ability to spell relatively simple and commonly used vocabulary. As the difficulty of the spelling questions increased, and in the higher stage assessments, children and young people were also asked questions to assess their knowledge of less familiar words. To answer the questions correctly, learners may have needed to rely on strategies, such as their knowledge of phonics, spelling patterns and rules, and at the highest level may have been asked to spell specialised vocabulary.

Throughout, spelling was assessed in context, using a range of formats. One format was the cloze style, in which learners had to select the correct spelling from a range of options, either in a single sentence or in a short passage. In a second question format, learners were asked to type in the correct spelling of a given word, also in context. In a third format, learners had to identify the incorrectly spelt word in a sentence.

Figure 20 shows a typical Spelling question from the P4 writing assessment.

Figure 20: Example of a P4 Spelling item, ‘Choose the incorrectly spelt word’

The question presented here required children to identify the incorrectly spelt word in a sentence, from a range of options. This question draws on the writing strategy of proofing and editing, which models for learners the importance of checking their writing to identify errors.

As can be seen in this P4 question, the context provided is a familiar classroom activity and all the words in the brief, grammatically simple sentence are common and relatively short.

To answer this question correctly, the learner must know the correct spelling of the common word ‘need’, or know that the other options given are all spelt correctly. The incorrect options increase the difficulty of this item, as the third and fourth options include some slightly less familiar words (‘plastic’ and ‘project’). This question was rated as having low difficulty in the 2017 to 2018 SNSA. It was presented to learners that had found the first phase of the assessment relatively challenging. A minority of these learners answered it correctly.

4.2.2 Grammar

This organiser focuses on general grammar points, addressing the skills, knowledge and understanding articulated in the Benchmarks. At P4, for example, it assessed children’s ability to link sentences using common conjunctions, such as ‘and’, ‘because’ or ‘but’, while at the higher stages, a fuller and more complex range of conjunctions was assessed.

Questions addressing the Grammar organiser also assessed usage of prepositions, verb forms, adjectives and pronouns. At P4, the questions tended to relate to simple sentences. At P7 and S3, the challenge was increased by introducing longer, more complex language structures, such as compound sentences, conditionals and negative clauses; or by asking questions for which the child or young person needed to identify the relationship between two pronouns, where there was some ambiguity. In summary, as well as the grammar itself becoming more challenging, the contexts in which learners were asked to demonstrate their skills became more complex in the higher stage assessments.

Figure 21 presents an example of an S3 Grammar question, a typical example of the cloze (or gap fill) writing questions. As in the reading assessments, SNSA writing questions were often presented as ‘units’: a group of questions based on a single piece of stimulus. These units enabled more sustained context to be provided, and were used to assess all three writing assessment organisers. This stimulus text uses quite commonplace, simple vocabulary, but the sentence structures are relatively complex, allowing for a range of grammatical forms to be addressed.

Figure 21: Example of an S3 writing unit with a Grammar question, ‘Oh Brother Question 1’

In this question, young people were asked to identify the correct adverb to fill the first space in the text, from a given range of options. Since all the options provided would be grammatical matches for the sentence and, apart from the last option (just), fit syntactically in the sentence, this question assesses the learners’ knowledge of the meaning of these cohesive devices and their correct application in context. This question was rated as having low difficulty in the S3 assessment. It was presented to learners who had found the first phase of the assessment relatively challenging. Most of these learners were able to answer this question correctly.

The question in Figure 22 is based on the same text as that in Figure 21, and asks about the second space in the given paragraph. In contrast to the first question, which requires an understanding of the meaning of the options presented, this second question clearly addresses knowledge of the correct lexical form, rather than meaning. Learners were asked to choose the participle required by the sentence structure.

Figure 22: Example of an S3 writing unit with a Grammar question, ‘Oh Brother Question 2’

This question was rated as having medium difficulty in the S3 writing assessment and was answered correctly by a majority of those learners to whom it was presented.

4.2.3 Punctuation

The 2017 to 2018 SNSA assessed the organiser Punctuation in a variety of ways. These included asking learners to identify the location of a given punctuation mark, to choose between sentences to identify the correctly punctuated example, and to choose the missing punctuation in a sentence from a range of options.

In the P4 assessment, questions from this organiser focused mainly on the full stop, question mark and exclamation mark, and the correct use of capital letters. This punctuation was assessed in relatively simple sentences. At P7, the range of punctuation was extended to include commas, parentheses and the uses of inverted commas and punctuation within speech, in more complex sentences. At S3, colons and semi-colons were added to the range of punctuation marks addressed, and learners were asked to identify correct usage of punctuation marks in increasingly complex sentences.

As for the other writing organisers, questions targeting the Punctuation organiser were presented either using stand-alone sentences, or in units that used a passage of text with several questions attached to it.

Figure 23 presents an example of a P7 writing question from the Punctuation organiser. It is a typical example of a question in which children were asked to choose the sentence with the correct punctuation from a range of four or five options.

Figure 23: Example of a P7 Punctuation question, ‘Choose the sentence with the correct use of inverted commas’

In this question, children had to identify the correct punctuation, where a single sentence of direct speech is broken up by information about who is speaking. To answer this question correctly, learners had to work through several steps: first, to recognise that Monika is the speaker and not part of the quoted speech; second, identify that the second quoted phrase is a fragment that forms part of a single quoted sentence; and third, show their knowledge of how to use punctuation to link a single sentence broken up by the internal placement of the speaker. This question was answered correctly by a minority of those learners who encountered it. Of the incorrect answers, learners were more likely to select the third option than either of the first two, suggesting that while most learners may have recognised that Monika was the speaker, many of them were not able to correctly punctuate the split quoted sentence.

Figure 24 shows another example of a question from the Punctuation organiser. In this question type, young people were presented with a sentence and asked to identify the correct location of the missing punctuation: in this case, parentheses. The question is from the S3 writing assessment.

Figure 24: Example of an S3 Punctuation question, ‘Identify the location of parentheses’

In this question, a relatively simple sentence uses the context of Scottish history. To answer this question correctly, young people had to identify that lifespan numerals should be put in parentheses. Of the incorrect answers, young people were most likely to select the fourth option. While these learners may have correctly identified that this phrase was not grammatically required by the sentence, they did not recognise the need for parentheses around the lifespan. This question was rated as having low difficulty. It was presented to learners who had found the first two phases of the assessment challenging. Among learners who encountered it, this question was answered correctly by a minority.

4.3 National results for writing

4.3.1 Overall capacity

Chart 19 shows the overall capacity for writing across all three year groups (P4, P7, and S3), both overall and when considering each of the three organisers Spelling (S), Grammar (G) and Punctuation (P). Regions show high, medium and low capacity, in line with SNSA reports for the 2017 to 2018 academic year, which are specific to each year group.

Chart 19: Writing Capacity by SNSA year

The chart shows that, with the exception of the results for the organiser Spelling in P4, across all three year groups the greatest proportion of learners tended to be found in the high capacity group, with the second-highest proportion in the medium capacity group. Few learners were observed in the low capacity group.

At P4, a majority of the learners demonstrated high capacity overall and for the Grammar and Punctuation organisers, while smaller proportions of learners showed low and medium capacity. When considering only Spelling, less than half of the learners demonstrated high capacity and a relatively large proportion of learners fell within the low capacity region.

Most learners at P7 demonstrated high capacity on their writing assessment, while only relatively small proportions demonstrated low capacity. There were somewhat smaller proportions of learners with high capacity when considering only the Punctuation organiser, as compared with Spelling and Grammar.

At S3, overall and for the three organisers individually, a majority of learners demonstrated high capacity, while only a few fell into the low capacity ranges. Among the organisers, the proportion of young people in the high capacity region was most pronounced for Grammar, with Punctuation showing a notably lower proportion at high capacity (although, still more than half fell into this category).

Given that learners tend to advance in their learning throughout the school year, but also at a different pace at different stages of schooling, we compared the results from the two SNSA norming studies^[9] carried out in Scotland during the 2017 to 2018 academic year, to review the differences in learning between the first and second halves of the year with regard to writing capacity acquisition among Scottish learners.

Chart 20: Writing capacity across norming study time periods

Chart 20 shows that between November and March, among learners assessed in the second norming study, at all three year groups, there were larger proportions with high capacity than among those assessed in the first norming study. Likewise, there were decreases in the proportion of learners with low capacity across the two norming studies in P7 and S3, but not at P4. The differences in the proportions of learners with high capacity between the two norming studies were relatively largest at P4 and smallest at S3.

4.3.2 Gender

Chart 21 shows the proportions of writing capacity among boys and girls for all three SNSA year groups, overall, and when considering each of the three organisers.

Chart 21: Writing capacity distribution by gender and SNSA year

For all year groups, the largest proportions of learners were found within the high capacity range, both overall and when considering each of the organisers separately. The only exception was the distributions for Spelling at P4. However, there were notable differences between the two gender groups, with consistently larger proportions of girls in the high capacity category, as compared with boys, and correspondingly smaller proportions of girls than boys in the low capacity region.

At P4, while a majority of girls attained an overall capacity in the high region, slightly less than half of boys fell into the same category. Similar differences between the two gender groups were observed when considering the three organisers separately, with the largest difference between proportions of high capacity learners in favour of girls recorded for Punctuation.

At P7, while in both gender groups most learners demonstrated high capacity, we observed notably higher proportions among girls than among boys. This finding holds overall and when considering the three organisers separately. Again, the largest differences between proportions with high capacity in favour of girls were observed for Punctuation.

At S3, in addition to the tendencies described for other year groups, there was a notably higher proportion of boys as compared with girls demonstrating low writing capacity overall. This difference was more substantial when only considering Punctuation, and the proportion of girls in the high capacity category for this organiser was notably larger than the proportion of boys.

4.3.3 Scottish Index of Multiple Deprivation

This section reviews the distribution of learners for all SNSA year groups across categories reflecting the Scottish Index of Multiple Deprivation (SIMD). To simplify the display of results and aid their interpretation, we used three categories to indicate levels of socioeconomic background, namely: 1–4, indicating the bottom socioeconomic quintile (that is, the most deprived children and young people, those in vigintiles 1 to 4); 5–16, indicating the three middle quintiles (that is, those in vigintiles 5 to 16); and 17–20, indicating the top quintile (that is, the least deprived children and young people, those in vigintiles 17 to 20).

Chart 22: Writing capacity distribution by SIMD and SNSA year

At each year group, it can be seen in Chart 22 that the proportions of learners in the group with high capacity was noticeably larger in the SIMD category which reflects a higher socioeconomic status (that is, less deprivation), while relatively larger proportions of learners with low capacity were observed in the SIMD category which reflects lower socioeconomic status. This pattern was present in all three year groups, overall and when considering only assessment items corresponding to each of the three writing organisers Spelling, Grammar and Punctuation.

The results illustrate that the differences in writing capacity between children from the bottom quintile and the top quintile of SIMD were similar across all three year groups (P4, P7 and S3). More children from the bottom quintile showed capacity corresponding to the low capacity region, whereas larger numbers of children in the top quintile demonstrated a high capacity. In P4, less than half of learners from the bottom SIMD quintile showed overall high capacity, while a majority of learners in the top SIMD quintile fell into the high capacity category. For the organiser Spelling, while less than half of learners across all SIMD groups demonstrated high capacity, there were marked differences between the lowest and highest quintile, in favour of the latter.

Among both P7 and S3 learners, the majority across all SIMD groups tended to demonstrate high capacity, but among those from the top quintile, there were much larger proportions of learners with high capacity than for the bottom quintile. With regard to Punctuation, a majority of S3 learners in the top SIMD quintile showed high capacity, while this was the case for less than half in the bottom SIMD quintile.

4.3.4 Ethnic background

This section looks at differences in writing capacity between learners with ‘White Scottish’ and other ethnic backgrounds. Chart 23 shows the reading/literacy results for these two groups.

Chart 23: Writing capacity distribution by Ethnic background and SNSA year

When comparing writing capacity across groups of learners with ‘White Scottish’ and other ethnic backgrounds, we found relatively small differences, both in terms of overall capacity and when considering results for each of the three organisers (Chart 23). The results show that slightly lower proportions of learners from a ‘White Scottish’ background demonstrated writing capacity in the high category, as compared with those from other ethnic backgrounds. However, at P7 and S3, in relation to Grammar, among learners from a ‘White Scottish’ background, there were slightly larger proportions with high capacity as compared with other groups.

4.3.5 Free School Meal Registered

Chart 24 shows the writing capacity of learners according to groups defined by registration for free school meals. This chart distinguishes those with registered entitlement from all other learners.^[10]

Chart 24: Writing capacity distribution by Free School Meal Registered and SNSA year

Chart 24 shows that at each of the three year groups there were notably larger proportions in the high capacity group among learners without entitlement than among those with FSE, and correspondingly higher proportions in the low capacity group among learners with FSE. This pattern was similar across all year groups and also when considering results by each of the three organisers.

At P4, while overall a majority of the children without FSE showed high capacity, only a minority of those with entitlement demonstrated capacity in this category. Differences were similar also when considering each of the three organisers separately, noting that for Spelling, in both groups, only a minority demonstrated high capacity.

At P7, the majority of learners in both groups demonstrated high capacity, overall and for each of the three organisers. However, we observed marked differences in favour of learners without FSE. Among learners at S3, differences between the proportions of learners with high capacity across the two groups were also similar. When considering only Punctuation, a minority of learners with FSE demonstrated high capacity compared to more than half of the learners without FSE.

4.3.6 Additional Support Needs

Chart 25 shows the proportions of learners with high, medium and low writing capacity across SNSA year groups, according to whether or not learners were identified as having Additional Support Needs (ASN).

Chart 25: Writing capacity distribution by ASN and SNSA year

Across all year groups, it can be seen that the proportions with high writing capacity were notably larger among learners with no registered ASN, while there were larger proportions reflecting low capacity among learners with ASN. This pattern is present in all year groups, both overall and for each of the three writing organisers.

At P4, a majority of learners without ASN had results corresponding to the high capacity range overall, while among learners with registered ASN, only a minority showed high capacity. When considering only Spelling, in both groups, less than half showed high capacity but the proportion of learners with registered ASN was noticeably larger in this category as compared with learners without ASN. Additionally, about half of learners with ASN demonstrated low capacity with regard to this organiser.

At P7, half or more among learners in both groups tended to show high capacity overall. However, there were marked differences in favour of those without registered ASN. Differences in the proportions of learners with high capacity were largest with regard to Spelling. At S3, similar differences were found between the two comparison groups. When considering only Punctuation, less than half of the learners with registered ASN showed high capacity, while a majority among those without ASN had results in this capacity range.

4.3.7 Looked After Children at Home and Looked After Children Away from Home

Chart 26 shows the proportions of learners with high, medium and low capacity in writing by categories of Looked After Children at Home (LAH) and Looked After Children Away from Home (LAA), in comparison with other learners, as classified within SEEMiS.

Chart 26 Writing capacity distribution by LAH/LAA and SNSA year

Across all three year groups, we found notably lower proportions demonstrating high capacity among those learners with LAH and LAA, while among other learners, there were larger proportions demonstrating high capacity. Similarly, among learners with LAH and LAA, there were also larger proportions with low capacity than was the case among other learners. This pattern was similar across all four year groups.

At P4, a majority among learners registered with LAH or LAA showed low capacity, with respect to Spelling, compared to less than half among other learners. Both at P7 and S3, less than half of learners with LAH and LAA demonstrated high capacity, with respect to Punctuation, while a majority of other learners fell within the high category for this organiser. At S3, this difference also held with regard to overall writing capacity for both groups.

4.3.8 English as an Additional Language

Chart 27 shows the writing capacity of learners according to groups defined by language background: English as an Additional Language (EAL) and all other children and young people. The category ‘Yes’ refers to those learners whose record in SEEMiS, the national database, showed that they had English as an additional language. The ‘Other’ category comprises both learners for whom there was a ‘No’ as the entry for EAL, and those for whom there was no entry in this field.

Chart 27: Writing capacity distribution by English as an Additional Language (EAL)

As Chart 27 shows, when comparing proportions of high, medium or low capacity between groups of learners for whom English is an additional language (EAL) and those with English as their first language, we observed no difference at P4, and only relatively small differences in favour of learners in the latter group at P7 and S3. Differences in favour of learners with English as their first language were larger when considering only Grammar. Similar patterns were observed across all three year groups.

Differences between the two groups with regard to Grammar were slightly larger at the higher year groups, and were largest at S3. However, at both P7 and S3, in relation to Spelling, there were only very small differences in favour of learners with English as their first language.