Scottish Survey of Literacy and Numeracy 2015 (Numeracy)

Chapter 3: Numeracy attainment over time

• The proportion of P4 pupils performing well or very well decreased between 2011 and 2013, and again slightly between 2013 and 2015.
• The proportion of P7 pupils performing well or very well decreased between 2011 and 2013 but remained constant between 2013 and 2015.
• The proportion of S2 pupils performing well or very well remained stable between 2011, 2013 and 2015.

A key objective of the SSLN is to monitor numeracy attainment over time at P4, P7 and S2. This chapter details the methodology used to assess numeracy attainment over time (2011, 2013 and 2015) and provides the trends in performance for the key measures from the survey.

3.1 Methodology

All analysis is based on the results of pupils who completed all three elements of assessment (two written booklets and the pupil-teacher interactive assessment). The effective sample size was 11,238 pupils for the 2011 survey, 10,561 pupils for the 2013 survey and 10,486 pupils for the 2015 survey (3,381 in P4, 3,368 in P7 and 3,737 in S2).

The SSLN design allows for a level of item release and replacement in order to provide examples of the tasks pupils are asked to undertake. They are included, for example, in this statistical report and in Education Scotland's PLRs. Therefore there are a proportion of assessment booklets that were not consistent between 2011, 2013 and 2015.

In order to check the item release and replacement strategy did not have an adverse effect on the ability to make comparisons, results were produced on two bases; firstly on all assessment booklets and secondly excluding booklets which had been released or replaced between cycles. A series of in-year and between year comparisons showed that excluding released and replacement booklets did not affect the overall picture of pupil performance and therefore that results across survey years are directly comparable.

Time series data in this report and the supplementary tables are based on an analysis of all booklets in each survey year. Statistical tests were used to test for significant differences in performance:

• between 2011 and 2013;
• between 2013 and 2015;
• between 2011 and 2015.

3.2 Overall distribution

Chart 3.1 shows the proportion of pupils who performed well or very well at the relevant level in 2011, 2013 and 2015. The overall pattern shown in the 2015 data was similar to the previous two years, with performance at primary higher than in secondary.

Chart 3.1: Proportion of pupils performing well or very well in numeracy in 2011, 2013 and 2015, by stage

The time series trend varies between stages, Table 3.1 shows which changes over time are statistically significant:

Table 3.1: Difference in proportion of pupils performing well or very well in 2011, 2013 and 2015, by stage

Stage	2011 to 2013	2013 to 2015	2011 to 2015
P4	Lower in 2013	Lower in 2015	Lower in 2015
P7	Lower in 2013	No difference	Lower in 2015
S2	No difference	No difference	No difference

The proportion of P4 pupils performing well or very well has reduced between each survey, however the rate of decline has slowed. The difference in the proportion of pupils performing well or very well between 2011 and 2013 and between 2013 and 2015 was eight and three percentage points, respectively, culminating in an overall reduction of 11 percentage points between 2011 and 2015. The decrease between 2013 to 2015 was solely due to a decrease in the proportion of pupils performing well, the proportion performing very well remained the same.

At P7, there was a decrease between 2011 and 2013 (six percentage points) but there was no difference between 2013 and 2015, with the proportion of pupils performing well or very well remaining the same at 66 per cent. The overall change between 2011 and 2015, therefore, was a decrease of six percentage points.

The proportion of pupils performing well or very well at S2 has remained the same (statistically similar) across the three surveys.

With respect to the proportion of pupils not yet working within the level, the proportions, and changes in proportions between 2011 and 2015, varied greatly between stages. There was no increase at P4 (negligible in all years) and a small increase at P7 from two percentage points to three percentage points. The largest increase was at S2 where the proportion of pupils not yet working within the level increased from 32 per cent to 36 per cent.

3.3 Attainment by gender

The pattern of performance by gender over time varied between stages, with only some changes in performance being statistically significant. These changes are displayed in Table 3.2.

Table 3.2: Difference in proportion of pupils performing well or very well in 2011, 2013 and 2015, by stage and gender

Girls

Stage	2011 to 2013	2013 to 2015	2011 to 2015
P4	Lower in 2013	No difference	Lower in 2015
P7	Lower in 2013	No difference	Lower in 2015
S2	No difference	Lower in 2015	Lower in 2015

Boys

Stage	2011 to 2013	2013 to 2015	2011 to 2015
P4	Lower in 2013	No difference	Lower in 2015
P7	Lower in 2013	No difference	Lower in 2015
S2	No difference	No difference	No difference

Chart 3.2: Proportion of pupils performing well or very well in numeracy in 2011, 2013 and 2015, by stage and gender

At P4 and P7, there was a decrease in the proportion of girls and boys performing well or very well between 2011 and 2013, with no statistically significant change between 2013 and 2015. The decrease in performance observed between 2011 and 2015 is due the changes between 2011 to 2013 only.

There has been no change to the proportion of S2 boys performing well or very well over the three years as shown in Chart 3.2. The S2 girls' performance has remained the same at 41 per cent in 2011 and 2013, before decreasing to 38 per cent in 2015.

It is worth noting that in 2013 boys outperformed girls at P7 only, whereas in 2015 boys outperformed girls at S2 only i.e. the gender gap in numeracy performance for the same cohort of pupils (P7 in 2013 and S2 in 2015) still exists.

3.4 Attainment by deprivation

The pattern of performance by deprivation over time is stable, in that a performance gap continues to exist across all stages.

Changes in performance of pupils within each deprivation category varied between stages, with only some changes in performance being statistically significant between the three surveys. These changes are displayed in Table 3.3.

Table 3.3: Difference in proportion of pupils performing well or very well in 2011, 2013 and 2015, by stage and deprivation

Least deprived category (pupils from the least deprived 30% of datazones)

Stage	2011 to 2013	2013 to 2015	2011 to 2015
P4	Lower in 2013	No difference	Lower in 2015
P7	No difference	No difference	No difference
S2	No difference	No difference	No difference

Middle deprivation category (pupils from the middle 40% of datazones)

Stage	2011 to 2013	2013 to 2015	2011 to 2015
P4	Lower in 2013	Lower in 2015	Lower in 2015
P7	Lower in 2013	No difference	Lower in 2015
S2	No difference	No difference	No difference

Most deprived category (pupils from the most deprived 30% of datazones)

Stage	2011 to 2013	2013 to 2015	2011 to 2015
P4	Lower in 2013	Lower in 2015	Lower in 2015
P7	Lower in 2013	No difference	Lower in 2015
S2	No difference	No difference	No difference

Between 2013 and 2015, the only groups of pupils where there was a change in performance was P4 pupils from most deprived areas and from the middle category, where performance was lower in 2015 than in 2013. Performance was stable for all other groups.

As the proportion of P4 pupils who performed well or very well decreased for both least deprived and most deprived pupils between 2011 and 2013, the performance gap between least and most deprived pupils remained statistically similar (13 and 14 percentage points respectively). The performance gap between least and most deprived pupils again is not statistically significantly different between 2013 and 2015 (14 and 21 percentage points respectively). However the cumulative change in the performance gap between 2011 and 2015 meant that the gap was found to be statistically significantly larger in 2015 than in 2011. (Chart 3.3)

Chart 3.3: Proportion of P4 pupils performing well or very well in numeracy for 2011, 2013 and 2015, by stage and deprivation category

At P7, performance by deprivation category was stable between 2013 and 2015. The reductions in performance of pupils from the most and middle deprived categories between 2011 and 2013 caused reductions in these categories between 2011 and 2015 overall: from 74 to 64 percent (middle deprivation category) and from 61 to 54 per cent (most deprived). (Chart 3.4)

Chart 3.4: Proportion of P7 pupils performing well or very well in numeracy for 2011, 2013 and 2015, by stage and deprivation category

While the P7 performance gap increased between 2011 and 2013 (from 16 to 24 percentage points), there was no change between 2013 and 2015 (24 and 22 percentage points respectively).

At S2, there were no statistically significant changes in performance when disaggregated by deprivation category, at any point between 2011 and 2015. Similarly there was no difference in the performance gap between these three years staying constant at 28 percentage points. (Chart 3.5)

Chart 3.5: Proportion of S2 pupils performing well or very well in numeracy for 2011, 2013 and 2015, by stage and deprivation category

3.5 Attainment by numeracy organiser

Attainment in P4 was lower across all organisers in 2013 compared to 2011, with the exception of 'chance and uncertainty'. Between 2013 and 2015, however, the picture is mixed, with no statistically significant changes in performance in five of the organisers and small reductions in three ('number and number processes', 'money' and 'time').

At P7, six out of the eight organisers saw lower performance in 2013 than in 2011, whereas between 2013 and 2015 performance had stabilised across all organisers apart from 'time' (a reduction of one percentage point).

At S2, the picture was more varied. Over the period 2011 to 2015 performance was slightly lower for four organisers, stable for three and marginally higher (one percentage point) for one organiser ('fractions, decimal fractions and percentages').