Evaluation of Scottish National Standardised Assessments against Curriculum for Excellence: FOI release

FOI reference: FOI/18/02535
Date received: 19 September 2018  
Date responded: 17 October 2018
 

 

Thank you for your request dated 19 September 2018 under the Freedom of Information (Scotland) Act 2002 (FOISA) for:

1. (a) What criteria were used to ensure ‘align[ment] with CfE experiences and outcomes’? Which specific Es and Os were invoked, and what statistical evidence was generated to compare them with the assessments? What criteria were used to ensure ‘culture appropriateness’?

(b) What statistical evidence was used to compare the conclusions of the assessments with the judgements formed by teachers?

2. What statistical evidence was gathered of teachers’ views about the use of the assessments for formative purposes?

3. What statistical evidence was used to measure the assessments according to ‘international standards of accessibility’?

1. (a) Experts and professionals at Education Scotland assessed each question in the Scottish National Standardised Assessments (SNSA) in reading, writing and numeracy against the Curriculum for Excellence (CfE) benchmarks and related Es and Os to ensure the right curricular concepts were being assessed appropriately. The attached document on Assessment Constructs sets out the list of organisers and subcategories per subject area that are included in the assessments which provides an overview of the assessment construction. Part of this document is commercially sensitive and we have therefore applied s33(1)(b) (commercial interests) of FOISA to that information. An exemption under section.

38(1)(b) (personal information) also applies to some of the information in that document. The reason why those exemptions apply is explained below.

In addition to quality assurance by professionals at Education Scotland, each new question is trialled with children and young people to assess psychometric value in advance of introduction to the SNSA in the following academic year.

Education Scotland professionals assessed all SNSA questions for cultural appropriateness to ensure texts, instructions and guidance are relevant to the Scottish context and that they are in accordance with learning and teaching practice within Curriculum for Excellence.

1. (b) The SNSA are designed to provide formative, diagnostic information to teachers to inform next steps in learning. We know that outcomes from the SNSA have also been used by teachers as one source of evidence to confirm their judgements about whether children and young people have achieved the relevant CfE level.  However, as the assessments are designed to provide supporting evidence across a selected range of concepts and skills from the CfE benchmarks not the CfE level as a whole, it would be inappropriate to simply align individual SNSA outcomes with overall professional judgement of achievement of a level.

Additionally, the Scottish Government does not gather teachers’ judgements of individual concept progressions or achievements, only overall achievement levels, and would not have the information to compare these outcomes.

2. During 2017/18 focus groups and classroom observations of the SNSA were conducted and were an opportunity for teachers and other professional representatives to provide open and frank feedback in relation to the assessments. The groups were conducted under ‘Chatham House Rules’ to encourage constructive criticism and comment to the Scottish National Standardised Assessment team and, as such, no formal written record of those discussions was kept. These sessions were however an integral part of the continuous improvement process and showed that some schools were starting to use the SNSA data in formative ways. Feedback from these discussions was used to inform the Scottish National Standardised Assessments User Review – Year 1 :

/publications/scottish-national-standardised-assessments-user-review-year-1-session-2017/

Further feedback was also requested from teachers working with the assessments in 2017/18 and this was published:

/publications/foi-18-01829/

Under section 25(1) of FOISA, we do not have to give you information which is already reasonably accessible to you. If, however, you do not have internet access to obtain this information from the website(s) listed, then please contact me again and I will send you a paper copy.

The Scottish Government has plans to systematically gather teachers’ views on a range of topics, including the use of the SNSA for formative purposes, during the course of 2018/19.

3. The SNSA have been assessed against the international Web Content Accessibility Guidelines (WCAG) AA standard by individual accessibility specialists and accessibility workshops continue to be held with independent specialists for the purposes of continuous improvement. Work is continuing to further improve accessibility in line with the WCAG AA standard.

Reasons for not providing information

Exemptions apply:

Section 38(1)(b) – applicant has asked for personal data of a third party

An exemption under section 38(1)(b) of FOISA applies to some of the information you have requested because it is personal data of a third party, ie names and contact details of individuals, and disclosing it would contravene the data protection principles in article 5(1) of the General Data Protection Regulation (GDPR) and in section 34(1) of the Data Protection Act 2018. This exemption is not subject to the ‘public interest test’, so we are not required to consider if the public interest in disclosing the information outweighs the public interest in applying the exemption.

Section 33(1)(b) – commercial interests

An exemption under section 33(1)(b) of FOISA (commercial interests) also applies to some of the information you have requested. This exemption applies because disclosure would (or would be likely to) prejudice substantially the commercial interests of any person or organisation, in this case the contractor who has delivered the SNSA system.  This exemption is subject to the ‘public interest test’. Therefore, taking account of all the circumstances of this case, we have considered if the public interest in disclosing the information outweighs the public interest in applying the exemption. We have found that, on balance, the public interest lies in favour of upholding the exemption. We recognise that there is a public interest in disclosing information as part of open and transparent government, and to help account for the expenditure of public money. However, there is a greater public interest in protecting the commercial interests of companies which enter into Scottish Government contracts, to ensure that we are always able to obtain the best value for public money.

About FOI
The Scottish Government is committed to publishing all information released in response to Freedom of Information requests. View all FOI responses at http://www.gov.scot/foi-responses