Annex C: Key monitoring and evaluation measures for Scottish public bodies
1. This annex outlines the monitoring and evaluation arrangements that will be put in place to provide assurance that Scottish public bodies are making progress towards the initial baseline, target and advanced stages of progression under the Scottish Public Sector Cyber Resilience Framework.
A. One-off monitoring and evaluation arrangements against initial baseline progression stage
2. On a one-off basis, all Scottish public bodies will be asked to provide information to the Scottish Government (and, if appropriate and upon agreement, NIS Competent Authority/ies) with regard to progress against key actions under the initial baseline progression stage of the Scottish Public Sector Cyber Resilience Framework. The wider public sector, including local authorities and the colleges and universities sector, will also be encouraged to participate in these monitoring arrangements wherever possible, although coordination of these wider monitoring and evaluation arrangements may be entrusted to other key partners.
3. There will be informal contact between the Scottish Government and public bodies at working level throughout the period of implementation of this action plan, to gauge progress and offer support where required. A formal request will be made at Board/Senior Management level to all Scottish public bodies at end June 2018 to provide one-off written updates , setting out progress on implementing the following key requirements under this action plan:
- Confirmation of (i) having undergone a Cyber Essentials pre-assessment, (ii) having taken a decision on whether to seek Cyber Essentials or Cyber Essentials Plus, and (iii) the expected timelines for achieving this. 
- Board/Senior Management – level commitment and basic governance arrangements in place.
- CiSP membership in place.
- Appropriate implementation of Active Cyber Defence measures in place.
- Appropriate training and awareness raising processes in place.
- Cyber incident response protocols in place, aligned with central mechanisms.
4. A further formal request will be made at Board/Senior Management level to all Scottish public bodies at end October 2018 to provide confirmation that Cyber Essentials or Cyber Essentials Plus certification has been achieved.
5. Appropriate information on progress will be made available to the public, Ministers, Parliament and the NCRLB, with due regard to cyber security considerations.
B. Development of ongoing monitoring and evaluation arrangements for target and advanced stages under Scottish Public Sector Cyber Resilience Framework
6. Monitoring and evaluation processes to support the implementation of the Scottish Public Sector Cyber Resilience Framework will be developed in tandem with the Framework. They will be designed to take account of, and align with, the requirements of the new competent authority or competent authorities that will be introduced to oversee compliance with the requirements of the EU NIS Directive, as well as those of Scottish Ministers and the National Cyber Resilience Leaders' Board. They will also ensure alignment with, and support for, existing arrangements in respect of Critical Infrastructure.
7. There will be an initial focus on evaluating how the Public Sector Cyber Catalysts are progressing towards the target and advanced progression levels of cyber resilience against this framework. However, the Scottish Government will also explore the potential for the development of a self-assessment and reporting tool that could assist all Scottish public sector organisations to assess and report against the Framework. The aim of this tool would be to:
- Assist Scottish public sector organisations to self-assess their progress against the initial baseline, target and advanced progression stages under the Framework in a standardised way, including against a simplified RAG status.
- Minimise additional reporting and compliance burdens by making clear the links between other standards, requirements or guidelines, and assisting in "translating" these into a self-assessment against the Framework. For example, where public bodies hold Cyber Essentials Plus, it may be assumed that this would provide a "green" RAG status in respect of certain criteria set out in the framework. Where public bodies are on the SCOTs  or other shared networks, it would be assumed that the standards achieved by providers of those networks apply to those connected to them.
- Produce standardised reports for submission to the Competent Authority/Authorities and Scottish Ministers on a mandatory (in the case of NIS operators of essential services) or voluntary (for all other organisations) basis.
8. In considering the feasibility of this self-assessment tool, the Scottish Government will work closely with the NCSC to build on a proposed Cyber Assessment Framework currently in development to support the NIS requirements. Subject to the successful development of the tool, consideration will be given to extending the requirement to report progress against the Framework to all Scottish public bodies in due course.
9. An indicative version of a monitoring and evaluation framework for the target and advanced stages of the Framework is set out below. This will be updated as the Framework is finalised and details of the NIS requirements become clear.