Open Government action plan 2021 to 2025 - commitment 3: data and digital commitment

Progress to June 2025

Commitment 3: open data and digital

Objective

Open up data relevant to other open government themes, such as key climate change datasets used by government for modelling and reporting, data on public transport and public sector expenditure.

Milestones

Continue to support the Fiscal Transparency Programme, climate change policy and other areas relevant to the open government themes to encourage open data reuse and share best practice on data standards, FAIR and open data principles.

Improve our understanding of the technological solutions for deliberation, to support the Scottish Government Participation Framework and to inform potential future applications to expand participation around public good uses of public data and AI.

Progress

We have completed our project with students from the University of Edinburgh Business School. A final presentation took place in March. The group delivered a detailed report outlining a framework for purpose-driven case studies illustrated with the Climate Action Platform. Outputs also included a pilot ‘Open Data City’ app. This interactive tool was designed to demonstrate the value of open data to a wide range of audiences, using QR codes, storytelling and gamification to make open data engaging, accessible and meaningful.

The project went beyond the original case study approach, offering a more innovative and public-facing model for open data engagement. It has generated new ideas about how to rework the narrative around open data beyond technical outputs and connect with different stakeholder groups to demonstrate value and impact.

We are continuing to work with internal and external stakeholders to explore how we can improve transparency in grants data using the 360Giving data standard, which is endorsed by the UK Data Standards Authority. A key challenge is balancing the complexities of opening up data with the value of creating a standardised open dataset.

The Accelerator phase of the CivTech process exploring how technology can support high-quality, scalable public participation in decision-making has been successfully concluded, with CrownShy delivering an MVP of Comhairle, an open-source-licensed platform that integrates many widely used engagement tools into flexible workflows.

Comhairle MVP features the integration of three key functionalities – a pre-existing open-source deliberative tool Polis, an open-source form builder HeyForms, and an in-person meeting planning / capture tool to facilitate the use of Comhairle in in-person settings.

At the MVP stage, Comhairle offers three options for public users to participate – using a Comhairle account (email and password), using a pseudo-anonymous ID, or using ScotAccount.

Comhairle MVP has now been used to support Scottish AI Alliance public engagement panel deliberation, and scale it beyond their usual audiences.

Preparations are underway for the pre-commercial agreement stage of the CivTech process, which would enable the Scottish Government and CrownShy to further develop Comhairle and roll it out.

Comhairle was showcased during the CivTech Demo Day on the 11th of June.

Next steps

• Civ-Tech demo day, filming day
• Civ-ech pre-commercial agreement stage

• publish blog: “Open Data: Working Together and Innovating for Impact” summarising the work of the Edinburgh University Business School project on developing a framework for open data case studies

Status

On schedule.

Objective

Run a CivTech challenge to evaluate if technology can make public sector data easy to find, assess outcomes and set out the way forward.

Milestones

Develop a considered approach to dataset sourcing and management for Find.Data.Gov.Scot to improve the discoverability of identified datasets, and by November 2025, add 10 more data sources to Find.Data.Gov.Scot.

Deliver an improved user feedback channel and run workshops with data producers and consumers to identify missing areas, address data quality issues and promote use. Investigate whether Find.Data.Gov.Scot can be linked to the UK Data Marketplace Government Data Catalogue.

Progress

The Find.Data.Gov.Scot data discovery search engine, developed through the CivTech process, has now been delivered. A contract is in place for the current year, with work focusing on automatically detecting and extracting metadata from additional sources to extend its coverage, resolving issues with duplicate data sources, and improving the quality and relevance of search results.

The supplier, Dtechtive, will also present a session on the importance of data and metadata quality for the Better Data Community of Practice series of events on data quality, which are taking place during June and July.

With the platform development complete, the team is now progressing with the contractual arrangements to support its wider use and promotion.

Next steps

• continue to add data sources to the site
• promote its use as ongoing business as usual

Status

On schedule.

Objective

Set up the Data Transformation Framework stating what ‘good data’ looks like and the process by which organisations can improve – this focuses on opportunity for organisations to improve data maturity, data literacy and adoption of standards, through collaboration and engagement with local government and other public sector bodies, to be useful for civil society.

Milestones

Launch the data maturity network for cohort alumni on KHub, to provide post-project support and networking; including published resources for data standards and data governance (complete and test by summer 2024)

Start to develop resources for data ethics, innovation and architecture and provide strategic guidance on the four foundations to data improvement that align to these pathways.

Develop and launch self-supporting data maturity guidance.

Complete cohorts 3 and 4 of the Data Maturity Programme by June 2024, launch cohort 5 in September 2024 (if funded).

Progress

The Data Transformation Framework continues to evolve in response to the insights gained from the Data Maturity Programme and the changing needs of organisations. The framework offers actionable next steps, with guidance pathways to support organisations to improve data management practices.

Guidance is being developed across the eight pillars of the data ecosystem, with updates reflecting changing knowledge of needs and technological advancements. This is particularly relevant for the innovation pillar where guidance pathways are being revised to address the growing interest in using artificial intelligence (AI). Similarly, the data standards pillar has been updated to include opportunities from accessing and publishing open data and has been renamed access and re-use.

Options are being explored to identify the most suitable platform/location to publish the framework guidance. A decision is expected in the coming months.

An information session for potential participants in the sixth cohort of the Data Maturity Programme (starting June 2025) was held on 21 May. A full cohort of ten organisations is expected to take part, including two NHS health boards and at least two further local authorities. Onboarding will start earlier than before to allow more time for organisations to establish a data advisory group, which is key to a successful data maturity assessment and follow delivery. The programme is now extended to nine months, and demand remains high. Work is underway to develop self-supporting resources for organisations who cannot take part or are seeking guidance for their activities.

With the programme now well established, attention is turning to evaluating its impact. Participants from cohorts 3-5 will take part in evaluation conversations over the coming months to assess whether it has helped them prioritise and resource data improvement activities.

The Data Maturity Network continues to provide peer support and a space for ongoing data improvement conversations. Topics are selected by the group with data culture covered in sessions with guest blogs during March-April and data governance featuring from May-June.  A theme covering data sharing and open data is in planning for the autumn.

Next steps

• identify a platform to publish guidance
• develop modular guidance
• conduct evaluation conversations

Status

On schedule.

Objective

Review the front end of our official statistics open data publishing platform, www.statistics.gov.scot.

Milestones

Initiate and run a discovery, which aims to gain a better understanding of the current service.

Complete discovery and produce a report which advises on options and recommendations for next steps.

Based on evidence and recommendations from the discovery, either move to next steps in improving the service in an alpha phase; or make small/limited changes to the service.

Progress

This year we have commenced an Agile alpha to build prototypes to test different ideas and explore new approaches of making data available. We have designed prototypes which are focused on addressing the high priority user needs that were identified during discovery, as well as addressing the key problems and challenges our users face. We are continually refining and developing our understanding of who are users are, what they need, and what our service should look like. We are involving users and data publishers through all stages of development and evaluation. For example, we have given a live demonstration of progress and created a slide pack outlining progress to date and next steps every 4 weeks. We have created an evaluation framework to test and access the viability of each solution in meeting needs. 

We are running usability sessions during June and July 2025 to understand what works and what doesn’t for our users. We have promoted these sessions through various groups to ensure a plurality of voices. These groups include the Open Data Scotland Slack Channel and the Trust and Transparency in Scotland Community bulletin.

As always, we are interested in hearing from you. You can get in touch with the Scottish Government’s Open Data team directly at statistics.opendata@gov.scot.

Next steps

• run usability testing to learn more about what our users need
• summarise our learnings from alpha into a findings and recommendations paper covering user and technology workstreams
• continue Beta planning and begin to transition from Alpha to Beta

Status

On schedule.

Objective

Increase the amount of Scottish public sector open data being published, through collaborations such as the Data Standards and Open Data Community of Practice.

Milestones

Publish the independent report and recommendations outlining a more strategic approach to supporting open data in Scotland.

During the remainder of 2024 initiate a project plan and develop a roadmap, with the outcome of creating a clear vision for purpose-driven impactful open government data in Scotland, and priorities for its implementation.

Set up an open data external reference group to provide accountability to the open government data commitment and to foster and long-term collaborative approach to open data.

Build on the series of open data events held with the Better Data Community of Practice in March 2024 with a series on data standards in May 2024 and encourage knowledge sharing about common challenges in making data better, smarter and more open.

Progress

We have finalised the discovery phase to take forward the recommendations from the Open data in Scotland: a blueprint for unlocking innovation, collaboration and impact report which was published in August 2024. The discovery has allowed the team to explore and investigate the opportunities arising from the recommendations from the report. We have reflected on the learnings we have achieved as part of the discovery, and in particular how they relate to the recommendations from the report. 

Below are some of the activities which have been completed within the discovery, all of which have been carried out in partnership with external organisations:

• we ran interviews with 8 informed stakeholders from the public, private, academic sectors and civil society, to assess what their view of the current state of play with open data is, and what we should focus on to ensure success. Priority areas have been identified as Awareness and Use of Data; Spreading the message through publishing and developing the landscape through capability building
• as mentioned above, we supported students from the Edinburgh Business School Entrepreneurship and Innovation MSc programme to develop a purpose-based framework for open data case studies
• we contributed to a steering group led by NatureScot and Alma Economics on Understanding the Need and Value for Land Cover and Habitat Data. This project has now been finalised and NatureScot prepared summary slides for which they developed a systematic framework to estimate the value of data. NatureScot intend to publish a report on their website

As part of its ongoing work, the Better Data Community hosted 7 events under the theme of Responsible and Inclusive Data Practices. This theme was inspired by the ODI Data Spectrum and encouraging rich discussions around the values and practices we should adopt in the public sector particularly those centred on transparency, participation, and inclusion.

Highlights from the events

Improvement Service – Data

A series of presentations by the Improvement Service which showcased how a national platform is created using data from every Scottish Local Authority and how data is harnessed to help decision making.

AWS Made Simple: A Beginner’s Guide for Data Experts

The SG Data Division delivered a beginner-friendly introduction to AWS, specifically tailored for those working in data roles across the public sector.

Practical Data Sharing: Why It’s Difficult (and What Can Help)

The SG Data Division led a session where they discussed some of the practical challenges faced in public sector data sharing, from issues of quality and clarity to trust and process inefficiencies. They further explored and showcased how innovative solutions, such as zero-knowledge proofs and virtual query layers, can help build trusted and efficient mechanisms for data sharing.

Oracle – A Corporate Data Story

The launch of Oracle has been a big undertaking. This is the story of the transformation from a data perspective and an opportunity to hear how the SG team developed data models to describe how People and Finance data hang together, and how this was the start of a much bigger Corporate Data story.

Maps in Public Services: New Guidance

SG colleagues shared the latest best practice guidance for creating maps in public services.

SEPA Open Data Hub

The event explored SEPA (Scottish Environment Protection Agency) approach to setting up a cloud-based spatial open data hub designed to share its environmental data for the benefit of businesses and users.

What Users Want & the Challenges They Face Accessing Environmental Data

The Digital Solutions Programme (DSP) shared insights from 2 years of user research exploring the needs of environmental data users across the UK’s four nations, towards building digital tools for NERC’s vast environmental data holdings.

The Community has now moved on to its next theme, Improving Data Quality. Upcoming sessions will feature speakers from the Office for National Statistics, Dtective.com, and the UK Government Digital Service, among others.

Next steps

• use the learnings from the open data policy discovery to plan out the next phase of work for open data policy in Scotland.  This will be done in conjunction with our planning for open data’s role in Scotland’s next Open Government Action Plan
• prepare a slide pack with the intention of presenting findings from the open data discovery along with a roadmap going forward
• begin work with NatureScot to test the data valuation methodology more widely using other datasets in other sectors 

Status

On schedule.

Objective

Develop a public register of AI algorithms.

Milestones

Begin a phased approach to the public sector wide roll out of the Scottish AI Register.

Initial stage to produce SG specific guidance and support materials for the creation of AI.

Identify existing use cases and new SG AI projects to be added.

Work with SG Data Science colleagues to ensure the adequate recording of all corporate AI use.

Work with procurement colleagues to build guidance into purchasing processes.

Continue promotion of the product through the Scottish AI taskforce to help future roll out.

Progress

Additional use cases have been added, and work is underway to finalise the process of moving from the old to new governance module.

Actively promoting with UKG and international governments to share best practice and the innovative approach the register supports.

Work is on-going to identify the elements regarded as mandatory and those to be shared with the public or held within the organisational governance model.

Sessions planned with SCS colleagues to share and promote the work of the Register.

Next steps

• on-going work to procure longer term contract for delivery of Register Infrastructure
• discussion of possible governance integration into an AI development environment, to build compliance in to delivery
• work with local authority digital colleagues to identify route to build it in to their processes.
• still facing challenge of the capacity required to build, promote and enforce compliance with the Register, plan to raise at a more senior level and consider options
• discussions with internal specialist groups (data, ethics, security etc. to integrate official policy guidance into governance

Status

On schedule.