Producing official statistics

Quality management policy

Purpose

This policy sets out the broad Quality Management (QM) approach used across the Scottish Government.

Information about the quality approach for individual statistical products, including data, analysis (including any coding undertaken) and outputs, as well as uncertainty and limitations inherent in the statistics, are published alongside releases.

Principles of Quality Management

The Scottish Government Quality Management approach is built on the following principles, aligned with Code 3.0:

Proportionality

The Code says that quality means that statistics fit their intended uses, are based on appropriate data and methods. This means that quality is not about perfection; it is about producing statistics to a sufficient level of accuracy that meets users’ needs.

European Statistics Code of Practice dimensions of quality

The European Statistics Code of Practice identifies five dimensions of data quality to assess the fitness for purpose of statistical outputs:

• relevance - the degree to which statistics meet the current and potential needs of users
• accuracy - the closeness between an estimated result and the (unknown) true value
• timeliness - the time gap between the publication date and the reference period for the statistics
• accessibility - the ease with which users can access the statistics and data
• comparability - the degree to which statistics can be compared over time, region or another domain

Not all data sources are equally strong in each of these dimensions. This allows us to consider trade-offs to help determine quality in the context of user need. The main trade-off we typically encounter is timeliness versus accuracy. By this we mean there is greater value to users in having timelier data which is potentially of lower accuracy.

Communicating uncertainty

In communicating uncertainty we aim to provide information to allow users to make a judgement if it’s fit for purpose for them. This can include information about:

• data quality strengths and limitations
• what the statistics do and do not say
• what they data can and cannot be used for
• indications of margins or error i.e. confidence intervals
• model assumptions
• any other context that may be relevant

Quality Assurance Processes

Statistical producers aim to improve the accuracy of their statistics in a number of ways. These may include:

• assessing the quality of source data, including by reviewing data providers’ governance, processes, and reliability
• independent checking of data by staff not directly involved in initial production
• reproducibility checks (ensuring outputs can be regenerated using documented processes)
• automated and manual validation tests to detect anomalies, outliers, or inconsistencies
• asking topic experts to review outputs prior to publication

Any issues identified are investigated and resolved. Where any known issues remain in statistical publications they are communicated transparently to users.

Further detail about how quality is managed for specific publications is published by statistical areas.

Oversight

Senior Analysts

Senior Statisticians and analysts in Scottish Government are responsible for the quality of statistics in their topic area. In relation to quality this includes:

• ensuring staff fully understand their quality management processes for their statistics and receive training where needed
• helping team members understand the user needs of their statistics
• developing a quality culture within their teams and understanding of the importance of the fitness for purpose of statistics relative to the user needs
• supporting teams to interpret and apply Code 3.0 standards

Office of the Chief Statistician (OCS)

The Office of the Chief Statistician is responsible for:

• ensuring statisticians and other professions have a high-level awareness of the requirements of quality required by the Code
• maintenance of guidance and good practice on quality through the Quality Community Group
• advice on quality matters
• any relevant training on quality approaches, including building statisticians’ confidence in taking the judgement calls associated with the concept of proportionate quality
• point of escalation for the profession on serious quality issues, and advice on mitigation, resolution and communication to users

Review

This policy will be reviewed annually or:

• after significant updates to the Code of Practice for Statistics
• following internal audit recommendations
• when new risks or technologies emerge requiring changes to quality processes

Further Information

For enquiries about Scottish Government quality management practices, contact: statistics.enquiries@gov.scot

Alternatively, you can email OSR on regulation@statistics.gov.uk or contact them via the OSR website.