Scottish Surveys Core Questions (SSCQ): Code of Practice compliance

3. Quality management approach

3.1 Scottish Government quality framework

The overall quality management approach for Scottish Government statistics is published centrally by the Office of the Chief Statistician (OCS). This sets out the organisation’s commitment to producing statistics that meet the standards of the Code of Practice for Statistics. 

This document complements the central framework by describing how quality is managed for the Scottish Surveys Core Questions (SSCQ), reflecting the specific nature of pooled survey data and the associated production processes. 

3.2 SSCQ quality management approach

We are committed to ensuring that SSCQ statistics are fit for purpose, methodologically robust and clearly communicated. Quality is managed throughout the full production process – from data acquisition and processing through to outputs and publication; in line with both the Scottish Government’s overarching quality framework (see 3.1) and the Code of Practice 3.0.  

Our approach combines: 

• Close collaboration with survey teams and data suppliers 
• Structured, reproducible production processes 
• Integrated and proportionate quality assurance 
• Ongoing monitoring of data, methods and outputs 
• Transparent communication of strengths, limitations and uncertainty 
• Continuous improvement informed by users and internal review 

3.2.1 Working with data suppliers and partners

SSCQ statistics are derived from three major Scottish Government surveys: the Scottish Household Survey (SHS), Scottish Health Survey (SHeS) and Scottish Crime and Justice Survey (SCJS). Maintaining strong working relationships with these survey teams and their contractors is central to managing quality. 

We work closely with survey teams throughout the year, including: 

• Regular scheduled meetings to discuss fieldwork progress, sampling, weighting, and emerging issues 
• Early engagement ahead of key production stages (e.g. sampling and weighting) to align expectations and timelines 
• Use of shared planning tools to anticipate risks and identify potential bottlenecks 
• Ongoing collaboration to resolve data queries during SSCQ production 

The Survey Strategy and Coordination team has responsibility for the core questions used across the surveys and works to ensure that these are implemented consistently. Where survey-specific design differences exist, we account for these within SSCQ processing and harmonisation. 

This collaborative approach helps ensure that underlying survey data are of high quality and that any issues are identified and addressed as early as possible. 

3.2.2 Managing quality across the production process

Quality assurance is embedded throughout the SSCQ production process, which is based on a structured and largely script-based workflow using SAS and R.

Data inputs and harmonisation

Data from the three surveys are combined through a process of harmonisation and validation, including: 

• Aligning variables and classifications across surveys 
• Incorporating additional data required to complete SSCQ variables where these are not included in standard survey extracts 
• Checking consistency across datasets and over time 

Weighting and methodological quality

Robust weighting and calibration are central to SSCQ quality. These include: 

• Calibration to National Records of Scotland population and household estimates 
• Separate weights for different analytical purposes (e.g. household, individual) 
• Survey-specific adjustments where required 

Weighting outputs are compared with previous years and assessed for plausibility to ensure consistency and representativeness. 

Production processes and reproducibility

Production is supported by scripted workflows which: 

• Standardise processing steps and reduce manual intervention 
• Enable reproducibility and transparency 
• Support validation of intermediate outputs 

While a combination of SAS and R is currently used, we are actively working towards increased use of open-source tools and reproducible analytical pipelines in line with wider Scottish Government priorities. 

Monitoring quality (inputs, methods and outputs)

We monitor quality across the full statistical lifecycle through a combination of automated checks, analytical validation and structured project management processes. 

This includes: 

• Comparing key estimates across years to identify unexpected changes 
• Assessing distributions and consistency across surveys 
• Evaluating weighting outputs, including design effects and calibration results 
• Cross-checking outputs against source survey publications 

Automated quality assurance processes are designed to highlight significant differences in estimates, prompting further investigation where required. 

Quality is further supported through formal project governance, including: 

• Detailed project planning 
• Issue logs with clear ownership and review timelines 
• Code change logs and version tracking 
• Lessons learned processes and post-publication reviews 

These processes ensure that quality issues are systematically identified, tracked and addressed, and that improvements are carried forward to future releases.

Quality assurance (QA) of outputs is carried out through layered internal and external processes. 

Internal QA includes: 

• Validation of outputs at each stage of the production  
• Cross-checking supplementary tables, technical report tables and underlying data 
• Verification of disclosure control procedures 
• Comparisons with previous SSCQ outputs and source survey results 

External QA involves engagement with: 

• Survey teams (SHS, SHeS, SCJS) 
• Policy teams and topic experts 

who review outputs and provide feedback prior to publication. This helps ensure that results are credible, interpretable and aligned with user needs. 

3.2.3 Communication of quality, uncertainty and limitations

We support appropriate use of SSCQ statistics by clearly communicating their strengths, limitations and uncertainty. 

As SSCQ outputs consist of data tables and a technical report (rather than a main narrative publication), the technical report is the primary vehicle for communicating quality. It includes: 

• Detailed descriptions of data sources and methods 
• Measures of uncertainty, including confidence intervals 
• Explanation of weighting and calibration approaches 
• Discussion of comparability between surveys 
• Information on coverage, accuracy and limitations 

Disclosure control procedures are applied to protect confidentiality, including suppression of small cell counts. Any revisions or corrections are clearly communicated through published errata, with explanations of the nature, cause and impact of the change.

3.2.4 Balancing quality dimensions and proportionality

In producing SSCQ statistics, we balance the key dimensions of data quality: relevance, accuracy, timeliness and punctuality, accessibility and clarity, and coherence and comparability. 

The SSCQ approach reflects the nature of pooled survey data. For example: 

• Pooling multiple surveys improves accuracy and enables robust analysis of smaller population groups, but introduces additional complexity and can impact timeliness 
• Harmonisation across surveys improves coherence and comparability, but requires careful handling of differences in survey design 

We prioritise producing accurate and coherent statistics that are suitable for detailed analysis, while ensuring that outputs remain accessible and clearly explained for users. 

We apply comprehensive quality assurance to all core SSCQ outputs to ensure the statistics are robust and capable of supporting a wide range of potential uses. For ad hoc analysis, quality assurance processes are applied proportionately to the intended use and impact of the outputs. 

As part of the ongoing SSCQ review programme, we are strengthening our understanding of these trade-offs and will continue to improve how they are communicated to users. 

3.2.5 Continuous improvement

We are committed to continuous improvement of SSCQ quality through: 

• Regular engagement with users and stakeholders 
• Ongoing collaboration with survey teams 
• Systematic review of methods and processes 
• Post-publication evaluation and lessons learned 

Current development work includes the SSCQ review programme, which is considering user needs, accessibility, harmonisation and methodology. Changes are assessed for their impact on quality and comparability and are communicated transparently to users.