SPS Management Information Technical Manual

These data are snapshots of a live system and as such have not been through the quality assurance processes for the time-lagged extracts used in the construction of the cellWise data which underpins the Accredited Official Statistics publications. The caveats around data quality for that source therefore apply equally or more so to statistics extracted from the SPS Daily MI.

In their favour, the live snapshots provide information not available or retained by system when it is overwritten or changed. A key example is the “Overall Sentence Length” field, which is prone to change whenever the active warrants in a person in prison’s file change.

Quality Assurance of Administrative Data

A comprehensive survey of the cellWise data source was conducted in March-May 2024 and documented in the Scottish Prison Population Statistics Technical Manual. Many of the findings of that exercise hold equally for information collected in the daily snapshots. However, due to the live nature of the data, new information will not typically have gone through the “quality assurance of retained values” stages and so are subject to greater error depending on their collection environment and input validation.

Delivery Errors

System errors or connection issues may result in the non-delivery of snapshots. These are typically spotted early, and replacement (if somewhat later) snapshots are taken to replace them. Where the time elapsed from the snapshot non-delivery to noticing the error is greater than a few hours, snapshots from neighbouring days are duplicated at the discretion of the analyst. This has typically occurred over weekends, when there is little movement into or out of the prison estate.