Criminal justice statistics user guide

2. Data sources and data standards

2.1 Criminal History System (CHS)

Data used in both bulletins comes from the Scottish Government Criminal Proceedings database. This is derived from data held on the Criminal History System (CHS), an administrative system used to track individuals through the criminal justice system. It is a central database used for the electronic recording of information on persons accused and/or convicted of committing a criminal act. The CHS is maintained by Police Scotland who are responsible for managing its operation and own the majority of the data. Chart 1 in the criminal proceedings bulletin depicts how people accused of committing a crime move through the criminal justice system. People can be disposed from the system in a variety of ways, including being dealt with directly by the police by fines or warnings, being fined or warned by the Crown Office and Procurator Fiscal Service (COPFS) or being proceeded against in court. At each of these stages information is logged on the CHS regarding the status of the accused. COPFS and the Scottish Courts and Tribunal Service (SCTS) make updates on their own systems which are fed back electronically to Police Scotland’s CHS. When an offender’s case reaches its conclusion or “disposal” and a sentence is given for guilty offenders, the case is considered completed on the CHS, and after this point, the data is sent to Scottish Government (JAS) in the next monthly return.

Individual-level returns are received in the CHS monthly. These are electronically submitted by Police Scotland for cases that are completed, or if a case has been further modified. Information on criminal trials that are on-going or have not been dealt with through the police or COPFS disposals are not held by the Scottish Government.

The CHS is designed for police officers use and not for statistical reporting. The Scottish Government only receives a subset of the CHS which is needed to produce the datasets for criminal proceedings and reconvictions.

The source of the statistical data on bail orders and undertakings is also drawn from the CHS.  Separate monthly files are received for this data.

As a court proceeding or police/ COPFS non-court disposal can be made up of more than one offence, production of the statistics at ‘persons’ level requires an intermediary processing stage to be carried out on the CHS data. Where a person is proceeded against for more than one crime or offence in a single proceeding, only the main charge is counted. The main charge is the one receiving the most severe penalty (or disposal) if one or more charges are proved, and is identified using a look-up table which ranks the disposal types in order of severity.

For example, custody is ranked higher than a monetary fine, so for a proceeding where there was a mixture of these two types of disposal, the main charge counted for this record would be the charge associated with the custody disposal rather than the charge related to the monetary disposal.

If two charges have the same disposal, then the charge mapped to the lowest numbered crime code is taken as the main crime. Generally, the lower the crime code, the more serious the crime would be considered to be. The lowest crime code is for murder and the highest for motor vehicle offences.

2.2 Scottish Offender Index (SOI)

Information on reconvictions is based on data held in the Scottish Offenders Index (SOI), which is derived from the data held on the Criminal Proceedings database.

The SOI currently contains a record of criminal proceedings against individuals (excluding companies) in Scottish courts as well as information on non-court disposals. The court convictions and non-court disposals are held in separate datasets by the Scottish Government and so are independent of each other.

The data in the SOI on court convictions currently covers all convictions where a sentence was imposed since the beginning of 1989, and the main offence involved was either a crime in Groups 1-5 or some of the offences in Group 6, in the Scottish Government’s classification of crimes (see User Guide to Recorded Crime Statistics in Scotland for further information about these classification groups). Minor offences (such as drunkenness, and almost all motor vehicle offences) are not included in the SOI. A list of which crimes and offences are included can be found in the User Guide to Recorded Crime Statistics in Scotland.

The data in the SOI on non-court disposals covers all crimes and offences, including motor offences, where a non-court disposal was given, back to 2008.

Each record on the SOI database includes information on the sex and date of birth of the offender, the dates of conviction and sentence, the main crime or offence involved and details of the sentence imposed. Information is also available on any crimes which were additional to the main crime. Each offender has a unique reference number, which allows individual convictions for that offender to be linked together. The SOI does not include the name and address of an offender, except the first half of an offender’s postcode. A Privacy notice is attached at the end of this document .

All but the most serious offences alleged to have been committed by children under the age of 16 are generally dealt with outside of the courts in the children’s hearings system, or by Early and Effective Interventions or other non-court disposals. The SOI does not hold data taken from the children’s hearings system.

2.3 Other data sources

Both Chart 1 and Table 1 in the criminal proceedings bulletin aim to summarise the entire criminal justice system. This includes using a range of summary statistics other than those derived from the CHS. This includes:

• Scottish Government’s Recorded Crime in Scotland statistics
• COPFS’ Case Processing Statistics
• Scottish Children’s Reporter Administration Official Statistics

Please see section 6 for a description on how the counting bases for these data sources differ.

Other external data are used in different parts of the criminal proceedings and reconviction rates bulletins. These include:

• population figures used to produce the conviction rates per 100,000 people (Table 5c in criminal proceedings) are the relevant mid-year estimates prepared by the National Records of Scotland (NRS)
• lengths of the punishment part of life sentences and Orders for Lifelong restriction data found in criminal proceedings are from a bespoke data extract provided to us by SCTS

Data on offence dates which are needed for the production of reconviction statistics are derived by getting proxy offence dates from the Police Oracle Database, which are derived from the case reference number, as the reference number is partly formed from the offence date. Then the police records that relate to the offences in the Scottish Offenders Index (SOI) are included in the reconvictions analysis. These are selected by merging with the Criminal Proceedings Persons dataset. If the actual offence date is found in the Crown Office data, then the proxy offence dates are overwritten with the actual offence dates.

2.4 Data standards for justice partners

Data standards are adhered to by organisations inputting data to the CHS in terms of the definitions of data items and their corresponding values. These standards are agreed under the Justice Digital Strategy and ensure there is consistency across the justice organisations in the information they collect. Further information on the data standards can be found on the Scottish Government’s Access to justice pages and in the Integration of Scottish Criminal Justice Information Systems (ISCJIS): data sharing manual.

The following protocols also ensure consistency in the data collected:

• the Scottish Courts and Tribunal Service protocol for the handling of errors that may occur in the transmission of data between justice partners’ databases
• the protocol for the investigation/resolution of disputed data between Police Scotland and the Crown Office and Procurator Fiscal Service
• the protocol for sharing electronic information between justice partners

The Scottish Government also has representation on a data quality group and is kept informed of any data quality issues relating to the CHS. This group meets around three times a year and includes representatives from Police Scotland, COPFS, Scottish Children’s Reporter Administrator and Scottish Courts and Tribunals Service.

The Scottish Government has implemented a crime code classification framework to ensure consistent and comparable reporting between criminal justice statistical outputs. Further details are available in the Scottish Government Guide to recorded crime Statistics in Scotland.