Queen Elizabeth University Hospital: case note review - overview report

3. Methodology

In this chapter we set out the approach we developed to access, collect and assess the data we believed were necessary for us to address the ToR governing our Review.

Data systems within NHS GGC were identified, access was negotiated and sources of other potentially important information sought and requested. Two data collection teams were formed to work in a complementary way to identify and extract different components of the clinical and microbiological information required to create a detailed timeline of clinical care for each eligible bacteraemic episode for every patient included in the Review. We created processes for documenting, collating and summarising data from multiple sources so as to inform the Panel discussions which assessed and determined outcomes.

Section 3.1 presents an overall timeline and also describes some of the constraints we encountered in our work. Sections 3.2 to 3.6 describe each of the steps in our processes and section 3.8 describes our approach to communication with stakeholder groups.

3.1 Overall timeline for the work undertaken for the Case Note Review

The overall process of the Review and the work of the Panel is summarised in the diagram provided as Appendix A.

There was an early assumption that the overall timeline to complete the work for the Case Note Review would begin in March 2020 and end in the summer of 2020. This view was held not only before the impact of COVID-19 became apparent but also before data collection commenced and we had begun to understand the challenges that lay ahead.

Communication and engagement with NHS GGC, requesting critical data for Panel consideration, began on 7.4.2020 and continued until a final set of data was received on 21.12.2020. A final meeting with NHS GGC was held on 4.2.21 to discuss late concerns about the data available to us.

Throughout the Review our aim was to communicate progress, and delays, to stakeholders by means of written updates and virtual meetings. The timeline illustrates these occurrences from March 2020 to February 2021 however the communication element of the Review will continue beyond publication of this report, particularly with patients and families (discussed further in Chapter 7).

3.1.1 Constraints on the work of the Panel

In planning for this Review, in February 2020, a number of individuals were being identified to work directly on aspects of the work but, by the end of May 2020, as a result of the competing demands of the COVID-19 pandemic, the number of those still available for the Review Team was significantly reduced. The 3 members of the Expert Panel had also each identified reduced capacity because of varying commitments to support COVID-19 related work in NHS England. At this point, the Review had reached a critical point: data extraction had been successfully established and Panel reviews of patient records were just starting. The last full meeting of the Panel had been on 26.5.2020 when the issue was discussed by the Core Project Team on 2.6.20. It was decided at that meeting to pause further Panel meetings for a period of time, but that data extraction could continue.

This became, however, a relatively short-term issue and by the second half of 2020, resource was adequate to fulfil the ToR for the Review (with, as is noted in section 8.1.3, the later appointment of additional IPC support). Panel meetings recommenced on 29.7.2020 although initially without full membership and by the time of the next meeting on 6.8.2020, the extent of data not yet available from NHS GGC to support the Review process was becoming fully apparent. This was discussed at the Core Project Team on 11.8.2020 and a mitigation plan agreed that resulted in the Panel scheduling Review meetings every week from 25.8.2020 to 15.12.20. Engagement with NHS GGC was increased to reinforce and clarify previous requests for data (see also section 8.1).

The Panel completed its primary review of all cases on 15.12.2020 but, by then, the need for a second review to assimilate late data received from NHS GGC had become apparent. This was completed in January 2021, but concerns emerged at the end of that month that there might be additional, potentially relevant data held by NHS GGC to which the Panel had not had access. Although this was subsequently not felt to be the case in relation to our ability to assess individual patients and episodes of infection, progress on the completion of the report was affected whilst this was investigated; a further additional short delay in the publication of this Report was therefore agreed with the Core Project Team on 10.2.21.

3.2 Selection criteria for inclusion of patients in the Review

The selection criteria for cases to be included in the Review were drafted and agreed by the Core Project Team after also inviting parents of the children and young people in the Review to comment on the proposals. These were approved by the Oversight Board and set out in a protocol document^[27]. This defined that the study population should include all patients cared for in the Paediatric Haematology Oncology service at the Royal Hospital for Children, NHS GGC who met one of the following criteria between May 2015 and December 2019:

• at least one positive blood culture of a Gram-negative bacterium associated with the environment (Group 1)
• at least one positive culture of an atypical Mycobacterium spp. (acid-fast environmental bacteria (Group 2).

It was nevertheless agreed that a flexible approach should be retained, and one patient who did not meet these criteria, but who nevertheless experienced severe infection with a Gram-negative environmental microorganism, although without proven bacteraemia, was included at the request of the family (Group 3).

All families were informed, in a letter sent from NHS GGC on 4.3.2020, of the inclusion criteria agreed by the Panel. Only one family responded to say they did not wish their child to be included in the Review.

3.2.1 Datasets and definitions used to identify patients for inclusion in the Review

The combined dataset used in a previous review by staff in HPS published in October 2019^[28] (and now ARHAI Scotland) formed the basis by which patients were identified to be included in the Review. For the HPS work, qualifying infection episodes were extracted from the following datasets:

• HPS dataset - Electronic Communication of Surveillance in Scotland (ECOSS) extract;
• NHS GGC Central Line Associated Blood stream Infection (CLABSI) Surveillance System;
• NHS GGC ECOSS extract; and
• NHS GCC Microbiology laboratory information management system (LIMS).

The data extract utilised for the previous HPS publication was extended to December 2019 and the final patient/episode list was cross-checked with NHS GGC before the start of the Review.

Positive blood cultures were identified for micro-organisms from the environment including enteric bacteria group. This included all species of the following: Achromobacter; Acinetobacter; Aeromonas; Brevibacillus; Brevundimonas; Burkholderia; Cedecea; Chryseobacterium; Chryseomonas; Citrobacter; Clavibacter; Comamonas; Cupriavidus; Delftia acidovorans; Elizabethkingia; Enterobacter; Flavimonas; Gordonia; Klebsiella; Pseudomonas; Pantoea; Pseudoxanthomonas; Psychrobacter; Ralstonia; Rhizobium; Rhodococcus; Roseomonas; Serratia; Sphingomonas; Stenotrophomonas and atypical mycobacteria.

A full breakdown of the grouping is detailed in Appendix B.

3.2.2 Case definition

In order to consider the diversity of bacteria likely to be identified if there is an environmental source, and to account for polymicrobial episodes, the following case definitions were used.

At the Species level - a positive blood culture of a single bacterium that has not been previously isolated from the patient’s blood within the same 14-day period (i.e. 14 days from date last positive sample obtained).

At the Episode level - a positive blood culture for an environmental including enteric bacteria group that has not been previously isolated with same or other environmental including enteric bacteria group organism in the patient’s blood within the same 14 day period.

In line with the case definition, and to align with other national bacteraemia surveillance, a standard 14 day rolling deduplication was applied to the HPS ECOSS dataset, and these episodes were cross-checked with NHS GGC data sets supplied.

All positive blood cultures were included with the exception of post-mortem blood, any quality test samples, foetal samples or non-human samples.

3.3 Epidemiology data collection

3.3.1 Objectives

The objectives of the epidemiological investigation were to:

• determine a timeline for each of the cases identified for review;
• characterise the cases in terms of time, place and person
  • Time: describe the episodes of blood stream infection over time and create a timeline for outbreak, including plotting of control measures against number of cases
  • Place: describe the location of patients (hospital, ward, bed/bay) and describe their movements in the hospital
  • Person: characterise the patients with infection in terms of intrinsic and extrinsic risk factors; outcomes; antimicrobial prophylaxis and treatment; and individual infection prevention and control measures in place; and
• describe the cases in the context of environmental risks and incidents including the use of environmental microbiological data and Healthcare Associated Infection – System for Controlling Risk in the Built Environment (HAI-SCRIBE)/other facilities data provided by NHS GGC.

3.3.2 Data extraction

A data extraction form was created to capture the data fields identified in a dataset agreed by the Core Project Team^[29] (this is shown in Appendix D).

Dates of inpatient, outpatient and day care attendance were provided by the NHS GGC TrakCare system, including bed location and movement data for inpatient stays. Extracts were linked with patient infection episodes and species level data and a bespoke MS Access database was built which incorporated these datasets.

Patient data were reviewed through direct access to NHS GGC Clinical Portal providing information from medical notes, nursing notes and observation charts, surgical procedures, drug charts, laboratory information and correspondence.

The process by which more detailed extraction of clinically relevant information required by the Panel, and by which the PTT was implemented, is described in section 3.4.

Microbiology management data and infection control actions were separately obtained from the NHS GGC Telepath and ICNet systems (section 3.5).

Although the time period of the Review was from May 2015 to December 2019, when necessary, patient records were reviewed outwith this period in order to obtain diagnostic information and other clinical details relevant to the Review, including accessing electronic notes that had been scanned into the patient record at a later date.

Data from the database were extracted and processed using R software^[30] to generate a report for each patient for review by the Panel.

3.3.3 Timelines

Timelines were created using data visualisation software (Tableau 2019.1). These were viewed via an online platform called Eviz, a secure Tableau server web space managed by National Services Scotland. Panel members were provided with individual password protected log-in details for access.

The timelines created were used to display:

• Patient admission/bed location with infection episodes. This allowed the species level microorganism list to be filtered so that all or only selected bacteria could be reviewed. Patients could be searched individually or collectively and locations of care could be separated by ward and room.
• Environmental water sample data provided by NHS GGC. This allowed the results to be filtered by positive and negative findings, by all or selected microorganisms and, where available, location could be searched at room level.
• Environmental ‘hard surface’ (this includes surfaces on items such as medical equipment, bathroom fittings and drains, and air conditioning units) sample data provided by NHS GGC. This allowed the results to be filtered by positive and negative findings, by all or selected organisms and, where available, location could be searched at room level.
• Facilities maintenance data provided by NHS GGC. This allowed maintenance activity to be viewed by clinical area, down to room level where available, and by type of work.

Time filters allowed data to be reviewed for the entire period of the Case Note Review or for selected periods within this.

3.4 Adverse Events and the Paediatric Trigger Tool

3.4.1 Background to national and NHS GGC Policy

It is internationally recognised that between 10-25% of episodes of healthcare (in general hospital, community hospital and general practice) are associated with an adverse event^[31].

Since 2013, NHS Scotland has used the National Reporting Framework for adverse events^[32]. The category I to III classification framework was in place since 2013, although the regulatory requirement to report all Significant Adverse Event Reviews commissioned for Category I events to Healthcare Improvement Scotland (HIS) was only applied in January 2020.

The NHS GGC Incident Management Policy (2020) details the organisational system to record and address adverse events and near misses. It covers all incidents, whether they involve patients, relatives, visitors, staff, contractors, volunteers or the general public, and indicates that a robust investigation will be conducted into all Significant Clinical Incidents. The purpose of the investigation is to determine whether there are learning points, locally or for the wider organisation.

The main route for reporting adverse events within NHS GGC is through Datix (a web-based incident reporting and risk management software for healthcare and social care organisations). A trigger list categorises adverse events in line with the national guidance and a risk assessment is undertaken to inform initial notification and its escalation. A risk matrix is used to determine the incident’s grade based on its impact and the likelihood of recurrence. The grades used by the matrix are designated: Insignificant, Minor, Moderate, Major and Extreme.

When an incident is scored Major or Extreme there must be an investigation, which investigates causation: one approach to this is Root Cause Analysis^[33]. If the severity is Moderate, there should at least be a local investigation, led by the line manager also using, if appropriate, a root cause analysis type approach.

We chose to explore the occurrence of adverse events by considering data both from the NHS GGC Datix system and from a tool specifically developed to detect adverse events in paediatric care (the PTT).

3.4.2 The Paediatric Trigger Tool (PTT)

A trigger tool is a method for identifying adverse events (AE). In adults, the rate of detection of AE with a trigger tool is typically ten-fold greater than the rate detected through spontaneous reporting systems^[34],[35]. Similar results have been reported with paediatric trigger tools in general wards^[36] and neonatal intensive care units^[37].

In 2014, the UK PTT was developed with the support of clinicians in nine hospitals across the UK in order to detect AE in paediatric care provided in district general hospitals, acute teaching hospitals and specialist paediatric centres^[38].

The intention of using the PTT as part of the methodology chosen for the Case Note Review was not to determine preventable or non-preventable harm but to create opportunities to learn from the AEs identified. The aims were to:

• identify all triggers and adverse events in all patients included in the Review;
• to describe the rate and severity of harm occurring in hospitalised children in this cohort; and
• to compare the rate and severity of harm occurring in the cohort with evidence from published studies

3.4.3 Adaptation of the UK PTT and its use in the Case Note Review

The checklist used for the implementation of the UK PTT is shown in Appendix C.

In preparation for the Review, the UK PTT was reviewed by Professor Hamish Wallace (Consultant Paediatric Oncologist at the Royal Hospital for Sick Children, Edinburgh and previously National Clinical Director of the Managed Service Network for Children and Young People with Cancer in Scotland); and by Professor George Youngson CBE (Emeritus Professor of Paediatric Surgery, Aberdeen University), a UK leader in patient safety practice. Following their review three additional triggers were recommended. These additions (PG12* Pain Score >7; PM9* Missed Doses; PM10* Antifungal treatment) were discussed and agreed by the Core Project Team.

The UK PTT is the same as the Canadian PTT. The validation study for the CPTT showed that inter-rater reliability was high when triggers were identified by a nurse and adverse events confirmed by a doctor^[39]. This is the method that we used. The positive predictive value of the additional triggers in the adapted PTT was high^[40] and we do not believe there is any reason to question the validity of the adapted UK PTT for detection of adverse events.

3.4.4 Data collection

The adapted UK PTT was applied to any episode of care for which the patient was an inpatient in QEUH/RHC for at least 24 hours. A systematic structured process was used to review the entire healthcare record. The process searched for ‘triggers’ within each episode of care as determined by the PTT check list. Once a trigger was identified, the reviewer used clinical expertise to examine the records in more detail to understand the circumstances around the event and record additional contextual narrative details. A second reviewer (a physician) reviewed, confirmed and validated all of the AE identified, recording the details within the PTT checklist and in accompanying additional narrative notes.

NHS GGC were asked to provide copies of all Datix reports for patients included in the Review, for the duration of the Review.

The National Framework in Scotland for learning from adverse events through reporting and review recommends that the following categories (and definitions) should be used to group adverse events:

• Category I – events that may have contributed to or resulted in permanent harm, for example unexpected death, intervention required to sustain life, severe financial loss (£>1m), ongoing national adverse publicity (likely to be graded as major or extreme impact on NHS Scotland risk assessment matrix, or as Category G, H or I on National Coordinating Council for Medical Error Reporting and Prevention (NCC MERP) index^[41]).
• Category II – events that may have contributed to or resulted in temporary harm, for example initial or prolonged treatment, intervention or monitoring required, temporary loss of service, significant financial loss, adverse local publicity (likely to be graded as minor or moderate impact on NHS Scotland risk assessment matrix, or Category E or F on NCC MERP index).
• Category III – events that had the potential to cause harm but no harm occurred, for example near miss events (by either chance or intervention) or low impact events where an error occurred, but no harm resulted (likely to be graded as minor or negligible on NHS Scotland risk matrix or Categories A, B, C or D on NCC MERP index).

The PTT uses the NCC MERP index, whereas Datix uses the NHS Scotland risk matrix to classify adverse events. We therefore converted these classes into the three categories advised by the National Framework for Scotland. We also applied these categories to data from published papers that use the NCC MERP index. An analysis and interpretation of the findings is given in section 8.6.

3.4.5 Literature review to obtain comparative data

Evidence from the literature about detection of AEs in paediatric inpatients using trigger tools was identified through searches in PubMed and Medline. Additional records were identified from published reviews and by searching bibliographies of full text articles. (Details of the literature search strategy, screening of articles and the studies included are in a report on Adverse Event Detection with the UK PTT separately submitted to the Chief Nursing Officer for Scotland.)

In comparing data with NHS GGC, hospitals identified from the literature review were classified according to the nature of the clinical services offered (secondary, tertiary).

As published studies used trigger tools in random samples from all admissions, for the comparison of event rates in NHS GGC with the published evidence, we only included adverse events that were not directly related to the infections causative of their inclusion in the Review.

3.5 Data relating to microbiology management and infection prevention and control

Telepath is the Laboratory Information Management System (LIMS) used by NHS GGC. The system is used to store laboratory sample results for patients (microbiology) and has the capacity to store patient notes (in the patient note pad - PNP) recorded by microbiologists. Communication between microbiologists and clinical teams are recorded in the PNP chronologically by date as a record of any discussions regarding advice provided by the microbiology team. This function allows any microbiologist to access the records and review previous conversations regarding patient specific issues relating to current or previous admissions, or positive samples.

ICNet is an electronic patient management system used by the Infection Prevention and Control Team (IPCT) to manage patients identified with possible or confirmed infection. The Telepath system sends microbiology results to the ICNet system every 15 minutes. This provides timely reporting to the IPCT.

The ICNet system has a pre-defined list of alert organisms (based on the list of alert organisms in chapter 3 of the national manual^[42]) which, if identified from the data transfer from Telepath, will automatically create a case in the ICNet system. This case alerts the local IPCT of a new referral to be reviewed and assessed. The IPCT also have the ability to create a case manually should the ward clinicians report patients with a possible infection where no microbiology results are available, or if separately alerted by a microbiologist. Once a case is created, the local IPCT would review the patient and assess the IPC needs in the same way as an automatically generated case. The ICNet system also receives regular information ‘pushes’ from the NHS GGC patient information system which allows the IPC team to identify patient location in the hospital during their stay. This is particularly helpful to assess any possible infection cross transmission risks and avoids the need to navigate multiple systems.

3.6 Expert Panel Review Process

Our overall process is summarised in Figure 3.2.

3.6.1 Anonymisation of patient data

Patients included in the Review were not identified to the Panel by name. A unique patient identifier (UPI) was created to link to the patient’s Community Health Index (CHI) number and was used by the data collection teams to present information to the Panel from each of the data sources accessed.

Data that came direct from NHS GGC (for example, environmental microbiology and facilities maintenance data) were anonymised by the substitution of patient identifiers with the UPI before being presented to the Panel.

3.6.2 Data collection

Two teams (the clinical/PTT team and the epidemiology team) accessed the NHS GGC Clinical Portal to view patient case note records. Access to other GGC systems was also required to collect further data required for the epidemiology data collection (section 3.3.2) and for the PTT and augmented clinical data collection (section 3.4). These data were collated into a single document created for each patient (case) and each infection/bacteraemia (episode). This was usually supplemented by a second document that provided narrative comments about clinical care and the microbiological management of the infection.

3.6.3 Data Synthesis

The data provided by the two collection teams were reviewed and integrated into a Data Synthesis file, which was created separately for each infection episode. The Data Synthesis File had three components: Dataset; Summary; and Conclusions.

The Dataset component recorded data obtained for the data items defined in the Expert Panel Dataset and was structured to allow queries to be raised about missing data or data requiring clarification.

The Summary component included the creation of a Clinical Timeline which set out the chronology of events around the infection episode. This component also included sections for completion by the Panel in relation to data provided from additional data sources.

The Conclusions component provided a framework to structure the Panel’s response to the key questions required of the Review.

The Data Synthesis files were reviewed at a weekly ‘mini Panel’ meeting with the data collection teams to identify and resolve queries before being passed on for full Panel review.

A copy of the Data Synthesis template is included at Appendix D.

3.6.4 Expert Panel Review

Once complete, Data Synthesis files were provided to us for review at a scheduled Panel Review meeting. All data were made available in individual files for each patient, identified by their UPI and stored in a secure MS Teams channel. In preparation for the Review meeting, in addition to the Data Synthesis file for each infection episode, we also had access to the source material utilised to create the clinical timeline; the Epidemiology timelines (via EViz - section 3.3.3); and to extracts from additional data sources (for example, extracts from the Telepath, ICNet and Datix systems).

Parents of the children involved in the Review had been invited to make submissions to the Panel if they wished, and a small number did so. When this was the case, these submissions also formed a part of the material made available to us as part of our Review.

Each case was first reviewed individually by one of us, to assess the adequacy of the data available and to make a provisional judgment on source/causality, impact and lessons learned. This initial assessment was shared and discussed amongst us at the Panel Review meeting when, after detailed review of the evidence, a consensus decision could usually be reached. In some cases, a decision could not be made pending the need for further information, in which case a further review took place at a subsequent meeting once all the information that could be obtained was available.

Some data (in particular, the results of environmental microbiology sampling, bacterial typing and facilities maintenance activity) only became available to us in a useful form in the later stage of the Review process. The reasons for this are further discussed in Chapter 8. As a consequence, we had to re-review all cases to ensure that our assessments were as informed as possible according to the information finally available. We also utilised the second review process to check for standardisation of our approach, and to review the basis of our initial decisions in the light of an evolving understanding of the issues we had been considering.

We recognised from the outset that we should need to use our judgement to assess and interpret the information available. We agreed, therefore, that our decisions should be justified by using the principle of the ‘balance of probabilities,’ i.e. that, on the evidence available, the conclusions we reached in the review of each case/episode were more likely to apply than not.

3.6.5 Final Outcome Reports

We recorded our final outcome within the data synthesis template for each episode of infection. In some cases, with more than one infection episode, one or more episodes were evaluated together, usually because of close time relationship and sometimes similar causative bacteria.

Prior to commencing our review meetings, we had defined the questions we needed to answer after reviewing each episode of infection. These were as follows:

1. Are the data provided sufficient to complete the review as intended and to reach a conclusion?

Answers: Yes; No

2. Does the infection episode fit within the criteria for the Review?

Answers: Yes; No

3. Is it possible to link this infection episode with the environment of the RHC/QEUH?

Answers: Unrelated; Possible; Probable; Confirmed; Unable to determine

The criteria we considered in determining the likelihood of a link between an infection episode and the environment of the hospital are discussed in section 3.6.6

4. Was there an impact on patient care and outcome in relation to the infection?

Answers: Yes; No; Unable to determine

5. If so, grade severity

Answers: These were initially scored by the Panel as None, Minor; Significant; Severe; Critical but for analysis were directly converted to Negligible, Minor, Moderate, Major, Extreme as used by the NHS Scotland Risk Assessment Matrix

We created a framework to assure a consistent approach in the allocation of a grade of severity (section 3.6.7).

6. What lessons might be learned from this case?

a) To strengthen IPC measures in the future?

b) In any other respect?

7. Are there any other points arising from this review?

8. The Panel’s response to any questions or comments raised by patient / family.

The data from the final outcome reports for all patients were entered into a data analysis spreadsheet to allow descriptive reporting of characteristics from the whole cohort of cases and episodes.

3.6.6 Categorising the likelihood of an environmental source for an infection

In considering the likelihood of the hospital environment being the source of each bacteraemia, we took into account all available (i.e. that which was provided to us) patient, clinical, infection prevention and control, microbiology, local investigations (including Datix and IMTs where available) and hospital environmental data.

The standard epidemiological way of determining causality of, and potential links between infections is according to ‘time, place and person’ information^[43]. The levels of certainty we agreed about a common source of infection (i.e. potentially from the hospital environment) were markedly influenced by whether clusters of episodes caused by the same bacterium occurred over successive days/weeks/months (time), affected different children (persons) in the QEUH and RHC (place). This was most pertinent for either large clusters (in time) and/or bacteraemias due to relatively uncommon bacteria.

We decided to categorise episodes into one of four levels of likelihood that the hospital environment was the source of a bacteraemia: Unrelated, Possible, Probable or Definite. This approach is discussed further in Chapter 5, section 5.6. In some cases, we thought we might be unable to determine likelihood because of inadequate or conflicting data. The allocation of these descriptors inevitably represented a position taken along a continuum of certainty and, for the two largest groups (Possible and Probable) we attempted to refine our position by further extending our categorisation into Weak Possible, Possible, Strong Possible, Probable and Strong Probable groupings. We did not feel we were able to distinguish between Probable and Weak Probable.

For the hospital environment to be classified as a Definite source of a bacteraemia, we required not only time, place and person data to confirm the opportunity for infection to be derived from the hospital environment, but also bacterial typing data (noting the limitations set out below) that matched a patient blood culture isolate to the same microorganism recovered from water or surface samples.

For cases that we considered to be Unrelated to the hospital environment, we agreed either that key issues such as a (relative) lack of opportunity to acquire bacteria from the hospital environment over a period of time consistent with the development of bacteraemia, and/or strong alternative hypotheses about the origin of the bacteraemia, had to be present. For example, if there was strong evidence of an endogenous source, including significant mucositis or typhlitis (both descriptors of damage to/inflammation of the bowel), in the absence of clear clusters of bacteraemias caused by the same bacterial species. Mucositis and typhlitis are known to be associated with an increased risk for the passage of bacteria from the bowel, where many different Gram-negative bacteria can be found, into the blood stream.

We found, as anticipated, that a distinction between the hospital environment being classified as a Possible or a Probable source of a bacteraemia was not straightforward. For a bacteraemia to have a Probable environmental source, we agreed that the information available supported a view that the environment was likely the source (on the grounds of probability), using a standard infection prevention and control assessment of the available data/information. In routine practice, such a conclusion would be made until/unless it was possible to confidently arrive at an alternative hypothesis for the cause/source of infection.

Clustering of cases caused by the same bacterial species was often a key factor in reaching a Probable conclusion – we discuss this further in section 4.3. Other factors included multiple/prolonged opportunities for contamination of intravascular catheters (which is a recognised cause of hospital acquired infection); bacteria that are uncommon causes of bacteraemia; repeated recovery of the same bacterial species from hospital environmental samples around the time of the bacteraemia(s), especially if such samples were taken close to where the patient was managed. The latter point was complicated by the often multiple placements (wards, units and rooms) used for both inpatient and outpatient care of each patient (including leave ‘on pass’ from in patient care). Given our remit, we focused on potential hospital sources of infection, but we acknowledge that community sources of infection were possible; we did take into account the extent of out of hospital exposure prior to a bacteraemia when assessing infection source likelihood.

The more of these criteria were present, the greater was our confidence in concluding a Probable environmental source of infection.

We also recognise that the chance of finding/proving that a microbe in the environment is the source of human infection is directly related to the frequency with which it is sought. This raises two issues: how commonly/systematically is the environment sampled, and are the samples obtained examined specifically for a microbe of interest, or simply to determine the overall number of microbes and/or whether one of a few commonly sought bacteria are present? It is therefore the case that not finding a bacterium in the hospital environment does not exclude the possibility that the latter could have been the source.

Different typing methods are used by reference laboratories to characterise different microbiological isolates and can be used to compare strains of the same bacterium taken from two or more different people or sites. However, it is also necessary to take into account the bounds of possibility around the observation that one strain is the same or closely related to another, given that bacterial DNA can vary in time. Thus, it is standard practice when considering such data, to ascribe limits of differences between strains under comparison, before concluding they are identical, indistinguishable or very closely related (making it highly likely that these are ‘the same’ bacterium) or distinct. One caveat, however, is that where reference laboratory reports merely state that an isolate/strain is ‘unique’, the interpretation depends on the knowledge of what the isolate in question was compared with – is it unique amongst two or amongst a much larger number of strains of the same bacterial species to which it had been compared?

We recognise there are well known constraints affecting any attempt to assess causality, i.e. the possibility that an association might be affected by chance, bias or confounding: we weighed up all these issues in considering the data presented for our assessment.

3.6.7 Standardising the assessment of the impact of infection on patient outcome

Assessing the consequences of the infection represented an important element of our work. In order to do so, we requested data related to the following specific areas:

1. Length of hospitalisation

2. Duration of antibiotic therapy

3. Removal of Central Venous Line (CVL)

4. Admission for intensive care (PICU)

5. Modification of the planned delivery of cancer treatment

6. Evidence of persisting toxicity

7. Death

We also considered

8. Any other impact on care highlighted by the PTT analysis or identified from the narrative of the case note records

9. Statements and insights submitted by parents about their perception of the impact of the infection episode on their child and themselves.

In order to standardise the approach taken, and to allow the generation of descriptive statistics for the final report, we developed a framework that defined a measure of the overall impact of each infection episode on an individual patient^[44].

The framework was informed by the approach taken by NHS Scotland to the categorisation of adverse events and to the definition of the impact/consequences that follow^[45], but it was tailored to utilise the specific outcome criteria we selected for use in the Case Note Review.

Early experience with data collected for the first 18 patients (24 episodes) in the Review was used to pilot a framework which related individual consequences to an overall category of severity. All the pilot episodes had been scored during the early phase of the review process by allocating an overall impact grade on a scale of 0–4 (initially defined^[46] as: 0 = None, 1 = Minor, 2 = Serious, 3 = Severe, 4 = Critical impact). These overall scores were plotted into a grid against the observed occurrence of the items numbered 1-7 in the list above. A single score allocation was then adjusted to achieve a degree of consistency across the differing measures of impact experienced by the whole pilot group. In the course of this, evidence for persisting toxicity (item 6 on the list above) was excluded from the model as data to define this was only readily identifiable in one case which, for other reasons, already met the grade of Critical impact.

The final version of the impact framework subsequently used to score all cases in the final outcome reports is shown in Figure 3.3. In utilising the framework, the occurrence of the specified outcome criteria for each infection episode were plotted onto the grid but could only be allocated into an unshaded field.

The overall impact was determined by the level of the highest impact grade recorded for each episode. For example, no patient who had their CVL removed as result of the infection could be graded as experiencing a ‘Minor’ impact; and no patient who was admitted to PICU because of the infection for >3 days could be graded as having anything less than ‘Critical’ impact.

Whilst the framework offered a standardised approach to allocating an impact grade, we retained flexibility to moderate the grade (up or down) by considering any other relevant information available at the time of our review.

Although the grade allocated implies a numerical level of impact on a 5-point scale, we also attributed a short descriptive identity to each, as follows: (this shown as our original descriptor with the NHS Scotland descriptor in brackets):

• Grade 1: None (Negligible) there is no discernible impact of the infection on the patient’s experience, or outcome.
  • Grade 2: Minor (Minor) whilst the infection had the potential to cause harm, the impact on the patient was limited to a short additional admission and / or to a non-significant delay to planned cancer treatment.
  • Grade 3: Significant (Moderate) the infection may have contributed to or caused temporary harm including any of the following: prolonged admission >7<15 days; removal of CVL; >7 day disruption to planned cancer treatment but without likelihood of long-term adverse consequences.
• Grade 4: Severe (Major)^[47] the infection caused significant disruption to patient experience and / or treatment with the potential for long term consequences. This includes any of the following: prolonged admission >14 days; >14 day disruption to planned cancer treatment; short (<3 day) PICU admission for higher level support.
• Grade 5: Critical (Extreme) this applied when the infection resulted in prolonged (>3 day) admission to PICU and/or if the infection is likely to have contributed to the patient’s death.

Finally, and importantly, we recognise that applying a numerical grade to define our assessment of the impact attributed to an infection episode may not necessarily reflect the ‘lived experience’ of the patient and family who were affected. In offering feedback to individual families at the end of the Review process (see section 7.2), emphasis will be placed as much on the descriptive detail of what we have observed as on the allocated grade.

3.7 Communication with stakeholders

This section provides more detail on who the stakeholders are (acknowledging differences within each group); what information was shared about the Review; the desired methods of communicating with them; and the sensitivities we considered when doing so. It also acknowledges and includes those who have indicated their preferences to not receive communications from the Review.

A summary of the meetings and other communications activity undertaken during our Review is shown the timeline in Appendix A.

3.7.1 Children, Young People, Parents and Families

Addressing individual questions from children, young people, their parents and families was a key driver for the Review. This section focuses on how we tried to understand individual circumstances and to ensure that our response took this into consideration.

There are different levels of engagement within this group; for example, some families did not want to receive communications about the Review. We also recognised the sensitivity required to address differences in perspective. For example, some families are affected by the death of their child whether or not this was thought to be related to infection or not. Others may feel their previously expressed concerns have not been ‘heard’ and/or still have unresolved questions relating to their child’s care. Others still may feel that their previous questions were not addressed in ways that instilled confidence or assured them that their concerns or dissatisfaction were understood. We recognise too that these positions may each overlap.

This section acknowledges the different elements to the communications work:

• introducing and setting out the background to the Review;
• contacting families and setting out the basis for case selection;
• providing families with the opportunity to highlight questions, issues or observations that they wished to make known to the Panel;
• addressing individual questions and providing appropriate updates on overall progress;
• ensuring that preferences for updates and discussion of the individual outcome for their child were elicited and delivered;
• communicating specific findings and responses to questions to those families/patients that wish to receive these; and
• ensuring that the core narrative supporting this Review was consistently reflected in communications and engagement – particularly reflecting Ministerial commitments to full, open, transparent and respectful engagement with parents and families.

Considerable engagement had already taken place with this group prior to the start of the Review, in particular, by information coming from NHS GGC and the clinical team working closely with the patients/families, supported by the Paediatric Haemato Oncology Closed Facebook page. Engagement has also been supported through the Scottish Government Oversight Board Communications and Engagement subgroup led by Professor Craig White, with the support of Patient and Family Representative, Professor John Cuddihy, and with whom we agreed a process for communication with families.

We were able to harness the established communication and engagement processes, to provide patients and families with quarterly written updates on the progress of the Review, to receive questions and information from families for consideration by the Panel, and to provide responses. Further information on information sent by families to the Panel is discussed in Section 7.1.

3.7.2 Core Project Team

The Core Project Team meetings, chaired by Professor Marion Bain, provided governance oversight for the Case Note Review. These meetings received an update on the progress of our work and provided an opportunity to discuss risks and issues arising from the Review process itself. These meetings also acted as the conduit to provide updates and escalate risks and issues to the Oversight Board.

3.7.3 NHS GGC Clinical and Medical Staff

This area of communications and engagement had been recognised as a particular risk in the Review. This group had been concerned with the appropriateness of (some of) the methods being applied for the Review, and there were particular sensitivities expressed with respect to any focus on the quality of care provided to these patients.

Steps were taken to address these concerns as far as was realistically achievable. This included quarterly virtual update meetings to which senior medical, nursing and management staff from the Paediatric Haematology Oncology service and RHC were invited. On occasions, a senior member of the Core Project Team also attended with a view to providing opportunities to raise concerns and ask questions.

3.7.4 NHS GGC Senior Management

Senior members of the NHS GGC Senior Leadership Team were appraised through the work of the Oversight Board to which Professor Marion Bain provided updates on progress of the Review following Core Project Team meetings while she was Director of Infection Prevention and Control at NHS GGC. In addition, we engaged frequently with the Head of Corporate Governance and Administration at NHS GGC regarding meetings to request and discuss data submissions for the Review.

3.7.5 Other NHS GGC staff

Through and with members of the wider Review team, we and other NHS GGC staff communicated frequently from April 2020 to December 2020 over requests for NHS GGC data, and to clarify data received. This spanned across various divisions in NHS GGC, for example, Estates and Facilities, Microbiology, Infection Prevention Control and Paediatric Haematology and Oncology.

3.7.6 Others

In line with the independent nature of the Case Note Review, we asked for meetings with, or sought written clarification from, a number of individuals who held technical, advisory or clinical positions within Scottish Government, HFS and NHS GGC. The purpose was to discuss background information and to clarify our understanding of specific points identified in our review. These meetings took place between November 2020 and January 2021.