Using Discrete Event Simulation to explore "what-if" waiting list scenarios in NHS Scotland

Methods

Process map

A process map provides a high-level visual overview of a process. For the case study the aim was to garner agreement on the key steps in the patient journey from initial contact with the health service, through to successful cataract operation. Information on the patient journey was gathered from the literature and from a range of stakeholders. This was then collated into a draft process map using Visio. A workshop involving key stakeholders was organised to gather feedback on the draft and collective agreement on the final product.

The process map in Figure 1 describes the possible patient journeys in ophthalmology and was used to build the structure in the Simul8 model. The numbers quoted are end of June 2024 figures, as reported in November 2024 (11).

Figure 1 : The agreed process map for the ophthalmology patient journey.

Numbers are from the waiting list statistics for April - June 2024, rounded to the nearest 1,000.

The process map provides a visual representation of the following journey. Patients generally arrive via Primary Care Services, which for ophthalmology is most often community optometrists. In April to June 2024 43,000 individuals were added to the New Outpatient List to wait for a consultation, the majority of which are assumed to be waiting to be considered for cataract surgery. During this time, around 33,000 individuals were seen at a consultation appointment, and another 10,000 were removed from the NOP list for other reasons. These removal reasons can include, for example, “treatment no longer required” and “referred back to GP”. Patients can be referred back to primary care for a number of reasons, including if they have failed to attend an appointment or because the referral to hospital was inappropriate. Treatment no longer required is often due to patients going to the independent sector for their surgery, or due to patient death.

Discrete event simulation

Discrete event simulation is a simulation method designed for looking at processes and optimising efficiency by testing changes to the processes in the model environment. The operation of a system is represented as a chronological sequence of discrete events of a predetermined timestep, such as a day. As each portion of time passes different activities take place and resources may move to different parts of the system. The benefits of this approach are that using simulation methods allows us to move away from deterministic estimates using averages, and better replicate reality by using realistic variation in demand and activity. This is achieved by having distributions for simulation variables rather than a fixed average value. For each run of the simulation the modelling software samples a random value for each variable from the respective distributions. The simulation is then run multiple times with new values sampled each time, and the outputs aggregated together. In this way individual simulation runs may yield completely different outputs, but running the simulation repeatedly provides a sense of the most likely outcome.

With simulation we can build a visual representation of the current patient journey based on available data and use this to test the impact of changes due to new policies and various “what-if” scenarios . This simulation method has previously been applied to healthcare processes (12 , 13, 14).

For the cataract case study, the process map was used to structure the DES which was subsequently built in the Simul8 software package. Published PHS data for quarterly additions, removals, attendances were used to fit the distributions for all the variables required in the simulation (2). The simulation was set up so that patients travelled through the system as work items, arriving at activities e.g. a consultation appointment, with a frequency based on historical distributions of arrivals per day. In this way it was not necessary to make assumptions on length of appointment, or staff availability.

The simulation model for the case study was run 30 times for each scenario tested. To quantify the impacts of the different “what-if” scenarios the simulated TTG list size was used as an indicator. The list size was written to a spreadsheet every two simulation weeks to capture the dynamics over the course of the simulation, for each run. Using the TTG list size as the core output also enabled the simulation to be verified and validated against published statistics for the TTG list.

Staff were not explicitly included in the Simul8 model as a constrained resource, partly due to the lack of suitable data. This meant that the impact of return outpatients were not considered in this case study. Theoretically this is something that Simul8 and DES modelling could forecast if the right data were available.

When simulating what-if scenarios where TTG activity is modified, this change is applied only to single lists (surgical lists which only have cataract cases) on the assumption that these are the surgical lists where productivity changes would be most feasible to implement.

Model verification and validation

Waiting list and activity data from Q2 2024 (April – June 2024) were used to set the parameter distributions in the model. These data were also used in a linear regression model to estimate the conversion rate. More detail about the distributions and the linear regression model can be found in the Annex.

Verification of the DES model involved checking that the simulated TTG list size for Q2 2024 aligned with the actual TTG list size. Figure 2 shows that over Q2 2024 the actuals land within the error bars for the model projection. The error bars represent the standard deviation of the projected TTG list size over 30 model runs.

The reported TTG list size over July – October 2024 was then used to sense check the short term projections of the DES model i.e. validate that the model can replicate real world dynamics . Figure 2 shows that the actuals continue to land within the error bars for the model projection.

Due to the uncertainty in the DES input distributions the standard deviation in

Figure 2 : Simul8 model output validated against TTG list size.

Simul8 model output verified against TTG list size for April to June 2024, and validating that the model continues to provide reasonable projections over July to September 2024. Error bars represent the standard deviation across 30 model runs. The Simul8 conversion rate was set using data from March 2022 onwards.

model runs increases over time as diverging dynamics are modelled. This is why the error bars grow over time. This allows the DES model to make reasonable projections that capture the July – October 2024 TTG list size, even though the Q3 2024 TTG waiting list growth was much higher than in Q2 2024.

During this process of verification and validation we found that the model is very sensitive to the initial parameter values sampled by Simul8. For example, there are some parameters combinations which result in an immediate downward trend for the TTG list size. This could be an avenue for further research into the details of the waiting list dynamics.

The modelled TTG list size was also sensitive to the conversion rate, as the main tuning factor in the model that adds patients to the TTG list. While being a modelling convention rather than an accurate measure of the flow of patients from NOP to TTG, it is still important to note that for ophthalmology the calculated conversion rate has not been constant in the last couple of years . This increases the variability in the distribution used in the model for the conversion rate.