Gender identity healthcare: evaluation of the impact of Scottish Government funding

3. Theory of Change Framework

We have developed a Theory of Change (ToC) to guide this evaluation project. This is a standard model for evaluating health interventions in complex systems. A ToC provides a framework for thinking about how an intervention works and is expected to lead to intended outcomes. ^[31] The ToC involves identifying assumptions behind the intervention, the steps involved as part of the intervention, and short and long-term impacts.

Our ToC is modelled through a set of steps, to understand how an intervention is carried out. The steps begin with the initial situation, then move to ‘inputs’ or resources, through to the activities involved, and finally outcomes and impacts. We understand that definitive claims of attribution are difficult to make in the complex systems. Thus, our focus is more on understanding an intervention, rather than seeking to prove correlation. We have structured our ToC in the following way:

Situation • Inputs • Activities • Preconditions • Outcomes • Impacts

Situation

The first step in the ToC is to assess the current situation within which gender identity services are being offered. Here, we consider three factors:

• The importance of the wider, structural context, whereby evidence has shown that GICs are becoming overextended to accommodate growing numbers of people seeking care, there are significant waiting lists, and a lack of specialists in this healthcare area. These barriers are contributing to mental health problems for people seeking gender identity care, who are often turning to private medical care, going abroad for treatment or self-medicating with hormones purchased online.
• Consideration of local context and need, whereby the delivery of gender identity services in Scotland is not a one-size-fits-all approach. Each GIC has developed its own approach to service delivery, based on the development of individual services, organisational structure, existing relationships, partners and resources. GICs are operating in different contexts with varying levels of demand. While some service users may live in an area with shorter waiting times and/or a smaller clinic, others may live in areas with longer waiting times.
• Fostering collaboration and relationships, which are important for holistic and wrap-around care. One of the commitments in the SG’s Strategic Action Framework for NHS gender identity services was to fund enhanced voluntary led support for people on waiting lists. Individual NHS Health Board proposals have generally included an emphasis on greater collaboration with the third sector. The aims are to improve the experience of people accessing gender identity healthcare, to alleviate pressures on staff, and to share learning.

Inputs

The second step in the ToC is to identify the inputs, that is, the resources that go into a service. In May 2022, the National Gender Identity Healthcare Reference Group agreed principles for the allocation of additional funding to support GICs to improve access to, and delivery of, gender identity healthcare.^[32] The SG Programme for Government 2021-22 included £9m to improve gender identity healthcare over the period 2022-24 (later extended to 2022-27). NHS Health Boards were invited to submit proposals for additional SG funding for GICs to address waiting lists and to better support people waiting to access services. The aims of the funding, as outlined by the SG, were:

• deliver care at all times based on the principles of Realistic Medicine
• fully consider clinical and administrative resource required to both reduce waiting times and increase support for those waiting to access services
• have robust clinical and operational governance structures, systems and frameworks, in line with the relevant Health Board requirements
• aim to be integrated across NHS Scotland and with other relevant organisations e.g. third sector support services
• consider the timing and location of appointments for people who may have to travel long distances for care, if not accessing virtual appointments
• adopt a human rights based, person centred and multidisciplinary approach, promoting respect, dignity and equality for all individuals accessing services

All four territorial NHS Boards with GICs won funding from the SG (NHS Grampian, NHSGGC, NHS Highland and NHS Lothian). The amount allocated directly to Health Boards with GICs was £915,000 in 2022-23.^[33] Each Board had latitude over the way it responded to the aims set out by the SG, the amount of funding sought, and how it would be used. In general, however, the main inputs enabled by the funding included hiring new staff to deliver services, extending hours of existing staff, and staff training.

Activities

The next step in the ToC is to capture the activities undertaken as part of the intervention to improve gender identity healthcare services. SG funding/inputs were intended to enable Health Boards to undertake a range of activities to improve gender identity services. These activities are detailed in the case studies. However, some cross-cutting themes from the funding proposals and plans may be identified:

• improve communication between the GIC and patients on waiting lists
• increase coordination and collaboration of all agencies across the pathway
• consider increased access to psychosocial and peer support
• strengthen collaborations across the Board, statutory services and third sector
• proactively seek to address health inequalities, including those resulting from intersectional identities that combine personal characteristics (i.e. race, age, class, ethnicity, religion, disability), with consideration given to people’s identity, background and circumstances and any barriers they may be experiencing
• leverage the funding to build out support with other NHS teams and partners
• implement innovative solutions to support people on waiting lists
• develop networks of support and knowledge exchange with other clinics

Preconditions

There are a number of preconditions which need to be in place before the activities funded by the SG and carried out by the GICs can lead to system-change outcomes and longer-term impacts. Based on our analysis, these preconditions include:

• data and evidence to support decision-making, delivery and resource allocation
• general training for NHS staff to reduce stigma and improve care for patients
• specialised training on gender dysphoria for clinical staff
• safe and fulfilling workplaces to support staff wellbeing and facilitate recruitment
• national leadership, clearly communicated, to enable system change
• local and regional leadership, to agree shared aims across delivery partners
• empowered and resourced staff who feel confident about delivering services

Outcomes

The penultimate step in the ToC framework is to consider system change outcomes as a result of the SG funding. The Strategic Action Framework includes a series of commitments^[34] that provide the foundation for outcomes of the ToC:

• improving waiting times to access services, and bringing gender identity services within national waiting times standards
• increasing support available for people waiting to access services, for instance through partnerships with the third sector
• supporting new multidisciplinary models of delivering care
• improving data collection on waiting times and patient outcomes
• developing national standards for gender identity services
• supporting staff through training and development
• providing a sustainable platform for gender identity service improvement

In addition to the commitments/outcomes identified in the Strategic Action Framework, individual NHS Health Boards have elaborated key outcomes they have sought to achieve from the SG funding, or have indicated positive unintended outcomes:

• increased visibility of GICs within Boards and focus on strategic improvement
• formalisation of governance structures around GICs
• buy-in from other NHS teams such as sexual health, psychology and GPs
• sharing of GIC data so NHS services can provide joined-up care

Impacts

The final step in the ToC is to identify long-term impacts for patients, staff and other partners working within the system. The Strategic Action Framework referred to the Realistic Medicine concept and its importance to delivering gender identity healthcare. This concept, endorsed by NHS Scotland^[35], “puts the person receiving health and social care at the centre of decisions made about their care.” The Realistic Medicine vision underpins the wider, long-term impacts identified in the ToC:

• Empowered Patients: patients feel fully empowered to discuss their needs, concerns and treatment with healthcare professionals
• Personalised Care: a person-centred approach, not a one-size-fits-all model;
• Informed Decisions: ensuring patients have all the information they need to make an informed choice about their care
• Joined-up and Collaborative Care: collaborative work between health professionals to provide joined-up care to meet patients’ needs, and shared decision making between professionals and patients about their care journey

In addition, based on our analysis, we have included the following additional impacts:

• Accessible and inclusive services: all patients, regardless of their personal/ protected characteristics or geographical location, have good access to care
• Pre-service support: as a longer-term impact, ensuring all patients are offered support from third sector or NHS services during referral wait times
• Educated workforce: healthcare professionals across NHS services are knowledgeable and trained to provide care and support to patients
• Well-resourced services: health professionals have the resources and support they need to deliver the highest-quality care
• Good governance: governance structures are strong and robust, with clear lines of communication, accountability and responsibility around service provision
• Strong networks: strengthening relationships across gender-identity clinics and services to facilitate knowledge exchange, learning and support
• Mainstreamed care: while some patient needs may require specialised care, other patient needs can be effectively treated by mainstream NHS services
• Reduced health inequalities: reducing stigma and discrimination experienced by patients, and reducing barriers to accessing healthcare, through clear treatment pathways and support, improved data and inclusive services
• Embracing intersectionality: developing an awareness of intersectional identities and how issues of race, class, and disability may exacerbate further inequalities that service users experience, and how to mitigate these

We have developed a ToC model based on these steps, summarised below. This ToC has guided the evaluation work. Towards the end of the project, we developed individual ToC models for each of the four NHS Health Boards, which provide variations on the general framework. The individual ToC are included in the case study chapters, and provide bespoke models tailored to each service. They discuss the outcomes that have and have not been achieved, and actions (linked to our best practice learnings) that may enable outcomes to be achieved.

Situation:

What is the situation, which the intervention seeks to change?

• Wider structural context (high demand for services, long waiting lists, overstretched staff, lack of specialists & training)
• Consideration of local context / need (insufficient local resource, geographical variation in access, variety of approaches)
• Fostering collaboration (importance of relationships with third sector, cross-clinic learning, partnerships with GPs / within Boards)

Inputs:

What resources have been deployed?

• Availability of SG funding to improve access to, and delivery of, gender identity services.
• NHS Health Boards are using the funding to increase their resources to support activities, by:
  • Hiring staff, especially with specialisation in gender dysphoria
  • Increasing the hours of existing staff to see more patients / build relationships
  • Staff training and development

Activities:

What activities are being undertaken to improve services?

• NHS Boards implementing plans to improve services, in line with SG funding aims:
  • Improve communications between GICs and patients
  • Increase collaboration across patient pathways
  • Build partnerships with 3rd sector to support patients
  • Address intersectional identities and health inequalities
  • Develop innovative solutions for pre-service support and for those on wait lists

Preconditions:

What needs to be in place within activities to achieve outcomes?

• Robust data & evidence
• Specialised training on gender identity for GIC staff and general training for mainstream NHS staff, to enable joined-up care
• Safe and fulfilling workplaces
• National leadership
• Local/regional leadership
• Empowered and resourced staff, able to confidently make decisions

Outcomes:

What are the system-change outcomes of the intervention?

• Changes to meet and respond to the needs of patients (improving waiting times, providing wrap-around support, and innovative models of gender identity care that is tailored to local systems)
• Changes to the understanding and knowledge of systems (improving data collection, providing staff training and development, and clear standards)
• Changes to the governance of systems (creating formal structures and clear decision-making processes, joined-up services across NHS, strong relationships across systems, and data sharing)

Impacts:

What are the long-term impacts on patients, staff and partners?

• Realistic Medicine vision:
  • Empowered patients, Personalised Care, Informed Decisions, Joined Up and Collaborative care
• Specific to GICs:
  • Accessible and inclusive services (including pre-service support), Educated workforce, Well-funded services, Good governance, Strong networks, Mainstreamed care, Reduced health inequalities