We are testing a new beta website for gov.scot go to new site

Effective Interventions Unit: Integrated Care Pathways Guide 4: Analysis and Review

DescriptionThis guide looks at ways of assessing the impact of an ICP and ensuring that it continues to improve the quality of care provided to the next client with similar presenting issues.
ISBN
Official Print Publication Date
Website Publication DateApril 22, 2004

    Listen

    Effective Interventions Unit

    Integrated Care Pathways
    Guide 4: Analysis and Review

    Logo

    Effective Interventions Unit
    Substance Misuse Division
    Scottish Executive
    St. Andrew's House
    Edinburgh EH1 3DG
    Tel: 0131 244 5117
    Fax: 0131 244 2689
    EIU@scotland.gsi.gov.uk
    http://www.drugmisuse.isdscotland.org/eiu

    This document is also available in pdf format (192k)

    WHAT IS THE PURPOSE OF THIS GUIDE?

    This is the fourth in a series of guides on developing and implementing Integrated Care Pathways (ICPs). This guide looks at ways of assessing the impact of an ICP and ensuring that it continues to improve the quality of care provided to the next client with similar presenting issues.

    WHO SHOULD READ IT?

    This guide is particularly aimed at those involved in developing, delivering and evaluating services for drug users. The Section on Review of effectiveness will also be of interest to those involved in the commissioning and planning of services.

    Analysis and Review are key processes in the evaluation of the effectiveness of an ICP. These processes also contribute to the cycle of continuous improvement and ensure that care and treatment is based on up-to-date evidence of effective practice.

    AN ICP CYCLE OF CONTINUOUS IMPROVEMENT

    diagram

    Source: National Electronic Library for Health (NELH) (2004)

    Earlier guides in the ICP series have set out the importance of having clear objectives for each pathway, identifying key indicators of change (ICP Guide 2) and of maintaining consistent documentation throughout the process (ICP Guide 3). This is all evaluation. For more information about evaluation see: www.drugmisuse.isdscotland.org/goodpractice/EIU_evaluationg3.pdf

    This Guide contains information on:

    • What is Analysis?

    • What is Review?

    • The Purpose of Analysis and Review

    • The Nature of the Exercise

    • Variance Tracking and Recording

    • Variance Analysis

    • Next Step

    WHAT IS ANALYSIS?

    This part of the process begins with an investigation into the different types of information available. It involves breaking down tables of figures and scrutinising recording sheets within individual case records. Finally it is an examination of all the facts and opinions put forward including the client's views, studied within the context of the original objectives of the pathway.

    WHAT IS REVIEW?

    One of the primary functions of a review process is to check whether the ICP does what you originally wanted it to do. Sometimes introducing change would provide better outcomes for the client but the pathway has been developed too rigidly to allow for that change to happen. Alternatively change may have been introduced to the structure of the pathway by one individual or department that has resulted in a reduction in the quality of care expected at the outset. For both these reasons we need to routinely reconsider whether the pathway continues to provide the best way of meeting its objectives.

    The review process also provides an opportunity to consider how new published evidence could be incorporated into the pathway. For example, emerging information on the treatment of psychostimulant use, dual diagnosis and opiate detoxification may influence the way in which existing processes of care are modified to maximise the effectiveness of the interventions.

    The review process should ask two clear questions:

    • To what degree was the pathway followed?

    • Did the implementation of this ICP result in improved outcomes for the clients?

    Supplementary questions would seek to ascertain whether, from the analysis of the information and evidence available, the ICP requires some modification in order to continue to provide the most appropriate and effective care and treatment. The ICP Development Group should consider:

    • Who should carry out the review?

    • How will the results of the review be shared?

    • Who will own the outcome of the review?

    • What resources are available to support any further development of the ICP?

    THE PURPOSE OF ANALYSIS AND REVIEW

    Before starting to gather the information required for the analysis it is important that as a group you discuss and agree on what you want out of the process. In order to do this you should clearly define the extent of the exercise and decide which questions you really want answers to. These may include:

    • Has the implementation of this pathway improved client care?

    • Has it provided the same level of care at less cost than previously?

    • Are there any changes that can be made which will improve its effectiveness?

    It is also important to identify learning opportunities arising during the course of a pathway, particularly those issues around partnership working when there are a number of service providers involved in delivering different aspects of care. Studying these situations may help the steering group and others to identify processes of care co-ordination or shared assessment that could be adopted or adapted to similar situations.

    ICP Guide 2 highlighted the importance of identifying key indicators of change while first developing an ICP. The Canadian Council on Health Services Accreditation defined indicators as, "Measurements, screens or flags used as a guide to monitor, evaluate and improve the quality of care, clinical services, support services, and organisational functions that affect client outcomes." 1

    Depending on the focus of your review there are a number of issues which could be explored. Middleton and Roberts 2 identify a number of possible areas to examine. These include:

    • Changes in documentation, i.e. accuracy/completeness and/or amount of time to complete

    • Changes in care delivery or process of care, e.g. introduction of guidelines, increased consistency, length of stay/contact, shortening time delays, reducing number of visits, changes in who does what

    • Changes in client and staff satisfaction

    • Changes in client outcomes, e.g. reduced drug use, criminality, improved health, social stability

    1. Getting started with integrated care pathways (2001) Health Service Utilisation and Research Commission
    2. Middleton S. & Roberts A. (2002) Integrated Care Pathways: A practical approach to implementation. Butterworth Heinemann (Oxford)

    NATURE OF THE EXERCISE

    Once you have agreed upon the type of information that you want to collect your group will need to spend some time deciding on which is the best method of collecting this. The EIU Evaluation Guide 3 ( www.drugmisuse.isdscotland.org/goodpractice/EIU_evaluationg3.pdf) provides further information on the different types of investigations and the methods of information collection involved.

    WHAT ARE THE TIMESCALES FOR THIS ANALYSIS AND REVIEW?

    There are a number of factors that will influence this:

    • The nature and extent of the exercise

    • The number of clients (and case records) involved

    • Whether there is a dedicated ICP project worker/Facilitator allocated to this project

    • Whether or not there is sufficient time allocated

    • Whether the information has been stored electronically or paper-based

    It might be useful for the group to develop a timetable for the analysis and review processes, setting out the individual tasks to be completed, the sequence in which these need to happen and the target start and finish dates of each task. This will help to show any differences between the 'planned' and 'actual' timescales.

    WHO IS ACCOUNTABLE FOR THE ANALYSIS AND REVIEW?

    There will be differing roles and degrees of involvement in the development, implementation, analysis and review of the ICP. Bearing in mind that the review may have a number of objectives; measuring the effectiveness of care provided, the cost-effectiveness of the interventions and identifying gaps in services, different people or groups (stakeholders) may take the lead on different parts of the analysis and review. It is the role of the ICP facilitator to ensure that all this activity is co-ordinated and implemented within an agreed framework.

    VARIANCE TRACKING AND RECORDING

    ICP guide 3 provided tips for improving documentation. It suggested that all ICPs should have a standard format which includes a recording sheet for tracking variances from the expected pathway. Variances are documented as they occur, along with information about what alternative action was taken.

    Categorising variances aids the analysis and review by grouping similar reasons for or types of variances. These could be coded 1-6 as above. Analysis of these categories highlights any patterns of variance that might necessitate corrective action.

    The tracking sheet should document:

    DATE

    DESCRIPTION OF VARIANCE

    REASON FOR VARIANCE

    ALTERNATIVE ACTION TAKEN

    CATEGORY OF VARIANCE

    SIGNATURE OF STAFF MEMBER

    VARIANCE ANALYSIS

    Integrated Care Pathways outline the optimal course of care for all clients who are undergoing a specific course of treatment based on specific presenting issues. There is a recognition and acceptance that there will be occasions where this is not the case and a different course of action needs to be followed. The use of ICPs is not intended to compromise anyone's professional judgement and any member of the team can decide to deviate the client's care from the pathway. However, there must be a valid reason for doing so, known as 'variations' which need to be recorded at the time. Reasons may include:

    1. The client's use of substances has changed significantly, e.g. increase in risk behaviour or managing to make positive changes ahead of plan

    2. The client sustains injury or illness which affects their ability to complete a section on a particular ICP

    3. The client's motivational state has changed, resulting in a review of their goals of care

    4. The client's social or legal situation has changed. Factors such as relationships, housing issues and economic status may all influence a client's ability to continue on a particular ICP

    5. The service provider is unable to meet their obligations within the ICP. This may be due to sickness, adverse weather conditions or changes in policy or practice

    6. New research or evidence indicates that a different approach would prove more effective than that mapped out in the pathway

    Variations are a possible outcome of individualised client care. A variance is the difference between what is expected to happen and what actually happens. Variances are deviations from an activity set out in an ICP. They feature in all ICPs and are recorded during the course of care. They allow you to review and update the process of care delivery, with attention to service processes and outputs and client outcomes.

    Often when variances occur as a result of a client's changing need these can be anticipated and 'exit points' built into the pathway. This would allow a person to leave a pathway at a particular point and enter another ICP at a point more appropriate to their changed need, thus providing seamless care.

    NEXT STEPS

    This guide concludes the core series of technical guides on developing and implementing ICPs. The EIU welcome comments on their work outputs.

    In order to further support services in the use of integrated care pathways in substance misuse services we plan to produce a number of guides that focus on particular topics. These include:

    • Community detoxification (May 2004)

    • Care of people with drug problems in acute psychiatric settings (July 2004)

    • Drug misuse in pregnancy (September 2004)

    • Care of people with drug problems in acute medical and surgical settings (November 2004)

    OTHER RESOURCES

    Previous guides in this series can be viewed on the EIU website: www.drugmisuse.isdscotland.org/eiu

    Thanks to the ICP Development Group and others for contributing to this published guide.