Scottish Inpatient Experience Survey 2016 Volume 2: Technical Report

Analysis and Reporting

Analysis

The survey data collected and coded by Quality Health Ltd were securely transferred to ISD Scotland via secure FTP and analysed using the statistical software package SPSS.

The analysis produced by ISD Scotland was transferred to the Scottish Government for inclusion in the national report.

Weighting the data

Estimates for Scotland and NHS Boards are weighted as the sampling is based on a stratified approach. The weight is calculated as the number of eligible inpatients (aged 16+ and therefore eligible for survey) as a proportion of the total number of eligible inpatients (Scotland or NHS Board).

Weighting the results in this way provides more representative results because the contribution of each hospital, to the national or NHS Board average, is proportional to the number of eligible patients treated there.

Percentage positive and negative

Percentage positive is frequently used in the reporting meaning the percentage of people who answered in a positive way. If people said they strongly agreed or agreed, these answers have been counted as positive responses. If people said they disagreed or strongly disagreed, these have been counted as negative. Appendix A details which responses have been classed as positive and negative for each question.

Inclusions and Exclusions

A description of people excluded from sampling is provided in section sampling frame.

Hospitals without an A&E department or a minor injury unit will not contain result for the A&E section of the hospital reports.

Reports for NHS Boards and hospitals are only produced if there are 50 or more responses. If a particular question had less than 20 responses, the results for that question were suppressed within the report.

Scotland Performs Healthcare Experience Indicator

The Healthcare Experience Indicator has been developed to measure the reported experience of people using the NHS. It is one of the 50 National Indicators in the National Performance Framework, which sets out the Government's outcomes based approach. Progress is reported in Scotland Performs: http://www.gov.scot/About/Performance/scotPerforms

The indicator is based on the reported experience from hospital inpatients, as a proxy for experience across the NHS. This has been chosen because:

a) the quality of hospital care is very important to people

b) the indicator involves the transitions to and from hospital, which depend on health and care services in the community

c) it includes the feedback of inpatients on experience in A&E which should reflect a much wider population of users and is an indicator of the system

The indicator is calculated by taking the mean scores for individual peoples' answers on the following questions in the inpatient survey and weighting them using total inpatient numbers to get a national score:

• Overall, how would you rate your admission to hospital (i.e. the period after you arrived at hospital until you got to a bed on the ward)?
• Overall, how would you rate the care and treatment you received during your time in the A&E?
• Overall, how would you rate the hospital and ward environment?
• Overall, how would you rate your care and treatment during your stay in hospital?
• Overall, how would you rate all the staff you came into contact with?
• Overall, how would you rate the arrangements made for your leaving hospital?

The score for each question for each patient is: 0 for very poor; 25 for poor; 50 for fair; 75 for good; 100 for excellent.

The mean of a person's scores for the six questions is used rather than the sum because not all people will have answered every question. The methodology will result in an indicator between 0 and 100 which is reported to one decimal place (Table 7).

The analysis was done using the SAS procedure proc surveymeans which calculates sampling errors of estimators based on complex sample designs.

Table 7: Example of how an individual person's answers are converted into a score for the Healthcare Experience Indicator

Question	Very poor (0)	Poor (25)	Fair (50)	Goo (75)	Excellent (100)	Score
Q9	Overall, how would you rate the care and treatment you received during your time in A&E?					-
Q13	Overall, how would you rate your admission to hospital (i.e. the period after you arrived at hospital until you got to a bed on the ward)?			√		75
Q20	Overall, how would you rate the hospital and ward environment?			√		75
Q34	Overall, how would you rate your care and treatment during your stay in hospital?				√	100
Q49	Overall, how would you rate all the staff you came into contact with?				√	100
Q60	Overall, how would you rate the arrangements made for your leaving hospital?				√	100
	Patient Score =	(75+75+100+100+100)/5 =				90

Quality Outcome Indicator

Twelve national Quality Outcome Indicators show progress towards the ambitions of the Quality Strategy. One of these indicators is the Healthcare Experience which combines the Scotland Performs Healthcare Experience Indicator, described above, with data from the Health and Care Experience Survey.

The indicator is calculated by taking the mean of the Scotland Performs Healthcare Experience Indicator and an indicator using the data from a survey of people registered with a GP practice. The latest value of the Healthcare Experience Quality Outcome Indicator is based on the 2016 Inpatient Experience survey and the 2015/16 Health and Care Experience survey ^[4] .

The GP practice component of the indicator is calculated by taking the mean scores for individual peoples' answers on the following questions in the Health and Care Experience survey and weighting them using GP practice populations to get a national score:

• Overall, how would you rate the arrangements for getting to see a doctor/and or nurse in your GP practice? As there are separate questions about doctors and nurses, the mean score of the answers are used
• Overall, how would you rate the care provided by your GP practice?

As for the Healthcare Experience Indicator, the score for each question for each patient is: 0 for very poor; 25 for poor; 50 for fair; 75 for good; 100 for excellent.

The analysis was done using the SAS procedure proc surveymeans which calculates sampling errors of estimators based on complex sample designs. The standard error of the indicator is calculated by combining the standard errors of the inpatient and GP components.

National report

The national level report was released on the 30 ^th August 2016. It presents national weighted results for each survey question and compares the results with the previous survey, showing statistically significant changes. It also provides results for the 14 NHS Boards and Golden Jubilee Foundation ( GJF), and reports on statistically significant differences from the previous survey at the 5% level.

NHS Board and hospital reporting

These reports provide information on the number of people who were sent a survey and the number who responded. They also provide, information on age, gender and whether respondents' day-to-day activities were limited by a health problem or disability. The local reports for individual hospitals and NHS Boards are available at: www.careexperience.scot.nhs.uk/Results2016.html

The format is similar to the 2012 and 2014 reports, with results shown as the positive percentage. This means the percentage of people who answered in a positive way. For example, when asked how much they agreed the main ward or room they stayed in was clean, if patients strongly agreed or agreed these have been counted as positive answers (see Appendix A).

Bar charts show the percentage positive as green, and the percentage negative as red. The results are compared with those from last year for the same NHS Board or hospital and are also compared with this year's Scottish average. Differences from Scotland or from the previous survey that are statistically significant are shown with a 's' next to the percentage change, e.g. +4s or -5s. Where a comparison has not been performed due to small numbers of responses in any category being compared this is denoted by " NT". An example is shown in Chart 1.

Chart 1: Example of NHS Board/ hospital bar charts

The variation between NHS Boards and Scotland are shown in the example shown in Chart 2. The green bar represents the NHS Board percentage positive result, the blue line represents the Scottish average with the grey bar displaying the range of percentage positive results from all NHS Boards.

Chart 2: Example of NHS Board variation chart

Details of the data sampling, quality assurance, data sources and confidentiality are can be found at the following link.