An Investigative Review into the process of establishing, managing and supporting Independent Reviews in Scotland

Chapter 8: The composition and production of a review report

237. Communicating the findings of a review is the culmination of the work undertaken in the investigation. It should reflect the remit and terms of reference. It should be clear and concise.^[103] If the subject matter is technical and jargon has to be used, there should be an accompanying explanation and/or a glossary of terms.

238. Members of a review should be given the opportunity to discuss the structure and content of any publication. This ensures that there is understanding of the direction of the work required to produce the report. It allows the members to be able to make a focused contribution to its content.

239. An awareness of who is going to be reading the report is also essential. Whilst it may have been commissioned by an individual government Minister, the interest in the outcome of a review is likely to be much wider and therefore so will the audience wishing to read it.

240. If the subject matter has generated a large amount of interest, then some consideration should be given to circulating the report in advance to key parties giving them the opportunity to prepare for any subsequent media interest.

241. There are mixed views on who has the responsibility for the publication of the report. Suggestions include that it is a matter for the chair and the commissioning agencies to assist with printing and distribution.^[104] Others believe that it is the decision of whoever commissioned the review.^[105] We agree that it may depend on the nature of the review and that either option can apply as long as it is made clear where the responsibility rests.

242. Consideration should also be given as to whether there is merit in producing an interim report. Depending on the subject matter, this may provide an opportunity to make any provisional recommendations, which will allow them to be addressed more quickly than waiting for a future publication.^[106] It also allows for an early indication of possible directions and conclusions of a final report to be made available to interested parties beyond just the membership of the review. It may prompt the submissions of additional evidence which may justify inclusion in the final report.^[107]

243. Irrespective of who is tasked with the drafting of the report, or individual sections of the report, one person should assume the responsibility of having final editorial control. This will ensure a consistency of writing style making any publication more fluid and readable.

The Mesh Review

244. There was a lack of consensus as to who were the target audience for the Reports. Some members saw the Reports being written for the Office of the Cabinet Secretary; others saw the Reports as a more technical guide for clinicians. Notwithstanding this uncertainty, the minutes from the meeting of May 2015 note that:

“The narrative in the Report referring to all tables must be concise summarising what we know, what we don’t know and use language appropriate for the intended audience.”^[108]

245. A lack of clarity as to how the Reports would be written and by whom was mentioned. One member said that they initially assumed a report would be written by a “senior civil servant”. The first chair also assumed that the report would be written by someone else but she quickly realised that ‘a writer wasn’t available to us’. It appears that a few members volunteered to write their respective parts of the reports; whereas others were specifically asked. The writing of the reports evolved in a seemingly arbitrary as the Review progressed, rather than following a pre-determined process.

246. There did appear to be a broad understanding that the petitioners would be involved in the creation of Chapter 3, often referred to as the ‘Patients Chapter’. One participant noted that, “we knew the [petitioners] would have input but did not know what their input would look like.” This was confirmed by another noting that:

“the patients also were promised to have their own chapter so they can tell their own story but that was not discussed at the beginning. That just happened.”

247. As the writing process progressed, one participant mentioned their concerns over a lack of “proper version control” of drafts during the drafting of the Final Report. This made it extremely difficult to identify and following changes across several drafts.

248. Access to different sections of the report appear to have been segregated, usually with only those directly involved in the chapter getting sight of drafts prior to the last meeting before the publication of the Final Report. Only exceptionally did the draft reports from the subgroups appeared to have been shared with the wider group membership.

249. This lack of consistency caused some frustration. One member described not appreciating having their work publicly critiqued when they could not, in turn; influence the work of other subgroups. Another summed it up saying, “Oh it was awful and there were bits chopped out and put into different places.”

Interim and Final Reports

Interim Report

250. The minutes are completely silent about how the decision to publish an Interim Review was taken and by whom. “Well, I never knew that there was going to be an Interim and Final Report.”

251. Members shared a variety of theories with us on why they believed an Interim Report was published. There was conjecture from some members that an Interim Report was issued to provide clarity on how to address the growing number of patients who were still waiting for treatment, with surgeons being unsure how to proceed.

252. The Report was also regarded by some as interim in nature due to the fact that the Group was awaiting the conclusions and publication of what were regarded as two key studies. The first was the final opinion of the European Commission’s Scientific Committee on Emerging and Newly Identified Health Risks (SCENIHR)^[109] on the use of mesh implants which was published on 3 December 2015. The second was the Prolapse Surgery: Pragmatic Evaluation and randomised Controlled Trials, known as the PROSPECT^[110] study. This was published In December 2016. Its lead author, Professor Catherine Glazener was a member of the Review, so, for some, at least, its findings were already known:

“You see we knew what was in the PROSPECT study and it needed to be published in an appropriate journal. It was submitted to the New England Journal of Medicine but not accepted. It was then submitted to the Lancet but that process takes so long.”

253. Finally, a few members alluded to the Interim Report coming about due to political pressure. One participant stated that the impetus to publish the Interim report was due to “the Minister” shortly “appearing before parliament.”^[111]

254. It was clear that the publication of an interim report was not considered at the outset of the review and that the reasons for its publication in 2015 were either not properly discussed with the Review Group or not fully understood by the Group.

255. Having said that, the Interim Report appeared to have been cautiously welcomed when it was published, it provided an opportunity to consider the conclusions that had been reached to date. We are of the view that it could also have usefully provided an opportunity for greater public engagement which might have been used to inform the conclusions of the Final Report although this does not seem to have been utilised. The petitioners produced a minority report urging that the recommendations made in the Interim Report should be actioned immediately.

256. Early actions to implement recommendations within an interim report can be valuable when these are providing an indication of the probable direction of the Final Report. There should be an awareness that such conclusions have been reached without the benefit of additional information or studies.

“Interim Reports are an under-utilised approach that can help inquiries deliver more rapidly on the key aim of preventing re-occurrence.”^[112]

Final Report

257. The Interim Report could have provided a base from which to progress to a final publication. Some members agreed seeing it as an “updating exercise” but others expected wider ranging changes.

258. During our conversations, there were concerns expressed that the Final Report was not very readable in general and that the boxes, which had been added to the start of each chapter, made it even more fragmented. One member questioned this approach but was told that “this was how updates were done” and that the boxes would help to draw attention to any updated materials appearing in the Final Report.

259. One member claimed that there was an increased governmental involvement in the late stages of the publication of the Final Report. They alluded to a lot of issues “behind the scenes”, and it “all getting a bit strange” and that, consequently, the process overall was becoming disrupted.

“I didn’t like how the second [Final] report came out. That was the way the Scottish Government did their reports so we were assured that was the best way and I went with that. It was not of a normal or easy reading way and I would certainly never have written it in that way.”

260. Changes also started to be made to phrasing and terminology which had been used and accepted in the Interim Report. Such changes did not seem to be supported by any rationale nor was there any discussion with the members of the Mesh Review Group. One controversial example was contained within a paragraph in the Interim Report discussing the concerns that some women had expressed to the clinical teams about their treatment, and included the phrase, “found that they were not believed.” The Final Report by contrast, subtly but significantly alters the language of this phrase to “feeling they were not believed.” The implication in the Interim Report was - that the women patients expressed concerns which were not accepted as medically well- founded. The petitioners’ view was that the Final Report’s shift of emphasis portrayed their concerns as something that they alone imagined.

261. A Scottish Government official suggested that the change was due to stylistic differences in proof reading staff, rather than an active decision. No one was able to recall a discussion over the change for that particular wording.

262. If this change was made simply due to the stylistic preferences of a different proof reader then it serves only to demonstrate the importance of having one person with editorial control. Given the obvious sensitivities of the subject matter, and the experiences of the petitioners in particular, is unfortunate that this particular phrase was changed without any active consideration behind it. There was no discussion with the petitioners to make them aware of the proposed change or to have an opportunity to comment on it. Whatever the reason, it is clear that this change caused hurt and dismay to the petitioners.

263. More generally concerns over the use of terminologies were raised. The use of Plain English was discussed at the second last meeting to be held prior to the publication of the Final Report, suggesting that plainer English be used where possible.^[113]

“Some concerns were raised that there were differences in terminology used depending on the report that was being referred to, and that this made it more difficult to compare the findings from each report.”^[114]

264. This resulted in an action for one of the members to “standardise terminology as much as is practicable, and look at the possibility of using plain English.”^[115]

265. Unfortunately, this was not followed through. When we enquired why not, we were told “it was due to the fact that ‘[we] ran out of time.”

266. As mentioned earlier, a few members commented on having issues with the approach taken in Chapter 3 which was seen as the “patients’’ chapter”. In particular, some had issue with the chapter being written “for the petitioners” rather than for all patient representatives.

267. The majority of those we spoke to were surprised to know that the patient representative, Isobel Montgomery who had experienced a good outcome, was not included in the evaluation or narrative in Chapter 3.

“I’m extremely surprised that the patient’s commentary or reportage would not have been included.”

268. Mrs Montgomery said that she had to argue for her testimony to be included in the Final Report, where it appeared as an appendix. She wondered why she was not asked to contribute to Chapter 3. She did enquire. She prepared a “long statement” and submitted it to Phil Mackie. He reviewed it and said that he would “extract from it what he thought was useful.” She believed it all to be useful and informed Mr. Mackie of this.

269. In a Review of this type, a chapter containing a qualitative account of individual patient experiences was relevant and could be valuable. However, it seems to be anomalous that there was a patient representative on the review with a good outcome and yet her testimony was not included in the narrative of chapter 3. We recognise that there were narratives of others who had a good outcome, but these came from individuals outwith the Mesh Review Group. Chapter 3 was instead a representation of experiences from women who were associated with the Scottish Mesh Survivors Group. It provided a powerful narrative, “voices needed to be represented” but it should have either been more balanced and included representation from other sources or titled more accurately.

270. Other members alluded to the tensions over the content of Chapter 6; referred to as “the clinicians’ chapter.” Concerns were raised between its presentation and form in the Interim Report and how it appeared in the Final Report.

"I wonder if the [lack of agreement] was a symptom of frustration. I don’t know. I think the trust was gone. I think that once people thought that stuff had been removed and it bore no resemblance to the first one that they had agreed with, it became destabilising and it was just a marriage that had just divorced.”

271. The petitioners made submissions to the Public Petitions Committee of the Scottish Parliament and to our Review suggesting that evidence had been “lost” or “destroyed” in the Final Report.^[116] Their concerns focused on a lack of publication of what they described as “patient friendly” tables, some of which contained “alternative evidence which did not favour mesh.” Only one of the tables was included in the body of the report, with the others “hidden in an Appendix or website amongst obscure data.”^[117]

272. We found no evidence to support the claim that evidence was deliberately concealed.

273. Whilst no-one we spoke to accepted that any form of evidence was “lost or destroyed”, several members did accept that moving certain information to the appendix in the Final Report might cause people to believe that it “decreases its importance” and made it harder to find.

274. We note that the rationale for including some evidence in Appendices was not fully explained in the Final Report. The second chair intimated to the Public Petitions Committee that certain evidence was presented this way because of a lack of consensus by the clinician members of the review.^[118]

275. We are not persuaded by this explanation. In our view this should not have precluded the inclusion of relevant information in the body of the Final Report. Instead any lack of agreement should have been reflected in the body of the Report with an explanation of why this was the case.

276. An appendix should be utilised for supplementary materials that would enhance understanding of what is contained in the main body of a report.^[119] Putting information in an appendix or website may infer that it is of lesser importance than including it in the main body of the Report.^[120]

277. The petitioners’ resignations followed a lack of resolution in relation to the matters described above. They have described the Final report as “diluted.” We asked members if they were surprised that the petitioners had resigned.

“I don’t think anything could have been done to have prevented that.” Others took a different view:

“A lot of work would have had to be done to get that and there were other pressures then to get it finished. I think that [the petitioners] could have been kept on board but it would have needed somebody else in the Chair and it would have needed different people to be talking to them. Talking in a different way”

“At this point there was no sense of let’s stop and get them [the petitioners] on board because this is just a document problem, it’s a substantive problem of content.”

278. Following the resignation of the petitioners, they requested that their contribution to the Final Report be removed. At a meeting on the 16 March 2017, attended by the petitioners and the then Cabinet Secretary for Health and Sport, Shona Robison, the petitioners asked that “all” of their input from the Final Report be removed.^[121] This was to include Chapter 3 and the petitioners’ Minority Report. The petitioners were advised that this would be conveyed to the second chair.

279. A series of communications then occurred between the Cabinet Secretary, the second chair and the petitioners.

280. On the 22^nd March, the Chair wrote to the petitioners by email saying that she understood from the Cabinet Secretary that they wished all their contributions to be removed. She listed what these were and asked them to confirm. They were asked to respond by 10.00 am on Thursday 24 March. Thursday was the 23 March, not the 24. The petitioners indicated that this email caused them distress and that they felt pressured by the deadline given to respond. They confirmed their request, the following day 23 March, to the chair and the Cabinet Secretary.

281. On the day of the publication of the Final Report, the petitioners received a communication from the Cabinet Secretary intimating that it was too late to meet their request. The Final Report was published on March 27 2017. The petitioners’ Minority Report had been removed but their input to Chapter 3 was still included in the Final Report.

282. The chair also sent a letter by post which was dated 23^rd, but posted on the 27^th and received on the 29^th March. The letter said that she had not received a response from the petitioners confirming that they wished all materials to be removed.

283. During our discussion with the Chief Medical Officer, Dr Catherine Calderwood, she accepted that the petitioners had wanted the Cabinet Secretary to make a request to the chair to remove the materials. Dr Calderwood took the view that the Cabinet Secretary, to maintain Ministerial independence, could not intervene. Instead, she said that the Chair “needed to make that decision for herself.” She accepted that this was not the petitioners understanding at the time.

284. These communications took place over a very short space of time. There were 11 days between the petitioners’ initial meeting with the Cabinet Secretary and the publication of the Report. The lines of responsibility appeared to be unclear regarding who should have had final say on whether the materials were to be included. The petitioners believed that this had been resolved following the initial meeting on March 16^th and that their request would be carried out.

285. We question why more time could not have allowed to resolve the matter, or at least to try and reach an understanding in relation to what would be included in the final publication and why this would be the case. Giving evidence to the Public Petitions Committee, the Chair said that the request to remove materials came after the report had been agreed by the remaining members.^[122] There are no minutes however, to show if or when this was discussed.

286. If it was not possible to reach an agreement between the remaining members of the Review and the petitioners, then this should have been acknowledged in the body of the Final Report. It should have been noted that such a request had been made by the petitioners but that a decision had been taken that it could not be removed.

287. Whilst differences in writing style and presentation were acknowledged by the first chair in her preface to both the Interim Report and Final Report, we believe that this greatly detracts from the overall cohesion of the Reports.^[123] A clear example of this can be seen when comparing the narrative and style of Chapter 3 in comparison to the intense clinical language and sophisticated presentation of data in Chapter 5.

288. There needed to be a stronger editorial responsibility and control over what and was not included and how it was presented.

289. The lack of editorial control over the content and structure resulted in a piecemeal Report which was ultimately difficult to both read and understand. Throughout both Reports there are issues with clinical language not being defined and a glossary of terms could have helped aid understanding.

290. An additional factor was the perceived increasing urgency to compete the Report. This left a number of matters unresolved. For example, it left a very short time to address and acknowledge in the Final Report that the petitioners had requested that their input be removed and provide reasons why this could not be done.

We recommend that it is clearly defined who has editorial control for the structure and composition of any report.

We recommend that there is a clear understanding of who has responsibility for the printing and publication of any report.