The Cochrane correspondence

The Cochrane review on exercise therapy for patients suffering from chronic fatigue syndrome has been the subject of ongoing controversy. In October 2018, a Reuters news article suggested that Cochrane would temporarily withdraw the review due to pressure and complaints from online activists. Publicly released correspondence between Cochrane’s former Editor in Chief and senior employees of the Norwegian Institute of Public Health offers a different perspective to the dispute.

A contested review

Cochrane is an international organization renowned for its systematic reviews of medical research, commonly regarded as the gold standard in evidence-based healthcare. [1] Its review on graded exercise therapy (GET) for patients suffering from chronic fatigue syndrome (CFS) however, is controversial and contested. [2]  

GET instructs patients to gradually increase the duration and intensity of exercise beyond their perceived limit to reverse deconditioning. [3] CFS patients, however, report a marked deterioration in health when they exceed a certain activity level, a phenomenon commonly referred to as post-exertional malaise. [4] The Cochrane review by Larun et al. suggests that GET is a safe and effective treatment for CFS [5], although its methodology and conclusions have been widely criticized. [6] 

Following a formal complaint about the review to the Editorial Board, Cochrane decided to conduct an internal audit to assess the validity of the critique. In October 2018, a Reuters news article reported that Cochrane would temporarily withdraw the contested review, a decision that was deemed “disproportionate and poorly justified” by the authors. [7] Publicly released correspondence between Cochrane’s former Editor in Chief David Tovey and senior employees at the Norwegian Institute of Public Health (NIPH), provides a different perspective to the controversy. NIPH hosts Cochrane Norway and employs two of the authors of the Cochrane review in question.

Justified criticism

The email exchange confirms that the review suffered from multiple methodological faults. Cochrane has added a notification to the review, claiming it is “substantially out of date and in need of updating.” [8] In the correspondence, Tovey stressed that the authors failed to address the criticism of their review.  An independent assessment of submitted comments indicated that these were cogent and that “there was a case to answer.” [9] The authors were given the opportunity to address the errors, but their resubmission was considered inadequate by multiple reviewers. As Tovey explained: 

“I asked my internal team to review the submission, and also three senior colleagues who I believe have the expertise and detachment required to provide an objective evaluation. As the attached documents show, all of these assessors continue to have important concerns […].” [10]

Larun et al. previously faced similar problems with a related Cochrane review that intended to use individual patient data (IPD) instead of aggregate outcomes on GET for CFS. The IPD manuscript also received negative reviews and had to be resubmitted. Again, multiple experts indicated that Larun et al. failed to grasp the methodological issues raised. Tovey announced Cochrane’s decision to cancel the IPD review as follows:

“[…] we were disappointed when the peer reviewers of the re-submitted review all agreed that the authors had not adequately addressed the methodological issues. We therefore decided that we had no option but to reject the IPD review.” [9]

Bias and activism

In total, Larun et al. erred on 5 different occasions while writing these two Cochrane reviews on GET for CFS. The email correspondence suggests the authors might have a bias in working towards a favorable conclusion. In an email of 25 October, Tovey wrote that “the overall tone of the conclusions appears to us to be overly optimistic.” [11] In another, he described the authors’ intention as “stretching the truth beyond its limit.” [12] The main author of the review, Lillebeth Larun has researched and promoted the use of exercise therapy for CFS before working on the Cochrane review. [13, 14] Another author, Kjetil Gundro Brurberg has since its publication been an outspoken defender of the treatment, claiming that “in the absence of evidence of efficacy for other interventions, health professionals should offer patients with CFS/ME the opportunity to try […] GET.” [15] Brurgberg wrote that some of the criticism towards the evidence for GET may be “counterproductive” and “discourage further research.” [15] Similar sentiments were expressed by Larun. In the email exchange, Tovey noted “that Lillebeth, and the trialists of the included reviews, tend to refer to all criticism of the trials and the review as being the result of ‘activism’.“ [9] Cochrane disagreed and emphasized that the criticism was “thoughtful and reasonable.“ [9] In an interview with Reuters, however, Larun expressed concern that the activists who have plagued her for years have now got to her editors. [16]

Leaked emails

In an email dated November 2, 2018, the Acting Director General of NIPH, Trygve Ottersen, advised Cochrane not to give in to the criticism:

“[…] on a controversial topic as CFS/ME, a continuous stream of criticism is to be expected. We hope however that the Cochrane editorial office will stay committed to Cochrane’s standard procedures also in this case […].” [17]

Shortly before that, internal emails were leaked to the press. On October 17, Reuters reported that Cochrane would temporarily withdraw the review “amid fierce criticism and pressure from activists and patients.”[7] It is unclear who leaked the information but the tone of the article aligns with the authors’ suggestion that Cochrane is conceding to unfounded but fierce criticism from the CFS community. Despite Tovey’s caution that “this not about patient pressure”, the article strongly suggests otherwise. Colin Blakemore, a former chief executive of Britain’s Medical Research Council, is quoted saying that the decision “set a worrying precedent for scientific evidence being over-ridden by the opinions of activists.” The Reuters article was taken over by multiple media outlets [18] and provoked anger and concern on social media. Clare Gerada, former chairperson of the Council of the Royal College of General Practitioners, commented on Twitter that “Cochrane has lost its way” [19] and that “we are entering an era of bullying scientists when lobby groups disagree with the results.” [20]

A second resubmission

The publicly released email correspondence indicates such criticism to be unfounded. Concerns about the review were raised by multiple experts. “When a collection of experienced and dispassionate colleagues are all making, in effect, the same criticisms”, Tovey wrote, “it is hard to ignore this.” [21] It is unclear if the leaked emails and public criticism impacted Cochrane’s handling of the matter. The review on GET for CFS, however, has not been withdrawn. On June 17, 2019, Cochrane announced that it has received the second resubmission of Larun et al. The amended review will be published during the next 2 months. [8]

[Edit: this blog post has been updated to correct spelling mistakes]

References

I received the email exchange documents through Kalliope, a member of the Science for ME forum and MARIT @MEMHJ, whose Blog gives an interesting overview of this subject.

[1] Cochrane Library: an improved online platform to guide health decision-making across the world. August 8, 2018. https://www.cochrane.org/news/cochrane-library-improved-online-platform-guide-health-decision-making-across-world

[2] Tuller D. Trial By Error: The Cochrane Controversy. Virology Blog. September 3, 2018. http://www.virology.ws/2018/09/03/trial-by-error-the-cochrane-controversy/

 [3] Clark LV, White PD. The role of deconditioning and therapeutic exercise in chronic fatigue syndrome (CFS). J. Ment. Health. 2005 June; 14(3):237-252. https://doi.org/10.1080/09638230500136308

 [4] Institute of Medicine. Beyond Myalgic Encephalomyelitis/Chronic Fatigue Syndrome: Redefining an Illness. The National Academies Press, Washington, D.C., 2015. https://www.ncbi.nlm.nih.gov/books/NBK274235/

[5] Larun L, Brurberg KG, Odgaard-Jensen J, Price JR. Exercise therapy for chronic fatigue syndrome. Cochrane Database Syst Rev. 2017 Apr 25;4:CD003200. https://doi.org/10.1002/14651858.CD003200.pub7

 [6] Struthers S. Reply to David Tovey’s letter about the Exercise for CFS review. February 18, 2019. https://healthycontrolblog.wordpress.com/2019/02/18/response-to-cochrane/

[7] Kelland K. Exclusive: Science journal to withdraw chronic fatigue review amid patient activist complaints. Reuters. October 17, 2018. https://www.reuters.com/article/us-health-chronicfatigue-dispute/exclusive-science-journal-to-withdraw-chronic-fatigue-review-amid-patient-activist-complaints-idUSKCN1MR2PI

[8] Cochrane Database of Systematic Reviews. Exercise therapy for chronic fatigue syndrome, see what’s new. https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD003200.pub7/information#whatsNew

[9] Email from David Tovey to Trygve Ottersen, dated October 4, 2018. Document: 36085(Ekspeder dokument for Innsyn sak 17/10566 dok 2-10). https://www.dropbox.com/s/0pdjqjd15s8i2vh/Cochrane%20correspondence%20from%2002-10-2018%20to%2002-11-2018.pdf?dl=0

[10] Email from David Tovey to Atle Fretheim, Lillebeth Larun and Kjetil Gundro Brurberg, May 24, 2019. https://www.dropbox.com/s/8jynkfcrtpliuyy/Cochrane%20correspondence%2024-05-2019.pdf?dl=0

[11] Email from David Tovey to Gunhild Wøien, Lillebeth Larun and Kjetil Gundro Brurberg, October 25, 2018. Document: 36085(Ekspeder dokument for Innsyn sak 17/10566 dok 2-10). https://www.dropbox.com/s/0pdjqjd15s8i2vh/Cochrane%20correspondence%20from%2002-10-2018%20to%2002-11-2018.pdf?dl=0

[12] Email from David Tovey to Atle Fretheim, date unknown. https://www.dropbox.com/s/8jynkfcrtpliuyy/Cochrane%20correspondence%2024-05-2019.pdf?dl=0

[13] Larun L, Malterud K. Treningsbehandling ved kronisk utmattelsessyndrom. Tidsskr Nor Laegeforen. 2011 Feb 4;131(3):231-6. doi: 10.4045/tidsskr.09.1475. https://tidsskriftet.no/2011/02/originalartikkel/treningsbehandling-ved-kronisk-utmattelsessyndrom

[14] Larun L, Malterud K. Finding the right balance of physical activity: a focus group study about experiences among patients with chronic fatigue syndrome. Patient Educ Couns. 2011 May;83(2):222-6. https://doi.org/10.1016/j.pec.2010.05.027

[15] Brurberg KG. A PACE-gate or an editorial without perspectives? The Mental Elf. July 19, 2017. https://www.nationalelfservice.net/other-health-conditions/chronic-fatigue-syndrome/a-pace-gate-or-an-editorial-without-perspectives/

[16] Kelland K. Sick and tired: Online activists are silencing us, scientists say. Reuters. March 13, 2019. https://www.reuters.com/investigates/special-report/science-socialmedia/

[17] Email from Trygve Ottersen to David Tovey, November 2, 2018. Document: 36085(Ekspeder dokument for Innsyn sak 17/10566 dok 2-10). https://www.dropbox.com/s/0pdjqjd15s8i2vh/Cochrane%20correspondence%20from%2002-10-2018%20to%2002-11-2018.pdf?dl=0

[18] Kelland K. Journal to Withdraw Chronic Fatigue Review Amid Patient Activist Complaints. Medscape. October 18, 2018. https://www.medscape.com/viewarticle/903621

[19] Clare Gerada #FBPE (@ClareGerada). “I think Cochrane has lost its way. Sad. As used to be most trusted source.” Twitter. October 18, 2018. https://twitter.com/ClareGerada/status/1052837095247613952

[20] Clare Gerada #FBPE (@ClareGerada).“withdrawal decision sets a worrying precedent for scientific evidence being over-ridden by the opinions of activists. (Colin Blakemore). > sad we are entering an era of bullying scientists when lobby groups disagree with the results.” Twitter. October 17, 2018. https://twitter.com/ClareGerada/status/1052636273180139520

[21] Email David Tovey to Atle Fretheim, May 27, 2019. https://www.dropbox.com/s/9w00sm7xkgyh5s2/Cochrane%20correspondence%2027-05-2019.pdf?dl=0