There's been a major breakthrough on PACE trial as the courts force Queen Mary University of London to release data from questionable research.
News | 19.08.16
Article Author: George Faulkner
In an important victory for ME/CFS campaigners fighting against misleading claims in medical research, an Information Tribunal has rejected Queen Mary University of London's £245,745 attempt to prevent release of data from the controversial PACE trial, the first piece of medical research to receive funding from the DWP.[1]
The PACE trial was assessing the value of biopsychosocial interventions at the same time that the DWP was using the biopsychosocial model of disability to help justify cuts to disability spending.[2,3] Mansel Aylward, former Chief Medical Officer at the DWP, and a key architect of the last two decade's reforms, had helped to secure funding for the trial and sat as an observer on the trial's steering committee.[2-5]
The director of Sense About Science USA, Trevor Butterworth, greeted news of the Tribunal's decision by stating that the "PACE trial is a fault line between the way we did medicine (secretive, clubbable) and the way we should do medicine (transparent, shared)" and "PACE is turning out to be the science controversy of the decade: it indicts the medical ecosystem of review".[6,7]
Comments on the PACE trial from critics within academia:
Professor Vincent Racaniello, Columbia University: "This is a flawed study, it has to be fixed and people are being harmed by it."[56]
Professor Arthur Reingold, University of California, Berkeley: “Under the circumstances, an independent review of the trial conducted by experts not involved in the design or conduct of the study would seem to be very much in order.”[8]
Professor Leonard Jason, DePaul University: “The PACE authors should have reduced the kind of blatant methodological lapses that can impugn the credibility of the research, such as having overlapping recovery and entry/disability criteria.”[8]
Professor Bruce Levin, Columbia University: “It calls into question the diagnosis of an illness whose patients already rate as ‘recovered’ or ‘within normal range.’ I find it nearly inconceivable that a trial’s data monitoring committee would have approved such a protocol problem if they were aware of it.”[8]
Professor James Coyne, University of Groningen: "The data presented are uninterpretable. We can temporarily suspend critical thinking and some basic rules for conducting randomized trials (RCTs), follow-up studies, and analyzing the subsequent data. Even if we do, we should reject some of the interpretations offered by the PACE investigators as unfairly spun to fit what [is] already a distorted positive interpretation of the results."[57]
Professor Rebecca Goldin, George Mason University: "How can we judge whether the improvements seen in primary and secondary outcomes associated with CBT and GET are “real” if “recovery” does not always require clinically meaningful improvement, and if the meaning of “normal range” includes averages for people in their late 70s and early 80s?"[58]
Dr. David Tuller, University of California, Berkeley: "This study is a piece of crap."[56]
The PACE trial was a large randomized trial of interventions for Chronic Fatigue Syndrome (CFS).[3] It cost over five million pounds of public money, yet problems with the design of the trial and the way in which it was conducted meant that results were always likely to be biased towards the PACE researchers' preferred approaches.[2,8] Impressive sounding results have been released to much fan-fare from the UK media, yet even ignoring problems with the trial itself, these results were not for the outcomes the PACE researchers had committed to in advance, but instead for ones which made it far easier to claim interventions had led to a worthwhile benefit for patients.[2,8-11] Clear factual inaccuracies underpinning some of these new outcome measures have been left uncorrected, even after they were reported to the trial's researchers.[2,8,12]
While patient groups have long been critical of the PACE trial and the way in which its results were presented, the stigma and prejudices which surround their condition seemed to lead to their complaints being dismissed, or even presented as harassment.[2, 10,13-21] More recently there has been growing academic support for patients and their criticism of the PACE trial, particularly following the publication of a series of pieces by David Tuller, an investigative journalist and academic coordinator of the concurrent masters degree program in public health and journalism at the University of California, Berkeley.[8]
The support from respected academics was noted by the Tribunal's majority decision: "The identity of those questioning the research, who had signed an open letter or supported it, was impressive."[1] A number of those academics speaking out about the problems with the PACE trial have expressed concern about the PACE researchers' failure to engage with criticism, and this was included in the Tribunal's majority decision as an explanation for why there was a public interest in releasing the requested data: "there is a strong public interest in releasing the data given the continued academic interest so long after the research was published and the seeming reluctance for Queen Mary University [the institution appealing against the order to release anonymised PACE trial data] to engage with other academics they thought were seeking to challenge their findings (evidence of [PACE trial researcher] Professor Chalder)."[1]
Queen Mary University of London (QMUL) appears to have attempted to use the prejudices which surround ME/CFS and patients' concerns about some researchers' work to justify their refusal to release data from the PACE trial.[1] If the PACE trial's critics could be portrayed as unreasonable, obsessive and dangerous, then there would be greater concern about releasing information to them, and even just their requests for information could be considered vexatious. However, neither the Information Commissioner nor the Tribunal were persuaded.
A summary of the Information Commissioner's submission argued that: "Professor Anderson's 'wild speculations' about the possibility of 'young men, borderline sociopathic or psychopathic' attaching themselves to the PACE trial criticism 'do him no credit'. Nor do his extrapolations from benign Twitter requests for information to an 'organised campaign' from an 'adversarial group' show that he has maintained the necessary objectivity and accuracy that he is required to maintain."[1] The Tribunal's majority verdict went on to conclude that: "It was clear that [Anderson's] assessment of activist behaviour was, in our view, grossly exaggerated and the only actual evidence was that an individual at a seminar had heckled Professor Chalder." Of Chalder's testimony, the Information Commissioner reported that "she accepts that unpleasant things have been said to and about PACE researchers only, but that no threats have been made either to researchers or participants. The highest she could put it was that some participants stated that they had been made to feel "uncomfortable" as a result of their contact with and treatment from her, not because of their participation in the trial per se." [emphasis in original][1]
Partly as a result of a prolonged media campaign, those ME/CFS patients who are concerned about the behaviour and quality of work of some of those researching their condition have faced a routinely stigmatising portrayal within the UK media.[2,10,13-26] Minutes from a 2013 meeting held at the Science Media Centre, an organisation that played an important role in promoting misleading claims about the PACE trial to the UK media, show these CFS researchers deciding that "harassment is most damaging in the form of vexatious FOIs [Freedom of Information requests]".[13,16, 27-31]
The other two examples of harassment provided were "complaints" and "House of Lords debates".[13] It is questionable whether such acts should be considered forms of harassment. One of the agreed action points from this meeting was to "collect evidence about the impact of harassment e.g. dealing with requests eats into research time. Feed into debate about misuse of the FOI Act". The government recently declined to exempt university researchers from the Freedom of Information Act in the way that PACE trial researchers had campaigned for, and it seems that the evidence of harassment collected was not sufficient for the Tribunal either.[1]
The patient who requested this trial data made specific reference to the Science Media Centre in his submission:
"Mr Matthees points to press releases from the Science Media Centre, a body working with PACE researchers, to the effect that they were "engineering the coverage" to “frame the narrative" in such a fashion to discredit those with legitimate criticisms as misguided extremists by sensationalising a small number of indefensible actions to the detriment of the vulnerable wider patient 'community'. This has been highlighted by respected scientists, and clinicians (see p8790/B7). Rather, no evidence of a 'silent majority' in support of the PACE trial has been put forward."[1]
In a previous justification for refusing to release separate data from the PACE trial QMUL had complained about the PACE trial's critics displaying what was, by the standards of QMUL, an "obsessional attention to detail", citing a 3,000 word response to an FOI refusal from Mr Matthees as evidence "supporting QMUL’s view of the unreasonableness and obsessiveness of such requesters".[34] In contrast, the Tribunal responded to the lengthy submissions made by Matthees and others by concluding that "The tribunal wish to thank all participants for the helpful manner in which they have presented their arguments and submissions. We have been provided with an extraordinary amount of ancillary and background information on and about the important subject matter under consideration and have considered all of it. There can be no doubt about the Public Interest in the subject matter which is evident throughout the course of this appeal, and beyond, and we are grateful for the assistance that has been given to us in this regard."[1]
QMUL Governance
Minutes from a meeting of the university's governing body, QMUL Council, show that the decision to appeal to the tribunal was discussed, but indicate little interest in the details of the case.[35] At this meeting it was stated that QMUL had only been ordered to release anonymous data, yet it was still claimed that this release would risk patients being less willing to participate in research. At the Tribunal QMUL attempted to argue that the data was not anonymous, but risked allowing the identification of participants.[1] The requested data included no identifiers such as age, location or sex, but was only final scores on standardised questionnaires and results from a walking test.[1] If the Tribunal ruled that even this limited data was not anonymised and should be treated as personal data this would pose a fundamental threat to attempts to improve the validity of medical research through data sharing.
The QMUL Council's minutes recorded no mention of the serious problems identified with the PACE trial or the way results were released, although presumably a reputation for producing poorly conducted and misleadingly presented research would concern potential research participants more than the release of anonymised trial data. There is no indication that the council discussed the ethics of using tactics at the Tribunal that seemed designed to take advantage of, and promote, prejudices about those suffering from a stigmatised health condition.
It is surprising that QMUL decided to continue with this approach, when there had already been signs that doing so could serve to attract further criticism. Following QMUL's decision to appeal against the Information Commissioner's judgement that PACE trial data should be released, James Coyne, a professor of health psychology, requested the data underpinning PACE results published in a journal which requires authors make data available to other researchers.[36,37] The refusal letter he received[38] argues that: "the active campaign to discredit the project has caused distress to the university’s researchers who hold legitimate concerns that they will be subject to public criticism and reputational damage." It is worrying that anyone would think that these are legitimate reasons for denying access to trial data which is having an important impact upon the lives of patients. The PACE team's refusal to honour their commitment to data sharing drew the ire of a wide range of campaigners concerned about problems with the reliability of medical research, leading to yet further requests for access to this data.[37,39] Coyne has claimed that he and the journal investigating this matter have "come under pressure from a number of sources, including Richard Horton, editor of the Lancet" and suggested that: "this is emerging as a major, maybe historic confrontation between the forces pushing for sharing of data and the British establishment".[40]
In December 2015 QMUL released a statement[41] responding to growing concerns about their failure to release PACE trial data. This claimed that they would seek "the advice of patients" about "appropriate access to relevant data". It is not clear if, or how, this advice was ever actively sought by QMUL, but in response twenty-four patient charities took the time to write to QMUL and ask for the requested data to be released.[42]
QMUL's 2015 statement also responds to concerns that the failure to share PACE trial data was part of an attempt to avoid independent scrutiny: "We have supplied requested, anonymised data to independent scientists, as part of normal research collaboration. Among others, these include a Cochrane review group".[41] When it was suggested during the Tribunal that a willingness to share anonymised data with independent scientists must mean that the data could be sufficiently anonymised for disclosure, the testifying PACE trial researcher stated "that disclosure to the Cochrane review does not count as disclosure to independent scientists as all three of the PACE principal investigators sat on the review panel."[1] QMUL's new statement in response to the Tribunal ruling states "QMUL has shared data from the PACE trial with other researchers",[43] no-longer using the word 'independent' and ignoring the vital role independent scrutiny should play in medical research.
QMUL spent £245,745 on outside legal advice for just this one Tribunal, yet neither of the statements released by QMUL, nor the reported witness evidence from QMUL's VicePrincipal, Steven Thornton, provide any indication that those in senior positions have taken the time to try to understand why the PACE trial has attracted so much criticism.[44,41,43,1] Without greater concern for the important details they complain the trial's critics obsess over, there is a danger that QMUL could choose to spend yet more public money on a further appeal, fighting on against the public release of important anonymised data which could help correct the misleading claims that have been made about this important and influential research.
The history of FOI requests for the PACE trial's original 'recovery' results
Those with health problems are understandably desperate to recover, and misleading claims about treatments leading to recovery can be an emotive topic. A commentary in The Lancet which accompanied the first publication of results claimed that treatment with cognitive-behavioral therapy (CBT) or graded exercise therapy (GET) had led to recovery rates of 30% and 28% using a “strict criterion” for recovery.[45] In fact, this criteria for recovery was so loose that patients could be classed as recovered even if they had reported declines on every outcome measure used in the trial.[3] These were not the only exaggerated recovery claims to come from the PACE trial, with the BMJ describing these patients as "cured" while a paper from different researchers cited the PACE trial to claim that "evidence from a recent trial of cognitive behavioural therapy and graded exercise therapy indicated a recovery rate of 30-40% one year after treatment."[47,48] The corresponding author for this paper, Esther Crawley, had been chair of the professional organisation for those providing CBT and GET to CFS patients, and when later discussing more conservative claims about the PACE trial's recovery rates she acknowledged the importance to patients of accurate information: “Every patient with CFS/ME wants to know how likely they are to recover."[30] All of the recovery criteria for which results from the PACE trial have so far been released are designed in such a way that patients could be classed as recovered even if they had reported declines on the trial's two primary outcomes.[2,3,8,49]
Troubled by the way results from the PACE trial were released, patient organisations made a joint Freedom of Information request for the results of those outcome measures laid out in the trial's protocol but not published in the trial's initial paper.[3,50,51] The response to this stated that the results for the protocol's recovery criteria were exempt from the Freedom of Information Act as they would soon be published in an academic journal.[52] This turned out not to be true. When these results had not been published over a year later, a new FOI request was made, which QMUL refused by stating that the PACE team had not calculated the requested results and therefore "the requested data relating to the recovery rates and positive outcomes do not exist."[53] A follow-up request that the trial's pre-specified outcomes be calculated from the trial data was then denied with the justification that the cost of conducting the calculations would be more than the £450 appropriate limit laid out by the Freedom of Information Act as “there is no longer a statistician employed by the PACE trial, [so] one would need to be recruited for this operation and trained.”[54] Mr Matthees began his FOI request by explaining that he was requesting anonymised data "in order to help ease the burden of staff having to perform the required calculations themselves" after previous requests for results were turned down.[55] It is surprising that a university does not already employ a statistician able to perform the requested analyses, and disappointing that QMUL then chose to spend over £245,000 fighting against the release of anonymised data that would allow for the independent calculation of the trials pre-specified primary and recovery outcomes.
NB: This article previously stated that QMUL spent £204,787 on outside legal advice, but this failed to account for VAT and has now been corrected to £245,745.
References:
1. Kennedy B, Stephenson D, Watson N (2016) First-Tier Tribunal, Information Rights.
Appeal Number: ΕΑ/2015/0269 http://www.informationtribunal.gov.uk/DBFiles/Decision/i1854/Queen%20Mary%20University%20of%20London%20EA-2015-0269%20(12-8-16).PDF
2. Faulkner G (2016) "In the Expectation of Recovery": Misleading Medical Research and Welfare Reform. Centre for Welfare Reform. http://www.centreforwelfarereform.org/uploads/attachment/492/in-the-expectation-of-recovery.pdf
3. White PD et al on behalf of the PACE trial management group (2011) Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet. http://www.ncbi.nlm.nih.gov/pubmed/21334061
4. DWP Central FoI Team (2011) Part funding of the PACE trial by the DWP. https://www.whatdotheyknow.com/request/part_funding_of_the_pace_trial_b
5. Shakespeare T, Watson N and Alghaib O A (2016) Blaming the victim, all over again: Waddell and Aylward’s biopsychosocial (BPS) model of disability. Critical Social Policy. http://csp.sagepub.com/content/early/2016/05/25/0261018316649120.abstract
6. Butterworth T (2016) "#PACE trial is a fault line between the way we did medicine (secretive, clubbable) and the way we shd do medicine (transparent, shared)." https://twitter.com/Butterworthy/status/765579942922948608
7. Butterworth T (2016) "@hangingnoodles This is why #PACE is turning out to be the science controversy of the decade: it indicts the medical ecosystem of review". https://twitter.com/Butterworthy/status/765911282537078784
8. Tuller D (2015) Trial By Error: The Troubling Case of the PACE Chronic Fatigue Syndrome Study. Virology Blog. http://www.virology.ws/2015/10/21/trial-by-error-i/
9. The Scotsman (2011) ME sufferers given hope as Scottish team leads way to new therapy. http://www.scotsman.com/news/me-sufferers-given-hope-as-scottish-team-leads-way-to-new-therapy-1-1501751
10. Pemberton M (2011) Protesters have got it all wrong on ME. The Daily Telegraph. An altered version of this article is available at: http://www.telegraph.co.uk/news/health/8641007/Protesters-have-got-it-allwrong-on-ME.html The updated version can be compared to what was originally published here: http://i64.photobucket.com/albums/h168/jgarcia3788/Telegraph.jpg~original
11. Laurance J (2011) Got ME? Just get out and exercise, say scientists. The Independent. http://www.independent.co.uk/life-style/health-and-families/health-news/got-me-just-get-out-and-exercise-sayscientists-2218377.html
12. Matthees A (2015) Assessment of recovery status in chronic fatigue syndrome using normative data. Quality of Life Research April, Volume 24, Issue 4, pp 905-907. http://www.ncbi.nlm.nih.gov/pubmed/25304959
13. Lothian A (2013) Minutes for 'CFS meeting 31st January 2013'. Science Media Centre. See copy of Minutes below (image).
14. Lewandowsky S (2015) Science and Society at the Crossroads: Skepticism vs. Denial and Elitism vs. Public Engagement. Royal Society. https://figshare.com/articles/RS_scienceandsociety_September_2015_pdf/2061696
15. Hanlon M (2013) This man faced death threats and abuse. His crime? He suggested that ME was a mental illness. 5 May. The Sunday Times.
16. Bithell C (2013) Review of the first three years of the mental health research function at the Science Media Centre. Science Media Centre. http://www.sciencemediacentre.org/wp-content/uploads/2013/03/Review-of-the-first-three-years-of-the-mental-health-research-function-at-the-Science-Media-Centre.pdf
17. Crawley E & Feilden T (2012) Threats of Persecution. Views From the Frontline. Science Media Centre. http://www.sciencemediacentre.org/wp-content/uploads/2012/10/SMC-Views-from-the-frontline-essays-on-the-SMC.pdf
18. Hawkes N (2011) Dangers of research into chronic fatigue syndrome. The British Medical Journal. 22;342:d3780. http://www.ncbi.nlm.nih.gov/pubmed/21697226
19. http://www.ayme.org.uk/news/-/asset_publisher/2uYbtahTJKdk/content/april-2011-analysis-of-pace-trial-results
20. Parry V (2011) Analysis of PACE Trial results. Association of Young People with ME. http://www.ayme.org.uk/news/-/asset_publisher/2uYbtahTJKdk/content/april-2011-analysis-of-pace-trial-results
21. Marchant J (2016) 'It was like being buried alive': battle to recover from chronic fatigue syndrome. The Guardian. http://www.theguardian.com/society/2016/feb/15/it-was-like-being-buried-alive-victim-ofchronic-fatigue-syndrome
22. Tuller D (2016) Trial By Error, Continued: A Few Words About “Harassment”. Virology Blog. http://www.virology.ws/2016/02/01/trial-by-error-continued-a-few-words-about-harassment/
23. O'Sullivan S (2015) It's All In Your Head: True Stories of Imaginary Illness. Chatto & Windus.
24. Blakemore C (2011) Disease of the brain, not the mind, has no stigma. The Times. http://www.thetimes.co.uk/tto/opinion/columnists/article3112079.ece
25. Liddle R (2015) Selfish Whining Monkeys: How we Ended Up Greedy, Narcissistic and Unhappy. Fourth Estate.
26. 205. Liddle R (2015) The ME lobby is just a symptom of our stupidity about mental illness. The Spectator. http://www.spectator.co.uk/2015/11/the-me-lobby-is-just-a-symptom-of-our-stupidity-about-mentalillness/
27. Science Media Centre (2011) Expert reaction to Lancet study looking at treatments for Chronic Fatigue Syndrome/ME. Science Media Centre. http://www.sciencemediacentre.org/expert-reaction-to-lancet-study-looking-attreatments-for-chronic-fatigue-syndromeme-2-2/
28. Sharpe M & Chalder T (2011) CFS/ME trial. Briefing for journalists. Science Media Centre. http://www.sciencemediacentre.org/cfsme-trial/
29. Sharpe M & McCrone P (2012) Do the best treatments for CFS cost more? Briefing for journalists. Science Media Centre. http://www.sciencemediacentre.org/do-the-best-treatments-for-cfs-cost-more-3/
30. Crawley E, Hamilton W & Miller A (2013) Expert reaction to new research into therapies for Chronic Fatigue Syndrome/ME [recovery paper]. Round up for journalists. Science Media Centre. http://www.sciencemediacentre.org/expert-reaction-to-new-research-into-therapies-for-chronic-fatiguesyndromeme/
31. Chalder T & Goldsmith K (2015) CFS/ME: The next step in the controversy. Briefing for journalists. Science Media Centre. http://www.sciencemediacentre.org/cfsme-the-next-step-in-the-controversy-2/
32. Fox F & White PD (2013) FOI Letter final. 31st October. Copy available at: https://mobile.twitter.com/maxwhd/status/674567886460076032
33. White P D (2015) On-line response to Freedom of Information Act review. A copy was previously available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/487271/Online_Responses_CitizenSpace.xls A copy of White comments are available at: http://forums.phoenixrising.me/index.php?threads/peter-white-complaining-about-pace-trial-foi-request-and-calling-for-changes-to-foi-laws.42027/
34. Queen Mary University of London (2016) Queen Mary University’s arguments regarding the application of section 14 to the request. Available at: https://www.mediafire.com/?9d5120lucdj594j. In relation to the information request made here: https://www.whatdotheyknow.com/request/fitness_data_for_pace_trial
35. Queen Mary University of London Council (2016) 05 April Confirmed Minutes. http://www.arcs.qmul.ac.uk/governance/council/committees/committee-papers/05-04-2016/178015.pdf
36. McCrone P, Sharpe M, Chalder T, Knapp M, Johnson AL, Goldsmith KA & White PD (2012) Adaptive Pacing, Cognitive Behaviour Therapy, Graded Exercise, and Specialist Medical Care for Chronic Fatigue Syndrome: A Cost-Effectiveness Analysis. PLoS ONE, 7(8), e40808. http://www.ncbi.nlm.nih.gov/pubmed/22870204
37. Oransky I & Marcus A (2015) To keep science honest, study data must be shared. STAT. http://www.statnews.com/2015/12/23/sharing-data-science/
38. Daley B (2015) King's College London response to data request from Prof Coyne. 11th December. https://dl.dropboxusercontent.com/u/23608059/PACE%20F325-15%20-%20Prof.%20James%20Coyne%20-%20Response-2.pdf
39. ME Action (2015) “Vexatious”: King’s College London dismisses James Coyne’s request for PLoS One PACE data. Meaction.net http://www.meaction.net/2015/12/12/vexatious-kings-college-london-dismissesjames-coynes-request-for-plos-one-pace-data/
40. Gelman A (2015) Tug of War: Epic battle over data in controversial paper on chronic fatigue syndrome. Statistical Modeling, Causal Inference, and Social Science. http://andrewgelman.com/2015/12/18/28362/
41. Investigators of the PACE trial (2015) Statement: release of individual patient data from the PACE trial. Queen Mary University of London. http://www.qmul.ac.uk/media/news/items/smd/168729.html
42. #ME Action (2016) 24 organisations in 14 countries tell QMUL: release PACE data. ME Action http://www.meaction.net/2016/03/13/24-organisations-in-14-countries-tell-qmul-release-pace-data/
43. Queen Mary University of London (2016) Statement: Disclosure of PACE trial data under the Freedom of Information Act. Queen Mary University of London. http://www.qmul.ac.uk/media/news/items/smd/179331.html
44. Peters J (2016) Using public money to keep publicly funded data from the public. Johnthejack.com https://johnthejack.com/2016/06/29/using-public-money-to-keep-publicly-funded-data-from-the-public/
45. Knoop H & Bleijenberg G (2011) Chronic fatigue syndrome: where to PACE from here? Lancet Mar 5;377(9768):786-8. http://www.ncbi.nlm.nih.gov/pubmed/21334060?dopt=Abstract
46. Sharpe MC, Archard LC, Banatvala JE, Borysiewicz LK, Clare AW, David A, Edwards RH, Hawton KE, Lambert HP & Lane RJ (1991) A report--chronic fatigue syndrome: guidelines for research. (The Oxford Criteria) Journal of the Royal Society of Medicine, 84(2), 118–121. http://www.ncbi.nlm.nih.gov/pubmed/1999813
47. The British Medical Journal (2011) All you need to read in the other journals. BMJ 2011;342:d1168 http://www.bmj.com/content/342/bmj.d1168
48. Collin SM, Crawley E, May MT, Sterne JA & Hollingworth W (2011) The impact of CFS/ME on employment and productivity in the UK: a cross-sectional study based on the CFS/ME national outcomes database. BMC Health Services Research, 11, 217. http://www.ncbi.nlm.nih.gov/pubmed/21923897
49. White PD, Goldsmith KA, Johnson AL, Chalder T & Sharpe M (2013) Recovery from chronic fatigue syndrome after treatments given in the PACE trial. Psychological Medicine, 43(10), 2227–2235. http://www.ncbi.nlm.nih.gov/pubmed/23363640
50. White PD, Sharpe MC, Chalder T, DeCesare JC, Walwyn R & the PACE trial group (2007) Protocol for the PACE trial: A randomised controlled trial of adaptive pacing, cognitive behaviour therapy, and graded exercise as supplements to standardised specialist medical care versus standardised specialist medical care alone for patients with the chronic fatigue syndrome/myalgic encephalomyelitis or encephalopathy. BMC Neurology, 7, 6. http://www.ncbi.nlm.nih.gov/pubmed/17397525
51. Riley N, Colby J & Colby N (2011) PACE Trial – text of joint letter calling for more information. http://www.meassociation.org.uk/2011/05/6171/
52. Smallcombe P (2011) Queen Mary University of London Response to FOI. http://www.meassociation.org.uk/wp-content/uploads/2011/06/FOI+from+Queen+Mary.pdf
53. Smallcombe P (2012) PACE Trial: Recovery Rates and Positive Outcome Rates. https://www.whatdotheyknow.com/request/pace_trial_recovery_rates_and_po#incoming-327302
54. Smallcombe P (2013) PACE Trial: Recovery Rates and Positive Outcome Rates (repeat request). https://www.whatdotheyknow.com/request/pace_trial_recovery_rates_and_po_2#incoming-463991
55. Matthees A (2014) Selected data on PACE Trial participants. https://www.whatdotheyknow.com/request/selected_data_on_pace_trial_part
56. Racaniello V and Tuller D (2016) Twiv 397: Trial by Error. This Week in Virology. http://www.microbe.tv/twiv/twiv-397/
57. Coyne JC (2015) Uninterpretable: Fatal flaws in PACE Chronic Fatigue Syndrome follow-up study. Mind the Brain, PLoS Blogs. http://blogs.plos.org/mindthebrain/2015/10/29/uninterpretable-fatal-flawsin-pace-chronic-fatigue-syndrome-follow-up-study/
58. Goldin R (2016) PACE: The research that sparked a patient rebellion and challenged medicine. Sense About Statistics. http://www.stats.org/pace-research-sparked-patient-rebellion-challenged-medicine/
