The following are excerpts from the BMJ Response Pages with the 329 Team responses. Some of the responses are expanded here.
Contents
18 September 2015 – Kim A. Papp
Restoring Study 329 illustrates three deficiencies of our medical literature.
While the authors concisely outline study design, endpoints, sample size determination, and ultimate failure of the study, their reanalysis fails by ignoring three principles of hypothesis testing. There are no pre-specified outcomes. There is no assessment of statistical power to detect a difference between the populations (Fisher RA 1925 Statistical Methods for Research Workers, Edinburgh: Oliver&Boyd) . Worse, no account is made for multiple comparisons (J Hsu Mltiple comparisons: theory and methods. CRC Press, Chicago, 1996, Bland JM, Altman DG. Multiple significance tests: the Bonferroni method. BMJ 1995; 310: 170). Neglecting multiple comparisons will results in fictitious associations (Austin PC, Mamdani MM, Juurlink DN, Hux JE Testing multiple statistical hypotheses resulted in spurious associations: a study of astrological signs and health. J CLin Epidemiol 59(9): 964-9 (2006)). The first deficiency of our medical literature is authors’ limited appreciation of statistics.
The second deficiency is readily demonstrated. Reviewers have a limited understanding of statistics.
Two good reasons for not permitting open access to clinical trial data are self- evident.
More telling and more damning is the third deficiency: the willingness of editors to headline unsupported claims of causal associations. No correction, no retraction, no apology, no comment. Shame on you.
A. Papp, MD, PhD, FRCPC
Canada
Competing interests: No competing interests
18 September 2015
Kim A. Papp
Physician
Waterloo, Canada
Team 329: Could this be competing interests?
17 September 2015 – Susan Molchan
Study 329 is a notorious study, with its perverted results well known to any psychiatrist paying attention. It is still well worth the detailed look at how the reported results came to be, and congratulations and thanks to the authors for doing so. RIAT is a great idea and showing specific examples like the 329 study illustrates how important trial transparency is. As Drs. Healy and Nardo know, psychiatric trials are rich with similar insults to science and patient care, in my opinion.
Competing interests: No competing interests
17 September 2015
Susan Molchan
psychiatrist
Bethesda, MD
17 September 2015 – Elizabeth Wager
Why is there no peer review history for this article? Given that this is a re-analysis of a previously published study, which, presumably the original authors still stand by, it would be especially interesting to see what the peer reviewers had to say about this new analysis and interpretation.
Competing interests: No competing interests
17 September 2015
Elizabeth Wager
Publications Consultant
Sideview
Princes Risborough
17 September 2015 – Larry D. Sasich
Paroxetine and Study 329 – What We Already Knew and When
The authors of the re-analysis of Study 329 [1] in which the original study examined paroxetine’s safety and tolerability in children and adolescents with depression using the Restoring Invisible and Abandoned Trials (RIAT) protocol miss some critical aspects of patient drug safety.[2,3]
Perhaps their most notable omission is that it has been known since at least 1998 from the US Food and Drug Administration (US FDA) approval package, posted free of charge on agency’s web site, that paroxetine had not been shown to be safe and effective in pediatric populations.[4] This information was available to the public three years before the original publication of Study 329.[2] The professional product label for paroxetine has been required to contain a box warning, the strongest type of safety alert the US FDA can require, that the drug is not approved for use in pediatric patients since the mid-2000s.[5]
More disturbing than the results of the Study 329 re-analysis is the continued prescribing of drugs, such as paroxetine, in populations in which there is no evidence at the level of rigorous regulatory scrutiny that the drug is beneficial. This leaves patients with only the possibility of harm. Prescribers must recognize that the phrase “safe and effective” has two meanings. One is the common usage that can be used in conclusions of medical journal articles with impunity and the other is the regulatory meaning that requires rigorous review of the evidence submitted to support the approval of or use of a drug in a specific population.
Thirty-five years ago evidence emerged suggesting that prescribing decisions may be based on commercial rather than scientific sources of information.[6] Prescribers are inundated with promotional materials that can also include some peer-reviewed medical journal articles claiming drugs are “safe and effective”. Advertising is no defense for prescribing drugs that have not been shown to be beneficial when public summaries exist based on rigorous evaluations of the known evidence supports avoiding the drug in this specific population. These public summaries would include US regulatory documents and the professional product information.
Any positive impact on patient safety from the re-analysis of Study 329 trial 14 years after the original study was published is questionable. A lesson that should be learned from the Study 329 tale is that patient drug safety may be better served by promoting and placing the information from regulatory summary documents in the hands of patients and their parents.
References
- Le Noury J, Nardo JM, Healy D, et al. Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. BMJ 2015;351:h4320.
- Keller MB, Ryan ND, Strober M, et al. Efficacy of paroxetine in the treatment of adolescent major depression: a randomized, controlled trial. J Am Acad Child Adolesc Psychiatry Jul 2001;40(7):762-772.
- Doshi P, Dickersin K, Healy D, Vedula SS, Jefferson T: Restoring invisible and abandoned trials: a call for people to publish the findings. BMJ 2013;346:f2865.
- Dubitsky GM. Food and Drug Administration Review and Evaluation of Paroxetine Controlled Released Tablets, July 17, 1998. At http://www.accessdata.fda.gov/drugsatfda_docs/nda/99/20-936_Paxil_medr_P…. Accessed August 24, 2015.
- Apotex Corp. Professional Product Label: Paxil CR (Paroxetine), July 12, 2015. At http://dailymed.nlm.nih.gov/dailymed/drugInfo.cfm?setid=483bd97f-c4d0-4e…. Accessed August 24, 2015.
- Avorn J, Chen M, Hartley R. Scientific versus commercial sources of influence on the prescribing behavior of physicians. Am J Med. Jul 1982;73(1):4-8.
Competing interests: No competing interests
17 September 2015
Larry D. Sasich
Pharmacist
Burlington, ON,
Canada L7S 15
19 September 2015 – Response to Dr Sasich
The dyad of the prescribing physician and the patient that actually takes the medication are both guided by versions of a risk-benefit ratio – a metric that’s under assault from every direction in the modern information age. Sasich’s proposal “that patient drug safety may be better served by promoting and placing the information from regulatory summary documents in the hands of patients and their parents” is certainly one antidote to the flood of biased information.
Our paper isn’t simply about this 14 year old study. It’s intended to serve notice to editors, authors, and sponsors that the medical literature is not a commercial advertising platform; that the raw data from clinical trials and other studies needs to accompany the journal articles so it can be examined directly; and that distorted or missing studies are fair game for other investigators.
The medical literature is the traditional venue for scientific information and there’s no reason at all to accept its increasing corruption by commercial interests. Our article is really about the fact that something like this situation must be assumed to be true across medicine for all indications and all ages where there is a lack of access to the underlying data.
Competing interests: An author on Restoring Study 329
19 September 2015
John M Nardo
Doctor
Retired
Jasper Georgia
23 September 2015 – Response to Dr Sasich
With great respect to Larry Sasich, FDA had to be told by outsiders to look under the emotional lability rock while appraising the submission of paroxetine for pediatric use. Even then they missed many of the suicidal events. As regards efficacy they agreed with what they had been previously told by GSK – that Study 329 did not show efficacy but issued an approvable letter stating: ” we agree that the results from … Study 329… failed to demonstrate the efficacy of Paxil in pediatric populations with MDD”. [but] “we agree that it would not be useful to describe these negative trials in the labeling”.[1]
Study 329 gave rise to a controversy about suicidality when it became clear that the medical reviewer of the issue was gagged by FDA leading to Congressional Hearings on the matter of his gagging.
What can we conclude? That all FDA reviews are vetted and reviewers self-censor so that no-one is being told the truth. That in the absence of access to the underlying data most journal articles on the results of RCTs and FDA reviews must be assumed to be compromised as Study 329 was as there is nothing in Study 329 that suggests this was anything other than standard industry practice.
1 https://davidhealy.org/wp-content/uploads/2015/09/Paxil-Approvable.pdf
Competing interests: Both authors on Restoring Study 329 with competing interests declared there.
23 September 2015
David Healy
John M Nardo
17 September 2015 – Wendy Rogers
In my view, this publication has significant implications for research ethics. Whether or not a clinical trial is considered ethical hinges on a number of factors, including risk to participants, the merit and integrity of the research, potential benefits such as producing important knowledge, whether or not individuals are respected and offered an informed choice regarding participation in the study, and whether the research meets justice requirements.1 The merit and integrity of the research are linked to the potential benefits: high quality research produces valid and reliable results that can usefully inform the care of future patients.
This ethical rationale for clinical research was turned on its head by the original study 329. Restoring Study 329 demonstrates with forensic detail just how the general justifying principles were ignored or subverted. The research was not conducted with integrity (the protocol was not followed, adverse events were not comprehensively reported), therefore the results were not reliable in guiding the care of future patients, leading to unquantified harms. Patients were not respected but rather their data were used to make a case for marketing commercial products. It is unknowable how many would have agreed to participate if this aim had been explicit in the patient information provided prior to seeking informed consent. Rather than serving the greater good, the original study met the economic interests of the pharmaceutical company.
Restoring Study 329 is both encouraging and deeply dismaying. It is encouraging because it shows that it is possible for a small number of extremely motivated people to identify research misconduct, and make comprehensive suggestions for avoiding future problems of this nature. It is dismaying because the ground has shifted in terms of undermining, perhaps terminally, the profession’s and the community’s confidence in the published findings of existing clinical research. And overcoming this seems unsurmountable given the resources required to do this kind of reanalysis, even if investigators (both commercially and publicly funded) open up their data.
However, a phoenix might rise from these ashes. The restoration of study 329 could be a trigger to demand that open access to protocols and data become the new norm. Human research ethics committees (institutional review boards) could demand open access as a requirement for ethics approval. Such a proposal might restore confidence in the clinical research enterprise, and thereby ensure that when patients enrol in research for the greater good, there is some chance of that good coming to pass.
- National Health and Medical Research Council. (2007; updates May 2015) National Statement on Ethical Conduct in Human Research. Available from: https://www.nhmrc.gov.au/guidelines-publications/e72
Competing interests: I have previously published with Jon Jureidini and Melissa Raven and have an ongoing project with them. I am on the Advisory Board of the Critical and Ethical Mental Health research cluster in the Robinson Institute at the University of Adelaide, where Prof Jureidini and Dr Raven are based
17 September 2015
Wendy Rogers
Academic; Professor of Clinical Ethics
Macquarie University
Department of Philosophy,
Macquarie University, Sydney