Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Imp Act Presentation


Published on

Published in: Health & Medicine, Business
  • Be the first to comment

Imp Act Presentation

  1. 1. Hiding in plain sight: Lessons from landmark clinical trials Steve Goodman, MD, PhD
  2. 2. This talk is brought to you by…
  3. 3. Project ImpACT: a little history <ul><li>Started in 2004 </li></ul><ul><li>PIs. S. Goodman, H. Marks </li></ul><ul><li>Coordinator: Karen Robinson </li></ul><ul><li>Research Assistants: >20 </li></ul><ul><li>Goal: To find (the most) “important” RCTs conducted in all of medicine and public health, 1948-Present. </li></ul>
  4. 4. Purposes <ul><li>To find and profile in depth the 100 “most important” RCTs in medicine and public health since 1948 (UK Streptomycin trial). </li></ul><ul><li>To create a resource of important trials to provide examples of the critical importance that RCTs have played in medicine and public health. </li></ul><ul><li>To learn from the history of RCTs lessons that have been forgotten or restricted to specialty fields of medicine, and to enhance cross-specialty learning. </li></ul><ul><li>Help establish CCT as an international center for the study of the history and methodology of clinical trials. </li></ul>
  5. 5. Dimensions of “importance” <ul><li>Trials that affect.... </li></ul><ul><li>Practice of medicine or public health </li></ul><ul><li>Methods of designing, conducting or analyzing trials </li></ul><ul><li>Course of subsequent research </li></ul><ul><li>Understanding of pathophysiology and biology </li></ul><ul><li>Regulations, law or policy </li></ul><ul><li>Trial ethics </li></ul>
  6. 6. ImpACT activities <ul><li>Field research </li></ul><ul><ul><li>Finding important trials within specific fields of medicine/PH </li></ul></ul><ul><ul><li>Soliciting nominations </li></ul></ul><ul><ul><li>Vetting nominations for profiles </li></ul></ul><ul><li>Profiling trials </li></ul><ul><ul><li>Structured 3-6 page summary of RCT CTSeminar/14_ECMO/ECMO_14/Profile_14_ECM0_13Sep06.doc </li></ul></ul><ul><li>Selecting ImpACT trials </li></ul><ul><ul><li>Conference with members of advisory board and international field experts to discuss and select important trials in that specialty </li></ul></ul><ul><li>In-depth “chapters” on selected ImpACT trials </li></ul>
  7. 7. ImpACT fields <ul><li>Oncology </li></ul><ul><li>Cardiology </li></ul><ul><li>GI </li></ul><ul><li>Neurology </li></ul><ul><li>Endocrine/metab </li></ul><ul><li>ID </li></ul><ul><li>Pulmonary </li></ul><ul><li>Hematology </li></ul><ul><li>Allergy/Immunology </li></ul><ul><li>Dermatology </li></ul><ul><li>Ophthalmology* </li></ul><ul><li>Ob/Gyn* </li></ul><ul><li>Neo/perinatology* </li></ul><ul><li>Pediatrics </li></ul><ul><li>Psychiatry </li></ul><ul><li>Surgery </li></ul><ul><li>Anesth./Crit. Care </li></ul><ul><li>Radiology </li></ul><ul><li>Health services </li></ul><ul><li>Public Health </li></ul>
  8. 8. ImpACT Website <ul><li> </li></ul><ul><li>Ca 180 trials profiled. </li></ul><ul><li>Ca 20 trials selected as ImpACT Landmark trials. </li></ul><ul><li>Last year SCT/ImpACT Trial of the Year award given for the first time. </li></ul>
  9. 9. Trials for today <ul><li>Polio (1954) </li></ul><ul><ul><li>Salk vaccine trial </li></ul></ul><ul><li>ECMO (1986) </li></ul><ul><ul><li>Extra-corporeal membrane oxygenation for neonatal pulmonary hypterension </li></ul></ul><ul><li>WHI (2003) </li></ul><ul><ul><li>Post-menopausal estrogen therapy for reduction of CAD risk </li></ul></ul><ul><li>NSABP-06 (1982) </li></ul><ul><ul><li>Lumpectomy vs. Mastectomy trial for early stage breast cancer </li></ul></ul>
  10. 10. Trials I would love to talk about <ul><li>Retrolental fibroplasia collaborative trial (1952) </li></ul><ul><li>Neonatal hypothermia (1959) </li></ul><ul><li>Coronary Drug Project (1966) </li></ul><ul><li>MR FIT (Multiple Risk Factor Intervention Trial) (1972) </li></ul><ul><li>UGDP (1974) </li></ul><ul><li>CRASH Trial (1992) </li></ul><ul><li>EC/IC (Extra-cranial/Intra-cranial bypass) </li></ul><ul><li>Macular Photocoagulation Study (1991) </li></ul><ul><li>Parkinson’s stem cell trial w/sham surgery. </li></ul><ul><li>NETT (National Emphysema Treatment Trial) (2004) </li></ul>
  11. 12. Background <ul><li>Poliomyelitis is a highly seasonal viral disease. 95% infections are asymptomatic. Small percentage cause permanent paralysis. </li></ul><ul><li>Polio peaked in 1952 with 58,000 cases (~37 per 100,000). Hardest struck were middle and upper class children. </li></ul><ul><li>The National Foundation for Infantile Paralysis (NFIP), or March of Dimes, mounted public donation campaign to raise funds to develop an effective vaccine. </li></ul><ul><li>Small field trial of the formalin-inactivated Salk vaccine (N=700) showed it to be safe and potent. </li></ul>
  12. 13. Salk vaccine trial timeline <ul><li>January, 1953 : NFIP Immunization committee considers Salk preliminary data and whether trial is needed. Salk wants to do more development and safety studies </li></ul><ul><li>May 1953 : O’Connor pushes trials and convenes committee w/o virologists to decide on design. Intense discussion of need for randomization, and nature of control. O’Connor and Salk opposed randomization, key scientists (Joseph Bell) supportive. Observed control preferred. </li></ul><ul><li>June 1953 : Tommy Francis, PhD, U Mich, chosen to direct trial. Insists on randomization with saline injected placebo. </li></ul><ul><li>November, 1953 : States contacted re participation. Several object to randomization, others object to observed controls. Negotiation starts with health authorities in every state. </li></ul><ul><li>January 11, 1954 : Final decision on trial design. 11 states agree to randomization, 33 states use observed controls. </li></ul>
  13. 14. Trial logistics <ul><li>Basil O’Connor, head of NFIP </li></ul><ul><li>Tommy Francis, Trial director </li></ul><ul><li>15,000 public schools </li></ul><ul><li>60,000 MDs, nurses, public health officials </li></ul><ul><li>64,000 teachers/principals </li></ul><ul><li>>200,000 lay volunteers </li></ul><ul><li>Trial funding: </li></ul><ul><li>Average public donation </li></ul>$7 Million <ul><ul><li>$0.27 </li></ul></ul>
  14. 15. Debate over randomization <ul><li>Observational design supported by Basil O’Connor, head of NFIP. Randomization pushed by some scientists, vehemently opposed by Salk: </li></ul><ul><ul><ul><li>“ . . .if we are aware of the fact that the presence of antibody is effective in preventing the experimental disease in animals and in man, then what moral justification can there be for intentionally injecting children with salt solution or some other placebo for the purpose of determining whether or not a procedure that produces antibody formation is effective. . .I would feel that every child who is injected with a placebo and becomes paralyzed will do so at my hands. . .The use of a placebo control, I am afraid, is a fetish of orthodoxy and would serve to create a ‘beautiful epidemiologic’ experiment but would make the humanitarian shudder and would make Hippocrates turn over in his grave”. </li></ul></ul></ul>
  15. 16. Time urgency <ul><li>Basil O’Connor, Fall 1953, during trial design debate: </li></ul><ul><li>“ I have just figured out that during this coming summer thirty or forty thousand children will get polio. About fifteen thousand of them will be paralyzed and more than a thousand will die. If we have the capacity to prevent this, we have a social responsibility here that none of you have been talking about. Let me remind you that we are supported by the people and it is our duty to save lives, no matter how many difficulties may be involved.” </li></ul><ul><li>Tom Rivers, MD, Chair of NFIP Immunization committee: </li></ul><ul><li>“ We all believed that the Foundation had to put on a field trial in the spring of 1954; if it didn’t, the lid would be off and the following year everybody and his aunt would be trying out their own vaccines unless the Public Health Service could put a muzzle on them.” </li></ul><ul><li>. </li></ul>
  16. 17. Public participation <ul><li>Thousands of informational meetings held with health officials, parents and children across the country. </li></ul><ul><li>NYC health commissioner, Leona Baumgartner, recalling talking to children at P.S. 61 in Manhattan’s Lower East Side about the placebo controls: </li></ul><ul><li>“ Those youngsters [gave] as good a description of a controlled experiment as I’ve ever heard. I was very excited, because it seemed to me that if you could teach a generation of kids about what a controlled experiment was, and about what science really was, this was a plus value regardless of whether the vaccine was any good or not.” </li></ul>
  17. 18. Salk vaccine trial timeline <ul><li>March, 1954 : Viral particles found in 4/6 vaccine lots. Winchell broadcasts warning against participation caused withdrawal of Wisconsin and 150K participants. </li></ul><ul><li>April 25, 1954 : Meeting to decide on whether to go forward on trial, with several committee members unable to sleep. </li></ul><ul><li>April 26, 1954 : Innoculations start, ending June 15. </li></ul>
  18. 19. Study Design <ul><li>Two different studies of school children, aged 6-9 years: </li></ul><ul><li>Randomized, double-masked, placebo-controlled trial: (455,474 1 st -3 rd graders from 11 states) </li></ul><ul><li>Observational Study (567,210 participants from 33 states) </li></ul><ul><ul><li>2 nd graders received vaccine </li></ul></ul><ul><ul><li>1 st and 3 rd graders served as controls </li></ul></ul><ul><li>On the parental consent form, the phrase ‘I give my permission’ was changed to ‘I hereby request,’ implying that not every child would be fortunate enough to be picked. </li></ul>
  19. 20. Conduct issues <ul><li>Due to the difficulty of diagnosing nonparalytic cases of polio, clinical evaluations initially performed by local physicians and physical therapists had to be reassessed by an advisory committee in order to standardize clinical definitions . </li></ul><ul><li>Approximately 600K children did not show up, and 200K did not have signed consents. </li></ul><ul><li>The evaluation center was under intense pressure to release the findings of the trial by the next polio season, shortening the timeline available for data analysis. Although the results were released to the public in the spring of 1955, the full study report did not appear for another 2 years. </li></ul>
  20. 21. Timeline, cont. <ul><li>April 12, 1955 : Results announced. Vaccine licensed by HHS 2 hrs later, shipments started 4 hrs hence. </li></ul><ul><li>April 13, 1955 : Nationwide immunization commences with stockpiled vaccine. </li></ul><ul><li>April 26, 1955 : Cutter incident (contaminated vaccine – >200 polio cases, 59 paralytic, 5 deaths) </li></ul>
  21. 22. Results <ul><li>Placebo Control Trial: </li></ul><ul><ul><li>Total cases reported differed significantly between vaccinated and controls (respective attack rates were 41 vs 81, P<0.001). </li></ul></ul><ul><ul><li>Difference in cases of paralytic polio was marked (33 among vaccinated vs 115 among controls (attack rates of 16 vs 57; P<0.001) </li></ul></ul><ul><li>Observational Study: </li></ul><ul><ul><li>Also measured significant difference in incidence of paralytic polio with 38 cases among vaccinated vs 330 cases among controls (attack rates were 17 vs 46; P<0.001). </li></ul></ul><ul><li>P olio incidence in the randomized study was similar to that in the observational study and neither study showed any appreciable difference between the two treatment arms in incidence of nonparalytic polio. </li></ul>MMWR. 41(55):46, Sept 1993, CDC, Atlanta, Georgia. Vaccine Cohort # Participants All Reported Cases Paralytic Polio Nonparalytic Polio # Cases Attack Rate # Cases Attack Rate # Cases Attack Rate Placebo-Controlled Trial Vaccinated 200,745 82 41 33 16 24 12 Placebo 201,229 162 81 115 57 27 13 Observational Control Trial Vaccinated 221,998 76 34 38 17 18 8 Controls 725,173 439 61 330 46 61 8
  22. 23. Impact <ul><li>Vastly accelerated the vaccine development process and undoubtedly saved hundreds of lives and thousands of paralytic cases. </li></ul><ul><li>Enhanced public support for medical research at the time and possibly the future (through the involved children.) </li></ul><ul><li>Improved adherence to vaccine manufacturing protocols and safety testing guidelines. </li></ul><ul><li>New understanding of poliovirus epidemiology and serology helped shape future vaccine studies. </li></ul><ul><li>The role in the Cutter incident of the recently formed EIS at the CDC established its reputation. </li></ul><ul><li>The process by which each community decided the terms of its participation gives new meaning to the term “consent”. </li></ul>
  23. 24. Lessons from the Salk trial (T. Burke, 2004) <ul><li>VERY large trials are possible. </li></ul><ul><li>Committed, effective leadership is essential </li></ul><ul><li>Precise case definitions are important </li></ul><ul><li>Clinical endpoints are key; correlates of immunity don’t suffice </li></ul><ul><li>Serious potential safety problems may not be detected in even a VERY large trial </li></ul><ul><li>A nongovernmental organization can do the job, </li></ul><ul><li>A successful phase III trial is just a beginning. </li></ul><ul><li>Sometimes observational trials must do; you cannot foist randomization on unwilling participants. </li></ul><ul><li>Meaningful public (aka consumer) involvement led to rapid public acceptance of both risks and benefits. </li></ul>
  24. 25. The ECMO controversy When tough cases don’t make good trials
  25. 26. The ECMO Circuit
  26. 27. Background <ul><li>ECMO had not improved survival in adults with acute respiratory failure, but it was hypothesized that it might be more effective in treating neonatal respiratory failure. </li></ul><ul><li>Robert Bartlett began treating critically ill infants with ECMO. Encouraging results (6/16 survived) </li></ul><ul><li>By the 1980s, Bartlett had achieved a 75% survival in patients judged to have a 5% survival when managed by conventional therapy. </li></ul>
  27. 28. Bartlett Article
  28. 29. Bartlett: Play-the-Winner Design ECMO Survived CMT Died 10 ECMO: survived 1 CMT: died
  29. 30. Trial Motivation <ul><li>Increasing demand for more rigorous evaluation of ECMO in neonates: </li></ul><ul><ul><li>Bartlett conducted a trial using a “play-the-winner” design </li></ul></ul><ul><ul><ul><li>1 patient received conventional therapy and died (smallest, most premature, lightest) </li></ul></ul></ul><ul><ul><ul><li>11 patients received ECMO and survived </li></ul></ul></ul><ul><li>Article published in same issue (Wung et al. Pediatrics, 1985) reported 100% survival in a cohort of similar neonates treated with new low intensity ventilation. </li></ul>
  30. 31. The ECMO Controversy “ Children’s Hospital made a serious error .” - Office for Protection from Research Risks, NIH
  31. 32. The Harvard Neonatal ECMO Trial Randomized newborns with PPHN to conventional therapy versus ECMO Conventional Therapy NICU: 7th Floor Neonatologists No patients had ever been offered ECMO Anti-ECMO ECMO PICU: 5th Floor Anesthesiologists & Surgeons Already had experience with ECMO for newborns with CDH Pro-ECMO
  32. 33. The Harvard Neonatal ECMO Trial: Randomized Consent <ul><ul><li>Eligible Newborn </li></ul></ul>RANDOMIZE Do not seek consent Seek consent for ECMO CMT CMT ECMO Yes No
  33. 34. Results <ul><li>Phase I: 19 children enrolled and randomized </li></ul><ul><ul><li>Conventional Medical Therapy Group: 6/10 survived </li></ul></ul><ul><ul><li>ECMO Group: 9/9 survived </li></ul></ul><ul><li>Phase II: 20 patients assigned to the ECMO group </li></ul><ul><ul><li>19/20 survived </li></ul></ul>
  34. 35. Newspaper coverage <ul><li>A Harvard study on newborns draws fire; </li></ul><ul><li>Doctors faulted for limiting life-saving treatment </li></ul><ul><li>August 7, 1989 </li></ul><ul><li>Richard A. Knox, Boston Globe </li></ul><ul><li>A Harvard University study involving mortally ill newborns is being challenged as unethical in a debate that raises important questions about how to do research on promising new therapies. </li></ul>
  35. 36. The Response to the ECMO Trial The hospital IRB “made decisions that rightfully belonged to the parents. They really blew it.” Charles McCarthy, Director of OPRR The NIH Office for Protection from Research Risks (OPRR) reprimanded the hospital: The doctors “were doing exactly what physicians did before we had a doctrine of informed consent - making decisions for parents.” George Annas, Boston University
  36. 37. A Justification
  37. 38. With comment… <ul><li>Donald A. Berry. : Ethics and ECMO </li></ul><ul><li>Robert E. Kass, Joel B. Greenhouse. A Bayesian Perspective. </li></ul><ul><li>Richard Royall. </li></ul><ul><li>Colin B. Begg. </li></ul><ul><li>Peter Armitage, D. Stephen Coad. </li></ul><ul><li>D. Y. Lin, L. J. Wei. </li></ul><ul><li>Richard G. Cornell. </li></ul><ul><li>Janis Hardwick. Recent Progress in Clinical Trial Designs that Adapt for Ethical Purposes. </li></ul><ul><li>James H. Ware. Rejoinder </li></ul>
  38. 39. The Problem of “Randomized Consent” &quot;It's clear to me they did not ask consent because it would be hard to get a control group otherwise. Properly informed parents would say 'No thank you.’ ” - Richard M. Royall, Johns Hopkins School of Public Health “ [Seeking consent from those randomized to the conventional medical treatment] would impose unacceptable psychological burdens on the parents of children not included in the ECMO arm of the study.” - Dr. Peter H. Wolff , Chair of Children’s Hospital IRB
  39. 40. The Problem of Adaptive Randomization &quot;The clear expectation was that more patients would die on conventional therapy. So the question is whether having an excess of deaths balances the worth of information gained . Since I believe such information is available without randomizing, my answer is a resounding no.” - Donald A. Berry, University of Minnesota &quot;The Harvard study was informed by the slightly hysterical view that we must immediately stop a study as soon as we have an idea which treatment might be better. If we're not careful, we will soon have a system in which we can establish nothing. ” - Paul Meier, University of Chicago
  40. 41. Are RCTs the only way to learn? <ul><li>“ The brilliant success of the RCT has now become a form of intellectual tyranny” Freireich </li></ul><ul><li>“ We should not proceed on the fallacious assumption that where there is no randomization, there is no truth.” Royall </li></ul>
  41. 42. Meinert on ECMO (Pediatrics, 1990) <ul><li>I have reservations about any trial that is designed to detect a treatment difference of “miracle” proportions, such as the one postulated in this trial. Generally, use of such a difference for sample size rationale means either that the investigators “shopped” for a set of specifications to yield a desired sample size or that the trial is being done to demonstrate or “prove” the value of a treatment. </li></ul><ul><li>… The desire to shield patients from upsetting discussions by using nontraditional approaches to consent is a natural one, but its pursuit should be reserved for settings in which there is a legitimate state of equipoise. </li></ul>
  42. 43. Chalmers: “A Belated Randomized Trial” <ul><li>“ A plague on all houses except those who reported and conducted [this] study…” </li></ul><ul><li>All new treatments should be administered under a randomized protocol. </li></ul><ul><li>“ IRBs should recognize the ethical and scientific deficiencies of uncontrolled pilot trials and innovative practice. They should demand… that physicians provide evidence for all that they do.” </li></ul><ul><li>“ Professional ethicists should get off the back of people performing good clinical research…In the absence of evidence of efficacy, both the withholding and delivery of a new treatment are equally less ethical than [RCTs].” </li></ul><ul><li>Payers should only pay for innovate therapies under protocol. </li></ul>
  43. 44. Chalmers, continued <ul><li>Trial registration would have to start. (KD cited) </li></ul><ul><li>Data monitoring and peer review boards would have to decide when to stop. </li></ul><ul><li>Electronic communication would have to be devised to keep track of rapid changes. </li></ul><ul><li>A portion of the money spent for care would be spent on experimentation. </li></ul>
  44. 46. Lantos and Fader, NEJM, 1991
  45. 47. NIH Consensus Conference <ul><li>Report of the Workshop on </li></ul><ul><li>Diffusion of ECMO Technology </li></ul><ul><li>Extracorporeal Membrane Oxygenation </li></ul><ul><li>May 31,1990—June 1,1990 </li></ul><ul><li>Rockville, Maryland </li></ul><ul><li>Sponsored by: </li></ul><ul><li>National Institute of Child Health and Human Development, NIH </li></ul><ul><li>Office of Medical Applications of Research, NIH </li></ul><ul><li>National Institute of Neurological Disorders and Stroke, NIH </li></ul><ul><li>National Heart, Lung, and Blood Institute, NIH </li></ul><ul><li>Agency for Health Care Policy and Research </li></ul><ul><li>Federal Drug Administration </li></ul>
  46. 48. NIH Consensus Recommendations <ul><li>Prevention and Therapeutic Alternatives </li></ul><ul><li>Alternative therapies and prevention have not been adequately explored. There is a serious lack of knowledge regarding epidemiologic factors that influence the requirement for ECMO therapy. </li></ul><ul><li>The actual incidence of conditions treated with ECMO.. is not well documented. …and the relative effectiveness of alternatives versus ECMO is unclear. … </li></ul><ul><li>Education and prevention strategies should be developed …. It is not yet known whether the use of ECMO can be reduced through education ….. </li></ul>
  47. 49. Additional research <ul><li>The increasing use of ECMO especially in new patient populations creates an urgent need for further research. Clinical studies are needed to determine when ECMO is the most appropriate treatment alternative and what technical improvements are safe and feasible. The short-term and long-term effects of ECMO on the nervous system, pulmonary system, cardiac system, and the blood in all age groups must be further defined. Quality of life as well as specific biologic parameters should be studied. </li></ul>
  48. 51. The UK Neonatal ECMO Trial <ul><li>1993-1995: 185 neonates randomized to ECMO vs CMT </li></ul><ul><li>Trial stopped early by DSMB, </li></ul><ul><ul><li>ECMO survival 60/93 = 65% </li></ul></ul><ul><ul><li>CMT survival 38/92 = 41%, p<0.0005 </li></ul></ul>
  49. 52. The Problem of Adaptive Randomization <ul><li>Phase I </li></ul><ul><ul><li>Infants assigned to ECMO or conventional medical therapy with 50:50 randomization </li></ul></ul><ul><ul><li>Randomized assignment continues until 4 deaths occur in either treatment group </li></ul></ul><ul><li>Phase II </li></ul><ul><ul><li>Infants assigned to the “winning” treatment group until 4 deaths or 28 successes occur </li></ul></ul><ul><ul><ul><li>Null hypothesis of 80% mortality in both groups </li></ul></ul></ul><ul><ul><ul><li>Alternative hypothesis of 80% mortality in conventional treatment group and reduction to 20% mortality in the ECMO group </li></ul></ul></ul><ul><ul><ul><li>alpha = 0.05, Power= 0.74 </li></ul></ul></ul>
  50. 53. Postscript, 2005 <ul><li>Extracorporeal Life Support: </li></ul><ul><li>History and New Directions </li></ul><ul><li>Robert H. Bartlett, MD </li></ul><ul><li>“ The use of ECLS peaked in 1992 and has been steadily decreasing since that time. The primary reasons are identification and avoidance of ventilator-induced lung injury and the use of inhaled nitric oxide as a pulmonary vaso dilator. In our experience, 40% of the patients who would have required support with ECMO in 1992 are now successfully treated with pressure-controlled (low stretch) ventilation and inhaled nitric oxide using standard or oscillation ventilator techniques.” </li></ul>
  51. 54. Issues raised <ul><li>How to evaluate promising new technology / interventions whose development produces results that disturb equipoise. </li></ul><ul><li>How to create a system – both professional and economic- that rewards evaluation as much as innovation. </li></ul><ul><li>Whether RCTs are necessary for breakthrough technologies. </li></ul><ul><li>How to evaluate interventions with possible large short term benefit and similar long term harm. </li></ul><ul><li>Damage can be done to the RCT enterprise if it is imposed on subjects, or a society, who are unwilling to accept it, or don’t understand it. </li></ul>
  52. 55. Women’s Health Initiative 1993 - 2002
  53. 57. Previous systematic review showed increased risk of cardiovascular events with HRT
  54. 58. Controversies <ul><li>Several top women epidemiologists complained about the size and complexity of the study </li></ul><ul><li>Huge budget ($625 million) was criticized as being unnecessary, that we were certain enough of the answer. </li></ul>
  55. 59. Results <ul><li>HR and 95% CI </li></ul><ul><li>CVD : 1.29 (1.02-1.63) </li></ul><ul><li>Stroke: 1.41 (1.07-1.85) </li></ul><ul><li>VTE: 2.11 (1.58-2.82) </li></ul><ul><li>Invasive breast cancer: 1.26 (1.00-1.59) </li></ul><ul><li>Colorectal cancer: 0.63 (0.43-0.92) </li></ul><ul><li>Hip fractures: 0.66 (0.45-0.98) </li></ul><ul><li>Global index: 1.15 (1.03-1.28) </li></ul>
  56. 60. Impact <ul><li>Results were shocking and extremely surprising </li></ul><ul><li>Significant reduction in initiation as well as a marked discontinuation of HRT </li></ul><ul><li>Reinforced the importance of conducting large scale RCTs </li></ul>
  57. 61. 06/06/09
  58. 62. Endpoints in WHI trial 06/06/09
  59. 63. WHI stopping rule <ul><li>Efficacy </li></ul><ul><li>Test CHD against an O-F upper 2.5%ile </li></ul><ul><li>If benefit for CHD, test Global index (any endpoint) against 5% upper boundary. </li></ul><ul><li>Safety </li></ul><ul><li>Lower 5% O-F Boundary for any major AE. </li></ul><ul><li>AND >1 SE in harm direction for Global index. </li></ul>06/06/09
  60. 64. 06/06/09
  61. 65. 06/06/09
  62. 66. 06/06/09
  63. 67. 06/06/09
  64. 68. 06/06/09
  65. 69. WHI DSMB recommendation 06/06/09
  66. 70. NIH Letter #1 06/06/09
  67. 71. Issues in stopping the ERT trial <ul><li>No stopping guidelines were crossed </li></ul><ul><li>No adverse CHD effect, beneficial (??) breast ca effect. </li></ul><ul><li>Global index showed equal risk, benefit. </li></ul><ul><li>BUT!!! </li></ul><ul><li>Strong evidence for high stroke risk. </li></ul><ul><li>WHIMS trial showed slight (unexpected) increase in dementia in both trials. </li></ul><ul><li>No CHD benefit. </li></ul>06/06/09
  68. 72. DSMB vote: May, 2003 <ul><li>“ Each one of us realized our vote was on a knife edge.” </li></ul><ul><li>5-4 vote to continue. </li></ul><ul><li>Degrees of confidence (0-100) were 45-55. </li></ul><ul><li>NIH convenes ad-hoc panel. 2 members of original DSMB observe as non-voting members. </li></ul><ul><li>July, 2003 : New panel votes to continue. </li></ul><ul><li>November, 2003 : DSMB meets again, augmented w/2 members of ad hoc panel dementia experts. Decide to continue trial w/letter to subjects explaining stroke risk and breast cancer benefit. </li></ul><ul><li>Unable to craft letter. </li></ul><ul><li>Re-vote: 5-4 to continue – different members in majority. </li></ul><ul><li>Recommendation to continue sent to NHLBI. </li></ul><ul><li>NHLBI stops trial on 2/04. </li></ul>
  69. 73. Lessons <ul><li>Importance of board independence. </li></ul><ul><ul><li>NIH and CC both had structural conflicts. </li></ul></ul><ul><ul><li>NIH replaced DSMB members and required permission for additional analyses, both inimical to optimal board function. </li></ul></ul><ul><ul><li>NIH reps sat in DSMB closed sessions. </li></ul></ul><ul><li>Monitor for both benefit and harm regardless of prior. </li></ul><ul><li>Need flexibility for weighting harms when monitoring multiple endpoints in long-term prevention trials. Is obligation to participants different in prevention than treatment trials? </li></ul><ul><li>“ JW adds special thanks to William Harlan who prevailed on her to join the DSMB when she knew that the trial was going to show that hormone replacement therapy reduced coronary disease and therefore that serving on this committee would be boring.” </li></ul>
  70. 74. Comparison of Total Mastectomy and Lumpectomy With or Without Radiation (NSABP B-06) 1976-1984 National Surgical Adjuvant Breast and Bowel Project Bernard Fisher, MD Chairman and Scientific Director, NSABP Carol K. Redmond, ScD Director, NSABP Biostatistics Center
  71. 75. Background <ul><li>Radical mastectomy—removal of the affected breast, underarm lymph nodes and both underlying chest muscles (en bloc dissection)—was the standard of care for breast cancer. </li></ul><ul><li>Two divergent hypotheses existed regarding tumor biology. Most believed that cancer was a local/regional disease spread by the lymph system to distant sites. Others proposed that breast cancer was a systemic disease better treated by systemic therapy. </li></ul><ul><li>During the 1970s, social pressure/advocacy groups were starting to raise awareness of breast cancer, advocating research and promoting less disfiguring surgery. </li></ul><ul><li>The NSABP—a clinical trials cooperative group supported by the NCI—had previously found that patients treated by total mastectomy without axillary-node dissection and pectoral muscle removal were at no higher risk of distant disease or death than those undergoing a Halstead operation (radical mastectomy). </li></ul><ul><li>This prospective, randomized, clinical trial wished to determine: </li></ul><ul><ul><li>The effectiveness of lumpectomy for breast preservation </li></ul></ul><ul><ul><li>The effect of post surgery radiation therapy on ipsilateral breast tumor incidence reduction </li></ul></ul><ul><ul><li>The clinical importance of tumor multicentricity </li></ul></ul>
  72. 76. Study Design <ul><ul><ul><li>In 1976, 2163 women with primary operable breast cancer and with tumors ≤ 4cm were randomized into: </li></ul></ul></ul><ul><ul><ul><ul><li>Mastecomy </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Lumpectomy </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Lumpectomy with radiation </li></ul></ul></ul></ul><ul><ul><li>All patients underwent an axillary lymph node dissection </li></ul></ul><ul><li>Standard randomized treatment assignments (stratified by participating institution) were changed to a Zelen“prerandomization” design during the course of the trial. </li></ul>
  73. 77. Issues <ul><li>Eight percent of women (174/2163) did not accept randomly assigned treatment but were followed. </li></ul><ul><li>Due to low enrollment rates, the standard randomization protocol (stratified by participating institution) was changed to a Zelen design (prerandomization) during the course of the trial. This change was accompanied by a six-fold increase in enrollment rates. This practice provoked considerable ethical debate. </li></ul>
  74. 78. Results <ul><li>At five years, 10% of patients initially treated with lumpectomy were found to have a tumor at the margins of resected specimens and therefore subsequently underwent total mastectomy. The incidence of positive margins was similar in both lumpectomy groups. Recurrences at the margin were positively related to the presence, number of positive nodes, and centrally located tumors. </li></ul><ul><li>Treatment by lumpectomy with or without breast irradiation resulted in disease-free, distant-disease-free and overall survival at five years that was no different than breast removal. </li></ul><ul><li>Distant-disease-free (p not significant) and disease-free survival (p=0.04) were higher in patients undergoing lumpectomy plus radiation as compared to the total mastectomy group. </li></ul><ul><li>Among women treated by lumpectomy, the addition of radiation therapy significantly decreased tumor recurrence (P<0.001) in the ipsilateral breast . The advantage of irradiation therapy was observed in both subjects with negative and positive nodes. </li></ul><ul><li>Coupling breast irradiation with lumpectomy also improved disease-free survival (p=0.02) when compared to lumpectomy alone. However, no significant differences between the two groups were found with respect to distant-disease-free or overall survival. </li></ul><ul><li>The fewest treatment failures were observed in the group receiving lumpectomy + radiation therapy. More than half of the first-reported treatment failures in all three treatment groups were due to distant recurrences. Second cancers and deaths from other causes were distributed equally across treatment groups. </li></ul>
  75. 79. Impact <ul><li>Findings along with B-04 trial repudiated the Halstedian principles of breast cancer treatment and provided support for the theory that breast cancer is a systemic disease. </li></ul><ul><li>Lumpectomy was shown to be an acceptable alternative to mastectomy for women with Stage I and II breast cancer. Radiation therapy was shown to be beneficial in reducing local recurrence of tumor in lumpectomy patients.. </li></ul><ul><li>At St. Luc Hospital, a Dr. Poisson admitted fraud after a routine audit revealed that records had been altered. He stated that he altered records to ease the entry requirements because he felt the treatment provided by the NSABP protocol was beneficial to potential participants in the study. </li></ul>
  76. 80. Consequences of “fraud” <ul><li>Fisher: “All Hell Broke Loose” </li></ul><ul><li>Fisher and Carol Redmond dismissed from NSABP </li></ul><ul><li>2 congressional hearings, one under John Dingell </li></ul><ul><li>New Clinical Trials Monitoring Branch at NCI </li></ul><ul><li>1993 Federal Register notice: Roger Poisson guilty of research misconduct </li></ul><ul><li>Poisson falsified or fabricated 111 separate instances of data in 14 NSABP clinical trials (<0.5% of total). </li></ul><ul><li>Chicago Tribune Front Page Story March 13, 1994: “Fraud in Breast Cancer Study: Doctor Lied on Data for Decade” </li></ul><ul><li>150 articles tagged “scientific misconduct” by NLM </li></ul>
  77. 83. Postscript <ul><li>In 1997, Bernard Fisher received an apology from the University of Pittsburgh, his rank as Distinguished University Service Professor reinstated, NCI and U. Pitt. restrictions lifted, and $2.75 million for wrongful punishment. </li></ul><ul><li>NCI said: &quot;Through his role as the scientific leader of the National Surgical Adjuvant Breast and Bowel Project (NSABP), [Fisher] has not only changed the way breast cancer is treated, but enlightened medical science to view breast cancer as not just a tumor confined to the breast, but as a systemic disease requiring more than surgical intervention,&quot; </li></ul>
  78. 84. Impact <ul><li>This trial raised ethical issues regarding both informed consent, fraud, and the appropriateness of actions against the investigators after the fraud had been detected. Thus, its impact and legacy was broad, affecting the understanding of biology, clinical practice, the ethics of randomization, the impact and handling of research misconduct, and the role of regulatory agencies in clinical trial oversight. </li></ul>
  79. 85. Advisory Board <ul><li>John Bailar III </li></ul><ul><li>David Brown </li></ul><ul><li>Sir Iain Chalmers </li></ul><ul><li>Kay Dickersin </li></ul><ul><li>Benjamin Djulbegovic </li></ul><ul><li>Sir Richard Doll </li></ul><ul><li>Susan Ellenberg </li></ul><ul><li>Norman Fost </li></ul><ul><li>P. J. Devereaux </li></ul><ul><li>Cynthia Mulrow </li></ul><ul><li>Barbara S. Hawkins </li></ul><ul><li>Brian Haynes </li></ul><ul><li>Joseph Lau </li></ul><ul><li>Curtis L. Meinert </li></ul><ul><li>Steven Piantadosi </li></ul><ul><li>David L. Sackett </li></ul><ul><li>Jonathan M. Samet </li></ul><ul><li>Harold Sox </li></ul><ul><li>Sean Tunis </li></ul><ul><li>Jan P. Vandenbroucke </li></ul><ul><li>Janet Wittes </li></ul>
  80. 86. ImpACT Staff Emily Evans Gila Neta* Alison Brown Sara Lowther* Olaide Odelola
  81. 87. ImpACT staff <ul><li>Current </li></ul><ul><li>Allison Brown </li></ul><ul><li>Emily Evans </li></ul><ul><li>Gila Neta </li></ul><ul><li>Olaide Odelola </li></ul><ul><li>Former </li></ul><ul><li>Nyasha Bakare (2006) </li></ul><ul><li>Juliana Cuervo (2007) </li></ul><ul><li>Lara Devgan (2006-2007) </li></ul><ul><li>Ishveena Duggal (2004) </li></ul><ul><li>Julia Gage (2005-2006) </li></ul><ul><li>Emily Henkle (2006) </li></ul><ul><li>Jay Herson (2005) </li></ul><ul><li>David Majure (2006-2007) </li></ul><ul><li>Tram Lam (2005-2006) </li></ul><ul><li>Sara Lowther (2005-2007) </li></ul><ul><li>Nancy Maldeis (2004-2005) </li></ul><ul><li>Todd Meyers (2005) </li></ul><ul><li>Kamal Patel (2006) </li></ul><ul><li>Sarah Post (2003) </li></ul><ul><li>Samara Rifkin (2006) </li></ul><ul><li>Jordana Rothschild (2004) </li></ul><ul><li>Lauren Singer (2004) </li></ul><ul><li>Ellen Wasserman (2003-2005) </li></ul><ul><li>Lashawn Worsley (2005) </li></ul>
  82. 88. Project leaders Harry Marks Karen Robinson
  83. 89. Final thoughts on the need for RCTs <ul><li>The fatal tendency of mankind to leave off thinking about a thing when it is no longer doubtful is the cause of half their errors. </li></ul><ul><ul><ul><li>John Stuart Mill ( On Liberty ,1859) </li></ul></ul></ul><ul><li>In theory there is no difference between theory and practice. In practice there is. </li></ul><ul><ul><ul><li>Yogi Berra </li></ul></ul></ul>