Put Down That Coffee!“Although the statistical association does not provethat coffee causes cancer, Dr. Brian MacMahon ofHarvard, leader of the research group, said he stoppeddrinking coffee a few months ago when the results ofthe study became clear. In a telephone interview, hesaid that he would not presume to advise others.”STUDY LINKS COFFEE USE TO PANCREAS CANCERNew York Times, March 12, 1981
Or Get A RefillThis otherwise excellent paper may be flawed in onecritical way, said a letter from Dr. Steven Shedlofsky ofthe Veterans Administration Hospital in White RiverJunction, Vt. He questioned the comparison ofpancreatic cancer patients with persons hospitalizedfor noncancerous diseases of the digestive system.CRITICS SAY COFFEE STUDY WAS FLAWEDNew York Times, June 30, 1981
Or Get A Refill“Such patients, he noted, might be expected to give upcoffee drinking because of their illness. This, he argued,would tilt the proportion of coffee drinkers away fromthe control group who were being compared withthe cancer patients. Amplifying the letter in aninterview, Dr. Shedlofsky said many patients withdigestive diseases give up coffee because they believeit aggravates their discomfort, and others do sobecause their doctors have advised them to.CRITICS SAY COFFEE STUDY WAS FLAWEDNew York Times, June 30, 1981
We Cured Cancer 15 Years Ago“Within a year, if all goes well, the first cancer patient willbe injected with two new drugs that can eradicate anytype of cancer, with no obvious side effects and no drugresistance -- in mice.”…Judah is going to cure cancer in two years, said Dr.James D. Watson, a Nobel laureate who directs the ColdSpring Harbor Laboratory, a cancer research center onLong Island. Dr. Watson said Dr. Folkman would beremembered along with scientists like Charles Darwin assomeone who permanently altered civilization.”The New York Times, May 3, 1998
Or Maybe We Didn’t. Here’s Why.The New York Times,February 11, 2013
Or Maybe We Didn’t. Here’s Why.“The study’s findings do not mean that mice are uselessmodels for all human diseases. But, its authors said,they do raise troubling questions about diseases likethe ones in the study that involve the immune system,including cancer and heart disease.”The New York Times, February 11, 2013
How Are The Media Doing?Schwitzer G. How do U.S. journalists cover treatments, tests, products,and procedures? An evaluation of 500 stories. PLoS Medicine 2008doi:10.1371/journal.pmed.0050095
207 stories• 83 (40%) did not report benefits quantitatively• 124 did, but- 103 (83%) reported relative benefits only,- 3 (2%) absolute benefits only,- 18 (15%) both absolute and relative benefits• 98 (47%) mentioned potential harm to patients• 63 (30%) mentioned costs• 170 stories cited an expert or a scientific study- 85 (50%) cited at least one source with disclosed financial ties- 33 (39%) disclosed these tiesMoynihan R et al. Coverage by the news media of thebenefits and risks of medications. N Engl J Med 2000;342:1645-1650How Are The Media Doing?
193 articles reporting at leastone benefit or harm of a drug100% mentioned at least one benefit132 (68%) did not mentionside effects or harms119 (62%) did not quantify benefits or harms– Of 510 mentions of benefits and harms, only 120 (24%) gave quantitativeinformation– In 26% (31/120) the magnitude was presented in relative terms37(19%) articles reported only surrogate benefits7 (4%) mentioned contraindications61 (32%) mentioned drug costs,89 (46%) mentioned drug alternatives30 (16%) mentioned nondrug options (such as exerciseor diet)Cassels A. Drugs in the news: an analysis of Canadiannewspaper coverage of new prescription drugs. CMAJ,April 29, 2003; 168 (9)How Are The Media Doing?
193 articles reporting at leastone benefit or harm of a drug120 (62%) quotedat least one interviewee.After exclusion of industry and governmentspokespeople, potentialfinancial conflicts of interest were reported for only 5 of 164 interviewees(3%)Of 57 articles covering studies, only 15 (26%) includedinformation onstudy fundingCassels A. Drugs in the news: an analysis of Canadiannewspaper coverage of new prescription drugs. CMAJ,April 29, 2003; 168 (9)How Are The Media Doing?
Wells: “Newspapers over-represent support for screeningmammography for ages 40 to 49”Reports would have been improved by• Identification of all sources for information cited• Less reliance on relatively few sources• Discussion of benefits in absolute termsMedical journalism may need standards similar to those used forreporting medical researchWells J. Newspaper reporting of screeningmammography. Ann Intern Med 2001;35:1029-1037.How Are The Media Doing?
FDA message: Use of pediatric antidepressants is linked to arisk of suicidality (as opposed to suicide itself)• Reported correctly in the vast majorityof news stories, BUT• Other key health messages in FDA warningoften missing• News stories morelikely to include anecdotes of children harmedversus childrenhelped by antidepressants• Quoted experts more likely to emphasize benefits over risksCoverage grew increasingly neutralover time, conveyingneither theimpression that the risks outweighed the benefits nor that benefitsoutweighed the risksBarry CL, Busch SH. News coverage of FDAwarnings on pediatric antidepressant use andsuicidality. Pediatrics 2010; 125:88-95.How Are The Media Doing?
In a national survey of U.S. health and medical journalists:• Nearly 70% had at least a bachelor’s degree• 19% reported having a master’s degree;• 4.5% had a doctorate; about 3% were M.D.s• Almost half had a degree in journalism• 13% had a degree in communications• 8% were ‘‘life sciences’’ majorsViswanath K et al: Occupational practices and the makingof health news: A national survey of U.S. health andmedical science journalists. Journal of HealthCommunication 2008; 13:759–777.Why Is It So Bad?
The ProblemFewer reporters are doing more stories, broadcasts, andblog posts• Sites chasing a smaller number of advertising dollars• Pressure to cover more and more, which places heavyreliance on journals and meetings
The ProblemMany health reporters feel it’s hard to findindependent experts willing to assist journalistsThey think editors need education in criticalappraisal of medical newsLarrson A. Medical messages in the media--barriers andsolutions to improving medical journalism. HealthExpectations 2003;6:323-31.
The ProblemBarriers to improving medical journalism• Lack of time, space and knowledge (the most commonobstacles)• Competition for space and audience• Difficulties with terminology• Problems finding and using sources• Problems with editors and commercialismLarrson A. Medical messages in the media--barriers andsolutions to improving medical journalism. HealthExpectations 2003;6:323-31.
But It’s Not Just JournalistsAcademic medical centers issue a mean of 49 press releases/yearAmong 200 randomly selected releases– 87 (44%) promoted animal or laboratory research, of which 64(74%) explicitly claimed relevance to human health– Among 95 releases about clinical research, 22 (23%) omittedstudy size and 32 (34%) failed to quantify results– 113 releases promoted human research• 17% promoted randomized trials or meta-analyses• 40% reported on uncontrolled interventions, small samples(<30 participants), surrogate primary outcomes, orunpublished data—yet 58% lacked the relevant cautionsWoloshin S et al. Press releases by academicmedical centers: not so academic? Ann InternMed 2009;150:613-618
Positive Publication Bias“The overall frequency of positive supportshas grown by over 22% between 1990 and2007, with significant differences betweendisciplines and countries.”“…the strongest increase in positive resultswas observed in disciplines—like ClinicalMedicine, Pharmacology & Toxicology,Molecular Biology”Fanelli, Scientometrics 2012.
How Often Are Studies Wrong?Ioannidis JPA. PLoS Med 2005; 2(8): e124