Important study facts and limitations often missing in media reports about medical research
News stories about medical research often omit basic facts about the study and fail to highlight important limitations, senior physicians warned in the latest issue of the Medical Journal of Australia. Such omissions can mislead the public, they said.
Dr Lisa Schwartz and Dr Steven Woloshin, both Associate Professors of Medicine at Dartmouth Medical School (New Hampshire and the VA Outcomes Group of Vermont), studied media coverage of research presented at scientific meetings.
"Scientific meetings are an important forum for researchers to exchange ideas and present work in progress. But much of the work presented is not ready for public consumption," says Dr Schwartz.
"The studies have undergone limited andfindings may change substantially by the time the final report is published in a medical journal. And some meeting presentations are never published at all."
Nonetheless, scientific meeting research receives extensive news media coverage. "Unless journalists are careful to provide basic study facts and highlight limitations the public may be mislead about the meaning, importance and validity of the research", says Dr. Woloshin.
For their study, three physicians with clinical epidemiology training analysed newspaper, TV and radio stories that appeared in the US for research reports from five major scientific meetings in 2002 and 2003 to see if basic study facts (eg., size, design) were reported; whether cautions about inherent study weaknesses were noted; and if the news stories were clear about the preliminary stage of the research.
The researchers found that basic study facts were often missing. For example, a third of reports failed to mention study size; 40% did not quantify the main result at all.
Important study limitations were often missing. For example, only 6% (1/17) of the news stories about animal studies noted that results might not apply to humans. And only 2 of 175 stories about unpublished studies noted that the study was unpublished.
Dr Schwartz says that reporters can and should do better. "Ideally this effort would begin with improving the media's sources. Press releases issued should routinely include balanced data presentations and study cautions."
Speaking about their own study, Drs. Schwartz and Woloshin point out 2 limitations of their own study, "First, we only examined 5 meetings. Second, as with any content analysis, some subjectivity is inherent in how we coded news stories."
The Medical Journal of Australia is a publication of the Australian Medical Association.