Top Researcher Finds Medical Studies to be Largely Wrong or Fraudulent

 Top Researcher Finds Medical Studies to be Largely Wrong or Fraudulent

by Tony Isaacs

Considering his background, Dr. John Ioannidis had good reason to expect that he might become a noted and respected researcher when he first entered the field of medical research. What he did not expect was that he would become known for challenging and exposing the bad science of his peers and finding that up to 80 percent of medical studies results are either wrong or fraudulent.

Ioannidis was unusually well prepared to enter medical research: he had been a math prodigy of near-celebrity status in high school and both of his parents were physician-researchers. He believed he would be able to follow his parents footsteps and use math to better support findings in a surprisingly sloppy field. "I assumed that everything we physicians did was basically right, but now I was going to help verify it," he said. "All we'd have to do was systematically review the evidence, trust what it told us, and then everything would be perfect."

It didn't turn out that way. When he pored over medical journals, Ioannidis was struck by how many findings of all types were later refuted and he was shocked at the range and reach of the reversals in everyday medical research.

Randomized controlled trials, which compare how one group responds to a treatment against how an identical group without the treatment fares, "had long been considered nearly unshakable" said Ioannidis. But they too ended up sometimes being wrong. "I realized even our gold-standard research had a lot of problems."

After working at Harvard, Tufts University, Johns Hopkins University and the National Institutes of Health, Ioannidis set up a base at the University of Ioannina in Greece. His team began producing a series of papers that pointed out specific ways certain studies were getting misleading results. In 2005 he published a paper which shook the foundations of medical research in the journal PLoS Medicine.

In the paper, Ioannidis laid out a detailed mathematical proof that, assuming modest levels of researcher bias, typically imperfect research techniques, and the tendency to focus on exciting rather than plausible theories, researchers will come up with wrong findings most of the time. His model, based on the rates in which studies had been overturned, predicted that 80 percent of non-randomized studies (by far the most common type), 25 percent of so-called gold-standard randomized trials, and as much as 10 percent of the platinum-standard large randomized trials turned out to be wrong.

The paper detailed how researchers were frequently manipulating data analyses and chasing career-advancing findings rather than good science, and even using the peer-review process to suppress opposing views. "The studies were biased," said Ioannidis. "Sometimes they were overtly biased. Sometimes it was difficult to see the bias, but it was there. At every step in the process, there is room to distort results, a way to make a stronger claim or to select what is going to be concluded. There is an intellectual conflict of interest that pressures researchers to find whatever it is that is most likely to get them funded."

Ioannidis noted that, in addition to the factors which doomed nutritional studies, drug studies had the additional corruptive force of financial conflict of interest - much to the detriment of doctors and patients. "Doctors need to rely on instinct and judgment to make choices," he said. "But these choices should be as informed as possible by the evidence," said Ioannidis.

He also noted that, "I'm not sure that more than a very small percentage of medical research is ever likely to lead to major improvements in clinical outcomes and quality of life."

Sources included:

http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/
http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124
http://jama.ama-assn.org/cgi/content/abstract/294/2/218