Drug Discovery Collapse: How to Fix Irreproducible Research?

Drug discovery, ever a challenge, can be likened to a tall structure. The upper stories rely on the stability of the floors below it, and a solid foundation is a must. No lofty heights will be reached when the base is not secure.

And so, it is not a surprise that a drug discovery project will collapse if its own base is, well, baseless. Unfortunately, this is not just a hypothetical scenario: There is an increasing realization that many published early stage investigations cannot be repeated and causing the rise of irreproducible research. In one study that underscores the extent of the problem, scientists at Amgen attempted to reproduce the findings of fifty-three “landmark” articles – and only succeeded with six.¹

What gives? Although it happens too often that a so-called research “finding” in fact stemmed from fraud, it is more often the case that irreproducible research results are caused by rushed, less-than-careful work by scientists under extreme pressure to publish, particularly due to funding and job security concerns. With a weak job market and tightened federal budgets, the pressure has been especially intense. Wait too long to announce results, and another group might reach the finish line first and “scoop” a project. In such an environment, researchers may pay less heed to internal doubts that call for further experiments. As Richard Feynman said, “The first principle is that you must not fool yourself, and you are the easiest person to fool.”

And even when an experiment is performed meticulously, crucial details might be left out of the manuscript. An experiment might only work in a narrow range of temperatures, for example, and others trying to repeat the experiment could flounder without this information. The writers of the original report may feel that there is not enough room in an article to list all the details, or, more nefariously, they don’t want competitors to learn the secrets to their success, and thus continuing the chain of irreproducible research.

Alarm bells are ringing. Nature – one of those top-tier journals that scientists break their backs to get into – wrote about the problem of irreproducible research in a series of articles.² Many concerned writers have weighed in about what to do. Although there are disagreements, one common theme emerges: transparency.

Many in the scientific community feel that journals should insist that authors provide every relevant experimental detail in a manuscript. A slight detail can mean a world of difference. Both positive and negative data should be reported, rather than trying to present a finding in the most exciting way possible. Some in the community have even suggested that at least part of the peer review process should be open to the readership. With questionable studies making it through peer review, it has been wondered if the quality of peer review suffers from its secrecy.

People in a position to award grants, tenure, promotions, and job offers can help by looking beyond an applicant’s number of publications in high-impact journals. What is the real quality of the research? After all, irreproducible research is not quality research. And those performing the research should remember that they are less likely to avoid embarrassing corrections and retractions if their experiments are well-rounded. Dedicated service providers through Scientist are available to assist. Irreproducibility in research has many causes and no simple answer, but making repairs is imperative. Drug discovery depends on it.