The classic model of scientific progress is that the field advances when new findings contradict or supersede old ones. But a new study reveals that this process isn’t working today—at least, not in scientific journals, where most data is shared with colleagues. Indeed, the researchers found that “rebuttals scarcely alter scientific perceptions about the original papers.”

For the study, a group of researchers looked at the citation rates on seven marine biology papers about fisheries. Citation rates are often used as a proxy for the “importance” of a scientific paper, with the notion that the more a paper is cited, the more influential it is. Each paper had been the subject of a rebuttal, also published in a scientific journal. The researchers wanted to know whether these rebuttals affected citation levels on the original papers—and, perhaps more importantly, whether they convinced people to question the interpretation of data in the original papers.

It turns out that rebuttals don’t seem to affect the scientific community’s understanding of the original papers in any way. “The original articles were cited 17 times more frequently than the rebuttals, an order of magnitude difference that overwhelms other factors,” write the study authors in Ecosphere. “Our test score results emphasize that rebuttals have little influence: even the rare few authors who happened upon the rebuttals were influenced only enough to move from whole-hearted support of the original article (a score of five) to neutrality (a score of three), despite the fact that all of the rebuttals argue that the interpretations of data in the originals were incorrect. Astonishingly, 8 percent of the papers that cited a rebuttal actually suggested that the rebuttal supported the claims of the original article, an observation which may give pause to those contemplating writing a rebuttal in the future.”