Peer review is intended to act as a gatekeeper in science. If working researchers deem a paper fit to be published, it should mean that the research is sound, rigorous, and accurate. But an experimental analysis of peer review suggests that it might also end up rejecting high-quality material. The analysis points to high levels of competition as the source of the problem.

Because peer review is a vastly complex system that can function quite differently in various disciplines, researchers Stefano Balietti, Robert L. Goldstone, and Dirk Helbing constructed an experimental game designed to mimic some of the primary features of peer review. Participants were divided into 16 groups of nine people each and tasked with creating a piece of “art” on a computer interface. The pieces could then be submitted to one of three “art exhibitions.”

Each participant was then given three pieces of other people’s art to review; pieces that averaged a score higher than five out of ten were accepted into the exhibition. Each group played 30 rounds of the game.