Scientific Siloes

In a series of papers we published over the past decade, we explored a question that seemed to us interesting: Why were there dramatically different schools of scientific thought about whether reducing population-level salt intake would result in fewer cardiovascular complications? 

The science has long been settled that for people with high blood pressure, salt reduction can be part of the risk reduction armamentarium. However, two competing National Academies of Science, Engineering, and Medicine reports came to rather different conclusions, highlighting the lack of consensus in the field about the utility of reducing salt for everyone. While one report noted that this was an open question for decades, the other report said that “For 40 years we have known about the relationship between sodium and the development of…life threatening diseases.” This leaves consumers confused and policymakers without direction.

What exactly was going on? In part, the answer, we suggested, was in the nature of science itself. In a meta-knowledge analysis of the literature, we showed that different scientific groups often did not include other scientists’ work as background to their own. They came to different conclusions because they looked away from whole segments of published information.

The salt saga illuminates one of the challenges that sometimes undermines the very heart of the scientific enterprise: How do we make sure that scientists include and critically appraise all work in the field? Perhaps it is easier to ask this question in the negative: Why do scientists sometimes not build on all we know to the end of advancing the field? Three principal forces come into play here.

[A]s the number of scientific papers continues to grow, keeping up becomes more, not less, of a challenge.

First, scientists, like all humans, develop their own frames of thought, seeing the world through a particular lens. It is indeed difficult to shake that lens and to embrace a perspective that is contrary to what you have been holding for a number of years. Tying it to the anchor illustration here, if you have been publishing evidence that population salt reduction matters, how willing are you going to be to challenge this belief as other evidence comes along? Will our ego as scientists even allow that?

Second, individual scientists do not work in a vacuum. We all work within mutually reinforcing systems. We publish science that journal editors want to publish, and we often are invited to conferences to present work that conforms with how the conference organizers think. The imperatives of funding bodies reinforce these orthodoxies, as organizations finance work that they see as aligned with their agendas. It is quite difficult for any of us to embrace idiosyncratically divergent perspectives that threaten our normative context, and it takes some doing for that context to change.

Finally, the universe of science continues to expand. That is all to the good, but it also means that scientists need to know and work harder to synthesize into their work the evolving state of the field, and, most importantly, to think critically about how this informs what they do. In other words, it is hard for scientists to keep up. Making this point elegantly, a recent altmetric analysis of user categories found that more than 50% of readers of scientific papers were doctoral students; professors accounted for less than 5% of paper readership. That of course does not mean that research groups, including doctoral students, could not be very well-informed, but it does hint at how difficult it is to stay abreast of rapidly evolving fields.

It is not particularly clear that there is an easy solution to any of this. But as the number of scientific papers continues to grow, keeping up becomes more, not less, of a challenge. Perhaps, as with many of the subjects of these Observing Science essays, a first step towards solutions is self-awareness. A second step would be acting on that self-awareness.

Previously in Observing Science: Challenging the Challenges to Science