Scientific ‘facts’ could be false
When is a scientific result true or false? Experiments always have a certain proportion of positive results and negative results, but scientific journals prefer to publish the positive results and new research from the Niels Bohr Institute shows that this can lead to false claims ending up being regarded as true facts. The results are published in the scientific journal, eLife.
Silas Boye Nissen is a PhD student in biophysics at the Niels Bohr Institute. (Credit: Ola Jakup Joensen)
Science is a process where researchers observe, experiment and develop theories to obtain a basic understanding of the natural world. When researchers examine a hypothesis, they perform a series of experiments. The hypothesis could be, for example, that a particular protein sits on a cell’s RNA molecule and cuts the RNA strand into smaller pieces. The fragments of RNA can then settle on other RNA molecules and this may interfere with the chemical processes of the cell and trigger disease. Researchers can then step by step carry out a series of experiments to test the hypothesis and the results can be published in scientific journals.
The likelihood of a hypothesis being true or false moves up and down a ‘ladder’ as more and more experiments on the hypothesis are published. The model shows that you have to publish a certain percentage of negative results (often 20-30 %) in order to ensure that hypotheses that are false do not end up being regarded as facts. (Credit: Nissen, NBI and Bergstrom, UW)
However, experiments are rarely 100 percent unambiguous. Some experiments show that the hypothesis is most likely true, while others show that it is most likely false and when research results are published, it is assumed that a certain percentage may be so-called false positive or false negative results.
Prefer positive results
In an earlier analysis, a group of researchers found that out of 74 studies of 12 antidepressants, 37 out of 38 submitted positive results were published, while only 3 of the 24 submitted negative studies were published. In addition, 8 submitted negative results were reformulated as positive for publication purposes. Studies with a negative result were thus only published at a tenth of a rate as the positive.
“In order to study the effect of the publication tendency (bias), we have used computer calculations to create a model for how a hypothesis can be perceived as true or false after repeated experiments based on previously published scientific articles on the hypothesis,” explains Silas Boye Nissen, PhD student in biophysics at the Niels Bohr Institute at the University of Copenhagen.
The probability that a hypothesis is true or false moves up and down a ‘ladder’ as more and more experiments relating to the hypothesis are published.
Necessary negative results
The figure shows a false negative rate of 0.2 (which is the standard that many fields of science use). The standard for the false positive rate is 0.05 (the white/yellow line), but there are indications that it is higher in practice. The figure shows that if you publish less than 20-30 percent negative results, there will be a high probability that a false hypothesis will be perceived as being a fact after repeated experiments and published articles. The scientific community only avoid this if they publish a greater proportion of negative results. (Credit: Nissen, NBI and Bergstrom, UW)
“Our model shows that if not enough negative results are published then hypothesis that are actually false could be perceived as being true facts. If, however, you increase the rate (often in the order of 20-30 percent) at which negative results are published, it would be easier to distinguish false hypothesis from true facts,” says Silas Boye Nissen, who made the calculations together with researchers at the University of Washington in Seattle, USA.
They conclude that if the scientific journals largely prefer to publish results that are positive and reject the more negative, it may lead to a growing belief that the hypothesis is true every time a positive result is published and eventually it will be declared as a fact. As the calculations show, the scientific journals have to publish negative results at a certain rate in order to ensure that the hypotheses that are false do not end up being regarded as true facts.
However, Silas Boye Nissen says: “Science remains our best way to understand the physical world, and well-studied topics such as the climate conflict and safety vaccines are far beyond the scope of our model”.
Silas Boye Nissen, PhD student in the Biophysics Complexity groups, StemPhys and Uni-Bio Lab at the Niels Bohr Institute, University of Copenhagen, +45 2186-2025, silas@nbi.ku.dk