Fake news flourish abundantly in science. Interest groups may distort research results in the media so they apparently support their attitude, and scientists can exaggerate the potential applications of their research to stir public interest that might lead to more founding money. Knowing the pitfalls where science news become problematic is essential when you get an idea for a story or evaluate your sources. The EUSJA journalistic team has made this step-by-step guide to help you assess whether a news story is scientifically substantiated or not and whether the results are worth believing.
By Gorm Palmgren, freelance science journalist writing for EUSJA’s NUCLEUS project
1) DOES IT SOUND TOO GOOD TO BE TRUE?
A headline should always be exciting and make the reader want to dive into the text, but if it sounds unrealistic, it should raise a red flag. Perhaps the author of the article does not understand what he writes about, or she may be trying to manipulate you to have a particular attitude. President Trumps infamous tweet “The concept of global warming was created by and for the Chinese in order to make U.S. manufacturing non-competitive” is a very bold statement, which really has to be backed by solid arguments. The tweet goes unaccompanied, so there is good reason to be sceptical.
2) DOES THE SENDER HAVE A PERSONAL INTEREST?
Although media, interest groups and businesses can be dead serious, they often have a strong interest in what they are writing about. It can affect their arguments and conclusions or it might influence what they choose to talk about. If the oil and coal industry is behind a story that calls off global warming, there is good reason to be sceptical. But if the same news comes from Greenpeace, the supporting evidence is probably paramount since it made environmentalist group change their opinion.
3) IS THE ARTICLE BASED ON OVERCONFIDENT POSTULATES?
Research results are never completely clear, but can be explained in many ways. Therefore, you should always be critical of results and conclusions that seem arrogant in their postulates. If the article itself is very skewed and does not question its own claims, then the content is all the more difficult to rely on. A claim like “The results show, that the new drug has no side effects” can by definition not be true and should make you question the validity of the whole article.
4) ARE THE ARGUMENTS BASED ON SOLID GROUND?
When researchers are to analyze their own test results, they often use statistics, and it usually only makes sense if there is a large dataset behind. If a new herbal drug against the flu is tested on five persons and all get cured within a week, the researchers – or the journalist reporting about the study – might be tempted to claim “100 percent efficiency”. But it would be more fair if they kept silent and repeated the experiment with much more patients as well as healthy controls.
5) ARE THE STATEMENTS CHALLENGED?
An article, based solely on a single research result or source, is completely unchallenged and can easily resemble a promotion of a particular point of view. It is always good if the claims are held up against other researchers’ comments which might put them in a critical light, or if the results are compared to other research that may reach different conclusions. A well balanced source allow you to better make your own opinion or just realize that the truth is complex.
6) DOES THE ARTICLE DESCRIBE THE SCIENTIFIC METHODS?
If the article describes how researchers have achieved their results, it is much easier to judge whether the results are trustworthy or just hot air. If the researchers’ methods are not mentioned, it is very difficult to evaluate the facts and you can easily be fooled into a false or biased conclusion. If the claims are based on a poll, it is important to look critically at the wording of the questions, who has been asked and how they have been chosen.