Copyright: artitcom / 123RF Stock Photo
It’s pretty well known that discussions about politics and religion can often go haywire. I mean, why wouldn’t they? People are arguing about their beliefs with other people and when someone suggests you’re wrong, that can be a gigantic blow to your ego.
So, it should come as no surprise that discussions about nutrition and health can become just as heated. Don’t believe me? Find a popular public post about nutrition on any social media platform and read the comments.
OK, what’s the problem?
Now, you shouldn’t be too concerned about the comments where people are just blatantly insulting each other because those aren’t worth paying attention to. What you should be paying attention to are discussions where studies are being referenced. These discussions can be a bit more civilized, but I’m proposing that they are more dangerous to the truth than the, “veganism is stupid arguments”.
Still confused about what I’m speaking about? Here, I’ll give you an example.
Person 1: “Dairy is bad for your health. It causes inflammation in the body and a buildup of mucus.”
Person 2: “That’s not true at all, look at this study showing that dairy is anti-inflammatory and all these mechanisms showing the effects of dairy on a buncha molecules, suggesting anti-inflammatory effects.”
Person 1: “Umm no, here’s a bunch of studies showing the negative effects of dairy on these outcomes. Also, your studies were funded by the dairy industry and have less than 1 million participants, therefore they’re invalid.”
Science discussions in a nutshell
Alright, you see my point now? Maybe you do, maybe you don’t. I guess it would depend on what circles you consider yourself a part of. Anyway, I like to describe the interactions above as, “PubMed Wars”. It involves building your case by citing as much evidence as possible and spamming another person on social media with those papers.
I’ll discuss specifically why these discussions tend to be more dangerous than your average ad hominem discussions, but before I do, I want to propose something else –– these kinds of interactions aren’t just limited to people who don’t understand science and who are on social media, they also happen a lot in the academic literature.
Why science researchers are part of the problem too
If you happen to be a researcher or just read scientific papers as a hobby, you’ll know that certain parts of a paper, most often referred to as the “Introduction” and “Discussion”, often contain references to other scientific papers in the literature. Researchers try to build a case around a hypothesis by citing the literature. The problem though is that some of them also engage in PubMed Wars, but in a more formal way.
People don’t recognize it because they like to appeal to authority, because they’re short on time, or because they’re just not skeptical enough. Here, I’ll give you a short example of how a scientific paper would look, when the author is partaking in the PubMed War:
“Inflammation is an important process of the human body. It allows the immune system to ward off infections, however, chronic inflammation is problematic and often associated with X and Y, which are bad. Current guidelines recommend the use of treatments A and B for lowering inflammation. Unfortunately, treatments A and B don’t have high quality evidence behind them and are associated with a million side effects.
Vitamin C has shown promise as an intervention for inflammation. It interacts with these molecules which are known to be associated with inflammation and lowers their concentrations. Bob et al found that Vitamin C administration in a buncha participants reduced biomarkers of inflammation. Rob et al also found this and Todd et al too! However, none of these studies looked at this population, which is what we’re doing.”
Why is this bad?
So, non-researchers and even researchers selectively choose evidence to back up their arguments with as many citations as possible, while excluding evidence that contradicts their narrative — which is known as confirmation bias.
Imagine this forest plot contains every study published on a topic.
The problem here is that there’s A LOT of published scientific literature. There are hundreds of papers being published every day, maybe even more. I don’t know. No one seems to have an accurate number. What I do know is that when people engage in PubMed Wars, they haven’t done comprehensive searches of the scientific literature to see whether there is any evidence contradicting their narrative, instead, they opened up Google or PubMed, typed in a couple of phrases, and cited the first few relevant papers. And this is bad because the papers likely aren’t accurate representations of the literature and people probably haven’t evaluated the QUALITY of these papers, rather, they’re focused on QUANTITY. Because more evidence = being right, right?
And I’ve seen this first hand from working in several labs. I’ve been asked by principal investigators/research coordinators or seen them ask others to find evidence to support a particular argument, but what I haven’t seen is them asking others to do comprehensive searches of the literature, unless it was for a systematic review. I’ve witnessed fellow lab members grabbing at whatever evidence they could, whether it be the product of blue skies research or a badly designed trial, to build the citations of a paper.
People aren’t always doing this intentionally, sometimes it’s just an unconscious behavior that happens and I’ll often find myself even doing this after having worked on a few systematic reviews and that’s alright, we all suffer from cognitive biases, but we have to acknowledge them if we hope to even overcome them.
Thorough searches of the literature shouldn’t only be reserved for systematic reviews and meta-analyses, they should be done when building a narrative using the published literature, because if you’re going to make a claim, you need to take into account the totality of the evidence, rather than a few papers that support your outcome.
In conclusion, it’s a good idea to do some extensive research before you make a claim — that means utilizing folks who know how to do this such as medical librarians or systematic reviewers. So, the next time you feel an urge to reference a scientific study, whether it be on a Facebook discussion or in a paper, at least spend some time doing a somewhat comprehensive search of the literature.