Never totally trust this phrase or any of its variants – “a study shows,” “scientists claim, ” etc.
Who funded the research, and how much influence did they have? What were their potential biases?
How many people were asked, examined, tested, and according to what methods?
Do any claimed findings show causation, or just correlation?
If they claim causation, what are the other possible explanations? There are always other possible explanations and often a lot of them. Were they considered? How convincingly were they ruled out?
Most importantly, has the research been validated by others and, where applicable, replicated? How many times? (By whom, with how many people. You get the idea.)
A by-product of the push-button publishing world we live in these days is that “research” is everywhere, and there is a lot of pressure to publish it. Certainly, that applies for traditional academics, but the thirst for research has multiplied exponentially with the growth of social media and content marketing. There is a continual and growing need to feed the beast.
And, the phenomenon extends well beyond social media and marketing. We live in an age in which one of the most popular forms of non-fiction is essentially to weave a tapestry from small studies – usually not replicated – that serve the larger story the author wants to tell. (Also known as confirmation bias.) Whenever you see “The Surprising Truth About …” or something similar in a title, proceed with critical caution.
This doesn’t mean we should never trust anything or start subscribing to conspiracy theories, but it does mean that, as serious lifelong learners, we need always to maintain a healthy skepticism, embrace epistemological modesty, and cultivate our critical thinking skills (including on this site!).
This, of course, is purely my opinion. I don’t know what the research says on this one.
And I probably wouldn’t trust it anyway.
Here are some other resources to help you apply a healthy level of skepticism to any research you encounter:
A great series from Dr. Peter Attia, a committed scientific thinker and one of my favorite sources on health and medicine.
- Studying Studies: Part I – relative risk vs. absolute risk
- Studying Studies: Part II – observational epidemiology
- Studying Studies: Part III – the motivation for observational studies
- Studying Studies: Part IV – randomization and confounding
- Studying Studies: Part V – power and significance