With the number of scientific articles about COVID-19 increasing, it seems we have entered a new era where our mantra must be “Reader Beware.” For reasons known only to their editorial boards, even reputed medical journals are falling prey to the temptation of publishing studies plagued with poor methodology, incomplete information, or conclusions that are not always justified by a careful analysis of the data.
Many classification schemes offer scientists help in judging the reliability and generalizability of study findings, as well as the value of conclusions authors draw from their study results1. Different types of research questions are answered by different types of research studies. Various levels of evidence, also known as a hierarchy of evidence, are assigned to studies based on their design, validity, and applicability to specific experimental or clinical scenarios.
In philosophy, the study of knowledge is called epistemology, which most agree requires three conditions: truth, beliefs, and justification. Even published statements, guidelines, and recommendations are rendered epistemologically more reliable when authors explicitly identify how both facts and opinions contribute to their conclusions. Evidence can be graded, and opinions themselves can be described, for example, as evidence-based, personal preference, as a result of consensus or compromise, or as grounded in conventional wisdom or moral convictions.
The value we attribute to such opinions often depends on their generalizability, as well as on whether they reflect feelings intrinsically shared with others. In this regard, they may be considered secure, vulnerable, or debatable. They may be based on graded factual claims, reasonable projections, or erroneous assumptions. They will also be viewed through the lens of culturally diverse populations, biases, presumptions, and experiences.
When the medical literature provides us repeatedly with questionable studies, it forces us to doubt the reliability of future publications. It mandates that we apply critical thinking, and not rely only on abstracts or potentially sensational sentences written into a well-composed conclusion. It reminds us also that complementing clinical training with learning elements of scientific writing, critical reading, and ethical research publication should be an essential part of professional training2.
In times of crisis, however, critical thinking may be neglected because of a rush to hope. Sometimes, experience suggests that what is best for a population of patients may not be what is in the best interests for a specific patient3. Furthermore, in a hierarchy built on tradition, some medical readers and practitioners may not share their doubts in the written word. Journals may not publish critical commentaries or provide explanatory retractions, and a herd mentality can prompt practices that potentially harm rather than help colleagues and patients alike.
- Lokker C. et al. A scoping review of classification schemes of interventions to promote and integrate evidence in to practice in healthcare. Implementation Science 2015;10:27.
- Eastwood S. Ethical Scientific Reporting and Publication: Training the Trainees. In, Ethical Issues in Biomedical Publication (Jones AH and McLellan F eds). Johns Hopkins University Press, NY, 2000, pgs 250-275.
- Brody H. Patient ethics and evidence-based medicine-the Good Healthcare Citizen. Cambridge Quarterly of Healthcare Ethics 2005;14:141.
Please subscribe to Colt’s Corner to automatically receive email notification of future blog posts.