EDITORIAL


https://doi.org/10.5005/jp-journals-10024-3615
The Journal of Contemporary Dental Practice
Volume 24 | Issue 12 | Year 2023

Re-evaluating the Epistemic Authority of Systematic Reviews and Meta-analyses in Contemporary Clinical Discourse


Shikha Yadav

Department of Dentistry, All India Institute of Medical Sciences, Mangalagiri, Andhra Pradesh, India

Corresponding Author: Shikha Yadav, Department of Dentistry, All India Institute of Medical Sciences, Mangalagiri, Andhra Pradesh, India, e-mail: drshikhayadav@gmail.com

How to cite this article: Yadav S. Re-evaluating the Epistemic Authority of Systematic Reviews and Meta-analyses in Contemporary Clinical Discourse. J Contemp Dent Pract 2023;24(12):919–920.

Source of support: Nil

Conflict of interest: None

Keywords: Evidence-based medicine, Evidence-based science, Meta-analyses, Systematic reviews and meta-analyses.

The prevailing enthusiasm for Systematic Reviews and Meta-analyses (SRMAs) as the epitome of evidentiary supremacy prompts a critical re-evaluation of their standing within contemporary clinical practice. The pervasive reliance on SRMAs as panaceas to clinical dilemmas warrants a nuanced exploration, with considerations undermining their unequivocal acceptance. The overemphasis on SRMAs may inadvertently sideline valuable insights from observational studies and other designs, limiting a comprehensive understanding of real-world treatment effects.

A balanced approach that embraces a spectrum of evidence sources is crucial for fostering a nuanced, patient-centered approach to evidence-based medicine (EBM). This calls for a shift from blind allegiance to SRMAs to a more discerning integration of diverse evidence types, ensuring a comprehensive and context-sensitive foundation for clinical decision-making.

In this editorial, we endeavor to highlight the deficiencies in the current system of crafting and publishing SRMAs. Our goal is not only to shed light on these shortcomings but also to propose viable solutions, striving to fortify and enhance the entire process of SRMA creation and dissemination.

EVOLUTION OF EVIDENCE-BASED MEDICINE

Once a stalwart guide, the classical pyramid of EBM has transformed into a trapezoid, challenging the hierarchical sanctity that positioned SRMAs at its zenith.1 The latest conceptualization suggests viewing all evidence through the prism of SRMAs, acknowledging the absence of clear-cut boundaries between different evidentiary tiers. Noteworthy is the variability inherent in study designs, with the grading of recommendations, assessment, development, and evaluation (GRADE) approach introducing undulating lines that reflect the nuanced rating of evidence based on various quality domains. Murad et al. examination in the Journal of Clinical Epidemiology unravels this complexity, underscoring the need for a prudent approach to discerning the true weight of evidence. Moreover, the American Heart Association’s attribution of level ‘A’ confidence to evidence derived from meta-analyses is questioned upon reevaluation using GRADE.1

Balshem et al. exploration in the Journal of Clinical Epidemiology illuminates the nuanced relationship between high-quality evidence and the strength of clinical recommendations, exemplified by the deliberations surrounding long-term warfarin administration post-initial deep venous thrombosis.2

Methodological Challenges in SRMAs

The burgeoning concern over the methodological robustness of SRMAs, particularly in specialized domains like sleep medicine, cannot be ignored. Xu et al.’s meta-epidemiological study exposes a paucity of methodological validity in a substantial number of SRMAs within this field.3 The implicit question arises: Can we steadfastly designate SRMAs as the pinnacle of evidence in light of such methodological fragility? The resounding answer is in the negative.

“Garbage In, Garbage Out”

The truism encapsulated in the phrase becomes particularly pertinent when appraising SRMAs. These analyses are inextricably linked to the pre-existing evidence landscape and are only as robust as the research they assimilate. The oft-overlooked in-depth data absent from certain publications casts a shadow on the comprehensiveness of SRMAs, prompting a skeptical inquiry into the integrity of the evidence they purportedly synthesize.

Biases and Heterogeneity

The omnipresent biases in publication and study selection pose formidable challenges. Positive results tend to overshadow inconclusive or negative findings, giving rise to a potential bias that necessitates redress. As exemplified in Dechartres et al. study in JAMA, the inherent heterogeneity in clinical, methodological, or statistical dimensions undermines the quest for meaningful conclusions from meta-analyses.4 The instability in findings accentuates the imperative for a cautious approach in basing treatment plans and policy changes solely on SRMA outcomes.

Database Access and Search Strategy

The access to multiple databases is often compromised, with SRMAs at times relying on a single database without a clear search strategy. The keys to a robust SRMA—explicitness, and reproducibility—remain elusive, raising legitimate concerns about the comprehensiveness and reliability of the synthesized evidence.

Proliferation of Unnecessary and Conflicted Reviews

The undeniable reality of the publish-or-perish ethos, coupled with the imperative to fill academic and professional requirements, has resulted in a surge of SRMAs. However, the quality of these instruments could be better. Ioannidis study in Milbank Quarterly unveils an alarming trend of unnecessary, misleading, and conflicted publications, underscoring the imperative for a recalibration in our reliance on such outputs. From January 1, 1986, to December 4, 2015, PubMed tags 266,782 items as “Systematic Reviews” and 58,611 as “Meta-analyses.” Between 1991 and 2014, annual publications increased by 2,728% for systematic reviews and 2,635% for Meta-analyses vs only 153% for all PubMed-indexed items. Currently, probably more systematic reviews of trials than new randomized trials are published annually.5

Employing state-of-the-art scientific tools and methodologies is essential for enhancing the quality of SRMAs. Some crucial tools and approaches include advanced statistical analyses, rigorous experimental designs, and the incorporation of cutting-edge technology, ensuring robust and reliable results. Additionally, maintaining transparency in methodology and adhering to standardized procedures further strengthens the validity and applicability of systematic reviews and meta-analyses. Some of the tools that can be used are:

  • Cochrane handbook for systematic reviews of interventions:

    • – Comprehensive guidelines for conducting SRMAs.

    • – Standardized methods for study selection and statistical analyses.

  • Grading of recommendations, assessment, development, and evaluation (GRADE):

    • – A structured framework for assessing evidence quality and grading recommendations.

    • – Enhances transparency and reliability of synthesized evidence.

  • ROBINS-I and ROBINS-E tools:

    • – Evaluate the risk of bias in non-randomized studies, ensuring a nuanced assessment beyond RCTs.

  • Review Manager (Rev Man) software:

    • – Facilitates systematic data management, statistical analysis, and creation of forest plots.

  • Comprehensive literature search tools:

    • – Use advanced literature search tools like PubMed, Embase, and specialized databases.

    • – Employ tools like EndNote or Zotero for reference management.

  • Covidence:

    • – Web-based tool to streamline the systematic review process.

    • – Aids in study selection, data extraction, and collaboration among reviewers.

  • GRADEpro guideline development tool (GRADEpro GDT):

    • – Assists in creating summary of findings (SoF) tables and evidence profiles based on GRADE assessments.

    • – Supports transparent presentation of the quality of evidence and recommendations’ strengths.

  • Rev Screen:

    • – Tool for systematic reviewers to expedite title and abstract screening.

    • – Uses machine learning algorithms for efficient identification of relevant studies.

  • Trial sequential analysis (TSA):

    • – Statistical tool to control the risk of random errors in cumulative meta-analyses.

  • Network Meta-Analysis (NMA) software:

    • – Tools like STATA, R (with the ‘netmeta’ package), or JAGS for complex comparisons.

In conclusion, integrating scientific tools elevates the SRMA process, enhancing methodological rigor and minimizing bias for superior evidence synthesis. Rather than positioning SRMAs as the definitive proof, they should be recognized as invaluable tools guiding refined research planning. Embracing nuance, acknowledging limitations, and advocating for diverse evidence sources and transparency are pivotal for advancing EBM. A commitment to “quality over quantity” is imperative for meaningful contributions to clinical practice and research decision-making. This editorial underscores the importance of a comprehensive perspective in shaping the future of evidence-based healthcare.

REFERENCES

1. Murad MH, Asi N, Alsawas M, et al. New evidence pyramid. Evid Based Med 2016;21(4):125–127. DOI: 10.1136/ebmed-2016-110401.

2. Balshem H, Helfand M, Schünemann HJ, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 2011;64(4):401–406. DOI: 10.1016/j.jclinepi.2010.07.015.

3. Xu C, Furuya-Kanamori L, Kwong JSW, et al. Methodological issues of systematic reviews and meta-analyses in the field of sleep medicine: A meta-epidemiological study. Sleep Med Rev 2021;57:101434. DOI: 10.1016/j.smrv.2021.101434.

4. Dechartres A, Altman DG, Trinquart L, et al. Association between analytic strategy and estimates of treatment outcomes in meta-analyses. JAMA 2014;312(6):623–630. DOI: 10.1001/jama.2014.8166.

5. Ioannidis JPA. The mass production of redundant, misleading, and conflicted systematic reviews and meta‐analyses. Milbank Q 2016;94(3):485–514. DOI: 10.1111/1468-0009.12210.

________________________
© The Author(s). 2023 Open Access. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted use, distribution, and non-commercial reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.