JBRA Assist. Reprod. 2026 2026;00(0):00-00
EDITORIAL

doi: 10.5935/1518-0557.20260147

Artificial Intelligence as a Partner in the Critical Appraisal of Medical Evidence: From Human Skepticism to Human-AI Collaboration

J.G. Franco Jr1,2,3

1Center for Human Reproduction Prof. Franco Jr, Ribeirao Preto, Brazil
2Paulista Center for Diagnosis - Research and Training, Ribeirao Preto, Brazil
3Editorial Consultant, JBRA Assisted Reproduction

CORRESPONDING AUTHOR
J.G. Franco Jr, Center for Human Reproduction Prof. Franco Jr, Ribeirao Preto, SP - Brazil
Email: franco@crh.com.br

CONFLICTS OF INTEREST
The author participated in the development of the TRUST family of appraisal instruments discussed in this editorial. No remuneration was received for this work. The author declares no financial conflicts of interest.

Artificial intelligence (AI) has entered medicine at remarkable speed. Its ability to search, summarize, compare, calculate, and interrogate scientific information creates opportunities that would have been difficult to imagine only a few years ago. At the same time, its incorporation into scientific evaluation has understandably generated skepticism. Can an algorithm critically appraise a medical publication or research question? Can it recognize methodological weaknesses behind an apparently convincing conclusion? And, perhaps more importantly, how much should clinicians trust an AIgenerated assessment of evidence?

These concerns are legitimate. Large language models may misinterpret incompletely reported methods, produce unsupported statements, or express uncertainty with inappropriate confidence. Scientific appraisal therefore cannot be delegated to AI as if it were an autonomous authority. However, rejecting AI because it is imperfect may be equally problematic. The rapidly expanding medical literature increasingly challenges the ability of clinicians, reviewers, editors, and guideline developers to examine every publication with the rigor and consistency required by evidence-based medicine.

The most useful perspective may therefore not be human versus artificial intelligence, but human-AI collaboration. Humans contribute clinical knowledge, methodological reasoning, contextual interpretation, ethical judgment, and the capacity to recognize when formal rules fail to capture clinical reality. AI contributes speed, consistency, systematic interrogation of large documents, and the ability to repeatedly apply a predefined framework across many publications. Neither contribution should be considered sufficient in isolation.

Our experience during the development of the TRUST family of instruments has progressively reinforced one principle: the quality of AI-assisted appraisal depends fundamentally on the quality of the instructions provided by humans. Asking an AI simply whether a paper is "good" or "trustworthy" invites an impressionistic answer. A more defensible approach is to define in advance what must be examined, which documentary evidence is required, how uncertainty should be handled, and when human adjudication becomes mandatory.

This philosophy underlies the evolving TRUST framework, including TRUST-MA for meta-analyses, TRUST-PAPER for scientific papers, TRUST-ABSTRACT for abstracts, TRUST-GUIDELINES for clinical guidelines, and TRUST-PS for position statements. These instruments are not intended to replace established methodological standards or expert review. Their purpose is to translate methodological principles into explicit operational rules that can be applied by human evaluators or AI systems, although a combination of the two is preferable.

TRUST-MA illustrates this approach particularly clearly. Its current operational version under testing requires evaluators to identify documentary anchors rather than award points from an overall impression. The clinical estimand and denominator are locked before scoring; points are awarded only when supported by locatable evidence; critical methodological flaws are assessed separately; and predefined criteria for disagreement or uncertainty trigger human adjudication. The objective is not to make AI independent of humans, but to make AIassisted evaluation more transparent, reproducible, and auditable.

Assisted reproduction provides a particularly relevant environment in which to explore this approach because several aspects of methodological interpretation are strongly domain-dependent. Universal principles such as protocol transparency, risk-of-bias assessment, appropriate statistical methods, heterogeneity, certainty of evidence, and proportional conclusions remain essential. Nevertheless, reproductive medicine also contains methodological complexities that require specific clinical understanding.

The denominator is a simple but important example. Live birth per embryo transfer, live birth per initiated cycle, cumulative live birth per oocyte retrieval, and cumulative live birth per patient do not represent the same clinical estimand. Conclusions may change according to the denominator selected. Similarly, repeated treatment cycles, multiple embryos originating from the same patient, multiple transfers, embryo-level outcomes, and patient-level cumulative outcomes may introduce dependencies that cannot be interpreted correctly without knowledge of the clinical structure of assisted reproduction.

Consequently, a generic AI instructed only to evaluate "methodological quality" may overlook issues that an assisted-reproduction specialist immediately recognizes as clinically important. This is not a failure of AI; it demonstrates why domain expertise must inform the design of the rules under which AI operates. Humans must determine which questions matter before asking machines to answer them.

Initial distrust of AI in this role should therefore not be dismissed. Skepticism is an important component of scientific progress. The appropriate response, however, is not blind acceptance or blind rejection, but verification. AIassisted evaluators should be required to show where the evidence supporting each judgment is located, distinguish unavailable source material from information that was not reported, avoid inferring unreported conduct, acknowledge uncertainty, and identify situations requiring expert review.

This approach also raises an interesting question about human evaluation itself. Scientific appraisal is not free from subjectivity. Two experienced clinicians may interpret the same meta-analysis differently; reviewers may disagree about methodological importance; and journal prestige or author reputation may unconsciously influence perceptions of credibility. Structured AI-assisted evaluation may therefore have value not because machines are intrinsically more objective, but because explicit rules can make both machine and human judgments more visible and contestable.

In this issue of JBRA Assisted Reproduction, two studies explore elements of this concept. One evaluates the reproducibility of TRUST-MA using independent dual-AI assessment (Franco Jr. et al., 2026a; Figure 1), while another examines whether journal impact predicts methodological trustworthiness using the TRUST-PAPER framework (Franco Jr. et al., 2026b; Figure 2). These studies should be interpreted cautiously. They do not establish AI as an autonomous scientific referee, nor do they constitute external validation of the entire TRUST family. Rather, they represent early steps in investigating whether AI systems operating under explicit instructions can contribute to structured, reproducible, and auditable critical appraisal.

 

Figure 1
Figure 1. QR codes for the Claude and ChatGPT resources associated with TRUST-MA.

 

 

Figure 2
Figure 2. QR codes for the Claude and ChatGPT resources associated with TRUST-PAPER.

 

The broader concept may be more important than any individual score. AI has already demonstrated its capacity to assist in generating scientific content. Its role in critically examining scientific information may become equally important. An AI system capable of rapidly identifying missing methodological anchors, inconsistencies between methods and conclusions, inappropriate denominators, statistical vulnerabilities, or unsupported clinical claims could become a valuable additional instrument for authors, reviewers, editors, guideline panels, and clinicians.

Such a future should remain collaborative. A useful conceptual pathway can be expressed as: human expertise → explicit methodological rules → AI-assisted evaluation → auditable output → human judgment. Human experts are therefore not removed from the process; they frame the evaluation at the outset and interpret its results at the end.

The TRUST family remains a developing approach. Further calibration, comparisons across AI models and with human experts, external validation, and evaluation across different medical domains are required before strong claims about its performance can be justified. This limitation should be explicit. Prudence is especially important when an instrument is intended to evaluate the trustworthiness of other scientific instruments and publications.

Nevertheless, collaboration between humans and artificial intelligence in evidence appraisal appears increasingly inevitable. The relevant question may no longer be whether AI will participate in the evaluation of medical literature, but how we can ensure that its participation improves rather than weakens scientific reasoning.

The answer will probably depend less on asking AI to think like a physician than on teaching it which methodological questions physicians, statisticians, and scientists have learned to ask.

Artificial intelligence should not be blindly trusted, nor reflexively distrusted. It should be instructed, constrained, audited, and interpreted.

Ultimately, AI may become extraordinarily useful in evaluating medical evidence. But first, humans must teach it what deserves to be trusted.

EDITORIAL NOTE
This article intentionally adopts a cautious position: the TRUST instruments are presented as developing, AI-assisted appraisal frameworks rather than validated replacements for established methodological tools or expert judgment.

USE OF ARTIFICIAL INTELLIGENCE
Claude (Anthropic) and ChatGPT (OpenAI) were used to assist in developing the TRUST appraisal system. The author retains responsibility for the methodological framework and the content of this editorial.

REFERENCES
Franco Jr JG, Petersen C, Vagnini LD, Massaro FC, Petersen B, Nicoletti A, Ricci J, Zamara C, Pouza RA, Matuella BC, Espirito-Santo EV, Meziara JB, Oliani AH, Oliveira JBA. Reproducibility of the TRUST-MA Framework for Assessing the Methodological and Clinical Reliability of Meta-Analyses: An Independent Dual-AI Evaluation. JBRA Assist Reprod. 2026a. In press.

Franco Jr JG, Petersen C, Vagnini LD, Massaro FC, Petersen B, Nicoletti A, Ricci J, Zamara C, Pouza RA, Matuella BC, Espirito-Santo EV, Meziara JB, Oliani AH, Oliveira JBA. Can Journal Impact Predict the Trustworthiness of Individual Studies in Assisted Reproduction? Development and Pilot Evaluation of the TRUST-PAPER Framework. JBRA Assist Reprod. 2026b. In press.