LLM-assisted systematic review of large language models in clinical medicine
Problem
The rapid flood of clinical LLM studies (roughly 3.2 papers per day) makes manual curation and review of the evidence base infeasible.
Why it matters
Clinicians and regulators need a clear evidence base to judge LLM usefulness and safety in medicine.
What the AI did
LLMs were used to assist a systematic review, identifying and characterizing 4,609 peer-reviewed clinical LLM studies from January 2022 to September 2025.
Results
4,609 studies identified; only 1,048 used real-world patient data and only 19 were prospective randomized trials; ChatGPT/OpenAI models were 65.7% of evaluated models.
Impact
Evidence supports LLM-assisted mapping of the clinical LLM evidence landscape at scale.
Record
| Venue | Nature Medicine |
|---|---|
| Published | 2026 |
| Work type | review |
| Authors | Sully F. Chen, Anton Alyakin, Andreas Seas, Eunice Yang, Joanne J. Choi, Jin Vivian Lee, Amelia L. Chen, Pranav I. Warman et al. |
| Citations | 67 (OpenAlex, at import time) |
| References | 17 |
| Model or method | not stated in source |
| Organization | not stated in source |
| Primary source | https://doi.org/10.1038/s41591-026-04229-5 |
| Open access | hybrid |
| Keywords | large language models, systematic review, clinical medicine, ChatGPT, evidence synthesis |
Importance rating
real but narrower advance
Abstract quantifies the flood of clinical LLM evidence (4,609 studies, ~3.2 papers/day) and scarcity of rigorous trials; meta-research advance aiding clinicians and regulators, narrower than direct care.