The emergence of large language models as tools in literature reviews: a large language model-assisted systematic review
Problem
Researchers lacked an overview of how large language models are actually being used to automate and assist scientific literature reviews.
Why it matters
Systematic reviews inform medical and policy decisions, and manually screening thousands of articles is slow and costly.
What the AI did
A GPT-4o-based LLM add-on in Covidence assisted screening and extraction; ChatGPT and Scite.ai helped clean data, generate figure code, and draft the manuscript.
Results
3788 articles retrieved, 172 eligible; GPT-based LLMs dominated review automation (n=126, 73.2%); only 26 papers (15.1%) were actual reviews
Impact
Shows LLMs are already embedded in parts of the review pipeline, though full automation remains rare.
Record
| Venue | Journal of the American Medical Informatics Association |
|---|---|
| Published | 2025 |
| Work type | review |
| Authors | Dmitry Scherbakov, Nina Hubig, Vinita Gangaram Jansari, Alexander Bakumenko, Leslie Andrew Lenert |
| Citations | 145 (OpenAlex, at import time) |
| References | 145 |
| Model or method | OpenAI GPT-4o, ChatGPT, Scite.ai |
| Organization | not stated in source |
| Primary source | https://doi.org/10.1093/jamia/ocaf063 |
| Open access | hybrid |
| Keywords | large language models, systematic review, review automation, GPT-4o, ChatGPT, literature screening |
Importance rating
real but narrower advance
Systematic reviews inform medical/policy decisions and manual screening of thousands of articles is slow/costly; abstract quantifies 3788 retrieved, 172 eligible studies.