Why do users override alerts? Utilizing large language model to summarize comments and optimize clinical decision support

Siru Liu, Allison B. McCoy, Aileen P. Wright, Scott D. Nelson, Sean S Huang, Hasan B. Ahmad, Stefano Carro, Jacob Franklin et al.

Journal of the American Medical Informatics Association 2024 article cited by 35

Source Open-access PDF More in Medicine & Health

importance 4/10 ai assisted Artificial Intelligence in Healthcare and Education clinical decision support stage: research proprietary model

Problem

Understanding why clinicians override CDS alerts required summarizing large volumes of user alert comments.

Why it matters

Alert overrides and poorly designed alerts can cause clinicians to ignore safety warnings.

What the AI did

GPT-4 independently summarized alert comments, and five CDS experts rated AI-generated versus physician-generated summaries on clarity, completeness, accuracy, and usefulness.

Results

Among the top 8 rated summaries, five were GPT-4-generated; AI summaries showed high clarity, accuracy, and usefulness.

Impact

AI summaries were rated comparably to physician summaries by CDS experts in this study.

Record

Venue	Journal of the American Medical Informatics Association
Published	2024
Work type	article
Authors	Siru Liu, Allison B. McCoy, Aileen P. Wright, Scott D. Nelson, Sean S Huang, Hasan B. Ahmad, Stefano Carro, Jacob Franklin et al.
Citations	35 (OpenAlex, at import time)
References	21
Model or method	GPT-4
Organization	Vanderbilt University Medical Center
Primary source	https://doi.org/10.1093/jamia/ocae041
Open access	hybrid
Keywords	GPT-4, alert override, clinical decision support, summarization

Importance rating

4 / 10

real but narrower advance

Alert overrides and poorly designed alerts can cause clinicians to ignore safety warnings; study covers one medical center and 8 alerts, a real but narrow patient-safety advance.

Basis: inferred. Judged from the source text, which gives no explicit scale. Stated scale: not stated in source. Method and limits.

More in Medicine & Health

#	Paper	Year	Citations	Importance
1	Multimodal deep learning of fundus abnormalities and traditional risk factors for cardiovascular risk prediction Yeong Chan Lee, Jiho Cha, Injeong Shim, Woong‐Yang Park, Se Woong Kang...	2023	107	10
2	Reliable Tuberculosis Detection Using Chest X-Ray With Deep Learning, Segmentation and Visualization Tawsifur Rahman, Amith Abdullah Khandakar, Muhammad Abdul Kadir, Khand...	2020	651	10
3	A comparative analysis of machine learning classifiers for stroke prediction: A predictive analytics approach Nitish Biswas, Khandaker Mohammad Mohi Uddin, Sarreha Tasmin Rikta, Sa...	2022	116	10
4	Cardiovascular Disease Diagnosis from DXA Scan and Retinal Images Using Deep Learning Hamada R. H. Al-Absi, Mohammad Tariqul Islam, Mahmoud Ahmed Refaee, Mu...	2022	89	10
5	Proteomics and Machine Learning Approaches Reveal a Set of Prognostic Markers for COVID-19 Severity With Drug Repurposing Potential Kruthi Suvarna, Deeptarup Biswas, Medha Gayathri J. Pai, Arup Acharjee...	2021	76	10