irAE-GPT: leveraging large language models to identify immune-related adverse events in electronic health records and clinical trial datasets - PubMed
3 hours ago
- #Large Language Models
- #Electronic Health Records
- #Immune-Related Adverse Events
- The study evaluated GPT models (GPT-3.5, GPT-4, GPT-4o) for identifying immune-related adverse events (irAEs) from unstructured patient notes in EHRs and clinical trial datasets.
- Using a zero-shot prompt, models were tested on 442 patients across three institutions, showing high sensitivity and specificity but moderate positive predictive values, indicating a tendency to overpredict irAEs.
- GPT-4o achieved the highest F1 scores, with best performance in haematological, gastrointestinal, and musculoskeletal/rheumatologic irAE categories.
- Limitations included difficulty in handling textual causation, specifically linking adverse events causally to immune checkpoint inhibitor therapy.
- The approach aims to automate irAE detection, reduce manual review burden, and enhance safety monitoring in healthcare datasets.