The Impact of Evaluation Strategy on Sepsis Prediction Model Performance Metrics in Intensive Care Data: Retrospective Cohort Study - PubMed
5 hours ago
- #intensive-care
- #machine-learning
- #sepsis-prediction
- Sepsis prediction models in intensive care units (ICUs) use different evaluation strategies: fixed horizon, peak score, and continuous evaluation.
- The study compares these strategies using a model trained on the Medical Information Mart for Intensive Care IV (MIMIC-IV) dataset and tested on BerlinICU, a German ICU dataset.
- Performance metrics vary significantly by evaluation strategy, with continuous evaluation showing an AUROC of 0.67, peak score similar, and fixed horizon reduced to 0.61.
- Onset matching minimally affects continuous and fixed horizon evaluations but increases peak score estimates.
- Shorter prediction horizons improve performance across all strategies.
- Continuous evaluation aligns best with real-world clinical monitoring, while fixed-horizon and peak score may skew results without proper length of stay matching.