External validation of machine learning models for predicting prehospital delay in acute ischemic stroke: a retrospective cohort study.
Prehospital delay remains a major barrier to timely reperfusion therapy in acute ischemic stroke (AIS). We externally validated eight prediction algorithms and examined whether complex machine-learning models added value beyond logistic regression.
This retrospective cohort study included consecutive patients with AIS from two hospitals in Kashgar, China (January 2019-December 2024). Hospital 1 (n = 1,377) was the development cohort and Hospital 2 (n = 1,015) the independent validation cohort. Prehospital delay was defined as onset-to-door time >4.5 h. LASSO selection and tuning were restricted to Hospital 1. Sensitivity analyses used all 36 encoded candidate features and repeated all models without SMOTE. Evaluation included discrimination, calibration, post-hoc recalibration, and decision curves.
Delay occurred in 80.9 and 83.2% of the two cohorts. In the prespecified LASSO-plus-SMOTE analysis, XGBoost had the highest validation AUC (0.771; 95% CI, 0.734-0.807), but did not significantly outperform logistic regression (difference, 0.015; 95% CI, -0.003 to 0.034; p = 0.107). Full-feature modeling produced no consistent gain. Omitting SMOTE improved calibration and often discrimination; logistic regression without SMOTE achieved AUC 0.774 and Brier score 0.122, versus 0.756 and 0.180 with SMOTE. The original SMOTE-trained logistic model underestimated absolute risk. Rural residence, non-emergency-channel presentation, referral, and non-ambulance transport were associated with absolute risk differences of 15.8, 21.5, 9.2, and 36.3 percentage points in Hospital 2.
Complex algorithms did not show a clear advantage over logistic regression. Because several predictors are available only after arrival, the framework is intended for retrospective health-system profiling and quality-improvement planning, not individual pre-arrival prediction. Universal interventions remain warranted given the high prevalence of delay.
This retrospective cohort study included consecutive patients with AIS from two hospitals in Kashgar, China (January 2019-December 2024). Hospital 1 (n = 1,377) was the development cohort and Hospital 2 (n = 1,015) the independent validation cohort. Prehospital delay was defined as onset-to-door time >4.5 h. LASSO selection and tuning were restricted to Hospital 1. Sensitivity analyses used all 36 encoded candidate features and repeated all models without SMOTE. Evaluation included discrimination, calibration, post-hoc recalibration, and decision curves.
Delay occurred in 80.9 and 83.2% of the two cohorts. In the prespecified LASSO-plus-SMOTE analysis, XGBoost had the highest validation AUC (0.771; 95% CI, 0.734-0.807), but did not significantly outperform logistic regression (difference, 0.015; 95% CI, -0.003 to 0.034; p = 0.107). Full-feature modeling produced no consistent gain. Omitting SMOTE improved calibration and often discrimination; logistic regression without SMOTE achieved AUC 0.774 and Brier score 0.122, versus 0.756 and 0.180 with SMOTE. The original SMOTE-trained logistic model underestimated absolute risk. Rural residence, non-emergency-channel presentation, referral, and non-ambulance transport were associated with absolute risk differences of 15.8, 21.5, 9.2, and 36.3 percentage points in Hospital 2.
Complex algorithms did not show a clear advantage over logistic regression. Because several predictors are available only after arrival, the framework is intended for retrospective health-system profiling and quality-improvement planning, not individual pre-arrival prediction. Universal interventions remain warranted given the high prevalence of delay.
Authors
Zeng Zeng, Yusufujiang Yusufujiang, Dang Dang, Zhu Zhu, Yasheng Yasheng, Wang Wang, Yan Yan, Yu Yu, Mamattursun Mamattursun, Aini Aini, Kang Kang, Li Li
View on Pubmed