¹Institute of Clinical Epidemiology and Biometry, Julius-Maximilians University Würzburg, Würzburg, Germany
²Department of Neurology, Heidelberg University Hospital, Heidelberg, Germany
³Department of Neurology, Alfried-Krupp Hospital, Essen, Germany
⁴Department of Brain Sciences, Imperial College London, London, United Kingdom

Text

Introduction: Intracerebral Haemorrhage (ICH) has a high mortality. Valid models for predicting risk of death in ICH patients 3 months after the event are lacking. Machine learning algorithms are increasingly proposed for improving risk prediction. We compared the results of machine learning with those of logistic regression to determine key predictors of 3-month mortality after ICH.

Methods: We analyzed a data set including ICH patients with atrial fibrillation, enrolled in 46 certified German stroke units in the investigator-initiated multicenter observational cohort study RASUNOA-prime (ClinicalTrials.gov-NCT02533960). Patients were followed up 3 months after index ICH. We compared 3 modeling approaches to predict 3-month mortality in patients with ICH: (1) logistic regression with variable selection via lasso regression, (2) logistic regression with backward selection, and (3) the xgboost algorithm. For the logistic regression approaches, multiple model variants were developed (e.g., using all analysis variables, or only univariately significant ones). Missing values were handled using multiple imputation. For xgboost, a 5-fold nested cross-validation was implemented. In each outer fold, models were trained on a training set and evaluated on an independent test set. Additionally, different hyperparameter tuning strategies were applied: default settings, grid search and random search. Model performance was assessed using the area under the receiver operating characteristic curve (AUC) for discrimination and calibration was evaluated using calibration slope, intercept and flexible calibration curves. The different methods were compared based on AUC and calibration curves.

Results: Between 2015 and 2021, 951 ICH patients were included in RASUNOA-prime. Of those, 811 patients with known 3-month survival status were analyzed. After 3 months, 394 patients (48.6%) were alive. Among 16 analyzed variables, age, neurological deficit (NIHSS score), hematoma volume and disability at admission (modified Rankin scale; mRS) were selected across all 3 methods as the most important predictors for 3-month mortality. Diabetes was never selected. Hypertension and external ventricular drainage were selected in almost all lasso models, but were less important in backward selection and xgboost models. Logistic regression models achieved AUCs between 88.5% and 89.9% regardless of the selection method. Xgboost models showed slightly lower AUCs (84.7% to 88.5%), depending on the cross-validation fold and hyperparameter tuning strategy.

Conclusion: Although there were differences in variable selection between the models, machine learning was not superior to logistic regression. All approaches showed similar performance based on AUC, calibration slope and intercept and identified the same key predictors of 3-month mortality. However, logistic regression models were easier to apply as it was not clear how to combine the results of xgboost into one final model due to the nested cross-validation results. Moreover, parameter tuning for the xgboost models was time-consuming and the lack of directly interpretable effect estimates makes clinical application challenging. Nevertheless, the xgboost approach may provide more robust results, as the test data was not used for training, while the logistic regression models were developed and validated on the same dataset.

Acknowledgements: Supported by an unrestricted research grant from Bayer Vital GmbH Germany, Bristol-Myers Squibb/Pfizer Alliance, Boehringer-Ingelheim Pharma GmbH&Co.KG, Daiichi Sankyo Europe GmbH.

The authors declare that they have no competing interests.

The authors declare that a positive ethics committee vote has been obtained.