Interpretable machine learning models for early prediction of acute kidney injury after cardiac surgery | BMC Nephrology

Comparison of characteristics and outcomes

A total of 2310 participants were included in the present retrospective cohort study. Table 1 shows the comparison of characteristics and outcomes in patients with and without postoperative AKI or severe AKI. In the whole cohort of cardiac surgery, 1020 (44.2%) patients developed PO-AKI and 286 (12.4%) developed severe AKI (stage 2 and 3 AKI). Univariate analyses indicated that presentation of hyperlipidemia, cerebrovascular accident, congestive cardiac failure, CCS angina class II-IV, NYHA class III-IV, CABG + valve/other surgery, CPB, aortic cross-clamping, IABP usage, intraoperative BPT, and longer assisted ventilation time were significantly associated with increased risk of PO-AKI (all P < 0.05). Notably, we found that the levels of preoperative SCr in the patients without PO-AKI were significantly higher than those of patients with PO-AKI (P < 0.001). Compared to the patients with PO-AKI, those without PO-AKI had higher percentages of preoperative medications use, such as intravenous nitroglycerin injection (54.1% vs. 43.3%; P < 0.001), β-blockers (50.1% vs. 37.8%; P < 0.001), ACEi/ARB (17.8% vs. 10.4%; P < 0.001), lipid-lowering agents (41.1% vs. 21.0%; P < 0.001)). There was no statistical significance between AKI and no-AKI groups regarding gender, age, BMI, smoker, diabetes mellitus, hypertension, chronic renal failure, COPD, peripheral vascular disease, arrhythmia, previous MI and cardiac interventions.

Development, validation and comparison of models

Using the sixteen preoperative and intraoperative predictors that were significantly associated with postoperative AKI, we developed five ML models and a conventional logistic regression model in the derivation set (n = 1848, 80%) and validated the models in the validation set (n = 462, 20%) for prediction of postoperative AKI and severe AKI, respectively. There were no significant differences among patient characteristics, postoperative outcome and complications between the derivation set and the validated set (Supplementary Table 1). Model performance metrics in the validated set were demonstrated in Table 2. The ROC curves of prediction models were also illustrated in Fig. 1(a) for AKI prediction and Fig. 1(b) for severe AKI prediction. Comparisons of AUCs among different models are presented in Supplementary Table 2.

Model performance illustrated by receiver operating characteristic curves for predicting (a) postoperative AKI and (b) postoperative severe AKI. AKI, acute kidney injury; AUC, area under receiver operating characteristic curves; CI, confidence interval

Among the five ML models, MLP and GNB, with similar AUCs (0.793, 95%CI: 0.735, 0.844 vs. 0.762, 95%CI: 0.701, 0.815, P_difference =0.179), performed best in predicting postoperative AKI. Compared to MLP, logistic regression had a higher AUC (0.812, 95%CI: 0.756, 0.860 vs. 0.793, 95%CI: 0.735, 0.844, P_difference =0.036), sensitivity (0.774, 95%CI: 0.719, 0.813 vs. 0.804, 95%CI: 0.749, 0.843), accuracy (0.753, 95%CI: 0.719, 0.781 vs. 0.745, 95%CI: 0.705, 0.778) and Youden index (0.513, 95%CI: 0.451, 0.573 vs. 0.460, 95%CI: 0.391, 0.536).

For postoperative severe AKI prediction, in terms of AUC, GBC (0.86, 95%CI: 0.808, 0.902) performed significantly better than the other four ML models, including DT (0.749, 95%CI: 0.688, 0.803, P_difference =0.011), RF (0.805, 95%CI: 0.748, 0.854, P_difference =0.027), GNB (0.734, 95%CI: 0.672, 0.790, P_difference =0.004) and MLP (0.718, 95%CI: 0.655, 0.775, P_difference <0.001). The AUC of GBC was larger than the AUC of conventional logistic regression (0.86, 95%CI: 0.808, 0.902 vs. 0.803, 95%CI: 0.746, 0.852) but no significant differences were found (P_difference =0.173). As illustrated in Table 2, the sensitivity (0.333, 95%CI: 0.224, 0.431 vs. 0.148, 95%CI: 0.078, 0.250), Youden index (0.304, 95%CI: 0.202, 0.424 vs. 0.138, 95%CI: 0.065, 0.260) and accuracy (0.896, 95%CI: 0.868, 0.917 vs. 0.892, 95%CI: 0.877, 0.905) of GBC seemed to be consistently greater than those of conventional logistic regression.

Model interpretation

For AKI prediction, the feature importance matrix plot for the best-performing logistic regression is shown in Fig. 2 (a). The top five most important predictors based on the SHAP values were assisted ventilation time, CPB, lipid-lowering agents, last preoperative SCr, and hyperlipidemia. To summarize the effects of all the features on logistic regression output at a global level, we plotted the SHAP values of every feature for every participant (Fig. 3 (a)). It revealed that CPB and longer assisted ventilation time increased the risk of AKI and vice versa for preoperative lipid-lowering agents and higher preoperative SCr. At a local level, we could get a set of SHAP values for specific participant’s feature values and visualize the impact of each feature on the output by force plot (Supplementary Fig. 1(a)). The color of the arrows shows how the feature impacts the model: a red arrow increases the model output predicted value in the positive direction while a blue arrow decreases it in the negative direction. The size of the color arrows for each feature value represents the impact magnitude of the SHAP value. The base value refers to the mean probability that would be predicted if we do not know any features for the current output. As shown in Supplementary Fig. 1(a), the base value here was 0.49 and the output predicted value was 0.61, lipid-lowering agents = 0 with red arrow and ACC = 1 with the blue arrow have similar impact magnitudes but in opposite directions. Red feature values contribute to postoperative AKI prediction, while blue ones push towards no AKI development.

Feature importance matrix plot based on SHAP for (a) postoperative AKI prediction using logistic regression and (b) postoperative severe AKI prediction using gradient boosting model. AKI, acute kidney injury; CCS, Canadian Cardiovascular Society; NYHA, New York heart association; CA, cerebrovascular accident; ACEi, angiotensin-converting-enzyme inhibitor; ARB, angiotensin II receptor blocker; ACC, Aortic cross-clamping; SCr, serum creatinine; CABG, coronary artery bypass grafting; CPB, cardiopulmonary bypass; IABP, intra-aortic balloon pump; BPT, blood product transfusion

SHAP summary plot for (a) postoperative AKI prediction using logistic regression and (b) postoperative severe AKI prediction using gradient boosting model. AKI, acute kidney injury; CCS, Canadian Cardiovascular Society; NYHA, New York heart association; CA, cerebrovascular accident; ACEi, angiotensin-converting-enzyme inhibitor; ARB, angiotensin II receptor blocker; ACC, Aortic cross-clamping; SCr, serum creatinine; CABG, coronary artery bypass grafting; CPB, cardiopulmonary bypass; IABP, intra-aortic balloon pump; BPT, blood product transfusion

Similarly, we combined the optimal GBC model for severe AKI prediction with SHAP analysis and presented the importance matrix plot in Fig. 2 (b). To get an overview of which features contribute most to the GBC model, we used the SHAP summary plot (Fig. 3 (b)). Of the top five most important variables, there were two risk factors (longer assisted ventilation time and hyperlipidemia) and three protective factors (higher last preoperative SCr, preoperative lipid-lowering agents and intravenous nitroglycerin intake). SHAP force plot (Supplementary Fig. 1(b)) showed the impact direction and magnitude of each feature value on the development of postoperative severe AKI by taking a specific patient case as an example. In this case, the blue-colored explanatory variable values (last preoperative SCr (mg/dl) = 0.49, no preoperative intravenous nitroglycerin injection, no preoperative intake of lipid-lowering agents, NYHA III-IV, no preoperative intake of β-blockers, hyperlipidemia) had a positive contribution towards severe AKI development and outweighed the negative impact of other blue-colored variable values, such as no CPB, assisted ventilation time (hr) = 45 and so on.