Hospital-acquired acute kidney injury (HA-AKI) frequently occurs in hospitalized patients, posing risks of chronic kidney disease, prolonged hospitalization, escalated healthcare expenses, and elevated mortality rates.
Image Credit: Thaiview/Shutterstock.com
As such, mitigating HA-AKI can significantly enhance outcomes for hospitalized individuals. Yet, predicting its onset proves challenging due to the multitude of contributing factors at play.
Mass General Brigham Digital researchers conducted a study assessing the efficacy of the Epic Risk of HA-AKI predictive model, a commercial machine learning tool, in predicting the risk of HA-AKI using patient data. The results indicated moderate success in prediction, albeit with lower performance compared to internal validation conducted by Epic Systems Corporation. This underscores the critical necessity of validating AI models rigorously before their clinical deployment.
The Epic model operates by evaluating adult inpatient encounters to gauge the risk of HA-AKI, characterized by predetermined rises in serum creatinine levels. Following the model's training with data sourced from Mass General Brigham hospitals, researchers subjected it to testing using data from approximately 40,000 inpatient hospital stays over a five-month span from August 2022 to January 2023.
The dataset was comprehensive, encompassing numerous data points pertaining to patient encounters. These included patient demographics, comorbidities, principal diagnoses, serum creatinine levels, and length of hospital stay. Two analyses were conducted, focusing on the performance of the model at both the encounter level and prediction level.
The investigators noted that the tool exhibited greater reliability in assessing patients with a lower risk of HA-AKI. While the model effectively identified low-risk patients who would not develop HA-AKI, it encountered difficulty predicting the onset of HA-AKI in higher-risk patients.
Moreover, the accuracy of predictions varied depending on the stage of HA-AKI under evaluation, with more successful predictions observed for Stage 1 HA-AKI compared to more severe cases.
The authors concluded that overall implementation of the tool might lead to elevated false-positive rates. They emphasized the need for further investigation into the tool's clinical impact.
We found that the Epic predictive model was better at ruling out low-risk patients than identifying high-risk patients. Identifying HA-AKI risk with predictive models could help support clinical decisions such as by warning providers against ordering nephrotoxic medications, but further study is needed before clinical implementation.
Sayon Dutta M.D., MPH, Study Lead Author, Clinical Informatics Team, Mass General Brigham Digital
Sayon Dutta is also an Emergency Medicine Physician at Massachusetts General Hospital.
Journal Reference:
Dutta, S., et al. (2024). External Validation of a Commercial Acute Kidney Injury Predictive Model. NEJM AI. doi.org/10.1056/aioa2300099.
Source: https://www.massgeneralbrigham.org/en