ML Struggles to Predict Severe Hospital-Acquired Kidney Injury

Hospital-acquired acute kidney injury (HA-AKI) frequently occurs in hospitalized patients, posing risks of chronic kidney disease, prolonged hospitalization, escalated healthcare expenses, and elevated mortality rates.

Focus on the hand of a patient in hospital ward

Image Credit: Thaiview/Shutterstock.com

As such, mitigating HA-AKI can significantly enhance outcomes for hospitalized individuals. Yet, predicting its onset proves challenging due to the multitude of contributing factors at play.

Mass General Brigham Digital researchers conducted a study assessing the efficacy of the Epic Risk of HA-AKI predictive model, a commercial machine learning tool, in predicting the risk of HA-AKI using patient data. The results indicated moderate success in prediction, albeit with lower performance compared to internal validation conducted by Epic Systems Corporation. This underscores the critical necessity of validating AI models rigorously before their clinical deployment.

The Epic model operates by evaluating adult inpatient encounters to gauge the risk of HA-AKI, characterized by predetermined rises in serum creatinine levels. Following the model's training with data sourced from Mass General Brigham hospitals, researchers subjected it to testing using data from approximately 40,000 inpatient hospital stays over a five-month span from August 2022 to January 2023.

The dataset was comprehensive, encompassing numerous data points pertaining to patient encounters. These included patient demographics, comorbidities, principal diagnoses, serum creatinine levels, and length of hospital stay. Two analyses were conducted, focusing on the performance of the model at both the encounter level and prediction level.

The investigators noted that the tool exhibited greater reliability in assessing patients with a lower risk of HA-AKI. While the model effectively identified low-risk patients who would not develop HA-AKI, it encountered difficulty predicting the onset of HA-AKI in higher-risk patients.

Moreover, the accuracy of predictions varied depending on the stage of HA-AKI under evaluation, with more successful predictions observed for Stage 1 HA-AKI compared to more severe cases.

The authors concluded that overall implementation of the tool might lead to elevated false-positive rates. They emphasized the need for further investigation into the tool's clinical impact.

We found that the Epic predictive model was better at ruling out low-risk patients than identifying high-risk patients. Identifying HA-AKI risk with predictive models could help support clinical decisions such as by warning providers against ordering nephrotoxic medications, but further study is needed before clinical implementation.

Sayon Dutta M.D., MPH, Study Lead Author, Clinical Informatics Team, Mass General Brigham Digital

Sayon Dutta is also an Emergency Medicine Physician at Massachusetts General Hospital.

Journal Reference:

Dutta, S., et al. (2024). External Validation of a Commercial Acute Kidney Injury Predictive Model. NEJM AI. doi.org/10.1056/aioa2300099.

Tell Us What You Think

Do you have a review, update or anything you would like to add to this news story?

Leave your feedback
Your comment type
Submit

While we only use edited and approved content for Azthena answers, it may on occasions provide incorrect responses. Please confirm any data provided with the related suppliers or authors. We do not provide medical advice, if you search for medical information you must always consult a medical professional before acting on any information provided.

Your questions, but not your email details will be shared with OpenAI and retained for 30 days in accordance with their privacy principles.

Please do not ask questions that use sensitive or confidential information.

Read the full Terms & Conditions.