Deploying and evaluating a machine learning intervention to improve clinical care and patient outcomes is a key step in moving clinical deterioration models from byte to bedside, according to a June 13 editorial in Critical Care Medicine that comments on a Mount Sinai study published in the same issue. The main study found that hospitalized patients were 43 percent more likely to have their care escalated and significantly less likely to die if their care team received AI-generated alerts signaling adverse changes in their health.
“We wanted to see if quick alerts made by AI and machine learning, trained on many different types of patient data, could help reduce both how often patients need intensive care and their chances of dying in the hospital,” says lead study author Matthew A. Levin, MD, Professor of Anesthesiology, Perioperative and Pain Medicine, and Genetics and Genomic Sciences, at Icahn Mount Sinai, and Director of Clinical Data Science at The Mount Sinai Hospital. “Traditionally, we have relied on older manual methods such as the Modified Early Warning Score (MEWS) to predict clinical deterioration. However, our study shows automated machine learning algorithm scores that trigger evaluation by the provider can outperform these earlier methods in accurately predicting this decline. Importantly, it allows for earlier intervention, which could save more lives.”
The non-randomized, prospective study looked at 2,740 adult patients who were admitted to four medical-surgical units at The Mount Sinai Hospital in New York. The patients were split into two groups: one that received real-time alerts based on the predicted likelihood of deterioration, sent directly to their nurses and physicians or a “rapid response team” of intensive care physicians, and another group where alerts were created but not sent. In the units where the alerts were suppressed, patients who met standard deterioration criteria received urgent interventions from the rapid response team.
Additional findings in the intervention group demonstrated that patients:
- were more likely to get medications to support the heart and circulation, indicating that doctors were taking early action; and
- were less likely to die within 30 days
“Our research shows that real-time alerts using machine learning can substantially improve patient outcomes,” says senior study author David L. Reich, MD, President of The Mount Sinai Hospital and Mount Sinai Queens, the Horace W. Goldsmith Professor of Anesthesiology, and Professor of Artificial Intelligence and Human Health at Icahn Mount Sinai. “These models are accurate and timely aids to clinical decision-making that help us bring the right team to the right patient at the right time. We think of these as ‘augmented intelligence’ tools that speed in-person clinical evaluations by our physicians and nurses and prompt the treatments that keep our patients safer. These are key steps toward the goal of becoming a learning health system.”
The study was terminated early due to the COVID-19 pandemic. The algorithm has been deployed on all stepdown units within The Mount Sinai Hospital, using a simplified workflow. A stepdown unit is a specialized area in the hospital where patients who are stable but still require close monitoring and care are placed. It’s a step between the intensive care unit (ICU) and a general hospital area, ensuring that patients receive the right level of attention as they recover.
A team of intensive care physicians visits the 15 patients with the highest prediction scores every day and makes treatment recommendations to the doctors and nurses caring for the patient. As the algorithm is continually retrained on larger numbers of patients over time, the assessments by the intensive care physicians serve as the gold standard of correctness, and the algorithm becomes more accurate through reinforcement learning.
In addition to this clinical deterioration algorithm, the researchers have developed and deployed 15 additional AI-based clinical decision support tools throughout the Mount Sinai Health System.
The Mount Sinai paper is titled “Real-Time Machine Learning Alerts to Prevent Escalation of Care: A Nonrandomized Clustered Pragmatic Clinical Trial.” The remaining authors of the paper, all with Icahn Mount Sinai except where indicated, are Arash Kia, MD, MSc; Prem Timsina, PhD; Fu-yuan Cheng, MS; Kim-Anh-Nhi Nguyen, MS; Roopa Kohli-Seth, MD; Hung-Mo Lin, ScD (Yale University); Yuxia Ouyang, PhD; and Robert Freeman, RN, MSN, NE-BC.