A machine learning-based model that enables medical institutions to predict the mortality risk for individual cardiac surgery patients has been developed by a Mount Sinai research team, providing a significant performance advantage over current population-derived models.

The new data-driven algorithm, built on troves of electronic health records (EHR), is the first institution-specific model for assessing a cardiac patient’s risk prior to surgery, thus allowing health care providers to pursue the best course of action for that individual. The team’s work was described in a study published in The Journal of Thoracic and Cardiovascular Surgery (JTCVS) Open. “The standard-of-care risk models used today are limited by their applicability to specific types of surgeries, leaving out significant numbers of patients undergoing complex or combination procedures for which no models exist,” said senior author Ravi Iyengar, PhD, the Dorothy H. and Lewis Rosenstiel Professor of Pharmacological Sciences at the Icahn School of Medicine at Mount Sinai, and Director of the Mount Sinai Institute for Systems Biomedicine. “Our team rigorously combined electronic health record data and machine learning methods to demonstrate for the first time how individual institutions can build their own risk models for post-cardiac surgery mortality.”

Prediction models based on machine learning algorithms have been generated across diverse fields of medicine, and some have shown improved results over their standard-of-care counterparts. In cardiac surgery, The Society of Thoracic Surgeons (STS) risk scores are considered the gold standard, and are routinely used to assess a cardiac surgery patient’s procedural risk. While they continue to provide important benchmarks for hospitals to evaluate and improve their performance, they are derived from population-level data and therefore may fail to accurately predict risk for specific patients with complicated pathologies who require tailored preoperative evaluations and complex surgeries.

AI models based on EHR data

Cardiovascular surgeons and data science specialists at The Mount Sinai Hospital, supervised by co-senior author Gaurav Pandey, PhD, Associate Professor of Genetics and Genomic Sciences at Icahn Mount Sinai, hypothesized that machine learning-based models using EHR data from their own institution could offer an effective solution. So they created a rigorous machine learning framework using routinely collected EHR data to develop a risk prediction model for postsurgical mortality that is both personalized to the patient and specific to the hospital – implicitly incorporating important information about Mount Sinai’s patient population, such as demographics, socioeconomic factors, and health characteristics. This is in contrast to population-derived models like STS, which are based on data from diverse health systems in different parts of the country. Further driving the performance of this methodology was a highly effective open-source prediction algorithm known as XGBoost, which builds an ensemble of decision trees by progressively focusing on harder-to-predict subsets of training data.

Credit: Mount Sinai Health System

Leading the study was co-corresponding author Aaron J Weiss, MD, PhD, formerly a cardiothoracic surgery resident at The Mount Sinai Hospital and a clinical research PhD graduate from Icahn Mount Sinai, who is now at Cleveland Clinic. Arjun Yadaw, PhD, Adjunct Assistant Professor of Pharmacological Sciences at Icahn Mount Sinai, and currently a Senior Data Scientist at the National Center for Advancing Translational Science, part of the National Institutes of Health (NIH NCATS), also co-led the work. These researchers used XGBoost to model 6,392 cardiac surgeries performed at The Mount Sinai Hospital from 2011 to 2016, including heart valve procedures; coronary artery bypass graft; aortic resection, replacement, or anastomosis; and reoperative cardiac surgeries, which have been shown to appreciably increase mortality risk. The team then compared the performance of its model to STS models for the same patient sets.

The study showed that the XGBoost model outperformed STS risk scores for mortality in all commonly conducted categories of cardiac surgery for which STS scores were designed. Prediction performance of the XGBoost model across all surgery types was also high, demonstrating the potential of machine learning and EHR data for building effective institution-specific models. “Accurate prediction of postsurgical mortality is critical to ensure the best outcomes for cardiac surgery patients, and our study shows that institution-specific models may be preferable to the clinical standard based on population data,” said Dr Pandey. “Just as importantly, we’ve demonstrated that it’s practical for healthcare institutions to develop their own predictive models through sophisticated machine learning algorithms to replace or complement the established STS template.”