Yale tested 6 early-warning algorithms used by health systems. Epic’s AI tool didn’t fare well

Anne Thompson

akstanwyck

More stories by Anne

‘Nosferatu’: How to Make a Robert Eggers Movie, with Help from Mel Brooks and Chris Columbus

In the Photorealistic

September 5 Director Tim Fehlbaum Didn

Time：2024-10-15 19:10

A patient on artificial lung ventilation lays next to monitors in an intensive care department -- health tech coverage from STAT — Adobe

By Katie Palmer

Oct. 15, 2024

Health Tech Correspondent

In the early days of the Covid-19 pandemic, hospitals were desperate for ways to manage the flood of seriously ill patients. Many turned to an artificial intelligence algorithm developed by Epic Systems, the electronic health record company, to predict which patients were most likely to rapidly deteriorate so they could get the critical care they needed.

Then, and now, many health systems have implemented this kind of proprietary AI algorithm without a clear sense of how well they perform. But in a rare head-to-head analysis, Yale’s health system evaluated the statistical performance of six early warning scores on the same clinical data from seven of its hospitals, publishing its results Tuesday in JAMA Network Open. It revealed that some clinical AI models aren’t all they’re cracked up to be.

“We didn’t set out to write a paper,” said co-author Deborah Rhodes, chief quality officer for Yale New Haven Health System and associate dean of quality for Yale School of Medicine. “We set out to find the best tool.” Across the country, Epic’s early warning score is widely used because it comes built into the company’s electronic health record. “My health system really wanted to go with the tool that was free,” said Rhodes.

STAT+ Exclusive Story

This article is exclusive to STAT+ subscribers

Unlock this article — and get additional analysis of the technologies disrupting health care — by subscribing to STAT+.

Already have an account? Log in

Monthly

$39

Totals $468 per year

$39/month Get Started

Totals $468 per year

Starter

$20

for 3 months, then $399/year

$20 for 3 months Get Started

Then $399/year

Annual

$399

Save 15%

$399/year Get Started

Save 15%

11+ Users

Custom

Savings start at 25%!

Request A Quote Request A Quote

Savings start at 25%!

2-10 Users

$300

Annually per user

$300/year Get Started

$300 Annually per user

View All Plans

To read the rest of this story subscribe to STAT+.

Subscribe Log In Artificial intelligence, health tech, STAT+ Submit a correction requestReprints

Katie Palmer
Health Tech Correspondent
Katie Palmer covers telehealth, clinical artificial intelligence, and the health data economy — with an emphasis on the impacts of digital health care for patients, providers, and businesses.