PhysEmp Insights

Insights

Clinical AI Gets Tested on the Wrong Work

Clinical AI is reaching tens of thousands of physicians, but a preprint of 127,833 clinician conversations found the median medical AI benchmark matches only 31 percent of real use. The validation those tests skip is landing on physicians at the point of…

HCA Healthcare expects about 10,000 of its physicians to be using ambient documentation AI by the end of this year, and Doximity says more than 190 health systems have now bought its clinical AI suite, up from 165 a quarter earlier. The tests used to judge these tools describe a different job. A preprint covering 127,833 clinician conversations with one academic system’s AI assistant found that the median published medical AI benchmark overlapped with only 31 percent of what clinicians actually asked it to do.

For AI in Physician Employment & Clinical Practice, that mismatch matters because the validation the benchmarks skip does not disappear. It moves to the physician using the tool, during the visit, on time nobody has budgeted.

This week’s rollout numbers came from the organizations doing the rolling out. HCA chief clinical officer Michael Schlosser, MD, told Becker’s the tool will reach more than 100 of the system’s roughly 189 hospitals for hospitalists and emergency physicians by year-end, plus about 3,000 outpatient physicians and APPs. He put the savings at about five minutes per note, or one to two hours per shift, and said outpatient clinics have added patients to schedules. Doximity CEO Jeff Tangney told investors the company’s AI suite now sits in 10 of the nation’s top 20 hospitals.

Physician uptake is moving more slowly than vendor footprints. A Doximity survey of 3,151 physicians, cited by PYMNTS, found ambient scribe use rose from 20 percent to 29 percent between two survey waves. Asked about their institution’s AI decision-making, 47 percent called it still evolving. Only 8 percent called it clear.

Even a Doximity researcher concedes the sequence is off. In a Doximity-partnered Modern Healthcare conversation, chief AI researcher Louis-Antoine Mullie, MD, a former intensivist, said that with some AI tools “deployment has come first.” His proposed minimum standard is plain: a physician using AI should provide safer care than a physician without it.

Continue reading →
Market Snapshot
Advertised ranges, latest PhysEmp Salary Report
“I've been doing this for over 30 years, and I'm having more fun than ever.”
The Sunday Read

Healthcare, Briefly

The physician workforce, weekly — without the 47-paragraph setup.

Subscribe Free →
For Residents & Early-Career MDs

Surviving Residency

Contracts, non-competes, and negotiation — the part nobody teaches. We fix that.

Subscribe Free →
More from Insights

We’ll get back to you shortly

By submitting your information you agree to PhysEmp’s Privacy Policy and Terms of Use…