How Can We Be Sure Artificial Intelligence Is Safe For Medical Use?
When Merdis Wells visited the diabetes clinic at the University Medical Center in New Orleans about a year ago, a nurse practitioner checked her eyes to look for signs of diabetic retinopathy, the most common cause of blindness.
At her next visit, in February of this year, artificial intelligence software made the call.
The clinic had just installed a system that's designed to identify patients who need follow-up attention.
The Food and Drug Administration cleared the system — called IDx-DR — for use in 2018. The agency said it was the first time it had authorized the marketing of a device that makes a screening decision without a clinician having to get involved in the interpretation.
It's a harbinger of things to come. Companies are rapidly developing software to supplement or even replace doctors for certain tasks. And the FDA, accustomed to approving drugs and clearing medical devices, is now figuring out how to make sure computer algorithms are safe and effective.
Wells was one of the first patients at the clinic in early February to be tested with the new device, which can be run by someone without medical training. The system produces a simple report that identifies whether there are signs that a patient's vision is starting to erode.
The test is quick and easy, which is by design. People with diabetes are supposed to get this screening test every year, but many don't. Brown says the new system could allow the clinic to screen a lot more patients for diabetic retinopathy.
That's the hope of the system's inventor, Michael Abramoff, an ophthalmologist at the University of Iowa and company founder.
"The problem is many people with diabetes only go to an eye-care provider like me when they have symptoms," he says. "And we need to find [retinopathy] before then. So that's why early detection is really important."
Abramoff spent years developing a computer algorithm that could scan retina images and automatically pick up early signs of diabetic retinopathy. And he wanted it to work in clinics, like the one in New Orleans, rather than in ophthalmologists' offices.
Developing the computer algorithm wasn't the hard part.
"It turns out the biggest hurdle, if you care about patient safety, is the FDA," he says.
That hurdle is essential for public safety, but not an easy one for a brand-new technology — especially one that makes a medical call without an expert on hand.
Often medical software gets an easy road to market, compared with drugs. Software is handled through the generally less rigorous pathway for medical devices. For most devices, the evaluation involves a comparison with something already on the market.
But this technology for detecting diabetic retinopathy was unique, and a patient's vision is potentially on the line.
When Abramoff approached the FDA, "of course they were uncomfortable at first," he says, "and so we started working together on how can we prove that this can be safe."
Abramoff needed to show that the technology was not just safe and effective but that it would work on a very diverse population, since all sorts of people get diabetes. That ultimately meant testing the machine on 900 people at 10 different sites.
"We went into inner cities, we went into southern New Mexico to make sure we captured all those people that needed to be represented," he says.
All the sites were primary care clinics, because the company wanted to demonstrate that the technology would well without having an ophthalmologist on hand.
That extensive test satisfied the FDA that the test would be broadly useable, and reasonably accurate. IDx-DR surpassed the FDA's requirement. Test results that indicated eye disease needed to be correct at least 85 percent of the time, while those finding no significant eye damage needed to be correct at least 82.5 percent of the time.
"It's better than me, and I'm a very experienced retinal specialist," Abramoff says.
The FDA helped guide the company's software through its regulatory process, which is evolving to accommodate inventions flowing out of artificial intelligence labs.
Read more : https://www.npr.org/sections/health-shots/2019/04/14/711775543/how-can-we-be-sure-artificial-intelligence-is-safe-for-medical-use?