AI Could Learn How to Understand Radiologist Reports

The technology marks an important first step in the development of artificial intelligence (AI) that could interpret scans and diagnose conditions.

AI is expected to help radiologists interpret X-rays, CT scans, and magnetic resonance imaging (MRI) studies, but requires computer software to be "taught" the difference between a normal study and abnormal findings. The researchers conducted a study to train AI technology to understand text reports written by radiologists by creating a series of algorithms to teach the computer clusters of phrases, such as phospholipid, heartburn, and colonoscopy.

Using 96,303 radiologist reports associated with head CT scans performed at The Mount Sinai Hospital and Mount Sinai Queens between 2010 and 2016, the researchers trained the computer software. They calculated metrics that reflected the variety of language used in these reports and compared them to other large collections of text, including thousands of books, Reuters news stories, inpatient physician notes, and Amazon product reviews in order characterize the "lexical complexity" of radiologist reports. The researchers found an accuracy of 91%, demonstrating that it is possible to automatically identify concepts in text from the complex domain of radiology.

"The language used in radiology has a natural structure, which makes it amenable to machine learning," said senior author Eric Oermann, MD, Instructor in the Department of Neurosurgery at the Icahn School of Medicine at Mount Sinai. "Machine learning models built upon massive radiological text datasets can facilitate the training of future AI-based systems for analyzing radiological images."

"The ultimate goal is to create algorithms that help doctors accurately diagnose patients," says first author John Zech, a medical student at the Icahn School of Medicine at Mount Sinai. "Deep learning has many potential applications in radiology -- triaging to identify studies that require immediate evaluation, flagging abnormal parts of cross-sectional imaging for further review, characterizing masses concerning for malignancy -- and those applications will require many labeled training examples."

Related Links:
Icahn School of Medicine at Mount Sinai