As artificial intelligence (AI) technologies are ever more tightly integrated into clinical practice, the work among professional imaging societies has begun in earnest to close the knowledge gap between those who are familiar, even comfortable, with its applications, and those who may not have progressed to develop even a functional understanding of what they are and are capable of doing.
At the end of 2021, the Radiological Society of North America (RSNA) introduced a new educational program, its Imaging AI Certificate, which is intended to deliver a pathway for radiologists to understand more about the various aspects of AI-driven technologies, and how to incorporate them within their radiology practices.
The course curriculum was developed by a trio of instructors: Matthew B. Morgan, the Medical Director of IT and Quality Improvement for Breast Cancer Imaging at the Huntsman Cancer Hospital in Salt Lake City, Utah, and an associate professor of radiology at the University of Utah School of Medicine; Linda Moy, a breast imaging specialist at NYU Langone; and George Shih, Professor of Clinical Radiology at Weill Cornell Medical College, both in New York, New York.
Morgan described the program as “a unique offering” among presently available training materials that cover the adoption and use of artificial intelligence technologies in the imaging space because it offers “hands-on” exercises in which students perform some of the work that goes into training and testing a machine learning model.
“This is a course designed for radiologists primarily, but will likely be of interest to other personnel within a radiology department,” Morgan said. “It introduces the concepts at a basic level and then proceeds to show how it can be implemented in the department.”
“I think it will be of great interest to radiologists who are just getting started, as well as to those who are mid-to-late-career,” he said. “It really has some nice hands-on features that get beyond the didactic type of presentation. They’ll be able to annotate images, train an AI model, and see how their model performs.”
Each of the learning modules are hosted online, and participants progress through them at their own paces, watching short videos of six to 12 minutes each, which are bundled into modules with interactive exercises. The modules span introductory information, data curation, data annotation and model building, model evaluation, AI ethics, and clinical implementation. Morgan said he hopes that, by getting under the hood of algorithm construction and testing, participants will be able to shed any apprehensions about the accessibility of AI technologies and their relevance to the day-to-day responsibilities of radiologists and their colleagues.
“I think a lot of people just see AI as something that only highly technical people will understand, and I think, after completing the course, people will be able to say, ‘Now I get what an AI algorithm is,’ rather than have it be something that feels kind of scary and nebulous,” he said.
“They will see what some of the pitfalls are in the process of creating these algorithmic models,” Morgan said. “An AI algorithm is essentially a choice of a method to use. Then when you apply that method to actual data, it becomes an AI model. You’ll learn that it’s humans putting circles around findings, and realizing that that could impact the results depending upon how the algorithm is trained. It matters how these models are trained because they’re only as smart as the training they receive.”
The ethics portion of the course helps students to unpack “the more difficult questions” surrounding AI, he said, including understanding that algorithms are created with mechanisms to guard against conscious and unconscious biases from their creators.
“For example, are there populations of people who are not included for some reason or another?” Morgan said. “We’ll give them a framework to be able to think about those things, to be able to talk to vendors about what they’ve done to mitigate for that kind of issue, and hopefully start to get people comfortable with the implementation component, which I think we’re barely starting to get into.”
Lessons on AI implementation help imaging professionals hone their judgment about what these technological offerings are, and how to go about intelligently selecting the right one for their institutions. Once selected, these lessons can help instruct them on how to incorporate AI technologies within their systems, to understand how the technologies affect workflow, and to measure the performance of a system to ensure it achieves its intended goals.
The genesis of the curriculum is rooted in the RSNA Informatics Committee, and the program took the better part of a year to develop, Morgan said. At least three additional modules will be created to supplement those available at the outset of the program, and the instructors will work to incorporate feedback from past students into subsequent offerings, Morgan said.
“It’s an asynchronous, self-paced type of approach, where people will be able to do it in the time they have available,” Morgan said. “This is for radiologists, by radiologists, and it has these hands-on components that I don’t think I’ve seen anywhere else.”
“We’ve seen so much hype around AI,” he continued. “It looks like the lower-hanging fruit of imaging findings that are time-sensitive — intracranial hemorrhage, pulmonary embolism, these sorts of things — people are ready to work with.
“As a field, we hope that there are more offerings that are not necessarily the computer trying to find things faster than the radiologist; offerings that are non-pixel-based,” Morgan said.
“There’s a world of other things to apply this to: non-imaging-based applications, such as natural language processing, or other novel things that could help with decision support for the radiologist, whether compiling data from the EMR or showing probabilistic diagnoses derived from text rather than from imaging. There are things that humans don’t see that a machine-learning-type system could be good at enhancing,” added Morgan.
Moy believes a foundational understanding of AI tools will help radiologists make better use of AI products that are in development or on the market currently.
“Are you going to invest in a technology that’s a black box?” Moy said. “These new AIs are very complex, and we don’t want to make the same mistake we did with computer-aided diagnosis (CAD),” These earlier AI tools did not deliver on the expectations that CAD could function as a second image reader. Not only did it not detect more breast cancers, but it actually prompted more false-positive findings.
“We were all excited about CAD,” Moy said. “Shortly after these CAD systems came out, CMS provided reimbursement for this technology. This reimbursement and the wide availability of digital mammography led to widespread dissemination of CAD. More recently, CAD was bundled into the fee for interpreting mammograms. As a result, there is concern about the return on investment for AI tools.
“To summarize, the ability to assess whether an AI model will work in clinical practice and issues regarding reimbursement for these AI tools have loom over radiologists who wish to purchase these AI products,” she said. “We are not going to be that naive again.”
For radiologists who do not have a background in computer science, Moy hopes the RSNA course will help professionals “to really see if the AI product will fit into their practice.”
“The unique aspects of our RSNA AI Certificate program are the short videos by experts, the case-based curriculum, and a hands-on approach with practice-based applications for radiologists,” she said. “This certificate will allow radiologists who are interested in AI to acquire fundamental competency in AI and assist them with accelerating their career. These skill sets are essential in order to understand how to use AI algorithms in clinical practice, and to develop realistic expectations of how AI software may change clinical workflows.”
After radiologists purchase an AI technology, it may be susceptible to “model drift,” Moy said; i.e., a change in performance after the initial model is established. As the gatekeepers of imaging technologies, radiologists must develop “this fundamental knowledge where you can peer beyond the hood of the car and figure out what is causing the change in performance,” she said.
“One of our goals is that upon completion of this program, radiologists will understand whether a product really makes its claim,” Moy said. “[We want] to get mainstream radiologists to peer inside the black box of AI to get some practical education.”
Although the program has been developed for radiologists, including those in private or academic practice, instructors believe it will be equally useful for medical students, residents and fellows.
Moy argues that the more that health care professionals are involved in the development of these technical tools, the more patient care can improve. She believes the AI Certificate course will open doors to greater knowledge and understanding of how AI tools work, how they’re defined, and what their limitations are.
“It’s going beyond what taking the vendor says at face value, and seeing whether these models really work for our own practices,” Moy said.
Bill McGinnis, executive director of business and sales at image analysis and AI specialist ContextVision of New York, New York, said he believes that any efforts to drive greater understanding of what AI means among medical imaging professionals is a welcome one from the vendor perspective.
“AI is such a generic term right now; it’s become more of a marketing term than anything from the technical side,” McGinnis said. “Any education we can get around AI in general is great. The more we can expose end-users to these different technologies, the more we can help to answer questions and alleviate fears.”
To McGinnis, that means explaining to prospective customers that “smart technologies” can help practitioners deliver more consistent results, make diagnostic choices more easily, and generally elevate the level of conversation around AI from the general to the specific.
“The more we can take that top of the cake off and get into the good stuff, the special things that we have, that to me is an advantage to everybody,” McGinnis said. “It makes RSNA look good, it helps their members, because they now have a basic understanding, and it helps the vendor and the OEM because we can focus on our pitch and then the specifics of what we have that’s different.”
Acknowledging that workflow support technologies were among the trendiest at RSNA in 2021, McGinnis pointed out that many AI-based imaging offerings are being developed for audiences other than radiologists. These might include clinicians who need assistance in decision support, but they’re just as likely to include people who encounter imaging technologies but who aren’t sonographers or X-ray technicians.
“The high-end AI techs are targeted for everybody; the hand-held items are not for traditional radiologists,” McGinnis said. “They could be for EMTs, nursing staff, somebody in the ED who may be a skilled doctor, but has never trained on ultrasound.”
“That’s a group that needs more education,” he said. “I have found more of a challenge closing a deal with that audience than with traditional radiologists. There might be a larger part of the audience, a larger segment of this market, that still needs to be educated. RSNA has been a driver of adoption before; hopefully more people follow the leader and we’re able to see this in more areas of medicine, not just in highly specialized societies.”
Cardiovascular and interventional radiologist Kris Kandarpa, director of research sciences and strategic directions at the National Institute of Biomedical Imaging and Bioengineering at the U.S. National Institutes of Health, said that he’s pleased that more education on AI-driven technologies is finding its way into the field.
“While the use of mature technologies, such as PACS and EHR, can be honed during residency/training years, it is still very important for a core group of radiologists (department leaders) to have in-depth knowledge and be involved in such purchasing decisions,” Kandarpa said. “AI is new to just about everyone in this field. There is little knowledge on what imaging-AI is, does, or how it can be best utilized in practice. There is a lack of trust in the technology, its eventual role, and a fear of erosion in the value of the radiology profession, especially amongst medical students and trainees.”
As McGinnis noted, learning how AI may affect health care delivery goes just as well for “non-imaging professionals, especially non-radiologist physicians,” he said, adding that the additional understanding will help them to trust decisions radiologists make that are based in the use of AI.
“Just as many radiologists learned MRI physics in their post-residency years, it is important for working radiologists to learn how AI may affect their practices,” Kandarpa said.
“It is far too early to relegate the imaging-AI knowledge to a select few,” he said. “The widespread dissemination of an understanding of what AI can do in practice will go a long way in alleviating fears and building trust in promising technology.”