TY - JOUR
T1 - Expert gaze as a usability indicator of medical AI decision support systems
T2 - a preliminary study
AU - Castner, Nora
AU - Arsiwala-Scheppach, Lubaina
AU - Mertens, Sarah
AU - Krois, Joachim
AU - Thaqi, Enkeleda
AU - Kasneci, Enkelejda
AU - Wahl, Siegfried
AU - Schwendicke, Falk
N1 - Publisher Copyright:
© The Author(s) 2024.
PY - 2024/12
Y1 - 2024/12
N2 - Given the current state of medical artificial intelligence (AI) and perceptions towards it, collaborative systems are becoming the preferred choice for clinical workflows. This work aims to address expert interaction with medical AI support systems to gain insight towards how these systems can be better designed with the user in mind. As eye tracking metrics have been shown to be robust indicators of usability, we employ them for evaluating the usability and user interaction with medical AI support systems. We use expert gaze to assess experts’ interaction with an AI software for caries detection in bitewing x-ray images. We compared standard viewing of bitewing images without AI support versus viewing where AI support could be freely toggled on and off. We found that experts turned the AI on for roughly 25% of the total inspection task, and generally turned it on halfway through the course of the inspection. Gaze behavior showed that when supported by AI, more attention was dedicated to user interface elements related to the AI support, with more frequent transitions from the image itself to these elements. When considering that expert visual strategy is already optimized for fast and effective image inspection, such interruptions in attention can lead to increased time needed for the overall assessment. Gaze analysis provided valuable insights into an AI’s usability for medical image inspection. Further analyses of these tools and how to delineate metrical measures of usability should be developed.
AB - Given the current state of medical artificial intelligence (AI) and perceptions towards it, collaborative systems are becoming the preferred choice for clinical workflows. This work aims to address expert interaction with medical AI support systems to gain insight towards how these systems can be better designed with the user in mind. As eye tracking metrics have been shown to be robust indicators of usability, we employ them for evaluating the usability and user interaction with medical AI support systems. We use expert gaze to assess experts’ interaction with an AI software for caries detection in bitewing x-ray images. We compared standard viewing of bitewing images without AI support versus viewing where AI support could be freely toggled on and off. We found that experts turned the AI on for roughly 25% of the total inspection task, and generally turned it on halfway through the course of the inspection. Gaze behavior showed that when supported by AI, more attention was dedicated to user interface elements related to the AI support, with more frequent transitions from the image itself to these elements. When considering that expert visual strategy is already optimized for fast and effective image inspection, such interruptions in attention can lead to increased time needed for the overall assessment. Gaze analysis provided valuable insights into an AI’s usability for medical image inspection. Further analyses of these tools and how to delineate metrical measures of usability should be developed.
UR - http://www.scopus.com/inward/record.url?scp=85200049584&partnerID=8YFLogxK
U2 - 10.1038/s41746-024-01192-8
DO - 10.1038/s41746-024-01192-8
M3 - Article
AN - SCOPUS:85200049584
SN - 2398-6352
VL - 7
JO - npj Digital Medicine
JF - npj Digital Medicine
IS - 1
M1 - 199
ER -