Assessment of noticing of classroom disruptions: a multi-methods approach

Teachers’ noticing as a basic precondition for effective teaching is characterized by focusing on relevant events in the classroom and ignoring the irrelevant. In recent years, many researchers have used eye-tracking methodology in classroom observations to gather information about the continuous attentional processes of teachers. Despite the general validity of the eye–mind assumption, methodological triangulation is necessary to draw conclusions about the where and why of the focus of attention. Although in previous studies, different data sources like gaze and verbal data have been used, the analyses were mostly conducted separately, instead of directly combining the data. In our study, we collected verbal data (retrospective think-aloud; RTA) and a reaction-based concurrent measure (keystroke) to assess the noticing process of novice and experienced teachers (N = 52) while they watched staged videos of classroom situations. For a direct triangulation, we combined these data with eye-tracking data. The aim of the study was to combine both measures with eye-tracking parameters that indicate attentional processes (fixation count, mean fixation duration, and revisits), and with expertise. We found that participants who were aware of the critical incidents in the videos (they gave a keystroke or mentioned the incident in the RTA), showed—as expected—a higher number of fixations and more revisits to the appropriate area, but a comparable mean fixation duration. However, expertise differences regarding accuracy in both measures could not be shown. We discuss methodological issues regarding the implementation of RTA and keystroke as measurements for the noticing process because—despite only partially significant results—both methods are promising as they allow complementation and possible correction of eye-movement-only data.