Deterioration of R-Wave Detection in Pathology and Noise: A Comprehensive Analysis Using Simultaneous Truth and Performance Level Estimation

Objective: For long-term electrocardiography (ECG) recordings, accurate R-wave detection is essential. Several algorithms have been proposed but not yet compared on large, noisy, or pathological data, since manual ground-truth establishment is impossible on such large data. Methods: We apply the simultaneous truth and performance level estimation (STAPLE) method to ECG signals comparing nine R-wave detectors: Pan and Tompkins (1985), Chernenko (2007), Arzeno et al. (2008), Manikandan et al. (2012), Lentini et al. (2013), Sartor et al. (2014), Liu et al. (2014), Arteaga-Falconi et al. (2015), and Khamis et al. (2016). Experiments are performed on the MIT-BIH database, TELE database, PTB database, and 24/7 Holter recordings of 60 multimorbid subjects. Results: Existing approaches on R-wave detection perform excellently on healthy subjects (F-measure above 99% for most methods), but performance drops to a range of F = 90.10% (Khamis et al.) to F = 30.10% (Chernenko) when analyzing the 37 million R-waves of multimorbid subjects. STAPLE improves existing approaches ( and Delta;F = 0.04 for the MIT-BIH database and and Delta;F = 0.95 for the TELE database) and yields a relative (not absolute) scale to compare algorithms' performances. Conclusion: More robust R-wave detection methods or flexible combinations are required to analyze continuous data captured from pathological subjects or that is recorded with dropouts and noise. Significance: STAPLE algorithm has been adopted from image to signal analysis to compare algorithms on large, incomplete, and noisy data without manual ground truth. Existing approaches on R-wave detection weakly perform on such data.