Comparative performance of PD-L1 scoring by pathologists and AI algorithms

Aim: This study evaluates the comparative effectiveness of pathologists versus artificial intelligence (AI) algorithms in scoring PD-L1 expression in non-small cell lung carcinoma (NSCLC). Immune-checkpoint inhibitors have revolutionized NSCLC treatment, with PD-L1 expression, measured as the tumour proportion score (TPS), serving as a critical predictive biomarker for therapeutic response. Methods and Results: In our analysis, 51 SP263-stained NSCLC cases were scored by six pathologists using light microscopy and whole-slide images (WSI), alongside evaluations by two commercially available software tools: uPath software (Roche) and the PD-L1 Lung Cancer TME application (Visiopharm). The study examined intra- and interobserver agreement among pathologists at TPS cutoffs of 1% and 50%, revealing moderate interobserver agreement (Fleiss' kappa 0.558) for TPS <1% and almost perfect agreement (Fleiss' kappa 0.873) for TPS ≥50%. Intraobserver consistency was high, with Cohen's kappa ranging from 0.726 to 1.0. Comparisons between the AI algorithms and the median pathologist scores showed fair agreement for uPath (Fleiss' kappa 0.354) and substantial agreement for the Visiopharm application (Fleiss' kappa 0.672) at the 50% TPS cutoff. Conclusion: These results indicate that while there is strong interobserver concordance among pathologists at higher TPS levels, the performance of AI algorithms is less consistent. The study underscores the need for further refinement of AI tools to match the reliability of expert human evaluation, particularly in critical clinical decision-making contexts.