The Correlational Agreement Coefficient CA(≤,D) - A mathematical analysis of a descriptive goodness-of-fit measure

The Correlational Agreement Coefficient, CA(≤,D), was introduced by J.F.J. van Leeuwe in 1974 within Item Tree Analysis (ITA), a data-analytic method to derive quasi orders (surmise relations) on sets of bi-valued test items. Recently, it has become of interest in connection with Knowledge Space Theory (KST). The coefficient CA(≤,D) is used as a descriptive goodness-of-fit measure to select out of competing surmise relations one with maximal CA(≤,D) value. Formal aspects like boundedness, decomposition, and the interplay between consistency of a surmise relation (with a binary data matrix) and the attainment of the maximum value of CA(≤,D) are investigated. Dependence of CA(≤,D) on trivial response patterns is quantified by a functional relationship that allows one to bunch the impact of trivial response patterns in a single "bias term". These considerations should warn against inconsiderate use of the coefficient. Mathematical reasons for failed, however, heuristically plausible, properties are presented.