Maximum likelihood estimation of agreement in the constant predictive probability model, and its relation to Cohen's kappa

Abstract

The alpha agreement parameter is defined as the proportion of a population of items that are classified identically "for cause" by two classifiers, the remaining items being classified at random. The parameters of the corresponding constant predictive probability model are shown to be estimable by the method of maximum likelihood, and a simulation study indicates applicability of the asymptotic results to finite samples. The new estimator tends to be larger than Cohen's kappa, except in the case of uniform margins. An application is made to the validity of cancer risk items included in a cancer registry.