Kappa coefficient calculation using multiple ratings per subject: a special communication

Abstract

The purpose of this special communication is to describe the application of the Kappa coefficient for the estimation of interobserver agreement. Kappa is a preferred statistic for the estimation of the accuracy of nominal and ordinal data in clinical research by physical therapists. A brief introduction to the properties of the Kappa coefficient is given, and a special case of Kappa for multiple ratings per subject is explained. A FORTRAN program, written specifically for the multiple-ratings situation, is described.