Coefficients of agreement between observers and their interpretation

Abstract

The measurement of agreement between ratings of patient symptomatology by two or more psychiatrists is discussed. It is noted that coefficients which allow for possible chance agreement are to be preferred, but they involve assumptions about the way in which chance factors may operate. Assumptions which involve prior probabilities of the incidence of a symptom appear to be too stringent, and a new coefficient, called the RE coefficient, is recommended in which is assumed to operate in a purely random way. Binary scales are discussed in detail, but methods of dealing with scales of wider range are also referred to.