Reliability analysis for continuous measurements: equivalence test for agreement

Abstract

In tandem with the rapid development of medical technology, methods for assessing intrarater and interrater reliability or agreement across tools for continuous measurements have become an increasingly important research topic. Thus far, a number of reliability assessment methods have been proposed. Among them, the limits of agreement and repeatability coefficients were found to be the most useful tools for assessing reliability when measurements are on a continuous scale. However, both are considered as descriptive methods. The concepts of consistency or conformity require an equivalence test without which judgment would be subjective. In this paper we will extend the repeatability coefficient approach and propose an equivalence test that can be used to confirm the agreement between two or more measurement tools or assess interrater and intrarater reliability. Using this approach, a formula to calculate sample size will also be suggested and examples will be provided to illustrate the method.