Ankylosing spondylitis antirheumatic drug trials: Effects of a standardized instructional viddeotape on the reliability of observer-dependent outcome measures

Abstract

Aims: A study was designed to assess the effects of a standardized instruction videotape on reducing interobserver variability for several commonly used observer-dependent outcome measures.

Methods: During a single day, six rheumatologists independently examined six patients with ankylosing spondylitis (AS) in a predetermined order using a Latin square design, before and after viewing a standardized videotape demonstrating 14 examination techniques. Reliability coefficients were calculated based on the variance components of the analysis of variance (ANOVA) table.

Results: Prestandardization reliability coefficients were <0.80 for three measures. Following standardization 12 reliability coefficients exceeded 0.80. For the majority of measures prestandardization reliability coefficients were high and no further improvement in reliability could be demonstrated. For one measure of cervical extension, but not another, an important and beneficial effect in reliability was noted. It was not possible to achieve adequate reliability in the performance of the chest excursion measurement.

Conclusions: It is usually assumed that serial measurement in clinical trials should be performed by the same assessor because of concern regarding interobserver variability. However, the high levels of prestandardization interobserver reliability observed in this study indicate that for these variables serial measurements in a clinical trial could be made by different assessors, assuming they were equally skilled. This observation has important implications for outcome measurement in AS clinical trials. Although high levels of prestandardization reliability precluded the demonstration of any significant effect, we speculate that the videotape might be effective in training less experienced assessors. Nevertheless, an alternative approach to standardization may be required for the chest excursion measurement. Reductions in observer variability have the potential to diminish sample size requirements for AS antirheumatic drug studies. The use of a videotape to achieve this goal offers cost and convenience advantages over one-on-one training procedures, and this method should be further assessed in a group of less experienced assessors.