Reporting of method comparison studies: a review of advice, an assessment of current practice, and specific suggestions for future reports

Abstract

Background: Anaesthetic journals frequently publish studies comparing measurement methods. A common method of analysis is the Bland and Altman plot, which relates the difference between paired measurements to the mean of the pair. Previous reviews have shown that key data are often omitted from reports using this method of analysis, and the analysis of more complex data is frequently insufficient.

Methods: We identified articles by searching reports, and subsequent citations, considering use of the method. We assembled a list of frequent and important criteria from these articles. These key features were tested by assessing articles in the yr 2013 and 2014, in five anaesthetic journals: Anaesthesia, Anesthesiology, Anesthesia and Analgesia, The British Journal of Anaesthesia, and The Canadian Journal of Anaesthesia.

Results: We found 29 features suggested for reporting such studies. Eight of these were frequently found. We chose 13 key features. In the journal articles reviewed to test these features, three features were almost always reported: the data structure, a plot of the bias, and the limits of agreement of the differences. Often, features required for adequate interpretation of the studies were absent, notably an a priori decision of acceptable limits of agreement, and an estimate of the precision of the limits of agreement.

Conclusions: Bland and Altman analysis remains poorly reported. Our formal list of key criteria will assist authors in providing all the relevant features of a study. We explain errors that may be made in reporting, and suggest methods for analysis, including easily available software.

Keywords: Accepted for publication; Editor’s key points; Research design; Software; Standards.