Assessing the quality of publications evaluating the accuracy of blood glucose monitoring systems

Abstract

Many studies determine the performance of blood glucose monitoring (BG) systems. Correct evaluation is, however, complex, and apparent contradiction of results creates confusion. This study aimed to provide an overview of frequently made errors and to develop easy-to-use checklists to verify the quality of such studies. Building on the work from Mahoney and Ellison and subsequent re-evaluation, study designs of accuracy studies were assessed, and best practice and internationally accepted norms were determined. Key issues were collated, and two simplified checklists were developed: one for the assessment of analytical accuracy studies and a second for guidance with studies assessing the influence of interferences. The checklists have been used in a feasibility study with 20 representative studies selected from a literature search between 2007 and 2012. This check revealed that limitations in the designs and methods of studies assessing the performance of BG systems are common. The use of the accuracy checklist with the 20 representative studies showed that only 20% were in agreement with most of the issues deemed important and that 40% showed clear nonconcordance with ISO 15197. The use of the interference checklist showed that only 50% of the publications were in good agreement with the quality checks. In agreement with previous studies, which concluded many evaluations are performed poorly and present questionable conclusions, the use of these checklists demonstrated that few publications adhered to international guidelines and recommendations. Taking this into consideration, it becomes obvious that the publications must be examined in more detail to establish their quality and the validity of conclusions drawn.

FIG. 1.

Number of publications assessing the performance of several blood glucose systems between 2004 and 2011.