Performance evaluations of continuous glucose monitoring systems: precision absolute relative deviation is part of the assessment

Abstract

Background: Even though a Clinical and Laboratory Standards Institute proposal exists on the design of studies and performance criteria for continuous glucose monitoring (CGM) systems, it has not yet led to a consistent evaluation of different systems, as no consensus has been reached on the reference method to evaluate them or on acceptance levels. As a consequence, performance assessment of CGM systems tends to be inconclusive, and a comparison of the outcome of different studies is difficult.

Materials and methods: Published information and available data (as presented in this issue of Journal of Diabetes Science and Technology by Freckmann and coauthors) are used to assess the suitability of several frequently used methods [International Organization for Standardization, continuous glucose error grid analysis, mean absolute relative deviation (MARD), precision absolute relative deviation (PARD)] when assessing performance of CGM systems in terms of accuracy and precision.

Results: The combined use of MARD and PARD seems to allow for better characterization of sensor performance. The use of different quantities for calibration and evaluation, e.g., capillary blood using a blood glucose (BG) meter versus venous blood using a laboratory measurement, introduces an additional error source. Using BG values measured in more or less large intervals as the only reference leads to a significant loss of information in comparison with the continuous sensor signal and possibly to an erroneous estimation of sensor performance during swings. Both can be improved using data from two identical CGM sensors worn by the same patient in parallel.

Conclusions: Evaluation of CGM performance studies should follow an identical study design, including sufficient swings in glycemia. At least a part of the study participants should wear two identical CGM sensors in parallel. All data available should be used for evaluation, both by MARD and PARD, a good PARD value being a precondition to trust a good MARD value. Results should be analyzed and presented separately for clinically different categories, e.g., hypoglycemia, exercise, or night and day.

Figure 1.

Glycemic profile of a given patient wearing two identical CGM systems over a period of 24 h. A, nighttime with 1–2 BG readings; B, daytime with ≥1 BG readings every hour; C, dynamic phase 1 BG reading every 15 min.

Figure 2.

Interpretation of SD.

Figure 3.

Anomalies seen in CGM recordings.

Figure 4.

Differences in two CGM recordings in the same patient in parallel, one day with significant differences.

Figure 5.

Glucose profiles registered with different CGM systems used in parallel in a given patient (unpublished data).

Figure 6.

Raw CGM and BG measurements for a patient wearing six sensors in parallel.

Figure 7.

Continuous PARD and MARD evaluation of the measurements of Figure 6.