Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Jan;11(1):59-67.
doi: 10.1177/1932296816662047. Epub 2016 Sep 25.

Significance and Reliability of MARD for the Accuracy of CGM Systems

Florian Reiterer  1 , Philipp Polterauer  1 , Michael Schoemaker  2 , Guenther Schmelzeisen-Redecker  2 , Guido Freckmann  3 , Lutz Heinemann  4 , Luigi Del Re  1
Affiliations

Significance and Reliability of MARD for the Accuracy of CGM Systems

Florian Reiterer et al. J Diabetes Sci Technol. 2017 Jan.

Abstract

Background: There is a need to assess the accuracy of continuous glucose monitoring (CGM) systems for several uses. Mean absolute relative difference (MARD) is the measure of choice for this. Unfortunately, it is frequently overlooked that MARD values computed with data acquired during clinical studies do not reflect the accuracy of the CGM system only, but are strongly influenced by the design of the study. Thus, published MARD values must be understood not as precise values but as indications with some uncertainty.

Data and methods: Data from a recent clinical trial, Monte Carlo simulations, and assumptions about the error distribution of the reference measurements have been used to determine the confidence region of MARD as a function of the number and the accuracy of the reference measurements.

Results: The uncertainty of the computed MARD values can be quantified by a newly introduced MARD reliability index (MRI), which independently mirrors the reliability of the evaluation. Thus MARD conveys information on the accuracy of the CGM system, while MRI conveys information on the uncertainty of the computed MARD values.

Conclusions: MARD values from clinical studies should not be used blindly but the reliability of the evaluation should be considered as well. Furthermore, it should not be ignored that MARD does not take into account the key feature of CGM sensors, the frequency of the measurements. Additional metrics, such as precision absolute relative difference (PARD) should be used as well to obtain a better evaluation of the CGM performance for specific uses, for example, for artificial pancreas.

Keywords: blood glucose; continuous glucose monitoring; diabetes therapy; glucose measurement; quality of measurement.

PubMed Disclaimer

Conflict of interest statement

Declaration of Conflicting Interests: The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: MS and GS are full-time employees of Roche Diabetes Care. GF is general manager of Institut für Diabetes-Technologie Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm, Ulm, Germany (IDT), which carries out studies evaluating BG meters and medical devices for diabetes therapy on its own initiative and on behalf of various companies. GF/IDT have received speakers’ honoraria or consulting fees from Abbott, Bayer, Berlin-Chemie, Becton-Dickinson, Dexcom, LifeScan, Menarini Diagnostics, Novo Nordisk, Roche Diabetes Care, Sanofi, and Ypsomed. LH is a consultant for a number of companies developing new diagnostic and therapeutic options for diabetes treatment and is a member of a Sanofi advisory board for biosimilar insulins. He is a partner of Profil Institute for Clinical Research, US and Profil Institut für Stoffwechselkrankheiten, Germany.

Figures

Figure 1.
Figure 1.
Measured CGM data, paired BG measurements, and calculated BG profiles (patient B1_01, day 2).
Figure 2.
Figure 2.
Exemplary CGM. Impact of the number of paired points on the uncertainty of MARD: upper and lower bounds of the confidence interval with probability γ = .95. The constant line represents the value to which it would converge.
Figure 3.
Figure 3.
Exemplary CGM. Confidence region of the MARD on the y-axis depending on the number of paired measurement points (see color code in the insert) and accuracy of the reference measurement system on the x-axis (black line: “true” MARD, purple line: MARD for number of paired points toward infinity).
Figure 4.
Figure 4.
Exemplary CGM. Effect of the relative error in the reference measurements on MARD (confidence interval, γ = .95). The width of the line indicates the confidence interval, the distance to the MARD0 (black dashed line) reflects the error introduced by the error in the reference measurement.
Figure 5.
Figure 5.
Combined effect of number of paired measurements and accuracy of the reference measurements on the uncertainty of MARD, here expressed by MRI.
Figure 6.
Figure 6.
Exemplary CGM. Effect of number and accuracy error of the reference measurements on the uncertainty of MARD (expressed by MRI indicated by the color code).
Figure 7.
Figure 7.
Effect of number and accuracy error of the reference measurements on the uncertainty of MARD (expressed by MRI), calculated for values of MARD0 between 8 % and 18 %.
See this image and copyright information in PMC

References

    1. Breton M, Kovatchev B. Analysis, modeling, and simulation of the accuracy of continuous glucose sensors. J Diabetes Sci Technol. 2008;2(5):853-862. - PMC - PubMed
    1. Kovatchev B, Patek S, Ortiz E, Breton M. Assessing sensor accuracy for non-adjunct use of continuous glucose monitoring. Diabetes Technol Ther. 2015; 17:177-186. - PMC - PubMed
    1. Bailey TS, Ahmann A, Brazg R, et al. Accuracy and acceptability of the 6-day Enlite continuous subcutaneous glucose sensor. Diabetes Technol Ther. 2014;16:277-283. - PubMed
    1. Damiano ER, El-Khatib FH, Zheng H, Nathan DM, Russell SJ. A comparative effectiveness analysis of three continuous glucose monitors. Diabetes Care. 2013;36:251-259. - PMC - PubMed
    1. Freckmann G, Pleus S, Link M, Zschornack E, Klötzer HM, Haug C. Performance evaluation of three continuous glucose monitoring systems: comparison of six sensors per subject in parallel. J Diabetes Sci Technol. 2013;7:842-853. - PMC - PubMed

Publication types

MeSH terms