Assessment of interchangeability rate between 2 methods of measurements: An example with a cardiac output comparison study

Abstract

The Bland-Altman (BA) and percentage error (PE) methods have been previously described to assess the agreement between 2 methods of medical or laboratory measurements. This type of approach raises several problems: the BA methodology constitutes a subjective approach to interchangeability, whereas the PE approach does not take into account the distribution of values over a range. We describe a new methodology that defines an interchangeability rate between 2 methods of measurement and cutoff values that determine the range of interchangeable values. We used a simulated data and a previously published data set to demonstrate the concept of the method. The interchangeability rate of 5 different cardiac output (CO) pulse contour techniques (Wesseling method, LiDCO, PiCCO, Hemac method, and Modelflow) was calculated, in comparison with the reference pulmonary artery thermodilution CO using our new method. In our example, Modelflow with a good interchangeability rate of 93% and a cutoff value of 4.8 L min, was found to be interchangeable with the thermodilution method for >95% of measurements. Modelflow had a higher interchangeability rate compared to Hemac (93% vs 86%; P = .022) or other monitors (Wesseling cZ = 76%, LiDCO = 73%, and PiCCO = 62%; P < .0001). Simulated data and reanalysis of a data set comparing 5 CO monitors against thermodilution CO showed that, depending on the repeatability of the reference method, the interchangeability rate combined with a cutoff value could be used to define the range of values over which interchangeability remains acceptable.

Figure 2

Graphical representations of range-based interchangeability rate between 2 methods of measurements for an RM with a coefficient of repeatability of 20% (Simulated data). (A) Bland–Altman (BA) graphical representation. The limits of agreements (1.70 units)(dark lines) and the bias (0.02 units) (red dashed line) are completed with an angular sector α of interchangeability (tan α = ) (red line) and the cutoff of interchangeability (using the calculated cutoff value = 5.42 ± 2.67 units). Interchangeable points and non-interchangeable points are depicted in green or red, respectively (N = 297 simulated data points). (B) Correlation graphical representation. The line of equality minus the bias (dark line) is completed by the angular sector β (red line) and the line of interchangeability and by the cutoff of interchangeability (red line [using the calculated cutoff value = 5.42 ± 2.67 units]). (C) Ranged based interchangeability rate according to the range. The cutoff of the interchangeability line is shown in red (using the calculated cutoff value = 5.42 ± 2.67 units).

Figure 3

X/Y plot and interchangeability rate over the range for Modelflow (A and B), Hemac (C and D), Wesseling cZ (E and F), LiDCO (G and H) and PiCCO (I and J), respectively. Interchangeable pairs of measurements are shown in green and non-interchangeable pairs of measurements are shown in red. (n = 199 measurement for each monitor).

Figure 4

Comparison between interchangeability rate of the 5 pulse contour monitors. Modelflow had the highest interchangeability rate compared to Hemac (^∗P = .022) and other devices (^∗∗∗P < .0001). Dotted lines show the limits for excellent (≥95%), good (≥90%), poor (75-90%), or not clinically relevant (<75%) interchangeability rates.