Exceeding the limits of liver histology markers

Abstract

Background/aims: Alternatives to liver biopsy for staging liver disease caused by hepatitis C virus (HCV) have not appeared accurate enough for widespread clinical use. We characterized the magnitude of the impact of error in the "gold standard" on the observed diagnostic accuracy of surrogate markers.

Methods: We calculated the area under the receiver operating characteristic curve (AUROC) for a surrogate marker against the gold standard (biopsy) for a range of possible performances of each test (biopsy and marker) against truth and a gradient of clinically significant disease prevalence.

Results: In the 'best' scenario where liver biopsy accuracy is highest (sensitivity and specificity of biopsy are 90%) and the prevalence of significant disease 40%, the calculated AUROC would be 0.90 for a perfect marker (99% actual accuracy) which is within the range of what has already been observed. With lower biopsy sensitivity and specificity, AUROC determinations > 0.90 could not be achieved even for a marker that perfectly measured disease.

Conclusions: We demonstrate that error in the liver biopsy result itself makes it impossible to distinguish a perfect surrogate from ones that are now judged by some as clinically unacceptable. An alternative gold standard is needed to assess the accuracy of tests used to stage HCV-related liver disease.

Figure 1

Family of receiver operating characteristic (ROC) curves and area under the ROC curve (AUROC) values of a surrogate marker vs. true disease used in the estimation of the validity (AUROC) of the surrogate marker vs. the liver biopsy. These AUROC values represent the full range of sensitivity and specificity of a surrogate marker vs. true disease at an infinte number of cut-offs for defining true disease.

Figure 2

The expected performance of a surrogate marker of a liver biopsy is shown as the area under the receiver operating characteristic curve (AUROC) and depicted in each graph in a third dimension (y2 axis) across a gradient of lowest (red) to highest (yellow) according to increasing prevalence of significant liver disease (metavir ≥2, y axis) and increasing validity of the surrogate marker as measured by the AUROC of the marker vs. true disease (x axis). Since the actual validity of liver biopsy itself is unknown, we provide 4 representations, graphs a-d, in which the sensitivity and specificity of the biopsy vary (80-90 %). Hashed lines demark the AUROC values of markers (compared to biopsy) that have been reported in previous studies.(6-12, 14) Black shading represents the set of conditions under which the AUROC values exceed what has already been observed, which by our calculations should only occur if the biopsy is ≥90% sensitive and specific and the surrogate marker has near perfect accuracy, graph d, upper right. If biopsy sensitivity and specificity are lower, even a perfect surrogate marker will have AUROC versus liver biopsy in the range of what has already been reported.