Volume of preclinical xenograft tumors is more accurately assessed by ultrasound imaging than manual caliper measurements

Abstract

Objective: The volume of subcutaneous xenograft tumors is an important metric of disease progression and response to therapy in preclinical drug development. Noninvasive imaging technologies suitable for measuring xenograft volume are increasingly available, yet manual calipers, which are susceptible to inaccuracy and bias, are routinely used. The goal of this study was to quantify and compare the accuracy, precision, and inter-rater variability of xenograft tumor volume assessment by caliper measurements and ultrasound imaging.

Methods: Subcutaneous xenograft tumors derived from human colorectal cancer cell lines (DLD1 and SW620) were generated in athymic nude mice. Experienced independent reviewers segmented 3-dimensional ultrasound data sets and collected manual caliper measurements resulting in tumor volumes. Imaging- and caliper-derived volumes were compared with the tumor mass, the reference standard, determined after resection. Bias, precision, and inter-rater differences were estimated for each mouse among reviewers. Bootstrapping was used to estimate mean and confidence intervals of variance components, intraclass correlation coefficients (ICCs), and confidence intervals for each source of variation.

Results: The average deviation from the true volume and inter-rater differences were significantly lower for ultrasound volumes compared with caliper volumes (P = .0005 and .001, respectively). Reviewer ICCs for ultrasound and caliper measurements were similarly low (1%), yet caliper volume variance was 1.3-fold higher than for ultrasound.

Conclusions: Ultrasound imaging more accurately, precisely, and reproducibly reflects xenograft tumor volume than caliper measurements. These data suggest that preclinical studies using the xenograft burden as a surrogate end point measured by ultrasound imaging require up to 30% fewer animals to reach statistical significance compared with analogous studies using caliper measurements.

Figure 1

Three-dimensional reconstructions of ultrasound imaging data for each xenograft tumor evaluated in the study. The segmented xenograft volume is shown in purple for a representative ultrasound reviewer and are ordered by TTV. Non-spheroid (irregularly shaped) xenografts, which are difficult to evaluate with calipers, are denoted by (*). Further details regarding each tumor can be found in Table 1.

Figure 2

Overall bias (solid lines) and standard error (dashed lines) are lower for ultrasound measurements (−53 mm³ ± 43 mm³) than caliper measurements (96 mm³ ± 88 mm³). Average bias for individual reviewers (X) was much lower in ultrasound than caliper, however, in both modalities, deviations per reviewer from the overall bias were small. Differences for each tumor are denoted as (●) for each reviewer.

Figure 3

Bias (solid lines) and 95% confidence intervals (dotted lines) were found to be smaller for ultrasound compared to caliper measurements. Data shown is from ultrasound Reviewer 1 (A) and caliper Reviewer 5 (B), both highly representative of other reviewer data sets. Individual xenograft difference from the true volume (●) was less in ultrasound measurements than caliper measurements. Lines are from nonparametric locally weighted scatterplot smoothing of the data points.

Figure 4

Inter-rater variability was significantly lower in ultrasound than caliper measurements. Tumors that resulted in large inter-rater differences for ultrasound did not correlate with tumors that resulted in larger inter rater differences for ultrasound (A). Inter-rater differences plotted against rank of true tumor volume for each mouse increased at faster rate for caliper measurements compared to ultrasound measurements as true volume increased (B). Trend lines feature nonparametric locally weighted scatterplot smoothing of the data points.

Figure 5

Median error (solid bars) for an ellipsoidal shaped tumor (xenograft 9) was small for both ultrasound and calipers compared to the true volume (dashed line). Median error for a non-ellipsoidal shaped tumor (xenograft 10) was small for ultrasound but large for calipers. The large error in caliper measurement for non-ellipsoidal tumors derives from the assumption that tumors are ellipsoidal inherent in the equation used to calculate tumor volume with caliper measurements.