Assessment of the reproducibility of postprocessing dynamic CT perfusion data

Abstract

Background and purpose: Commercially available software programs for the conversion of dynamic CT perfusion (CTP) source data into cerebral blood volume (CBV), cerebral blood flow (CBF), and mean transit time (MTT) maps require operators to subjectively define parameters that are used in subsequent postprocessing calculations. Our purpose was to define the variability of CBV, CBF, and MTT values derived from CTP maps generated from the same source data postprocessed by three different CT technologists (CTTs).

Methods: Raw data derived from dynamic CTP examinations performed in 20 subjects were postprocessed seven times by three experienced CTTs. Parenchymal regions of interest derived from each map (CBV, CBF, and MTT) were compared. The CBF maps generated by each technologist were also qualitatively assessed. Decisions made by each analyzer during postprocessing were assessed.

Results: The intraclass correlation coefficients were 0.73 (95% CI, 0.64-0.81), 0.87 (0.83-0.91) and 0.89 (0.85-0.93), for the CBV, CBF, and MTT parenchymal regions of interest, respectively. All individual correlation coefficients between data sets were significant to a P value <.05. Measurement error, made solely on the basis of different technologists postprocessing the same source data and expressed as the coefficients of variation, were 31%, 30%, and 14% for CBV, CBF, and MTT, respectively. The selection of the arterial input function (AIF) region of interest, venous function region of interest, and preenhancement interval were very reproducible. The technologists differed significantly with respect to the selection of the postenhancement image (PoEI) (P <.01). A retrospective review of the individual CBF maps indicated that variance in the PoEI selection accounted for much of the variation in the qualitative appearance of the CBF maps generated by different technologists. The PoEI was selected to demarcate the baseline of the AIF time-attenuation curve. It is likely that this method of PoEI selection significantly contributed to intra- and interanalyzer variability.

Conclusion: There is a high degree of correlation between parenchymal regions of interest derived from CBV, CBF, and MTT maps generated from the same dynamic CTP source data postprocessed by different operators. The level of agreement, however, may not be sufficient to incorporate quantitative values into clinical decision making. Quantitative differences between parenchymal regions of interest were not infrequently manifest as significant differences in the qualitative appearance of the CBF maps. It is likely that, with optimization of postprocessing parameter selection, the degree of variability may be substantially reduced.

Fig 1.

Example of the selection of postprocessing parameters (15). Dynamic CTP data are derived from 89 sequential contrast-enhanced CT images. A and B are magnified images selected from a series of sequential enhanced CT images performed during a dynamic CTP examination. An AIF is selected by placing a small circular region of interest (1–4 mm²) within the earliest appearing and most densely enhancing artery (usually one of the anterior or middle cerebral arteries). A depicts a small circular region of interest (circle, labeled “1”) placed within the A2 branch of the right anterior cerebral artery. A venous function is selected by placing a circular region of interest (2–8 mm²) within one of the dural venous sinuses. B depicts a small circular region of interest (circle labeled “2”) placed within the posterior third of the superior sagittal sinus/torcula region. The AIF and VF regions of interest define time (image number)–attenuation curves, which depict the time course of the dynamic enhancement of the artery and vein, respectively (C; ordinate: Hounsfield units; abscissa: image number). The AIF curve (labeled “1”) is of smaller amplitude and appears earlier than the VF curve (labeled “2”). The PrEI is defined as the interval from the first image to the image just preceding the upslope of the AIF curve. The vertical dashed line depicts the demarcation for the last image of the PrEI—images 1–13. The PoEI is defined as the first image after the AIF returns to baseline. The vertical dot-dashed line depicts the demarcation for the PoEI—image 58. The selection of the AIF, VF, PrEI, and PoEI represent the four decisions made by the analyst during the postprocessing of CTP data.

Fig 2.

Intraobserver variability. Points composing the region of interest data sets generated by the same observers during different postprocessing trials were paired as ordinate and abscissa values (region of interest data sets 1 versus 4, 4 versus 7, 1 versus 7, 2 versus 5, and 3 versus 6) and graphed as a scatter plot. A simple regression analysis was then performed to fit the data points for (A) CBV (r = 0.77), (B) CBF (r = 0.89), and (C) MTT (r = 0.91). The best fit linear regression line (single short-dash line), 95% confidence intervals for the regression line (paired solid lines), and 95% confidence intervals for the data points (paired long-dash lines) are superimposed on the scatter plot.

Fig 3.

Interobserver variability. Points composing the region of interest data sets generated by the different observers during the same trials were paired as ordinate and abscissa values (region of interest data sets 1 versus 2, 1 versus 3, 2 versus 3, 4 versus 5, 4 versus 6, 5 versus 6) and graphed as a scatter plot. A simple regression analysis was then performed to fit the data points for (A) CBV (r = 0.78), (B) CBF (r = 0.86), and (C) MTT (r = 0.88). The best fit linear regression line (single short-dash line), 95% confidence intervals for the regression line (solid lines), and 95% confidence intervals for the data points (paired long-dash lines) are superimposed on the scatter plot.

Fig 4.

Distribution of the individual parenchymal region of interest values derived from (A) CBV, (B) CBF, and (C) MTT maps as a function of the average value for each corresponding parenchymal region of interest. Ninety-five percent confidence intervals (solid lines) are defined by the squares of the geometric standard deviations for the data sets. The confidence intervals define the range of measurements that would be expected for a given “true value.” If the data generated by CTT 1 are excluded from the analysis (see Results), the 95% confidence intervals are improved substantially for the (D) CBV and (E) CBF data, with little change in the variability of (F) MTT.

Fig 5.

CBF maps generated from a single dynamic CTP data set by three different CTTs (A–C). Dynamic enhancement curves for the AIF (arrowheads) and VF generated by each technologist are depicted (D–F) below the corresponding CBF maps. The three larger circular regions of interest placed in identical locations on each map (frontal lobe white matter, deep gray matter, and temporal lobe mixed cortical-subcortical white matter) represent the prospectively designated parenchymal CBF region of interest measurements made by each technologist. Although these CBF maps were all generated from the same dynamic CTP data set, the results are qualitatively very different. All three maps demonstrate asymmetric CBF within the cerebral hemispheres, left greater than right. A, Map depicts focal regions of decreased CBF within the right posterior temporal lobe and right frontal lobe. B, Map demonstrates a similar region of decreased flow within the right frontal lobe, while the posterior temporal lobe CBF appears more normal (green > blue with some scattered foci or yellow-red). C, Map demonstrates normal (right hemisphere) and supranormal (left hemisphere) CBF without any foci of decreased CBF. The AIF and VF regions of interest were placed within the ACA and posterior third of the superior sagittal sinuses, respectively, by each of the technologists. The corresponding AIF curves (labeled “1”) generated by each technologist are essentially identical. The VF curve (labeled “2”) generated by the third observer (F) does not reach the amplitude of those of the other two observers (D, E). The PrEI chosen by the three technologists were 12, 14, and 11 for A, B, and C, respectively. The PoEI selections chosen by the three technologists were 57, 51, and 42, respectively for A, B, and C, respectively. Thus, while the selections of AIF region of interest, VIF region of interest, and PrEI were very homogeneous between technologists, the selection of the PoEI differed markedly. These data indicate that ambiguity with respect to the selection of the most appropriate PoEI represents a significant source of variability in the calculation of CBF maps. Fig 6. Ambiguity of postenhancement image selection. This dynamic enhancement curve (ordinate: Hounsfield units; abscissa: image number (1–89) provides an example of the ambiguity that is frequently involved in the selection of the most appropriate postenhancement image. The downslope of the AIF curve (arrow) is gradual and never completely returns to a baseline over the sequential 89 images. A large range of PoEI selections would be expected if this curve were submitted to multiple different technologists for postprocessing. It is also evident that different selections of the PoEI would result in the exclusion of a variable segment of the downslope of the VF time-attenuation curve from the analysis. The exclusion of segments of the VF time-attenuation curve will result in significant variation in the measured values of CBV and CBF, with larger values calculated when larger segments of the VF curve are excluded from the analysis.