Institutional patient-specific IMRT QA does not predict unacceptable plan delivery

Abstract

Purpose: To determine whether in-house patient-specific intensity modulated radiation therapy quality assurance (IMRT QA) results predict Imaging and Radiation Oncology Core (IROC)-Houston phantom results.

Methods and materials: IROC Houston's IMRT head and neck phantoms have been irradiated by numerous institutions as part of clinical trial credentialing. We retrospectively compared these phantom results with those of in-house IMRT QA (following the institution's clinical process) for 855 irradiations performed between 2003 and 2013. The sensitivity and specificity of IMRT QA to detect unacceptable or acceptable plans were determined relative to the IROC Houston phantom results. Additional analyses evaluated specific IMRT QA dosimeters and analysis methods.

Results: IMRT QA universally showed poor sensitivity relative to the head and neck phantom, that is, poor ability to predict a failing IROC Houston phantom result. Depending on how the IMRT QA results were interpreted, overall sensitivity ranged from 2% to 18%. For different IMRT QA methods, sensitivity ranged from 3% to 54%. Although the observed sensitivity was particularly poor at clinical thresholds (eg 3% dose difference or 90% of pixels passing gamma), receiver operator characteristic analysis indicated that no threshold showed good sensitivity and specificity for the devices evaluated.

Conclusions: IMRT QA is not a reasonable replacement for a credentialing phantom. Moreover, the particularly poor agreement between IMRT QA and the IROC Houston phantoms highlights surprising inconsistency in the QA process.

Figure 1

IROC Houston’s head and neck phantom. The imaging/dosimetry insert is disassembled, showing the 2 targets in brown and a small avoidance structure in olive.

Figure 2

Truth table for institutional IMRT QA results versus IROC Houston phantom results for head and neck phantom plans. (a) All plans were assumed to pass institutional IMRT QA unless the institution explicitly stated otherwise. (b) Institutional IMRT QA results were determined by IROC Houston using the common criteria of 3% dose or >90% of pixels passing 3%/3 mm gamma.

Figure 3

Percent differences between dose measurements and treatment planning system calculations for institutional IMRT QA compared with the TLD in the IROC Houston phantom. The linear trend line should ideally have a slope of 1, but instead is nearly flat.

Figure 4

Percent of pixels passing gamma for institutional IMRT QA compared with the IROC Houston phantom films. The linear trend line should ideally have a slope of 1, but instead is nearly flat.

Figure 5

ROC curves for the ion chamber (a), film (b), and MapCheck (c), devices, indicating the sensitivity and specificity of the devices across thresholds (dose difference for the ion chamber and percent of pixels passing 3%/3 mm gamma for film and MapCheck). The area under the curve (AUC) is shown for each device.