Independent recalculation outperforms traditional measurement-based IMRT QA methods in detecting unacceptable plans

Abstract

Purpose: To evaluate the performance of an independent recalculation and compare it against current measurement-based patient specific intensity-modulated radiation therapy (IMRT) quality assurance (QA) in predicting unacceptable phantom results as measured by the Imaging and Radiation Oncology Core (IROC).

Methods: When institutions irradiate the IROC head and neck IMRT phantom, they are also asked to submit their internal IMRT QA results. Separately from this, IROC has previously created reference beam models on the Mobius3D platform to independently recalculate phantom results based on the institution's DICOM plan data. The ability of the institutions' IMRT QA to predict the IROC phantom result was compared against the independent recalculation for 339 phantom results collected since 2012. This was done to determine the ability of these systems to detect failing phantom results (i.e., large errors) as well as poor phantom results (i.e., modest errors). Sensitivity and specificity were evaluated using common clinical thresholds, and receiver operator characteristic (ROC) curves were used to compare across different thresholds.

Results: Overall, based on common clinical criteria, the independent recalculation was 12 times more sensitive at detecting unacceptable (failing) IROC phantom results than clinical measurement-based IMRT QA. The recalculation was superior, in head-to-head comparison, to the EPID, ArcCheck, and MapCheck devices. The superiority of the recalculation vs these array-based measurements persisted under ROC analysis as the recalculation curve had a greater area under it and was always above that for these measurement devices. For detecting modest errors (poor phantom results rather than failing phantom results), neither the recalculation nor measurement-based IMRT QA performed well.

Conclusions: A simple recalculation outperformed current measurement-based IMRT QA methods at detecting unacceptable plans. These findings highlight the value of an independent recalculation, and raise further questions about the current standard of measurement-based IMRT QA.

Keywords: IMRT QA; Mobius; patient-specific QA; recalculation.

Figure 1

Regression analysis of quality assurance (QA) result [Institutional intensity‐modulated radiation therapy (IMRT) QA or Mobius3D] vs the actual dose delivered to the phantom. This was for IMRT QA based on gamma analysis (a) and for point dose measurements (b). [Color figure can be viewed at wileyonlinelibrary.com]

Figure 2

Receiver operator characteristic (ROC) curves for all phantom results as evaluated by the Imaging and Radiation Oncology Core (IROC) recalculation. The higher line shows the ROC for the failing phantom result cohort area under the curve (AUC = 0.78) while the lower line shows the ROC for the poor phantom result cohort (AUC = 0.60). [Color figure can be viewed at wileyonlinelibrary.com]

Figure 3

Receiver operator characteristic (ROC) curves for the 245 phantom results evaluated by institutional intensity‐modulated radiation therapy (IMRT) quality assurance (QA) using an array device and a 3%/3 mm criterion. The upper line shows the ROC curve for this cohort evaluated using the Imaging and Radiation Oncology Core (IROC) recalculation (area under the curve [AUC] = 0.79) where the tolerance for dose disagreement in the PTV was varied. The lower line shows the ROC curve for this cohort evaluated using institutional IMRT QA (AUC = 0.60) where the percent of pixels passing gamma was varied. Sensitivity and specificity were relative to identification of phantoms that failed IROC criteria. [Color figure can be viewed at wileyonlinelibrary.com]