Definition of a framework for volumetric modulated arc therapy plan quality assessment with integration of dose-, complexity-, and robustness metrics

Abstract

Background and purpose: Conventionally, the quality of radiotherapy treatment plans is assessed through visual inspection of dose distributions and dose-volume histograms. This study developed a framework to evaluate plan quality using dose, complexity, and robustness metrics. Additionally, a method for predicting plan robustness metrics using dose and complexity metrics was introduced for cases where plan robustness evaluation is unavailable or impractical.

Materials and methods: The framework and prediction models were developed and validated using 103-bronchial Volumetric Modulated Arc Therapy (VMAT)-plans. The application of the framework was demonstrated using 25-VMAT-plans. To identify significant metrics for plan evaluation, 122-metrics were analysed and narrowed down using multivariate Spearman correlation. Metric limits were set with Statistical process control (SPC). Robustness metrics were predicted using multivariable or single linear regression models based on dose-and complexity-metrics.

Results: Twenty-five-metrics were selected based on the amount and strength of correlations. R₉₅(dose coverage) and HI_95/5(homogeneity index) stood out among the dose-metrics, while the complexity-metrics showed similar correlations. Average scenarios dose at 95 % Clinical Target Volume D95_mean(CTV) and Errorbar-based Volume-Histograms (EVH) were notable for robustness metrics. Approximately 99 % of evaluated metrics fell within established SPC limits. The prediction model for D95_mean(CTV) showed good performance (adjusted R² = 0.88, mean squared error (MSE) = 3.84 × 10^-6), while the model for EVH demonstrated moderate reliability (adjusted R² = 0.52, MSE = 0.2). No statistically significant differences were found between the predicted (using dose-and complexity-metrics) and calculated robustness metrics (EVH (p-value = 0.9) and D95_mean(CTV) (p-value = 1)).

Conclusions: The developed framework enables early detection of sub-optimal, complex and non-robust treatment plans. The predictive model can be used when robustness evaluations are impractical.

Keywords: Dose metrics; Plan complexity metrics; Plan robustness metrics; Robustness prediction; Statistical process control.

Fig. 1

Established workflow used to assess plan quality. This workflow has two processes: (a) baseline creation (red framed) and (b) an example of the framework application (blue framed). The “baseline creation” refers to the selection of metrics and the calculation of SPC limits needed for plan evaluation. The baseline is created with n VMAT treatment plans (e.g., n = 103 plans). The baseline results are saved in a shared database. The “framework application” uses an existing baseline, saved in a database, to evaluate a treatment plan. The step 'optimise & calculate plan” is not part of the framework in this study. Note: The treatment plan can refer to any plan requiring evaluation. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 2

Multivariable Spearman correlation analysis of the selected dose, plan complexity, and plan robustness metrics based on the CTV and the OAR. Each cell in the plot shows the correlation between the corresponding row and column variables. An asterisk (*) indicates statistically significant correlation coefficients (α = 0.05). Blue indicates a negative correlation, while red indicates a positive correlation. The lightest blue/red colours represent coefficients with no correlation, and the colour intensity increases according to the categorisation of the coefficients: very strong/high (0.90–1.00), strong/high (0.70–0.89), moderate (0.50–0.69), weak/low (0.30–0.49), and no/little (0.00–0.29). The number of coefficients >0.3 in Figure is lower than those listed in Table 2 because some significant coefficients were counted but not selected. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 3

Statistical Process Control (SPC) chart of four calculated metrics. Blue circles represent baseline treatment plans, the filled black diamonds indicate test plans, red dashed lines mark the upper and lower control limits (UCL/LCL), and the black solid line indicates the centre line (CL). The control chart for the dose metrics “CI_paddick” (a) and complexity metrics “PMU” (b) are in control and are normally distributed. The left-skewed robustness metric “D95_mean(CTV)” (c) shows plans TP120, TP124, and TP127 outside the limits. The right-skewed robustness metric “EVH” (d) values are within the limits. TP116, TP119, TP120, and TP124 are the calculated test plan metrics between the EVH values 6–7. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 4

The black squares show the comparison between the calculated and predicted values from the validation dataset (20 % of the randomly split baseline dataset) for D95_mean (CTV) (a) and EVH (b), used for model validation. The red fit line in (a) and (b) represents a perfect fit, where calculated equals predicted. Important metrics to validate the prediction models are stated in the plot. The calculated robustness metrics (blue circles) and predicted robustness metrics (black triangles), along with confidence and prediction intervals, are shown for the test plans. The robustness metrics D95_mean(CTV) (c) and EVH (d) are presented. A low EVH indicates a more robust plan, and D95_mean(CTV) should be high. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)