Interobserver variability in radiation therapy plan output: Results of a single-institution study

Abstract

Purpose: We investigated the sources of variability in radiation therapy treatment plan output between planners within a single institution.

Methods and materials: Forty treatment planners across 5 campuses of an institution created a plan on the same thoracic esophagus patient computed tomography scan and structure set. Plans were scored and ranked based on the planner's adherence to an ordered list of target dose coverage and normal tissue evaluation criteria. A runs test was used to identify whether any of the studied planner qualities influenced the ranking. Spearman rank correlation was used to investigate whether plan score correlated with years of experience or planned monitor units.

Results: The distribution of scores, ranging from 80.24 to 135.89, was negatively skewed (mean, 128.7; median, 131.5). No statistically significant relationship between plan score and campus (P = .193), job title (P = .174), previous outside experience (P = .611), or number of gantry angles (P = .156) was discovered. No statistical correlation between plan score and monitor unit or years of experience was found.

Conclusions: Despite clear and established critical organ dose criteria and well-documented planning guidelines, planning variation still occurs, even among members of the same institution. Because plan consistency does not seem to significantly correlate with experience, career path, or campus, investigation into alternate methods beyond additional education and training to reduce this variation, such as knowledge-based planning or advanced optimization techniques, is necessary.

Figure 1

CT scan used for this study. PTV_{Low Dose} (bold contour) and PTV_{High Dose} (thin contour) are superimposed on the image.

Figure 2

Graphical illustration of point system used to assign scores to each submitted plan based on how well it met the various clinical criteria for this anatomical site.

Figure 3

Distribution of plan scores for all 40 submitted plans

Figure 4

Scatter plots of plan score versus (a) total MU, (b) date of submission, (c) self reported years of experience and (d) MU versus self reported years of experience. For the Spearman rank correlation calculation the two plans created for the incorrect prescription dose, marked as “x’s”, were omitted as outliers.