Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Feb:179:109449.
doi: 10.1016/j.radonc.2022.109449. Epub 2022 Dec 22.

The relation between prediction model performance measures and patient selection outcomes for proton therapy in head and neck cancer

Artuur M Leeuwenberg  1 , Johannes B Reitsma  2 , Lisa G L J Van den Bosch  3 , Jeroen Hoogland  2 , Arjen van der Schaaf  3 , Frank J P Hoebers  4 , Oda B Wijers  5 , Johannes A Langendijk  3 , Karel G M Moons  2 , Ewoud Schuit  2
Affiliations
Free article

The relation between prediction model performance measures and patient selection outcomes for proton therapy in head and neck cancer

Artuur M Leeuwenberg et al. Radiother Oncol. 2023 Feb.
Free article

Abstract

Background: Normal-tissue complication probability (NTCP) models predict complication risk in patients receiving radiotherapy, considering radiation dose to healthy tissues, and are used to select patients for proton therapy, based on their expected reduction in risk after proton therapy versus photon radiotherapy (ΔNTCP). Recommended model evaluation measures include area under the receiver operating characteristic curve (AUC), overall calibration (CITL), and calibration slope (CS), whose precise relation to patient selection is still unclear. We investigated how each measure relates to patient selection outcomes.

Methods: The model validation and consequent patient selection process was simulated within empirical head and neck cancer patient data. By manipulating performance measures independently via model perturbations, the relation between model performance and patient selection was studied.

Results: Small reductions in AUC (-0.02) yielded mean changes in ΔNTCP between 0.9-3.2 %, and single-model patient selection differences between 2-19 %. Deviations (-0.2 or +0.2) in CITL or CS yielded mean changes in ΔNTCP between 0.3-1.4 %, and single-model patient selection differences between 1-10 %.

Conclusions: Each measure independently impacts ΔNTCP and patient selection and should thus be assessed in a representative sufficiently large external sample. Our suggested practical model selection approach is considering the model with the highest AUC, and recalibrating it if needed.

Keywords: Head and neck cancer; Individualized treatment decisions; Normal tissue complication probability models; Prediction performance measures.

PubMed Disclaimer

Conflict of interest statement

Declaration of Competing Interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: J.A. Langendijk reports a relationship with Dutch Cancer Society: funding grants. J.A. Langendijk reports his department has research contracts with IBA, RaySearch, Siemens, Elekta, Leoni, and Mirada. J.A. Langendijk reports a relationship with Global Scientific Advisory Board of IBA, RayCare International Advisory Board of RaySearch that includes: board membership, consulting or advisory, and speaking and lecture fees. J.A. Langendijk reports a relationship with Netherlands Society for Radiation Oncology that includes: board membership. J.B. Reitsma and E. Schuit are involved as methodologist in the development of indication protocols for patient selection for proton therapy in the Netherlands. Conflicts of Interest J.B. Reitsma: Involvement as methodologist in the development of indication protocols for patient selection for proton therapy in the Netherlands. J.A. Langendijk: Department has research contracts with IBA, RaySearch, Siemens, Elekta, Leoni, and Mirada. Received grants from Dutch Cancer Society and EU. Member of Global Scientific Advisory Board of IBA. Member of RayCare International Advisory Board of RaySearch. Chair of the Netherlands Society for Radiation Oncology. E. Schuit: Involvement as methodologist in the development of indication protocols for patient selection for proton therapy in the Netherlands.

Publication types

LinkOut - more resources