External validation and comparison with other models of the International Metastatic Renal-Cell Carcinoma Database Consortium prognostic model: a population-based study

Abstract

Background: The International Metastatic Renal-Cell Carcinoma Database Consortium model offers prognostic information for patients with metastatic renal-cell carcinoma. We tested the accuracy of the model in an external population and compared it with other prognostic models.

Methods: We included patients with metastatic renal-cell carcinoma who were treated with first-line VEGF-targeted treatment at 13 international cancer centres and who were registered in the Consortium's database but had not contributed to the initial development of the Consortium Database model. The primary endpoint was overall survival. We compared the Database Consortium model with the Cleveland Clinic Foundation (CCF) model, the International Kidney Cancer Working Group (IKCWG) model, the French model, and the Memorial Sloan-Kettering Cancer Center (MSKCC) model by concordance indices and other measures of model fit.

Findings: Overall, 1028 patients were included in this study, of whom 849 had complete data to assess the Database Consortium model. Median overall survival was 18·8 months (95% 17·6-21·4). The predefined Database Consortium risk factors (anaemia, thrombocytosis, neutrophilia, hypercalcaemia, Karnofsky performance status <80%, and <1 year from diagnosis to treatment) were independent predictors of poor overall survival in the external validation set (hazard ratios ranged between 1·27 and 2·08, concordance index 0·71, 95% CI 0·68-0·73). When patients were segregated into three risk categories, median overall survival was 43·2 months (95% CI 31·4-50·1) in the favourable risk group (no risk factors; 157 patients), 22·5 months (18·7-25·1) in the intermediate risk group (one to two risk factors; 440 patients), and 7·8 months (6·5-9·7) in the poor risk group (three or more risk factors; 252 patients; p<0·0001; concordance index 0·664, 95% CI 0·639-0·689). 672 patients had complete data to test all five models. The concordance index of the CCF model was 0·662 (95% CI 0·636-0·687), of the French model 0·640 (0·614-0·665), of the IKCWG model 0·668 (0·645-0·692), and of the MSKCC model 0·657 (0·632-0·682). The reported versus predicted number of deaths at 2 years was most similar in the Database Consortium model compared with the other models.

Interpretation: The Database Consortium model is now externally validated and can be applied to stratify patients by risk in clinical trials and to counsel patients about prognosis.

Funding: None.

Figure 1

Results of Kaplan-Meier analysis of overall survival for the Database Consortium model

Figure 2. Comparison of the Database Consortium model with other models for prognosis of metastatic renal-cell carcinoma

Panels show reclassification calibration comparison of the Database Consortium model with the CCF model (A), the IKCWG model (B), the French model (C), and the MSKCC model (D). The x-axis includes nine groups from the 3 × 3 cross-tabulation table of risk groups of the two models being compared. A smaller reclassification calibration χ² suggests a better fit. CCF=Cleveland Clinic Foundation. IKCWG=International Kidney Cancer Working Group. MSKCC=Memorial Sloan-Kettering Cancer Center. DCM=Database Consortium model. Fav=Favourable. Intm=Intermediate.

Figure 3. Net reclassification improvement comparing the Database Consortium model with other models for prognosis of metastatic renal-cell carcinoma

Panels show comparison of the DCM with the CCF model (A), the IKCWG model (B), the French model (C), and the MSKCC model (D), calculated according to patient’s survival status at 2 years after start of anti-VEGF targeted treatment. 489 patients were included, of whom 161 were alive and 328 were dead. 183 who had not reached 2 years follow-up were excluded. As an example, comparing the French model with the DCM, 14% of alive patients were incorrectly moved to a worse risk group and 36% were correctly moved to a better risk group by the DCM, resulting in a 22% (36% − 14%) improvement. 17% of patients who had died were correctly moved to a worse risk group and 16% were incorrectly moved to a better risk group by DCM, resulting in a 1% (17% − 16%) improvement. The overall net reclassification improvement was 23% (22% + 1%). CCF=Cleveland Clinic Foundation. IKCWG=International Kidney Cancer Working Group. MSKCC=Memorial Sloan-Kettering Cancer Center. DCM=Database Consortium model. Fav=Favourable. Intm=Intermediate. RI=reclassification improvement. NRI=net reclassification improvement.