Discrepancies of assessments in a RECIST 1.1 phase II clinical trial - association between adjudication rate and variability in images and tumors selection

Abstract

Background: In imaging-based clinical trials, it is common practice to perform double reads for each image, discrepant interpretations can result from these two different evaluations. In this study we analyzed discrepancies that occurred between local investigators (LI) and blinded independent central review (BICR) by comparing reader-selected imaging scans and lesions. Our goal was to identify the causes of discrepant declarations of progressive disease (PD) between LI and BICR in a clinical trial.

Methods: We retrospectively analyzed imaging data from a RECIST 1.1-based, multi-sites, phase II clinical trial of 179 patients with adult small cell lung cancer, treated with Cabazitaxel compared to Topotecan. Any discrepancies in the determination of PD between LI and BICR readers were reviewed by a third-party adjudicator. For each imaging time point and reader, we recorded the selected target lesions, non-target lesions, and new lesions. Odds ratios were calculated to measure the association between discrepant declarations of PD and the differences in reviewed imaging scans (e.g. same imaging modality but with different reconstruction parameters) and selected lesions. Reasons for discrepancies were analyzed.

Results: The average number of target lesions found by LI and BICR was respectively 2.9 and 3.4 per patient (p < 0.05), 18.4% of these target lesions were actually non-measurable. LI and BICR performed their evaluations based on different baseline imaging scans for 59% of the patients, they selected at least one different target lesion in 85% of patients. A total of 36.7% of patients required adjudication. Reasons of adjudication included differences in 1) reporting new lesions (53.7%), 2) the measured change of the tumor burden (18.5%), and 3) the progression of non-target lesions (11.2%). The rate of discrepancy was not associated with the selection of non-measurable target lesions or with the readers' assessment of different images. Paradoxically, more discrepancies occurred when LI and BICR selected exactly the same target lesions at baseline compared to when readers selected not exactly the same lesions.

Conclusions: For a large proportion of evaluations, LI and BICR did not select the same imaging scans and target lesions but with a limited impact on the rate of discrepancy. The majority of discrepancies were explained by the difference in detecting new lesions.

Trial registration: ARD12166 ( https://clinicaltrials.gov/ct2/show/NCT01500720 ).

Keywords: Inter-observer variability; Phase II; Response evaluation criteria in solid tumors; Small cell lung carcinoma; Tumor imaging.

Fig. 1

Flowchart of the ARD12166 trial. Using a common database of imaging scans, LI and BICR performed RECIST 1.1 evaluations. In case of evaluations discrepancies, an adjudicator was solicited to perform a third evaluations blinded from previous assessments

Fig. 2

Example of one non-measurable lesion. Both readers targeted the same region of interest, but their perception of tumors’ boundary was. One reader (right) considered the hepatic tumor as coalescent, while the other (left) considered two distinct tumors. The inter-reader proportional difference was 73%. It should be noted that readers enabled different window levels

Fig. 3

Number of selected target lesions. Number of selected target lesions by LI (blue bars) versus BICR (red bars) readers. Data are ordered first according to the higher number selected by BICR, and then by the higher number selected by the LI. The number of target lesions selected by BICR was significantly higher than that by the LI (pVal< 0.05, Wilcoxon test)

Fig. 4

Anatomical sites where target lesions have been selected. Pie chart displaying the proportion of sites where target lesions have been selected. Orange: pulmonary lesions; grey: hepatic lesions; yellow: nodal lesions; light blue: adrenal; navy blue: brain; green: undefined. Undefined lesions are in bone, spleen, pancreas, muscle, and kidney. Left: Location of LI target lesion selection. Right: Location of BICR target lesion selection