Blinded Independent Central Review (BICR) in New Therapeutic Lung Cancer Trials

Abstract

Background: Double reads in blinded independent central reviews (BICRs) are recommended to control the quality of trials but they are prone to discordances. We analyzed inter-reader discordances in a pool of lung cancer trials using RECIST 1.1.

Methods: We analyzed six lung cancer BICR trials that included 1833 patients (10,684 time points) involving 17 radiologists. We analyzed the rate of discrepancy of each trial at the time-point and patient levels as well as testing inter-trial differences. The analysis of adjudication made it possible to compute the readers' endorsement rates, the root causes of adjudications, and the proportions of "errors" versus "medically justifiable differences".

Results: The trials had significantly different discrepancy rates both at the time-point (average = 34.3%) and patient (average = 59.2%) levels. When considering only discrepancies for progressive disease, homogeneous discrepancy rates were found with an average of 32.9%, while readers' endorsement rates ranged between 27.7% and 77.8%. Major causes of adjudication were different per trial, with medically justifiable differences being the most common, triggering 74.2% of total adjudications.

Conclusions: We provide baseline performances for monitoring reader performance in trials with double reads. Intelligent reading system implementation along with appropriate reader training and monitoring are solutions that could mitigate a large portion of the commonly encountered reading errors.

Keywords: RECIST 1.1; blinded independent central review; clinical trial; lung cancer.

Figure 1

Workflow of the BICR with double reads and adjudication.

Figure 2

Warning limit for monitoring the discrepancy rate in the DOP. Limit of acceptable discrepancy rate measured as a function of the number of patients read in the trial. In orange: a discrepancy rate higher than 50% after reviewing more than 50 patients warns of suboptimal performances in the trial.

Figure 3

Reader endorsement rates after adjudication: Adjudication was based on discrepancy in the DOP. The dashed green line at 25% indicates a significant difference from the average reader endorsement (50%) for readers involved in more than 25 adjudications.

Figure 4

New micro- or ground-glass lesions. Multiple new lung nodules marked in red circles (partial solid or ground-glass opacities) appeared in week 7, were determined as equivocal in the same week, and resolved in week 18.

Figure 5

New sclerotic lesions. Newly appearing sclerotic lesions in the CT and uptakes in the bone scan in week 12, but with no evidence of lesions at baseline (marked in blue circles).

Figure 6

Variability in target lesion measurements. Reader 1 and reader 2 selected the same target lesion at baseline. Even though the two readers performed very similar subsequent measurements, at the third visit, a small measurement difference triggered discrepant responses. At the final visit, one reader declared a stable disease (SD) whereas the other reader declared a progressive disease (PD).