Added Value of a Blinded Outcome Adjudication Committee in an Open-Label Randomized Stroke Trial

Abstract

Background and purpose: Blinded outcome assessment in trials with prospective randomized open blinded end point design is challenging. Unblinding can result in misclassified outcomes and biased treatment effect estimates. An outcome adjudication committee assures blinded outcome assessment, but the added value for trials with prospective randomized open blinded end point design and subjective outcomes is unknown. We aimed to assess the degree of misclassification of modified Rankin Scale (mRS) scores by a central assessor and its impact on treatment effect estimates in a stroke trial with prospective randomized open blinded end point design.

Methods: We used data from the MR CLEAN (Multicenter Randomized Clinical Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands). The primary outcome was the mRS at 90 days. Standardized, algorithm-based telephone interviews to assess the mRS were conducted from a central location by an experienced research nurse, unaware but not formally blinded to treatment allocation (central assessor). Masked reports of these interviews were adjudicated by a blinded outcome committee. Misclassification was defined as an incorrect classification of the mRS by the central assessor. The effect of endovascular treatment on the mRS was assessed with multivariable ordinal logistic regression.

Results: In MR CLEAN, 53/500 (10.6%) of the mRS scores were misclassified. The degree and direction of misclassification did not differ between treatment arms (P=0.59). Benefit of endovascular treatment was shown on the mRS when scored by the central assessor (adjusted common odds ratio, 1.60 [95% CI, 1.16-2.21]) and the outcome adjudication committee (adjusted common odds ratio, 1.67 [95% CI, 1.21-2.20]).

Conclusions: Misclassification by the central assessor was small, randomly distributed over treatment arms, and did not affect treatment effect estimates. This study suggests that the added value of a blinded outcome adjudication committee is limited in a stroke trial with prospective randomized open blinded end point design applying standardized, algorithm-based outcome assessment by a central assessor, who is unaware but not formally blinded to treatment allocation. Registration: URL: https://www.isrctn.com; Unique identifier: ISRCTN10888758.

Keywords: algorithm; clinical trial; ischemic stroke; odds ratio; telephone.

Figure 1.

Agreement between the central assessor and adjudicators of the outcome committee. mRS indicates modified Rankin Scale.

Figure 2.

Cross-tabulation of mRS (modified Rankin Scale) scores by the central assessor and outcome committee according to treatment allocation. Values are numbers (percentages): data of the intervention arm (A) and the control arm (B). The green cells indicate no misclassification, the orange cells indicate misclassification towards better mRS scores by the central assessor, and the blue cells indicate misclassification towards worse mRS scores by the central assessor. *This patient died at 90+1 d after treatment. The outcome committee assigned a score of 5 on the mRS to the patient because the patient was alive at exactly 90 d.

Figure 3.

Treatment effect of endovascular treatment (EVT) on the modified Rankin Scale (mRS) according to the central assessor alone and the outcome committee. acOR indicates adjusted common odds ratio; aOR, adjusted odds ratio; and cOR, common odds ratio. *Values were adjusted for age; National Institutes of Health Stroke Scale score at baseline; time from stroke onset to randomization; status with respect to previous stroke, atrial fibrillation, and diabetes; and occlusion of the internal carotid artery terminus (yes/no).

Figure 4.

Flowchart to assess added value of an outcome adjudication committee in trials with prospective randomized open blinded end point (PROBE) design. Flowchart for differential misclassification (A) and nondifferential misclassification (B). *The likelihood of unblinding during outcome assessment is low. †The likelihood of unblinding during outcome assessment is high. ‡The likelihood of unblinding is lower for a central assessor than for on-site assessors. §The acceptable rate of correctly indicated treatment allocations by the assessor depend on the number of treatment arms. For example, in a trial with 2 treatment arms, the assessor should not be able to indicate the correct treatment allocation in significantly more than 50% of the cases. ∥The nondifferential misclassification rate can be reduced by standardized outcome assessment. #The impact of nondifferential misclassification also depends on the size of the treatment effect.