A method for reducing misclassification in the extended Glasgow Outcome Score

Abstract

The eight-point extended Glasgow Outcome Scale (GOSE) is commonly used as the primary outcome measure in traumatic brain injury (TBI) clinical trials. The outcome is conventionally collected through a structured interview with the patient alone or together with a caretaker. Despite the fact that using the structured interview questionnaires helps reach agreement in GOSE assessment between raters, significant variation remains among different raters. We introduce an alternate GOSE rating system as an aid in determining GOSE scores, with the objective of reducing inter-rater variation in the primary outcome assessment in TBI trials. Forty-five trauma centers were randomly assigned to three groups to assess GOSE scores on sample cases, using the alternative GOSE rating system coupled with central quality control (Group 1), the alternative system alone (Group 2), or conventional structured interviews (Group 3). The inter-rater variation between an expert and untrained raters was assessed for each group and reported through raw agreement and with weighted kappa (kappa) statistics. Groups 2 and 3 without central review yielded inter-rater agreements of 83% (weighted kappa = 0.81; 95% CI 0.69, 0.92) and 83% (weighted kappa = 0.76, 95% CI 0.63, 0.89), respectively, in GOS scores. In GOSE, the groups had an agreement of 76% (weighted kappa = 0.79; 95% CI 0.69, 0.89), and 63% (weighted kappa = 0.70; 95% CI 0.60, 0.81), respectively. The group using the alternative rating system coupled with central monitoring yielded the highest inter-rater agreement among the three groups in rating GOS (97%; weighted kappa = 0.95; 95% CI 0.89, 1.00), and GOSE (97%; weighted kappa = 0.97; 95% CI 0.91, 1.00). The alternate system is an improved GOSE rating method that reduces inter-rater variations and provides for the first time, source documentation and structured narratives that allow a thorough central review of information. The data suggest that a collective effort can be made to minimize inter-rater variation.

FIG. 1.

Forty-five trauma centers were randomly divided into three study groups balanced by each center's past experience in TBI trials. Group 1 used the alternative GOS/GOSE rating system coupled with central quality control, in which the raters were required to complete six sets of pre- and post-injury narratives according to six sample transcripts prior to outcome assessment. Group 2 used the alternative system with no central quality control, in which the raters used six sets of pre-specified narratives to rate the outcome. These narratives contained information, strictly transferred from the original interview transcripts by an expert, which allowed the validation of GOS/GOSE assessment without errors introduced by incorrect narratives. Group 3 used conventional structured interviews in which the raters were required to fill out the structured GOSE interview questionnaires based on the same six transcripts, and to provide an overall GOSE rating of the case (GOS, Glasgow Outcome Scale; GOSE, extended Glasgow Outcome Scale; TBI, traumatic brain injury).