Nodular Leptomeningeal Disease-A Distinct Pattern of Recurrence After Postresection Stereotactic Radiosurgery for Brain Metastases: A Multi-institutional Study of Interobserver Reliability

Abstract

Purpose: For brain metastases, surgical resection with postoperative stereotactic radiosurgery is an emerging standard of care. Postoperative cavity stereotactic radiosurgery is associated with a specific, underrecognized pattern of intracranial recurrence, herein termed nodular leptomeningeal disease (nLMD), which is distinct from classical leptomeningeal disease. We hypothesized that there is poor consensus regarding the definition of LMD, and that a formal, self-guided training module will improve interrater reliability (IRR) and validity in diagnosing LMD.

Methods and materials: Twenty-two physicians at 16 institutions, including 15 physicians with central nervous system expertise, completed a 2-phase survey that included magnetic resonance imaging and treatment information for 30 patients. In the "pretraining" phase, physicians labeled cases using 3 patterns of recurrence commonly reported in prospective studies: local recurrence (LR), distant parenchymal recurrence (DR), and LMD. After a self-directed training module, participating physicians completed the "posttraining" phase and relabeled the 30 cases using the 4 following labels: LR, DR, classical leptomeningeal disease, and nLMD.

Results: IRR increased 34% after training (Fleiss' Kappa K = 0.41 to K = 0.55, P < .001). IRR increased most among non-central nervous system specialists (+58%, P < .001). Before training, IRR was lowest for LMD (K = 0.33). After training, IRR increased across all recurrence subgroups and increased most for LMD (+67%). After training, ≥27% of cases initially labeled LR or DR were later recognized as nLMD.

Conclusions: This study highlights the large degree of inconsistency among clinicians in recognizing nLMD. Our findings demonstrate that a brief self-guided training module distinguishing nLMD can significantly improve IRR across all patterns of recurrence, and particularly in nLMD. To optimize outcomes reporting, prospective trials in brain metastases should incorporate central imaging review and investigator training.

Fig. 1.

Sample image from the training module providing guidance on how to differentiate different patterns of recurrence for the study. This is one of the images highlighting common features in patients with nodular leptomeningeal disease.

Fig. 2.

Schematic overview of data generation workflow for each physician rater. The 30 anonymized magnetic resonance imaging (MRI) cases were randomized before each review. Pattern of recurrence labeling was multiple choice. Pretraining recurrence options: local recurrence, distant parenchymal recurrence, and leptomeningeal disease. Posttraining recurrence options: local recurrence, distant parenchymal recurrence, nodular leptomeningeal disease, and classical leptomeningeal disease. Raters assigned a confidence score to each answer choice on a scale of 0 to 4.