Reliability of the Diagnosis of Cerebral Vasospasm Using Catheter Cerebral Angiography: A Systematic Review and Inter- and Intraobserver Study

Abstract

Background and purpose: Conventional angiography is the benchmark examination to diagnose cerebral vasospasm, but there is limited evidence regarding its reliability. Our goals were the following: 1) to systematically review the literature on the reliability of the diagnosis of cerebral vasospasm using conventional angiography, and 2) to perform an agreement study among clinicians who perform endovascular treatment.

Materials and methods: Articles reporting a classification system on the degree of cerebral vasospasm on conventional angiography were systematically searched, and agreement studies were identified. We assembled a portfolio of 221 cases of patients with subarachnoid hemorrhage and asked 17 raters with different backgrounds (radiology, neurosurgery, or neurology) and experience (junior ≤10 and senior >10 years) to independently evaluate cerebral vasospasm in 7 vessel segments using a 3-point scale and to evaluate, for each case, whether findings would justify endovascular treatment. Nine raters took part in the intraobserver reliability study.

Results: The systematic review showed a very heterogeneous literature, with 140 studies using 60 different nomenclatures and 21 different thresholds to define cerebral vasospasm, and 5 interobserver studies reporting a wide range of reliability (κ = 0.14-0.87). In our study, only senior raters reached substantial agreement (κ ≥ 0.6) on vasospasm of the supraclinoid ICA, M1, and basilar segments and only when assessments were dichotomized (presence or absence of ≥50% narrowing). Agreement on whether to proceed with endovascular management of vasospasm was only fair (κ ≤ 0.4).

Conclusions: Research on cerebral vasospasm would benefit from standardization of definitions and thresholds. Dichotomized decisions by experienced readers are required for the reliable angiographic diagnosis of cerebral vasospasm.

FIG 1.

Illustrative case from the portfolio. The degree of vessel narrowing was assessed on the right-sided Index images (A2, B2, C2).

FIG 2.

Illustrative cases of maximal agreement and maximal disagreement. Two cases showing A) maximal agreement (17/17 readers rated vasospasm to be “severe” with 50% narrowing threshold), B) maximal disagreement (only 8/17 readers rated vasospasm to be “severe”).

FIG 3.

Total number of dichotomized assessments of severe vasospasm and choice for endovascular treatment. Along the x-axis, the graph shows patients primarily ordered according to increasing proportion of severe vasospasm votes. Patients are secondarily ordered within each bracket according to number of votes for endovascular treatment. Circles show patients who received angioplasty in real life. Note that perfect agreement occurs only at the tails of the distribution: cases 1–92 (all no) and 212–221 (all yes).