Development and Reliability of a User-Friendly Multicenter Phenotyping Application for Hemorrhagic and Ischemic Stroke

Abstract

Background: Annotation and Image Markup on ClearCanvas Enriched Stroke-phenotyping Software (ACCESS) is a novel stand-alone computer software application that allows the creation of simple standardized annotations for reporting brain images of all stroke types. We developed the ACCESS application and determined its inter-rater and intra-rater reliability in the Stroke Investigative Research and Educational Network (SIREN) study to assess its suitability for multicenter studies.

Methods: One hundred randomly selected stroke imaging reports from 5 SIREN sites were re-evaluated by 4 trained independent raters to determine the inter-rater reliability of the ACCESS (version 12.0) software for stroke phenotyping. To determine intra-rater reliability, 6 raters reviewed the same cases previously reported by them after a month of interval. Ischemic stroke was classified using the Oxfordshire Community Stroke Project (OCSP), Trial of Org 10172 in Acute Stroke Treatment (TOAST), and Atherosclerosis, Small-vessel disease, Cardiac source, Other cause (ASCO) protocols, while hemorrhagic stroke was classified using the Structural lesion, Medication, Amyloid angiopathy, Systemic disease, Hypertensive angiopathy and Undetermined (SMASH-U) protocol in ACCESS. Agreement among raters was measured with Cohen's kappa statistics.

Results: For primary stroke type, inter-rater agreement was .98 (95% confidence interval [CI], .94-1.00), while intra-rater agreement was 1.00 (95% CI, 1.00). For OCSP subtypes, inter-rater agreement was .97 (95% CI, .92-1.00) for the partial anterior circulation infarcts, .92 (95% CI, .76-1.00) for the total anterior circulation infarcts, and excellent for both lacunar infarcts and posterior circulation infarcts. Intra-rater agreement was .97 (.90-1.00), while inter-rater agreement was .93 (95% CI, .84-1.00) for TOAST subtypes. Inter-rater agreement ranged between .78 (cardioembolic) and .91 (large artery atherosclerotic) for ASCO subtypes and was .80 (95% CI, .56-1.00) for SMASH-U subtypes.

Conclusion: The ACCESS application facilitates a concordant and reproducible classification of stroke subtypes by multiple investigators, making it suitable for clinical use and multicenter research.

Keywords: Africa; DICOM application; Stroke; developing country; ischemic stroke; phenotyping; reporting software intracerebral hemorrhage.

Figure 1. Flowchart for ACCESS development

Schematic process involved in obtaining code for lexicon terms for stroke phenotyping.

Figure 2. Screenshot of the ACCESS software application

Screenshot of the ACCESS software application showing axial non-contrast (left) and reformatted sagittal (right) CT images of a stroke patient with acute hemorrhage in right putaminal region with associated mass effect evidenced by compression of the ipsilateral lateral ventricle and effacement of the cerebral sulci. Assessment of neuro-images, CT/MRI is usually initialized by clicking (A) to open the aim template (B) on the right of the screen. Clicking on (C) selects the modality and reporting begins by selecting appropriate drop down buttons from (D). After all buttons that characterize the lesion are selected, an annotation is automatically created by clicking on (E) which may then be used for analysis on an Excel spreadsheet or any other statistical package.