Multicentre comparison of a diagnostic assay: aquaporin-4 antibodies in neuromyelitis optica

Abstract

Objective: Antibodies to cell surface central nervous system proteins help to diagnose conditions which often respond to immunotherapies. The assessment of antibody assays needs to reflect their clinical utility. We report the results of a multicentre study of aquaporin (AQP) 4 antibody (AQP4-Ab) assays in neuromyelitis optica spectrum disorders (NMOSD).

Methods: Coded samples from patients with neuromyelitis optica (NMO) or NMOSD (101) and controls (92) were tested at 15 European diagnostic centres using 21 assays including live (n=3) or fixed cell-based assays (n=10), flow cytometry (n=4), immunohistochemistry (n=3) and ELISA (n=1).

Results: Results of tests on 92 controls identified 12assays as highly specific (0-1 false-positive results). 32 samples from 50 (64%) NMO sera and 34 from 51 (67%) NMOSD sera were positive on at least two of the 12 highly specific assays, leaving 35 patients with seronegative NMO/spectrum disorder (SD). On the basis of a combination of clinical phenotype and the highly specific assays, 66 AQP4-Ab seropositive samples were used to establish the sensitivities (51.5-100%) of all 21 assays. The specificities (85.8-100%) were based on 92 control samples and 35 seronegative NMO/SD patient samples.

Conclusions: The cell-based assays were most sensitive and specific overall, but immunohistochemistry or flow cytometry could be equally accurate in specialist centres. Since patients with AQP4-Ab negative NMO/SD require different management, the use of both appropriate control samples and defined seronegative NMOSD samples is essential to evaluate these assays in a clinically meaningful way. The process described here can be applied to the evaluation of other antibody assays in the newly evolving field of autoimmune neurology.

Figure 1

Study design. The coded clinical and serological data from 209 patients were sent to the University of Lübeck, Germany. The coded serum (199) or plasma samples (10) were sent to Euroimmun, Lübeck, Germany. Sixteen samples were excluded due to insufficient volume. In total, 193 samples were recoded, aliquoted (90–100 μL) and sent on dry ice to 15 European Centres to run 21 AQP4 assays with a 17-week deadline. Each centre entered its own data online to a web-based server maintained at the Institute of Quality Assurance, Lübeck, Germany. All clinical data, assay results and sample codes were sent to Oxford, UK and Lyon, France for initial analysis. A blinded overview was sent to each centre before unblinding the study in a meeting in Paris, France, where all groups were represented.

Figure 2

Assay specificity based on results from the 92 randomised control patient samples. These comprised 37 multiple sclerosis (35 relapsing remitting, 2 primary progressive), 1 connective tissue disease, 1 neuromyotonia, 1 progressive encephalomyelitis with rigidity and myoclonus, 4 clinically isolated syndrome, 1 acute disseminated encephalomyelitis, 1 Susac syndrome, 2 tumour (1 B-cell lymphoma, 1 colon carcinoma), 5 myasthenia gravis and 39 headache. Each column represents an individual assay (see table 2 for assay details) except for the first column which shows the serostatus assigned by the participating centre. Assays are grouped on the basis of their specificity in this cohort: assays on the left-hand side have 0 or 1 false-positive results (12 assays), whereas assays on the right-hand side have more than 1 false-positive result (9 assays). The assays are numbered 1–21: 1–3 are live cell-based assays (CBAs), whereas 4–6 are fixed CBA performed in-house at Euroimmun, 7–13 are fixed CBA performed at other European centres, 14–17 are flow cytometry assays, 18–20 are immunohistochemistry assays with detection based on enzymatic colour change (18) or fluorescence (19–20), and 21 is a commercially available ELISA (Iason). Each row represents a single serum sample. Positive results are coloured pink to red with a semiquantitative score from ‘1’ to ‘4’ inserted, whereas a negative result is white.

Figure 3

Defining the serostatus of patients with neuromyelitis optica (NMO). Results in patients with NMO, defined by the 2006 Wingerchuk criteria excluding AQP4 serostatus. Results are presented as in figure 2, with each column representing an individual assay, apart from the first column that shows the serostatus submitted with the sample, and each row represents an individual serum. A positive result is graded in colour from pink to red with a semiquantitative score from ‘1’ to ‘4’ inserted. A negative result is displayed as a white box. Two individual results from patients with seropositive NMO were considered unevaluable by the testing centre and were scored negative. In total, 32 of 50 NMO samples are considered seropositive as they were positive on at least two of the specific assays. The remaining 18 samples were defined as seronegative NMO, including one that was submitted as seropositive. The numbers at the bottom of the figure show the assay sensitivity (%) based on these clinically defined patients.

Figure 4

A heatmap of the entire data set presented in a similar fashion to figures 2 and 3. Each column is an individual assay. They are in the same order as in figures 2 and 3. Each row is an individual serum sample. Results are based on the semiquantitative scores; negative results are blue and positive results range from yellow (low positive) to red (high positive). The control samples are shown in (A) and the neuromyelitis optica (NMO) and NMO spectrum disorder (NMOSD) samples in (B). In total, 34 of 51 samples in the NMOSD are considered seropositive and 17 seronegative. The final serostatus of the NMO and NMOSD samples is listed on the right-hand side in B. The heatmap was generated using GENE-E V.3.0.204 (http://www.broadinstitute.org/cancer/software/GENE-E/).

Figure 5

Overall metrics of the AQP4 assays. Sixty-six samples were considered seropositive: 32 NMO and 34 NMOSD. Using the semiquantitative scores of 0–4 for each assay result, there was no difference in the average assay score across 21 assays between the NMO and NMOSD groups (A). (B–D) The assays are grouped by assay type on the x-axis with the study assay number in parentheses. The sensitivity (B) of assays was based on the samples classified as AQP4-antibody positive NMO or NMOSD (66 in total). The specificity (C) is based on the 92 control samples and the 35 seronegative NMO/SD samples. The accuracy (D) calculation was based on the categories described above: (((true positive+true negative)/total tests)×100). CBA, cell-based assay; IHC, immunohistochemistry; NMO, neuromyelitis optica; NMOSD, NMO spectrum disorder; SD, spectrum disorder.