Defining Seropositivity Thresholds for Use in Trachoma Elimination Studies

Abstract

Background: Efforts are underway to eliminate trachoma as a public health problem by 2020. Programmatic guidelines are based on clinical signs that correlate poorly with Chlamydia trachomatis (Ct) infection in post-treatment and low-endemicity settings. Age-specific seroprevalence of anti Ct Pgp3 antibodies has been proposed as an alternative indicator of the need for intervention. To standardise the use of these tools, it is necessary to develop an analytical approach that performs reproducibly both within and between studies.

Methodology: Dried blood spots were collected in 2014 from children aged 1-9 years in Laos (n = 952) and Uganda (n = 2700) and from people aged 1-90 years in The Gambia (n = 1868). Anti-Pgp3 antibodies were detected by ELISA. A number of visual and statistical analytical approaches for defining serological status were compared.

Principal findings: Seroprevalence was estimated at 11.3% (Laos), 13.4% (Uganda) and 29.3% (The Gambia) by visual inspection of the inflection point. The expectation-maximisation algorithm estimated seroprevalence at 10.4% (Laos), 24.3% (Uganda) and 29.3% (The Gambia). Finite mixture model estimates were 15.6% (Laos), 17.1% (Uganda) and 26.2% (The Gambia). Receiver operating characteristic (ROC) curve analysis using a threshold calibrated against external reference specimens estimated the seroprevalence at 6.7% (Laos), 6.8% (Uganda) and 20.9% (The Gambia) when the threshold was set to optimise Youden's J index. The ROC curve analysis was found to estimate seroprevalence at lower levels than estimates based on thresholds established using internal reference data. Thresholds defined using internal reference threshold methods did not vary substantially between population samples.

Conclusions: Internally calibrated approaches to threshold specification are reproducible and consistent and thus have advantages over methods that require external calibrators. We propose that future serological analyses in trachoma use a finite mixture model or expectation-maximisation algorithm as a means of setting the threshold for ELISA data. This will facilitate standardisation and harmonisation between studies and eliminate the need to establish and maintain a global calibration standard.

Fig 1. Typical results from an ELISA plate.

Specimens are sorted by increasing OD values and are each represented by a separate diamond. The mean values of the controls tested in triplicate are represented by coloured horizontal lines.

Fig 2. Threshold values for Laos (1–9 year olds) data.

Panel A shows the threshold as determined by visual inflection point analysis by 12 volunteer individuals. Volunteers had access only to the data presented in the leftmost panels, which shows sorted OD₄₅₀ values. The second panel in A shows the density of data points for the sample while the third panel in A shows a box and whisker plots with the range of threshold values that were selected by the 12 volunteers. The box shows the inter-quartile range for the values, with the thick horizontal line marking the median value. Whiskers show the upper quartile plus 1.5x the range between the 1^st and 3^rd quartiles. Outliers are shown by an open circle. Panel B shows the thresholds set by the finite mixture model and expectation-maximisation algorithm. Density plots of normalised OD values and thresholds, showing the FMM estimated distribution functions of ‘seronegative’ specimens in red and ‘seropositive’ specimens in green. Vertical lines show the threshold values determined by the finite mixture model (right-most line) and the expectation-maximisation algorithm (left-most lines). Panel C compares the threshold specifications by four different methods. Scatterplots show the normalised and sorted OD₄₅₀ values with horizontal lines marking the thresholds specified by VIP (OD₄₅₀ = 0.619), EM (OD₄₅₀ = 0.650), FMM (OD₄₅₀ = 0.696), ROC curve maximising Youden’s J-index (OD₄₅₀ = 0.870), ROC curve with sensitivity >80% (OD₄₅₀ = 0.968) and ROC curve with specificity>98% (OD₄₅₀ = 1.951).

Fig 3. Threshold values for Uganda (1–9 year olds) data.

Panel A shows the threshold as determined by visual inflection point analysis by 12 volunteer individuals, as detailed in Fig 2. Panel B shows the thresholds set by the finite mixture model and expectation-maximisation algorithm, as described in Fig 2. Panel C compares the threshold specifications by four different methods. Scatterplots show the normalised and sorted OD₄₅₀ values with horizontal lines marking the thresholds specified by VIP (OD₄₅₀ = 0.641), EM (OD₄₅₀ = 0.450), FMM (OD₄₅₀ = 0.554), ROC curve maximising Youden’s J-index (OD₄₅₀ = 0.870), ROC curve with sensitivity >80% (OD₄₅₀ = 0.968) and ROC curve with specificity>98% (OD₄₅₀ = 1.951).

Fig 4. Threshold values for Gambian (all ages) data.

Panel A shows the threshold as determined by visual inflection point analysis by 12 volunteer individuals, as detailed above in Fig 2. Panel B shows the thresholds set by the finite mixture model and expectation-maximisation algorithm, as described in Fig 2. Panel C compares the threshold specifications by four different methods. Scatterplots show the normalised and sorted OD₄₅₀ values with horizontal lines marking the thresholds specified by VIP (OD₄₅₀ = 0.570), EM (OD₄₅₀ = 0.570), FMM (OD₄₅₀ = 0.672), ROC curve maximising Youden’s J-index (OD₄₅₀ = 0.870), ROC curve with sensitivity >80% (OD₄₅₀ = 0.968) and ROC curve with specificity>98% (OD₄₅₀ = 1.951). Note that the thresholds set by VIP and EM are identical (0.570 OD₄₅₀) and overlap on the graph.

Fig 5. Receiver Operating Characteristic (ROC) curve showing the relationship between sensitivity, specificity and threshold values.

Three different thresholds were specified to meet the requirements of: (A) an assay (threshold = 0.870 OD_450, specificity = 93.9%, sensitivity = 91.4%, PPV = 89.8%, NPV = 92.4%) with balanced sensitivity and specificity (maximal Youden’s J value); (B) an assay (threshold = 0.965 OD_450, specificity 94.8%, sensitivity = 89.4%) with at least 80% sensitivity and (C) an assay (threshold = 1.951 OD₄₅₀, specificity = 98.3%, sensitivity = 43.9%, PPV = 66.7%, NPV = 95.0%) with at least 98% specificity.