Predicting the infecting dengue serotype from antibody titre data using machine learning

Abstract

The development of a safe and efficacious vaccine that provides immunity against all four dengue virus serotypes is a priority, and a significant challenge for vaccine development has been defining and measuring serotype-specific outcomes and correlates of protection. The plaque reduction neutralisation test (PRNT) is the gold standard assay for measuring serotype-specific antibodies, but this test cannot differentiate homotypic and heterotypic antibodies and characterising the infection history is challenging. To address this, we present an analysis of pre- and post-infection antibody titres measured using the PRNT, collected from a prospective cohort of Thai children. We applied four machine learning classifiers and multinomial logistic regression to the titre data to predict the infecting serotype. The models were validated against the true infecting serotype, identified using RT-PCR. Model performance was calculated using 100 bootstrap samples of the train and out-of-sample test sets. Our analysis showed that, on average, the greatest change in titre was against the infecting serotype. However, in 53.4% (109/204) of the subjects, the highest titre change did not correspond to the infecting serotype, including in 34.3% (11/35) of dengue-naïve individuals (although 8/11 of these seronegative individuals were seropositive to Japanese encephalitis virus prior to their infection). The highest post-infection titres of seropositive cases were more likely to match the serotype of the highest pre-infection titre than the infecting serotype, consistent with antigenic seniority or cross-reactive boosting of pre-infection titres. Despite these challenges, the best performing machine learning algorithm achieved 76.3% (95% CI 57.9-89.5%) accuracy on the out-of-sample test set in predicting the infecting serotype from PRNT data. Incorporating additional spatiotemporal data improved accuracy to 80.6% (95% CI 63.2-94.7%), while using only post-infection titres as predictor variables yielded an accuracy of 71.7% (95% CI 57.9-84.2%). These results show that machine learning classifiers can be used to overcome challenges in interpreting PRNT titres, making them useful tools in investigating dengue immune dynamics, infection history and identifying serotype-specific correlates of protection, which in turn can support the evaluation of clinical trial endpoints and vaccine development.

Fig 1. Mean (point) ± standard deviation (error bar) of the pre- and post-infection log PRNT titre, by infecting DENV serotype as quantified by RT-PCR (combined for DENV seronegative and DENV seropositive individuals).

PRNT: plaque reduction neutralisation test. Dengue virus (DENV-1: red, DENV-2: blue, DENV-3: green, DENV-4: dark blue. Japanese encephalitis virus (JEV): orange. RT-PCR: reverse transcriptase polymerase chain reaction. Seronegative individuals were defined as pre-infection titres < 10 for all four DENV serotypes. Seropositive individuals were defined as those with a pre-infection titre ≥ 10 for at least one DENV serotype.

Fig 2. Highest pre- and post-infection log PRNT titres in cases in seropositive individuals (N = 169).

(a) The frequency of subjects whose highest post-infection titre was against the infecting serotype, a non-infecting serotype or whose highest titre was tied between one or more serotypes due to maximum PRNT dilution. (b) The percentage of subjects whose highest pre- and post-infection titres were against each serotype and JEV, stratified by the infecting serotype. Tied highest titres are split. Seropositive individuals were defined as those with a pre-infection titre ≥ 10 for at least one DENV serotype. PRNT: plaque reduction neutralisation test. RT-PCR: reverse transcriptase polymerase chain reaction. DENV: dengue virus. JEV: Japanese encephalitis virus.

Fig 3. Greatest change in log PRNT titres following infection with a DENV serotype, as quantified by RT-PCR in (a) cases in seropositive individuals and (b) cases in DENV seronegative individuals.

Seronegative individuals were defined as pre-infection titres < 10 for all four DENV serotypes. Seropositive individuals were defined as those with a pre-infection titre ≥ 10 for at least one DENV serotype. The plot area is proportional to the number of seropositive / seronegative cases. Points indicate a single case. PRNT: plaque reduction neutralisation test. DENV: dengue virus. JEV: Japanese encephalitis virus. RT-PCR: reverse transcriptase polymerase chain reaction.

Fig 4. Algorithm of model development and validation.

For each scenario and each classifier (1), the data were randomly split into 90% train and 10% test sets (2). The train data were pre-processed and the model hyperparameters were tuned using leave-one-out-cross-validation (3–4). Performance on the train data was calculated (5) and the set of hyperparameters with the highest kappa statistic were chosen for the final model (6). This model was then applied to the test data (7) and the test performance metrics were calculated (8). Steps 2–8 were repeated 100 times using bootstrap samples of the test and train sets, and the mean and 95% CI were calculated for test and train performance metrics (9). Scenario A: all titre predictor variables (pre- and post-infection PRNT titres and change in titre against all four dengue virus serotypes and Japanese encephalitis virus, and the number of days between measurement of the pre- and post-infection titres and the date of infection). Scenario B: all predictor variables (titre predictor variables plus age, year, and school). Scenario C: post-infection PRNT titre of the dengue serotypes were predictor variables. Scenario D: all titre predictor variables but only predicted seropositive cases. Seropositive individuals were defined as those with a pre-infection titre ≥ 10 for at least one DENV serotype. CI: confidence interval. PRNT: plaque reduction neutralisation test.

Fig 5. Comparison of regression and machine learning models for predicting the infecting DENV serotype.

(a) Performance of each model in predicting the infecting serotype across four scenarios using a 90/10 train/test split of the data. Scenario A and scenario D predictor variables were pre- and post-infection PRNT titres against DENV 1–4 and JEV, change in PRNT titre and number of days between infection and measurement of pre- and post-infection titres. Scenario B used the same predictor variables as A and C, plus year, age, and school. Scenario C predictor variables were post-infection PRNT against DENV 1–4. Scenario’s A, B, and C predicted all cases, scenario D predicted cases in seropositive individuals only. Seropositive individuals were defined as those with a pre-infection titre ≥ 10 for at least one DENV serotype. (b) Comparison of different train/test proportions (90/10, 80/20, 70/30, 60/40, 50/50) for predicting the infecting dengue virus serotype (scenario A), using the GBM classification model. For each performance metric, the mean (point) and 95% confidence interval (error bar) were calculated using 100 bootstrap samples of the test and train sets. Test performance was calculated by predicting on the hold-out-sample. Train performance was calculated using leave-one-out cross validation. ANN: artificial neural network. MLR: multinomial logistic regression. SVM: support vector machine. GBM: gradient boosting machine. RF: random forest. PRNT: plaque reduction neutralisation test. DENV: dengue virus.

Fig 6. Post-infection (a) and change (b) in log PRNT titres of the predicted infecting DENV serotype (y axis) and the true infecting DENV serotype (x axis) in cases that were misclassified by the gradient boosting machine model.

N = the number of times a case was misclassified out of 100 bootstrap samples of the 10% test set. Predictor variables were pre- and post-infection PRNT titres against DENV-1-4 and JEV, change in PRNT titre and number of days between infection and measurement of pre- and post-infection titres (scenario A). PRNT: plaque reduction neutralisation test. DENV: dengue virus. JEV: Japanese encephalitis virus.