Development of parallel forms of a brief smell identification test useful for longitudinal testing

Abstract

Although there are numerous brief odor identification tests available for quantifying the ability to smell, none are available in multiple parallel forms that can be longitudinally administered without potential confounding from knowledge of prior test items. Moreover, empirical algorithms for establishing optimal test lengths have not been generally applied. In this study, we employed and compared eight machine learning algorithms to develop a set of four brief parallel smell tests employing items from the University of Pennsylvania Smell Identification Test that optimally differentiated 100 COVID-19 patients from 132 healthy controls. Among the algorithms, linear discriminant analysis (LDA) achieved the best overall performance. The minimum number of odorant test items needed to differentiate smell loss accurately was identified as eight. We validated the sensitivity of the four developed tests, whose means and variances did not differ from one another (Bradley-Blackwood test), by sequential testing an independent group of 32 subjects that included persons with smell dysfunction not due to COVID-19. These eight-item tests clearly differentiated the olfactory compromised subjects from normosmics, with areas under the ROC curve ranging from 0.79 to 0.83. Each test was correlated with the overall UPSIT scores from which they were derived. These brief smell tests can be used separately or sequentially over multiple days in a variety of contexts where longitudinal olfactory testing is needed.

Keywords: Anosmia; Hyposmia; Longitudinal testing; Olfactory test; UPSIT.

Fig. 1

Optimization criteria for different machine learning methods. Note. Optimization criteria (arithmetic mean of the accuracy, sensitivity, specificity, and AUC metrics) achieved during the feature selection strategy with increasing odorant counts for each machine learning method. Leave-one-out cross-validation was used to calculate the classification performance metrics

Fig. 2

Sequential feature selection performance metrics for the LDA model. Note. Odorants appear in the order of their inclusion to the selected feature set during the sequential feature selection algorithm. Ten-fold cross validation was used to quantify the variability. Shaded areas show the range of performance values observed in ten-fold cross validations. Odorant names indicate which of the UPSIT® odorant/response sets were employed

Fig. 3

Comparison of sequential test score with UPSIT^® score. Note. Total score of the sequential tests are summed up as serial test score which is correlated with UPSIT score (r = 0.84, p < 0.0001). Area under the curve of the receiver operating characteristics curve analysis was 0.91 in differentiating normosmics from olfactory compromised subjects. The right subpanels compare each parallel smell test with UPSIT^® score with the AUCs of the ROCs