Sample-to-answer platform for the clinical evaluation of COVID-19 using a deep learning-assisted smartphone-based assay

Abstract

Since many lateral flow assays (LFA) are tested daily, the improvement in accuracy can greatly impact individual patient care and public health. However, current self-testing for COVID-19 detection suffers from low accuracy, mainly due to the LFA sensitivity and reading ambiguities. Here, we present deep learning-assisted smartphone-based LFA (SMART^AI-LFA) diagnostics to provide accurate decisions with higher sensitivity. Combining clinical data learning and two-step algorithms enables a cradle-free on-site assay with higher accuracy than the untrained individuals and human experts via blind tests of clinical data (n = 1500). We acquired 98% accuracy across 135 smartphone application-based clinical tests with different users/smartphones. Furthermore, with more low-titer tests, we observed that the accuracy of SMART^AI-LFA was maintained at over 99% while there was a significant decrease in human accuracy, indicating the reliable performance of SMART^AI-LFA. We envision a smartphone-based SMART^AI-LFA that allows continuously enhanced performance by adding clinical tests and satisfies the new criterion for digitalized real-time diagnostics.

Fig. 1. Workflow of SMART^AI-LFA for COVID-19 diagnostics.

a Experimental design of SMART^AI-LFA, enabling sample-to-answer for COVID-19 with the aid of deep learning-assisted determination. The architecture mainly consists of object finding and binary classification, which are trained over the dataset containing 16,919 COVID-19 tests (11,468 positive and 5451 negatives). b The main validation of SMART^AI-LFA by the blind test (n = 1500) from untrained individuals, human experts, and SMART^AI-LFA, validating SMART^AI-LFA’s clinical predictability. SMART^AI-LFA deep learning-assisted smartphone-based LFA, LFA lateral flow assay.

Fig. 2. Algorithm optimization.

a The algorithm of SMART^AI-LFA consists of object findings and classification. b, c Algorithm #1: Object finding; b ROC curves, and c prediction accuracy of object findings using three different approaches. Detection of only the test line reveals a higher accuracy than the whole LFA cassette and LFA cassette window. d–f Algorithm #2: Classification; d RMSE values from seven different frameworks, showing ResNet-18 and 50, provide a high-accuracy diagnosis. e Training of SMART^AI-LFA; we prepared five data sets and represented acquiring data and its data augmentation, allowing enhanced accuracy of deep learning-based smartphone assay. f Examples of data augmentations. SMART^AI-LFA deep learning-assisted smartphone-based LFA, ROC receiver operating characteristic, LFA lateral flow assay, RMSE root mean square error.

Fig. 3. Clinical validation via blind tests, cross-reactivity, concentration prediction, and daily monitoring.

a–e Blind test; a Workflow of the clinical validation process using a blind test using 1500 test images (n = 1000 patients and n = 500 healthy controls). Using SMART^AI-LFA, we carried out blind tests (n = 1500) of untrained individuals (n = 10) and human experts (n = 10) every 150 test images and compared it with SMART^AI-LFA results, showing great enhancement in sensitivity, specificity, and accuracy using a SMART^AI-LFA. b The ROC curve and c prediction accuracy. d, e Answers for three positive clinical sample images, clarifying the AI’s decision ability. f Cross-reactivity using different respiratory viruses, revealing no cross-reactivity. g The concentration prediction ability of SMART^AI-LFA using a heat map, representing the ability of quantitative analysis. h The sample concentration prediction with clinical patient sample (female, 33 y) according to dilution factors. i Daily COVID-19 test of clinical sample (Male, 27 y), showing the ability of daily monitoring of virus titers via SMART^AI-LFA. SMART^AI-LFA deep learning-assisted smartphone-based LFA, ROC receiver operating characteristic, AI artificial intelligence.

Fig. 4. Clinical tests with a smartphone application.

a Smartphone application and b schematic diagram depicting the data flows from a smartphone to the server (AWS), where the two algorithms are located. c The ROC curves according to training data (standard only (n = 8914) and additional clinical data (n = 8005)). d The ROC curves and confusion matrix of the two algorithms (SMART^AI-LFA and xRcovid) for clinical tests (n = 3278). e, f Universality test. e The accuracy of the 135 app-based tests with different users/smartphones, showing 98% accuracy. f The total averaged sensitivity and specificity from eight different LFA models (LFA model 1 and 7 different models) were determined as 94.8% and 90.9%, respectively. g The tunability of sensitivity and specificity according to the training data. h The accuracy according to test data (n = 3), including three different low-titer ratios (31, 61, and 91%), showing more reliable performance of SMART^AI-LFA than humans. Error bars represent standard deviation from the mean. AWS Amazon Web Services, ROC receiver operating characteristic, SMART^AI-LFA deep learning-assisted smartphone-based LFA.