Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comment
. 2024 Apr 1;178(4):401-407.
doi: 10.1001/jamapediatrics.2024.0011.

Development and Validation of an Automated Classifier to Diagnose Acute Otitis Media in Children

Nader Shaikh  1 , Shannon J Conway  1 , Jelena Kovacevic  2 , Filipe Condessa  3 , Timothy R Shope  1 , Mary Ann Haralam  1 , Catherine Campese  1 , Matthew C Lee  1 , Tomas Larsson  4 , Zafer Cavdar  4 , Alejandro Hoberman  1
Affiliations
Comment

Development and Validation of an Automated Classifier to Diagnose Acute Otitis Media in Children

Nader Shaikh et al. JAMA Pediatr. .

Abstract

Importance: Acute otitis media (AOM) is a frequently diagnosed illness in children, yet the accuracy of diagnosis has been consistently low. Multiple neural networks have been developed to recognize the presence of AOM with limited clinical application.

Objective: To develop and internally validate an artificial intelligence decision-support tool to interpret videos of the tympanic membrane and enhance accuracy in the diagnosis of AOM.

Design, setting, and participants: This diagnostic study analyzed otoscopic videos of the tympanic membrane captured using a smartphone during outpatient clinic visits at 2 sites in Pennsylvania between 2018 and 2023. Eligible participants included children who presented for sick visits or wellness visits.

Exposure: Otoscopic examination.

Main outcomes and measures: Using the otoscopic videos that were annotated by validated otoscopists, a deep residual-recurrent neural network was trained to predict both features of the tympanic membrane and the diagnosis of AOM vs no AOM. The accuracy of this network was compared with a second network trained using a decision tree approach. A noise quality filter was also trained to prompt users that the video segment acquired may not be adequate for diagnostic purposes.

Results: Using 1151 videos from 635 children (majority younger than 3 years of age), the deep residual-recurrent neural network had almost identical diagnostic accuracy as the decision tree network. The finalized deep residual-recurrent neural network algorithm classified tympanic membrane videos into AOM vs no AOM categories with a sensitivity of 93.8% (95% CI, 92.6%-95.0%) and specificity of 93.5% (95% CI, 92.8%-94.3%) and the decision tree model had a sensitivity of 93.7% (95% CI, 92.4%-94.9%) and specificity of 93.3% (92.5%-94.1%). Of the tympanic membrane features outputted, bulging of the TM most closely aligned with the predicted diagnosis; bulging was present in 230 of 230 cases (100%) in which the diagnosis was predicted to be AOM in the test set.

Conclusions and relevance: These findings suggest that given its high accuracy, the algorithm and medical-grade application that facilitates image acquisition and quality filtering could reasonably be used in primary care or acute care settings to aid with automated diagnosis of AOM and decisions regarding treatment.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Shope reported receiving grants from the National Institutes of Health, Merck Foundation, and the American Academy of Pediatrics outside the submitted work. Mr Larsson reported receiving personal fees from University of Pittsburgh to Dcipher Analytics for services rendered in developing the application programming interface that gives programmatic access to trained artificial intelligence models during the conduct of the study. Mr Cavdar reported receiving personal fees from University of Pittsburg to Dcipher Analytics for services rendered in developing the application programming interface that gives programmatic access to trained artificial intelligence models during the conduct of the study. Dr Hoberman reported having a patent for a pediatric oral suspension formulation of amoxicillin–clavulanate potassium and the method for its use (licensed to Kaizen Biosciences). Drs Hoberman, Shaikh, and Kovačević are listed as inventors under US patent number 9636007 covering automated diagnosis of AOM using the specific features predicted by the decision support tool to diagnose AOM vs no AOM reported in this manuscript. No other disclosures were reported.

Figures

Figure 1.
Figure 1.. Computation Flow and Model Architecture
The learning and decision-making paradigm of a deep residual-recurrent neural network (DR-RNN), shown in panel A, is similar to visual cognition of a human brain. Each layer’s responsibility can be simplified as follows: The DR-RNN is like a team of experts who specialize in recognizing different parts of an image and passes on their findings (eg, colors, edges, shapes, objects) to the next expert. If 1 of the experts is not certain, they can also ask the previous expert for help. This way, the team gets really good at recognizing all the details of an image. For the long short-term memory recurrent neural network (LSTM), the diagnosis decision is not based on information in a single image but on a sequence of images. One frame may tell more about color while another frame gives translucency information. LSTM remembers what has been seen in the previous frames and pays attention to the sequence of observations. The attention layer is like a spotlight and helps the next layers of the model focus on the most important parts of the information remembered by LSTM. It looks at clues found earlier, and it says, "Pay extra attention to this part right here!" to the fully connected neural network. The fully connected neural network is like a team of experts who take all the important clues and put them together. They talk to each other, share their thoughts, and come to a conclusion about features or diagnosis. Decision tree models (B) work by making a series of binary decisions based on the input features, ultimately leading to classification. Our decision tree model takes expert feature estimations from the DR-RNN model, computes the importance of these features for the final decision (diagnosis), and creates decision-making rules that maximize the accuracy.
Figure 2.
Figure 2.. Development of the Quality Filter
The figure shows the visual landscape with tympanic membrane frames clustered based on similarity (A), an example of 1 cluster evaluated by experts (B), and final landscape with expert determinations (C). The X and Y axes in panel C represent the transformed coordinates of the high-dimensional image features.
Figure 3.
Figure 3.. Flow Diagram of Combined, Training, and Test Study Populations
See this image and copyright information in PMC

Comment in

References

    1. Schappert SM, Rechtsteiner EA. Ambulatory medical care utilization estimates for 2007. Vital Health Stat 13. 2011;(169):1-38. - PubMed
    1. Young DE, Ten Cate WJ, Ahmad Z, Morton RP. The accuracy of otomicroscopy for the diagnosis of paediatric middle ear effusions. Int J Pediatr Otorhinolaryngol. 2009;73(6):825-828. doi:10.1016/j.ijporl.2009.02.012 - DOI - PubMed
    1. Finkelstein JA, Metlay JP, Davis RL, Rifas-Shiman SL, Dowell SF, Platt R. Antimicrobial use in defined populations of infants and young children. Arch Pediatr Adolesc Med. 2000;154(4):395-400. doi:10.1001/archpedi.154.4.395 - DOI - PubMed
    1. Pichichero ME, Poole MD. Assessing diagnostic accuracy and tympanocentesis skills in the management of otitis media. Arch Pediatr Adolesc Med. 2001;155(10):1137-1142. doi:10.1001/archpedi.155.10.1137 - DOI - PubMed
    1. Pichichero ME. Diagnostic accuracy, tympanocentesis training performance, and antibiotic selection by pediatric residents in management of otitis media. Pediatrics. 2002;110(6):1064-1070. doi:10.1542/peds.110.6.1064 - DOI - PubMed

Publication types