Automatic Clinical Assessment of Swallowing Behavior and Diagnosis of Silent Aspiration Using Wireless Multimodal Wearable Electronics

Abstract

Dysphagia is more common in conditions such as stroke, Parkinson's disease, and head and neck cancer. This can lead to pneumonia, choking, malnutrition, and dehydration. Currently, the diagnostic gold standard uses radiologic imaging, the videofluoroscopic swallow study (VFSS); however, it is expensive and necessitates specialized facilities and trained personnel. Although several devices attempt to address the limitations, none offer the clinical-grade quality and accuracy of the VFSS. Here, this study reports a wireless multimodal wearable system with machine learning for automatic, accurate clinical assessment of swallowing behavior and diagnosis of silent aspirations from dysphagia patients. The device includes a kirigami-structured electrode that suppresses changes in skin contact impedance caused by movements and a microphone with a gel layer that effectively blocks external noise for measuring high-quality electromyograms and swallowing sounds. The deep learning algorithm offers the classification of swallowing patterns while diagnosing silent aspirations, with an accuracy of 89.47%. The demonstration with post-stroke patients captures the system's significance in measuring multiple physiological signals in real-time for detecting swallowing disorders, validated by comparing them with the VFSS. The multimodal electronics can ensure a promising future for dysphagia healthcare and rehabilitation therapy, providing an accurate, non-invasive alternative for monitoring swallowing and aspiration events.

Keywords: dysphagia; kirigami structure; machine learning; silent aspiration; swallowing disorder; wearable electronics.

Figure 1

Design, architecture, and swallowing detection process using a wireless multimodal wearable electronic system. A) Illustration of the portable wireless wearable system mounted on the neck to detect swallowing behavior and abnormality without special facilities and trained personnel. B) Exploded views of the device, including two EMG electrodes, a microphone, a wireless circuit, and stretchable interconnectors encapsulated by a soft membrane. C) Photo of the wearable device mounted on the skin with three extended sensing pads. D) Schematic anatomy of the human neck related to the swallowing movements and the locations where the sensors are positioned. E) Flowchart showing the process from physiological signal detection to data classification. Signals are acquired from EMG electrodes and a microphone to a digitized analog‐to‐digital converter and sent via Bluetooth for deep learning analysis, automatic swallowing classification, and abnormal aspiration detection.

Figure 2

Kirigami‐structured device for enhanced conformal contact and more consistent signal detection. A) Photo of fabric with kirigami cuts for embedding serpentine electrodes, designed for enhanced strain distribution and better skin contact. B) FEA results showing the comparison of strain distribution in electrodes with different kirigami patterns under tension, highlighting the best performance from the Y‐shape. C) Negligible electrical resistance changes of the device under 20% tension capturing the strain‐damping effect of the Y‐shaped kirigami patterns. D) Comparison of the maximum strain applied to the electrode according to the perforated pattern when attached to the patch spherical dummy (inset: photos of the actual patch attachment state). E) Skin‐electrode contact impedance on the chin demonstrating the stable contact of the patch despite jaw movement (open and close), showing the advantages of kirigami patterns for consistent signal recording. F,G) EMG signal comparison from the digastric and sternohyoid muscles during swallowing action, using the kirigami patch and commercial Ag/AgCl electrodes. The results show similar data quality and SNR, although the commercial one shows a slightly higher signal due to a much bigger electrode contact area.

Figure 3

Performance of a soft material‐enabled wearable sensor for sound detection. A) Schematic of a testing setup, showing the microphone consistently exposed to white noise and a 200 Hz chirp to simulate a hospital environment while measuring swallowing sound profiles through an artificial esophagus. B) Graph showing SNR values according to the curvature of the skin replica, comparing the presence and absence of a gel layer for sound isolation. C,D) 2D plots and spectrograms depicting the placement of the microphone on a skin replica, illustrating waveform collection with a gel layer and the detrimental effects on SNR from gaps without the gel on increased curvatures. Scale bar: 5 mm. E) Application of the microphone with gel on the neck, demonstrating optimal conformity over the thyroid cartilage for efficient sound capture. F) A rigid commercial device on the skin with an air gap when attached to the skin, introducing ambient noise intrusion. G,H) Comparative spectrograms showing the soft platform‐based microphone (G) and the commercial device (H), demonstrating clear differences in swallowing event detection and sound quality. The rigid one has many motion‐related noise disturbances in signal recording.

Figure 4

Physiology of swallowing and detection of abnormal swallowing – silent aspiration by using the wearable system. A) Schematic images showing swallowing phases. During the swallowing motion, the digastric and sternohyoid muscles, connected to the hyoid bone, repeat contraction and relaxation to transport the consistency from the mouth to the esophagus. B) X‐ray image of a normal swallow of 5 mL water, captured during VFSS. The airway is well protected while the bolus passes through the esophagus. Scale bar: 3 cm. C) Data measured by the wearable system with three sensors, showing a well‐synchronized sequence of muscle activities and sounds. D) X‐ray image of an abnormal swallowing of 5 mL water – silent aspiration. Since the airway is not closed well, the bolus penetrated or aspirated to the airway. Scale bar: 3 cm. E) Multimodal sensor data showing that when silent aspiration happens, muscle activities and swallowing sound are out of sync, leading to failure in closing the airway at the exact time.

Figure 5

Development of a deep‐learning algorithm for automated classification of swallowing status and detection of abnormality with multiple post‐stroke patients. A) Schematic overview of the data processing and machine learning for swallowing status classification and silent aspiration detection. B) Machine learning architectures for data classification; Conv: Convolution, FC: Fully Connected layers, DO: Dropout, and BN: Batch Normalization. C) Measured swallowing data from the multimodal wearable system during seven classes, including rest, liquid, soft food, dense food, silent aspiration, cough, and chewing. D) Confusion matrix showing the model classification test accuracy of 89.47% of seven classes for swallowing status classification.