A Robust Cross-Platform Solution With the Sense2Quit System to Enhance Smoking Gesture Recognition: Model Development and Validation Study

Abstract

Background: Smoking is a leading cause of preventable death, and people with HIV have higher smoking rates and are more likely to experience smoking-related health issues. The Sense2Quit study introduces innovative advancements in smoking cessation technology by developing a comprehensive mobile app that integrates with smartwatches to provide real-time interventions for people with HIV attempting to quit smoking.

Objective: We aim to develop an accurate smoking cessation app that uses everyday smartwatches and an artificial intelligence model to enhance the recognition of smoking gestures by effectively addressing confounding hand gestures that mimic smoking, thereby reducing false positives. The app ensures seamless usability across Android (Open Handset Alliance [led by Google]) and iOS platforms, with optimized communication and synchronization between devices for real-time monitoring.

Methods: This study introduces the confounding resilient smoking model, specifically trained to distinguish smoking gestures from similar hand-to-mouth activities used by the Sense2Quit system. By incorporating confounding gestures into the model's training process, the system achieves high accuracy while maintaining efficiency on mobile devices. To validate the model, 30 participants, all people with HIV who smoked cigarettes, were recruited. Participants wore smartwatches on their wrists and performed various hand-to-mouth activities, including smoking and other gestures such as eating and drinking. Each participant spent 15 to 30 minutes completing the tasks, with each gesture lasting 5 seconds. The app was developed using the Flutter framework to ensure seamless functionality across Android and iOS platforms, with robust synchronization between the smartwatch and smartphone for real-time monitoring.

Results: The confounding resilient smoking model achieved an impressive F₁-score of 97.52% in detecting smoking gestures, outperforming state-of-the-art models by distinguishing smoking from 15 other daily hand-to-mouth activities, including eating, drinking, and yawning. Its robustness and adaptability were further confirmed through leave-one-subject-out evaluation, demonstrating consistent reliability and generalizability across diverse individuals. The cross-platform app, developed using Flutter (Google), demonstrated consistent performance across Android and iOS devices, with only a 0.02-point difference in user experience ratings between the platforms (iOS 4.52 and Android 4.5). The app's continuous synchronization ensures accurate, real-time tracking of smoking behaviors, enhancing the system's overall utility for smoking cessation.

Conclusions: Sense2Quit represents a significant advancement in smoking cessation technology. It delivers timely, just-in-time interventions through innovations in cross-platform communication optimization and the effective recognition of confounding hand gestures. These improvements enhance the accuracy and accessibility of real-time smoking detection, making Sense2Quit a valuable tool for supporting long-term cessation efforts among people with HIV trying to quit smoking.

International registered report identifier (irrid): RR2-10.2196/49558.

Keywords: confounding gestures; convolutional neural networks; e-cigarette; mobile application; mobile health; mobile phone; people living with HIV; real-time monitoring; smartwatch; smoking cessation; smoking gestures; vaping; wearable technology.

Figure 1

Overview of the Sense2Quit smartphone and smartwatch system architecture.

Figure 2

User interface screenshots of the Sense2Quit smartphone app, showcasing the (A) login, (B) home, (C) smoking trends, (D) list of games, (E) Tetris game, and (F) tips screens.

Figure 3

User interface for the data collection screen demonstrating the states before data collection on the left and after data collection is completed on the right.

Figure 4

Workflow of the automatic smoking detection system, from sensor data collection and processing to using the smoking detection model to provide user feedback.

Figure 5

Smoking detection model architecture illustrating the underlying layers.

Figure 6

The Sense2Quit dashboard for research staff provided usage information to participants enrolled in this study.

Figure 7

Confusion matrices for models (A) CRS, (B) baseline CNN, (C) baseline 1, and (D) baseline 2. CNN: convolutional neural network; CRS: confounding resilient smoking.

Figure 8

ROC curves for the smoking detection model. The blue lines represent the CRS model, the green lines correspond to the Baseline CNN, and the yellow and red lines represent the state-of-the-art baseline 1 and 2 models, respectively. Solid lines represent smoking detection, and dotted lines represent nonsmoking detection. The AUC fractions indicate the performance for smoking (first value) and nonsmoking (second value) detection. AUC: area under the curve; CNN: convolutional neural network; CRS: confounding resilient smoking; ROC: receiver operating characteristic.

Figure 9

Confusion matrix for 16-class classification: red highlights indicate false negatives and false positives for the target class “smoking.”.

Figure 10

Visualization of raw accelerometer and gyroscope data of various confounding gestures (smoking, eating, drinking, yawning, waving, and scratching the head).

Figure 11

Smartwatch power consumption during active and baseline states.