A YOLOv11-based AI system for keypoint detection of auricular acupuncture points in traditional Chinese medicine

Abstract

Objective: This study aims to develop an artificial intelligence model and web-based application for the automatic detection of 21 commonly used auricular acupoints based on the YOLOv11 neural network.

Methods: A total of 660 human ear images were collected from three medical centers. The LabelMe annotation tool was used to label the images with bounding boxes and key points, which were then converted into a format compatible with the YOLO model. Using this dataset, transfer learning and fine-tuning were performed on different-sized versions of the YOLOv11 neural network. The model performance was evaluated on validation and test sets, considering metrics such as mean average precision (mAP) under different thresholds, recall, and detection speed. The best-performing model was subsequently deployed as a web application using the Streamlit library in the Python environment.

Results: Five versions of the YOLOv11 keypoint detection model were developed, namely YOLOv11n, YOLOv11s, YOLOv11m, YOLOv11l, and YOLOv11x. Among them, YOLOv11x achieved the highest performance in the validation set with a precision of 0.991, recall of 0.976, mAP⁵⁰ of 0.983, and mAP^50-95 of 0.625, though it exhibited the longest inference delay (19 ms/img). On the external test set, YOLOv11x achieved an ear recognition accuracy of 0.996, sensitivity of 0.996, and an F1-score of 0.998. For auricular acupoint localization, the model achieved an mAP⁵⁰ of 0.982, precision of 0.975, and recall of 0.976. The model has been successfully deployed as a web application, accessible on both mobile and desktop platforms to accommodate diverse user needs.

Conclusion: The YoloEar21 web application, developed based on YOLOv11x and Streamlit, demonstrates superior recognition performance and user-friendly accessibility. Capable of providing automatic identification of 21 commonly used auricular acupoints across various scenarios for diverse users, it exhibits promising potential for clinical applications.

Keywords: YOLO; artificial intelligence; auricular acupoints; deep learning; keypoint detection.

FIGURE 1

Illustrative samples and distribution of dataset images for the AI model of automatic auricular acupoint keypoint detection. (A) Representative images from the dataset; (B) distribution of image quantities across the training, validation, and test sets.

FIGURE 2

Annotation process and representative examples for the AI model of automatic auricular acupoint keypoint detection. (A) Step 1: Medical staff from various hospital departments collect ear images from individuals of different ages, genders, and occupations using different devices. (B) Step 2: Two acupuncture specialists annotate the collected images using the LabelMe 5.3.1 graphical annotation tool and perform cross-checking. (C) Step 3: A senior acupuncture specialist with 15 years of experience reviews the annotations and makes the final decision on the labeling. (D) Example of Auricular Acupoint Image Annotation. The training and validation sets were annotated by one group of acupuncture specialists, while the external test set was independently annotated by a separate group of clinicians.

FIGURE 3

Keypoint detection system for the automatic recognition of 21 common auricular acupoints.

FIGURE 4

Flowchart of the AI model for automatic detection of auricular acupoint keypoints.

FIGURE 5

The variation trends of loss functions during the training process of different YOLOv11 model versions. (A) The variation trend of Bounding Box Loss across training epochs; (B) The variation trend of Keypoint Loss across training epochs.

FIGURE 6

Performance Metric Trends of Different YOLO Models During Training. (A) Variation trend of bounding box precision; (B) Variation trend of bounding box recall; (C) Variation trend of bounding box mAP50; (D) Variation trend of keypoint precision; (E) Variation trend of keypoint recall; (F) Variation trend of keypoint mAP50. B: Bounding Box, K: Keypoint, mAP⁵⁰: mean average precision at 50% Intersection over Union threshold.

FIGURE 7

Performance comparison of different YOLOv11 model versions. The x-axis represents the inference time required for processing a single image by different YOLOv11 models under the PyTorch framework, measured in milliseconds (ms), where positions further to the left indicate faster processing speeds; the y-axis displays the mean average precision at 50% Intersection over Union threshold (mAP50) obtained by the models on the validation set, with higher positions indicating greater mAP50 values.

FIGURE 8

Grad-CAM Visualization of the AI Model’s Decision-Making Process. (A) Original images; (B) AI model recognition results; (C) Overlay display of activation heatmaps on original images.

FIGURE 9

Prediction results of the model on the test set. (A) Confusion matrix of the AI model for ear detection task, (B) displays the original ear images, (C) presents the manually annotated images, (D) shows the AI model’s prediction results. The model’s predictions (Figure 9D) closely resemble the physician’s annotations (Figure 9C). In Figure 9D, the confidence score of 0.9 for the model’s prediction of the ear is displayed in the upper left corner of the predicted bounding box.

FIGURE 10

Web application developed based on the optimal model and its use cases. By scanning the QR codes in the figure, viewers can observe two real-time detection cases of the AI model performing ear recognition. (A) Operational interface of the web application; (B) Case 1; (C) Case 2.