A Comprehensive Review of Deep Learning in Computer Vision for Monitoring Apple Tree Growth and Fruit Production

Abstract

The high nutritional and medicinal value of apples has contributed to their widespread cultivation worldwide. Unfavorable factors in the healthy growth of trees and extensive orchard work are threatening the profitability of apples. This study reviewed deep learning combined with computer vision for monitoring apple tree growth and fruit production processes in the past seven years. Three types of deep learning models were used for real-time target recognition tasks: detection models including You Only Look Once (YOLO) and faster region-based convolutional network (Faster R-CNN); classification models including Alex network (AlexNet) and residual network (ResNet); segmentation models including segmentation network (SegNet), and mask regional convolutional neural network (Mask R-CNN). These models have been successfully applied to detect pests and diseases (located on leaves, fruits, and trunks), organ growth (including fruits, apple blossoms, and branches), yield, and post-harvest fruit defects. This study introduced deep learning and computer vision methods, outlined in the current research on these methods for apple tree growth and fruit production. The advantages and disadvantages of deep learning were discussed, and the difficulties faced and future trends were summarized. It is believed that this research is important for the construction of smart apple orchards.

Keywords: apple tree growth; computer vision; fruit production; smart orchard; target recognition.

Figure 1

Summary of literature searches for monitoring apple tree growth and fruit yield based on deep learning and computer vision techniques.

Figure 2

The PMD for identification of pests. (a) The structure of the EfficientDet neural network. (b) The PMD. (c) Automatic counting results for PMD models (red border box—detecting classe malifoliella, green border box—detecting classes other insects) [27].

Figure 3

Accurate localization and identification of apple leaf diseases using YOLOV5-CBAM-C3TR. (A) The overall architecture of YOLOV5-CBAM-C3TR. (B) The detailed structure of the added optimization module: (1) CBAM, (2) Transformer. (C) Comparison of recognition results of different models in real scenes: (3) original image, (4) recognition results using YOLOV5, and (5) recognition results using YOLOV5-CBAM-C3TR [39].

Figure 4

The improved YOLOV7 model for identifying apple blossoms. (a) Parameter configuration and overall structure diagram of the improved YOLOV7 model. (b) Comparison of the accuracy of different models for detecting apple blossoms using thermodynamic force maps [57].

Figure 5

Counting apple production in the orchard using YOLOV7 + MAM. (A) The specific architecture of the YOLOV7 + MAM model. (B) Performance demonstration of YOLOV7 + MAM model for apple fruit recognition: (1) the original image in orchard, (2) the image of YOLOV7 + MAM model for identification apples in orchard. (C) The apple tracking results of the model for different video frames: (3) apple tracking results for Video ID1 using YOLOV7 + MAM + ByteTrack, (4) apple tracking results for Video ID2 using YOLOV7 + MAM + ByteTrack, (5) apple tracking results for Video ID3 using YOLOV7 + MAM + ByteTrack [75].

Figure 6

(A) Flowchart of apple fruit defect detection using ASDINet. (B) The comparison of AP metrics versus time with other SOTA models. (C) The comparison of loss metrics with other SOTA models. (D) Class activation map: (1) original image, (2) recognition results of apple fruit defects by U-Net [85].