Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Sep 3;23(9):1160.
doi: 10.3390/e23091160.

A Crop Image Segmentation and Extraction Algorithm Based on Mask RCNN

Shijie Wang  1 , Guiling Sun  1 , Bowen Zheng  1 , Yawen Du  1
Affiliations

A Crop Image Segmentation and Extraction Algorithm Based on Mask RCNN

Shijie Wang et al. Entropy (Basel). .

Abstract

The wide variety of crops in the image of agricultural products and the confusion with the surrounding environment information makes it difficult for traditional methods to extract crops accurately and efficiently. In this paper, an automatic extraction algorithm is proposed for crop images based on Mask RCNN. First, the Fruits 360 Dataset label is set with Labelme. Then, the Fruits 360 Dataset is preprocessed. Next, the data are divided into a training set and a test set. Additionally, an improved Mask RCNN network model structure is established using the PyTorch 1.8.1 deep learning framework, and path aggregation and features are added to the network design enhanced functions, optimized region extraction network, and feature pyramid network. The spatial information of the feature map is saved by the bilinear interpolation method in ROIAlign. Finally, the edge accuracy of the segmentation mask is further improved by adding a micro-fully connected layer to the mask branch of the ROI output, employing the Sobel operator to predict the target edge, and adding the edge loss to the loss function. Compared with FCN and Mask RCNN and other image extraction algorithms, the experimental results demonstrate that the improved Mask RCNN algorithm proposed in this paper is better in the precision, Recall, Average precision, Mean Average Precision, and F1 scores of crop image extraction results.

Keywords: Mask RCNN; deep learning; instance segmentation; sobel operator.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Mask RCNN algorithm network structure.
Figure 2
Figure 2
FPN network architecture.
Figure 3
Figure 3
ROIAlign bipolar interpolation.
Figure 4
Figure 4
ROI generated before and after the RPN network improvement. (a) ROI generated before the RPN network optimization; (b) ROI generated after the RPN network optimization.
Figure 5
Figure 5
Improved FPN network structure.
Figure 6
Figure 6
Bottom-up feature map structure.
Figure 7
Figure 7
Adding of micro fully connected layer branch to the mask.
Figure 8
Figure 8
Image samples of some crops in the Labelme annotated Dataset: (a) Original image; (b) manual segmentation mask map.
Figure 8
Figure 8
Image samples of some crops in the Labelme annotated Dataset: (a) Original image; (b) manual segmentation mask map.
Figure 9
Figure 9
Image segmentation of multiple crops: (a) Original image; (b) method of this article; (c) Mask RCNN; (d) U-net; (e) FCN.4.4.
Figure 9
Figure 9
Image segmentation of multiple crops: (a) Original image; (b) method of this article; (c) Mask RCNN; (d) U-net; (e) FCN.4.4.
Figure 10
Figure 10
AP value of some crops: (a) Beetroot; (b) Granadilla; (c) Kaki; (d) Onion White.
Figure 10
Figure 10
AP value of some crops: (a) Beetroot; (b) Granadilla; (c) Kaki; (d) Onion White.
Figure 11
Figure 11
Example of partial crops segmentation: (a) Original image; (b) method of this article; (c) Mask RCNN; (d) U-net; (e) FCN.
Figure 11
Figure 11
Example of partial crops segmentation: (a) Original image; (b) method of this article; (c) Mask RCNN; (d) U-net; (e) FCN.
See this image and copyright information in PMC

References

    1. Uddin M.A., Ayaz M., Mansour A., Aggoune E.M., Sharif Z., Razzak I. Cloud-connected flying edge computing for smart agriculture. Peer Peer Netw. Appl. 2021:1–11. doi: 10.1007/s12083-021-01191-6. - DOI
    1. Agarwal M., Gupta S.K., Biswas K.K. Development of Efficient CNN model for Tomato crop disease identification. Sustain. Comput. Inform. Syst. 2020;28:100407. doi: 10.1016/j.suscom.2020.100407. - DOI
    1. Iraji M.S. Comparison between soft computing methods for tomato quality grading using machine vision. J. Food Meas. Charact. 2019;13:1–15. doi: 10.1007/s11694-018-9913-2. - DOI
    1. Pratapa A., Doron M., Caicedo J.C. Image-based cell phenotyping with deep learning. Curr. Opin. Chem. Biol. 2021;65:9–17. doi: 10.1016/j.cbpa.2021.04.001. - DOI - PubMed
    1. Minaee S., Boykov Y.Y., Porikli F., Plaza A.J., Kehtarnavaz N., Terzopoulos D. Image segmentation using deep learning: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 2021 doi: 10.1109/TPAMI.2021.3059968. - DOI - PubMed

LinkOut - more resources