Phenotyping of Panicle Number and Shape in Rice Breeding Materials Based on Unmanned Aerial Vehicle Imagery

Abstract

The number of panicles per unit area (PNpA) is one of the key factors contributing to the grain yield of rice crops. Accurate PNpA quantification is vital for breeding high-yield rice cultivars. Previous studies were based on proximal sensing with fixed observation platforms or unmanned aerial vehicles (UAVs). The near-canopy images produced in these studies suffer from inefficiency and complex image processing pipelines that require manual image cropping and annotation. This study aims to develop an automated, high-throughput UAV imagery-based approach for field plot segmentation and panicle number quantification, along with a novel classification method for different panicle types, enhancing PNpA quantification at the plot level. RGB images of the rice canopy were efficiently captured at an altitude of 15 m, followed by image stitching and plot boundary recognition via a mask region-based convolutional neural network (Mask R-CNN). The images were then segmented into plot-scale subgraphs, which were categorized into 3 growth stages. The panicle vision transformer (Panicle-ViT), which integrates a multipath vision transformer and replaces the Mask R-CNN backbone, accurately detects panicles. Additionally, the Res2Net50 architecture classified panicle types with 4 angles of 0°, 15°, 45°, and 90°. The results confirm that the performance of Plot-Seg is comparable to that of manual segmentation. Panicle-ViT outperforms the traditional Mask R-CNN across all the datasets, with the average precision at 50% intersection over union (AP₅₀) improved by 3.5% to 20.5%. The PNpA quantification for the full dataset achieved superior performance, with a coefficient of determination (R ²) of 0.73 and a root mean square error (RMSE) of 28.3, and the overall panicle classification accuracy reached 94.8%. The proposed approach enhances operational efficiency and automates the process from plot cropping to PNpA prediction, which is promising for accelerating the selection of desired traits in rice breeding.

Fig. 1.

Study sites located at the China National Rice Research Institute, Hangzhou, Zhejiang Province, China, 98 d after rice sowing (A) and at the Jiaxing Academy of Agricultural Sciences, Jiaxing, Zhejiang Province, China, 104 d after rice sowing (B).

Fig. 2.

Plot segmentation in agricultural remote sensing images via the Mask R-CNN.

Fig. 3.

Panicle dataset categorization diagram, where FHtM represents the full heading to the maturity phase.

Fig. 4.

Dataset images at different stages of the FHtM. (A) Original image and (D) annotated image at the first third stage of the FHtM. (B) Original image and (E) annotated image at the second third stage. (C) Original image and (F) annotated image at the final third stage.

Fig. 5.

The architecture of the Panicle-ViT (A), the architecture of the backbone (B), multiscale patch embedding (C), and multipath transformer block (D).

Fig. 6.

Panicles with 0° angle (A), 15° angle (B), 45° angle (C), and 90° angle (D).

Fig. 7.

Overall results of Plot-Seg in field 1 in 2022 (A) and details of partial plot segmentation (B).

Fig. 8.

Accuracy of PNpA based on Mask R-CNN (orange points) and Panicle-ViT (blue points). All-stage images (A), the first third stage (B), the second third stage (C), and the final third stage (D). R², RMSE, and rRMSE represent the coefficient of determination, root mean square error, and relative RMSE, respectively.

Fig. 9.

Confusion matrix of the Res2Net50 classifier for the testing dataset.

Fig. 10.

Comparison between ultrahigh-resolution (A to C) and high-resolution images (D to F).

Fig. 11.

Validation of Panicle-ViT predictions for the PNpA on the basis of the 2023-Val dataset.

Fig. 12.

Mapping examples of the predicted PNpA at field 1 in 2022 on day 125 after sowing (A) and at field 2 in 2023 on day 104 after sowing (B), along with the time series analysis for PNpA in 3 plots from field 1 in 2022 (C), and 3 plots from field 2 in 2023, with the left dashed line indicating the day when 10% heading occurred and the right dashed line indicating the day when 80% heading was achieved (D).