Leaf-Movement-Based Growth Prediction Model Using Optical Flow Analysis and Machine Learning in Plant Factory

Abstract

Productivity stabilization is a critical issue facing plant factories. As such, researchers have been investigating growth prediction with the overall goal of improving productivity. The projected area of a plant (PA) is usually used for growth prediction, by which the growth of a plant is estimated by observing the overall approximate movement of the plant. To overcome this problem, this study focused on the time-series movement of plant leaves, using optical flow (OF) analysis to acquire this information for a lettuce. OF analysis is an image processing method that extracts the difference between two consecutive frames caused by the movement of the subject. Experiments were carried out at a commercial large-scale plant factory. By using a microcomputer with a camera module placed above the lettuce seedlings, images of 338 seedlings were taken every 20 min over 9 days (from the 6th to the 15th day after sowing). Then, the features of the leaf movement were extracted from the image by calculating the normal-vector in the OF analysis, and these features were applied to machine learning to predict the fresh weight of the lettuce at harvest time (38 days after sowing). The growth prediction model using the features extracted from the OF analysis was found to perform well with a correlation ratio of 0.743. Furthermore, this study also considered a phenotyping system that was capable of automatically analyzing a plant image, which would allow this growth prediction model to be widely used in commercial plant factories.

Keywords: circadian clock; lettuce; machine learning; optical flow; phenotyping; plant factory.

Figure 1

Multiple plant imaging (MPI) system in commercial plant factory. (A) Production line of commercial plant factory in Osaka Prefecture University. Successive operations, including greening, nursing, cultivation, and cutting, are performed. (B) Nursing room. This room has three levels (left-hand upper figure shows the third level. The lower figure shows the first level). This nursery line (right-hand figure) has a capacity to accommodate 10⁵ seedlings. (C) MPI system for acquiring feature values for each seedling based on time-series photographs captured on the nursing line. (D) Simultaneous measurement of seedlings on nursing panel using MPI system.

Figure 2

Summary of analyses performed, and conceptual drawing of normal-vector analysis. The flowchart on the left shows the preprocessing process for the dataset. The process with the blue background shows the type of data which were focused, and the red background shows the type of analysis method used for preprocessing. The description in the gray box shows the definition used in this study. The image on the bottom right shows the concept of our study. The green circle shows the detection of the panel using HT. The blue vector is the OF vector, and the red circle shows the local-normal vector.

Figure 3

Visualization result of normal-vector analysis. Visualization result for single lettuce at (A) t = 168 h and (B) t = 180 h. Blue vector represents p_ijk Red vector represents n_ijk. White vector represents . The visualization of was performed by at 20 × the original .

Figure 4

Time-series analysis result for single plant using OF. (A,B) Time-series analysis result for single lettuce from t = 0–201 h. Red, green, and blue lines represent , S_k, and θ_k, respectively. The white and black bar at the top of the figure indicates the light and dark conditions. (C–F) shows the image data and visualization results for |n_ijk| as well as the visualization results for sgn(θ_ijk) at t = 24, 36, 120, and 132 h, respectively. The red part of sgn(θ_ijk) represents the positive-direction vector, relative to the center of the seedling. The blue part of sgn(θ_ijk) represents the negative-direction vector, relative to the center of the seedling.

Figure 5

Predicted fresh weight using machine-learning method. (A) Correlation ratio between fresh weight at the harvest and PA at t = 180 h. (B) Correlation ratio between the fresh weight and PA from t = 0–201 h. The dark green and light green indicate the different sets of experiments (dark green: 141 plants, light green: 92 plants). (C) Correlation ratio between observed fresh weight and predicted fresh weight using gradient boost regression (GBR). (D) Feature importance analysis of 115 features using GBR. (E) Correlation ratio between observed fresh weight and predicted fresh weight using support vector regression (SVR).