Measuring crops in 3D: using geometry for plant phenotyping

Abstract

Using 3D sensing for plant phenotyping has risen within the last years. This review provides an overview on 3D traits for the demands of plant phenotyping considering different measuring techniques, derived traits and use-cases of biological applications. A comparison between a high resolution 3D measuring device and an established measuring tool, the leaf meter, is shown to categorize the possible measurement accuracy. Furthermore, different measuring techniques such as laser triangulation, structure from motion, time-of-flight, terrestrial laser scanning or structured light approaches enable the assessment of plant traits such as leaf width and length, plant size, volume and development on plant and organ level. The introduced traits were shown with respect to the measured plant types, the used measuring technique and the link to their biological use case. These were trait and growth analysis for measurements over time as well as more complex investigation on water budget, drought responses and QTL (quantitative trait loci) analysis. The used processing pipelines were generalized in a 3D point cloud processing workflow showing the single processing steps to derive plant parameters on plant level, on organ level using machine learning or over time using time series measurements. Finally the next step in plant sensing, the fusion of different sensor types namely 3D and spectral measurements is introduced by an example on sugar beet. This multi-dimensional plant model is the key to model the influence of geometry on radiometric measurements and to correct it. This publication depicts the state of the art for 3D measuring of plant traits as they were used in plant phenotyping regarding how the data is acquired, how this data is processed and what kind of traits is measured at the single plant, the miniplot, the experimental field and the open field scale. Future research will focus on highly resolved point clouds on the experimental and field scale as well as on the automated trait extraction of organ traits to track organ development at these scales.

Keywords: 3D plant scanning; Parameterization; Plant model; Plant traits; Point cloud.

Fig. 1

The hierarchy of the introduced 3D measuring techniques which are most relevant for plant phenotyping (highlighted in color) is presented. Laser triangulation, structured light approaches, time of flight sensing, structure from motion and light field imaging are shown in their technical connection (a). The two most important techniques laser triangulation (b) and structure from motion (c) are introduced in detail to show the procedure of point measuring

Fig. 2

Laserscanning accuracy—reference experiment using a photogrammetric method as reference to evaluate the accuracy of the Laserscanning device and the Leafmeter as a device for measuring leaf area [35, 36]. Both methods show a high correlation compared to the reference method (a). The comparison between the laser scanner using different point resolutions and the introduced reference method is visualized in addition (b). The transparent color in both plots indicates the confidence intervals (95%). The black line describes the bisecting line of the angle as the line of highest correlation

Fig. 3

Traits that can be extracted from a 3D point cloud of a young barley plant. From the XYZ point cloud (a) non-complex parameters like plant height (b), plant width (c), the convex hull (d) and the projected leaf area (e) can be extracted. Furthermore the leaf area density (f) can be derived. The number of leaves (g) and the respective leaf length (h) can be measured after identifying the individual plant organs. For each point the inclination and its height can be calculated resulting in a inclination (i) or height map (j)

Fig. 4

A common 3D processing pipeline including the use of a region of interest and outlier handling to extract non-complex parameters as height, width and volume (1). The use of routines like machine learning/deep learning enables the identification and parameterization of plant organ parameters (2). Using multiple recordings over time monitoring of development and differentiation between growth and movement is possible (3)

Fig. 5

A combination of 3D point cloud and hyperspectral image data is possible by calibrating the sensor setup including the 3D imaging sensor and the hyperspectral camera. The top (a) and side (b) view of a combined point cloud is shown for combination of 3D- and VISNIR-data (911 nm, a) as well as for 3D and SWIR data(1509 nm, b). The VISNIR and the SWIR spectrum can be investigated at the same point in the 3D point cloud (c)