Estimating leaf area index of maize using UAV-based digital imagery and machine learning methods
Experimental designA 2-year field experiment was conducted at the Modern Agricultural Research and Development Base of Henan Province (113° 35′–114° 15′ E, 34° 53′–35° 11′ N). In order to enhance the diversity of LAI data, a split-plot design with a variety of field management measures and three replications was selected for the experiment (Fig. 1). The size of each experiment plot was 40 m2, the soil texture was predominantly sandy loam and sandy clay loam, as determined by textural analysis of soil samples collected before planting. Maize cultivar Dedan-5 was used in the experiment, which was planted on June 12, 2019, and June 20, 2020, with a row spacing of 42 cm and a planting density of 7 seedlings·m−2. The soil and cultivar in field experiments were representatives of those in the region. The irrigation, pesticide, and herbicide control practices followed local management for maize production.Figure 1The experimental design.Full size imageLAI measurements and UAV-based image acquisitionThe measurements of LAI were conducted at four growth stages including the tasseling stage (TS), flowering stage (FS), grain-filling stage (GS), and milk-ripe stage (MS) of maize in 2019 and 2020, a total of 264 LAI data of maize were collected during the 2-year field trial (Table 1). In order to reduce the impact of plant variability, the random sampling method was used to collect LAI samples. For each plot, three plants were randomly selected to measure the total green leaf area with the non-destructive portable leaf area meter (Laser Area Meter CI-203; CID Inc.). And the average leaf area of selected plants represented the single plant leaf area in each experiment plot. The LAI of each plot wasTable 1 Description of samplings.Full size table$$mathrm{LAI}=mathrm{LA}*mathrm{D}$$
(1)
where (mathrm{LA}) is the leaf area of a single plant in each plot; (mathrm{D}) is the planting density in one square meter.PHANTOM 4 PRO (DJI-Innovations Inc., Shenzhen, China) is a multi-rotor UAV equipped with a 20-megapixel visible-light camera that was employed to capture digital images. Aerial observations were conducted on the same dates as the LAI measurements, which was between 10:30 a.m. and 2:00 p.m. local time when the solar zenith angle was minimal. The UAV was flown automatically based on preset flight parameters and waypoints, with a forward overlap of 80% and a side overlap of 60%. A three-axis gimbal integrated with the inertial navigation system stabilized the camera, the automatic camera mode with fixed ISO (100) and a fixed exposure was used during the flight. Altogether, 4192 images were taken in eight flights from a flight height of 29.36 m above ground, with a spatial resolution of 0.008 m.The measurements of maize LAI were carried out with permission from the Modern Agricultural Research and Development Base of Henan Province. All experiments were carried out in accordance with relevant institutional, national, and international guidelines and legislation.Image pre-processingDJI Terra (version 2.3.3) was used to generate ortho-rectified images based on the structure from motion algorithms and a mosaic blending model. The main procedures are as follows: (1) extract feature points and match features according to the longitude, latitude, elevation, roll angle, pitch angle, and heading angle of each image; (2) build dense 3D point clouds by using dense multi-view stereo matching algorithm; (3) build a 3D polygonal mesh based on the vector relationship between each point in the dense cloud; (4) establish a 3D model with both external image and internal structure by merging the mosaic image into the 3D model; (5) generate digital orthophoto map (DOM).Vegetation indices (VIs) derived from the UAV-based digital imageryDigital imagery records the intensity of visible red (R), green (G), and blue (B) bands in individual pixels24. In order to enhance the vegetation parameters contained in the digital image, fourteen commonly used RGB-based VIs were collected, and their correlation with the LAI of maize at different growth stages was evaluated. Table 2 shows the detailed information of the selected RGB-based VIs.Table 2 RGB-based VIs for LAI estimation.Full size tableCentered on the point where LAI was measured, regions of interests (ROIs) with a size of 100*100 were clipped from the digital image. Python 3.7.3 was used for extracting the R, G, B information of maize and computing the RGB-based VIs from ROIs. In order to reduce the effects of light and shadow, the R, G, B color space of the image was normalized according to the followings:$$mathrm{r}=frac{R}{R+G+B}$$
(2)
$$g=frac{G}{R+G+B}$$
(3)
$$b=frac{B}{R+G+B}$$
(4)
where r, g, and b are the normalized values. R, G, B are the pixel values from the digital images based on each band.Pearson correlation analysisBefore regression analysis, the Pearson correlation analysis was performed to determine the relationship between maize LAI and different RGB-based VIs extracted from the digital image. Pearson correlation coefficient ((mathrm{r})) reflects the degree of linear correlation between two variables, which is between − 1 and 1. The calculation formula of Pearson correlation coefficient was expressed as follows:$$mathrm{r}= frac{sum_{i=1}^{n}left({X}_{i}-overline{X }right)left({Y}_{i}-overline{Y }right)}{sqrt{sum_{i=1}^{n}{left({X}_{i}-overline{X }right)}^{2}}sqrt{sum_{i=1}^{n}{left({Y}_{i}-overline{Y }right)}^{2}}}$$
(5)
where (X), (mathrm{Y}) are variables, (n) is the number of variables.Regression methodsLinear regression (LR)Linear regression is an approach for modelling the relationship between dependent and independent variables. The case of one independent variable is called unary linear regression (ULR), the expressions can be expressed as follows:$$mathrm{y}={beta }_{0}+{beta }_{1}x+varepsilon $$
(6)
where (varepsilon ) is deviation, which satisfies the normal distribution. (x), (mathrm{y}) are variables. ({beta }_{0}), ({beta }_{1}) are the intercept and slope of the regression line, respectively.For more than one independent variable, the regression process is called multiple linear regression (MLR), the expressions can be expressed as:$$mathrm{y}={beta }_{0}+{beta }_{1}{x}_{1}+{beta }_{2}{x}_{2}+dots +{beta }_{n}{x}_{n}$$
(7)
where ({x}_{1}),( {x}_{2}), …, ({x}_{n}), (mathrm{y}) are variables, ({beta }_{0}), ({beta }_{1}), ({beta }_{2}), …, ({beta }_{n}) are coefficients that determined by least square method and gradient descent method38.The RGB-based VIs with the highest Pearson correlation coefficient was used to establish the ULR model, and VIs with a correlation coefficient higher than 0.7 were used to establish the MLR model. In each growth stage, 70% of observation data were randomly selected for establishing models, and the remaining 30% of data were used as the testing dataset to assess the model performance.Back propagation neural networks (BPNN)In this study, a three-layer BPNN model was established for LAI estimation (Fig. 2). RGB-based VIs with a correlation coefficient higher than 0.7 were selected as the input variables. Tan-Sigmoid activation function was used in the hidden layer, and the Levenberg–Marquardt algorithm was selected as the training function. The maximum epoch of BPNN training was set to 1000, the learning rate was set to 0.005, and the MSE was set to 0.001. The observation data set was split into the training set and the testing dataset with a ratio of 7:3. The training dataset was used to fit the weights and bias of the BPNN model, the testing dataset was used to evaluate the model performance. Before training, data normalization was conducted for the input and output variables, and the denormalization was required to convent the output variable back into the original units after training.Figure 2Three-layer BPNN model.Full size imageRandom forest (RF)RF is a non-parametric ensemble ML method that operates by constructing a multitude of decision trees at training time and outputting the average prediction of the individual trees (Fig. 3). The bootstrapping approach was used to collect different sub-training data from the input training dataset to construct individual decision trees.Figure 3Random forest model.Full size imageThe construction process of RF regression model is as follows:
(1)
The value of (mathrm{n}_mathrm{estimators}) was tested from 50 to 1000 in increments of 50, and the value of 500 was finally selected according to higher R2 and lower RMSE.
(2)
At each node per tree, (mathrm{m}_mathrm{try}) RGB-based VIs was randomly selected from all 14 vegetation indices, and the best split was chosen according the lowest Gini Index. (mathrm{m}_mathrm{try}) was tested from 3 to 10, and the final value was 6.
(3)
The other parameters in the RF model were kept as default values according to the (mathrm{RandomForestRegressor}) function in (mathrm{Scikit}-mathrm{learn library}).
(4)
For each tree, the data splitting process in each internal node was repeated from the root node until a pre-defined stop condition was reached.
(5)
Similar with LR and BPNN model, the RGB-based VIs with a correlation coefficient higher than 0.7 were selected as the input variables, and the output variable is LAI.
Data analysis and performance evaluationThe repeated random sampling validation method was used to evaluate the generalization performance of different models. The training and testing dataset were randomly split 500 times. For each split, the LR, BPNN, and RF models were fitted to the training dataset, and the estimation accuracy was evaluated using the testing dataset. The coefficient of determination (R2), root mean square error (RMSE), and Akaike information criterion (AIC) of the training dataset were used for the assessment of models39, and the estimation accuracy was evaluated by R2 and RMSE of the testing dataset. Mathematically, a higher R2 corresponds to a smaller RMSE, and thus represents better model performance. The procedures of LAI inversion using UAV-based digital imagery and ML methods were shown in Fig. 4.Figure 4Flowchart of LAI inversion using UAV-based remote sensing and ML methods.Full size imageThe construction and evaluation of models was performed using Python 3.7.3 in Windows 10 operating system with Intel Core i7-9700 processor, 3.00 GHz CPU, and 32 GB RAM. The processing software is Spyder. The statistical analysis and figure plotting were performed in R × 64 4.0.3. More
