Visible-NIR hyperspectral classification of grass based on multivariate smooth mapping and extreme active learning approach
Study areaGrassland herbage samples are from Shaerqin base, institute of grassland research of CAAS (Chinese Academy of Agricultural Sciences). We obtained the permission of the institution to take HSI of the grassland sample. Our work did not cause damage to grassland. Researcher Weihong Yan of the institute provided us with relevant information about grassland. The land use type in the study area is mainly grassland, which is composed of forage species, most of which are representative species of typical grassland. We take this area as an example to conduct research on grass classification. By enriching the relevant recognition technology, it can also be used as a reference for the pastures of other grasslands. The grass species Grass1 for the experiment is shown in Table 1. The official introduction of plant materials is detailed in the flora of China15.Table 1 Samples information for Grass1 dataset.Full size tableThe field hyperspectral platformWe assemble a system for collecting HSI in the field: HyperSpec©PTU-D48E HSI instrument, high-precision scanning PTZ, tripod, data analysis software Hyperspec, etc. The light source is natural light. The imaging instrument is in line scanning mode. Table 2 shows the technical parameters.Table 2 Technical parameters of hyperspectral instrument.Full size tableData collectionIn July 2021, the data was collected during the lush grass growth period. Collect data from 11:00 a.m. to 2:00 p.m. every day. At this time, it is sunny, cloudless and the wind force does not exceed level 2. So as to ensure the consistency of the acquisition time line and avoid the influence of different degrees of light on the reflectivity as far as possible. The measuring points are arranged facing the sun and the opposite direction of the shadow. We collect data from different angles of the grassland, which is based on the growth of various types of forages, and selects relatively concentrated places within the study area. Each shot is a single category of grass. The image resolution is 1166 × 1004 pixels (Fig. 1). The imaging spectrometer is fixed with scanning head when shooting. Data acquisition and transmission are executed on Hyperspec software. Then save it as a BIL file. The ENVI5.3 software was used to extract the forage spectrum to establish the dataset Grass1. Well balanced regions with a clear image, uniform spectral distribution are selected for further segmentation. The average value of spectral reflectance of grass pixels was taken as the reflectance spectrum of a single type of grass.Figure 1True color map of grass samples.Full size imageMethodologyIn Fig. 2, we present the framework of visible-NIR hyperspectral classification of grass based on multivariate smooth mapping and extreme active learning (MSM–EAL). Specifically, we first introduce the proposed MSM algorithm for global enhanced spectral reconstruction, which utilizes smooth manifold projection technology to alleviate the problems of difficult feature selection and redundant data. Then, the EAL framework is proposed to address the matter of hyperspectral labeled samples and spectral classification. In the following, each step of this method will be presented in detail.Figure 2Proposed MSM–EAL framework for grass HSI classification.Full size imageThe proposed MSM algorithmIn the process of field HSI acquisition, on the one hand, the surface distribution of grass is uneven and the plant height is different, causing certain scattering effect and coverage spectrum change. On the other hand, HSI is easy to be disturbed by external natural factors such as light, wind and shadow, resulting in a certain degree of distortion. Multiplicative scatter correction (MSC) is a scattering correction effect, which helps to eliminate the scattering effect caused by the above reasons and enhance the spectral variability. The moving window smooth spectral matrix (Nirmaf) belongs to the smooth effect, which improve the signal-to-noise ratio of the spectrum and reduce the influence of random noise16,17. Preprocessing methods are different and related to each other. We design an enhanced preprocessing multivariate smooth (MS) method that fusing MSC and smooth Nirmaf to target grass spectral signal features. In the follow-up, a model will be established to verify the validity of MS.Most of the high-dimensional spatial data have the characteristics of being embedded in a manifold body, so the manifold learning isometric feature mapping (Isomap) based on spectral theory is adopted. Isomap preserves the global geometric features of the initial data and extracts features by reconstructing the underlying smooth manifold of HSI. It is nonlinear dimensionality reduction based on linear and multidimensional scaling transformation18. Isomap has been applied in image and HSI classification19,20, but there is no report on visible-NIR hyperspectral classification of grass.In view of the above, we proposed the multivariate smooth mapping (MSM) spectral reconstruction algorithm, which can be represented as follows:$$ MSM_{z} { } = { }frac{{left( {P_{j} – b_{j} } right)left( {2n + 1} right) + n_{j} cdot mathop sum nolimits_{j = – n}^{n} C_{j} P_{k + j} }}{{n_{j} left( {2n + 1} right)}} + V_{Z} F_{Z}^{frac{1}{2}} { } $$
(1)
where Pj, bj, and Cj represent the raw reflectance value of spectrum j, baseline shift amount, and weight factor, respectively, k and nj represent the polynomial degree and offset, respectively. MSMz is the feature cube reconstructed to Z dimension from the spectrum calculated by 2n + 1 moving window width, V eigenvector matrix and F eigenvalue matrix.In Isomap equidistant mapping, the shortest path of edge Pi Pj needs to be solved, and the representation matrix is:$$ D_{G} = [d_{G}^{2} (P_{i} ,P_{j} )]_{i,j = 1}^{n} $$
(2)
where d (Pi, Pj) is the weight of the edge Pi Pj calculated from the neighborhood graph G and its side Pi Pj.The proposed EAL frameworkLabeling hyperspectral samples is expensive in terms of time and cost, at the same time, the lower spatial resolution and more bands increase the difficulty of labeling. Active learning (AL) provides an efficient labeling strategy, which only needs to label a relatively small number of samples to learn a more accurate model21. The pool-based AL selects the most informative samples according to the query strategy for limited labeling through iteration, so as to facilitate model improvement. Commonly used query strategies are uncertainty criteria, such as least confidence22, the bayesian active learning disagreement (BALD), the entropy sampling23, etc.Due to there is still an over-fitting problem, different strategies such as hybrid prediction and regularization need to be used for non-recursive datasets24. The research25 proposed that extreme gradient boosting algorithm (XGBoost) based on gradient boosting. As a classification method, XGBoost has been successfully applied in Kaggle competition and other fields. Its most important feature for visible-NIR hyperspectral classification is that can easily and directly classify according to features, and the physical interpretation of features can help understand the electronic nature behind spectral classification. XGBoost is a machine learning algorithm based tree structure that integrates multiple weak classifiers to achieve flexible and high-precision classification. It is an upgraded version of gradient boosting decision tree. The optimization process of XGBoost entailed: (1) Expanding the objective function to the second order, and finds a new objective function for the new base model to improve the calculation accuracy. (2) L2 regularization term is added to the loss function to prevent over-fitting. (3) Using blocks storage structure realize automatic parallel computing26,27. The algorithm steps are as follows:The objective function:$$ Lleft( Phi right) = mathop sum limits_{i} lleft( {y^{i} ,widehat{{y^{i} }}} right) + mathop sum limits_{k} Omega left( {f_{k} } right) $$
(3)
In formula (3), the first and second terms are the loss function term and the regularization term, respectively. Where,$$ Omega left( {f_{k} } right) =upgamma {text{T}} + frac{1}{2}lambda left| w right|^{2} $$
(4)
γ and λ are regularization parameters which are used to adjust complexity of the tree.Next, second derivative Taylor expansion of the objective function. Where (g_{i}) and (h_{i}) are the first derivative and second derivative, respectively.$$ L^{left( t right)} = mathop sum limits_{i = 1}^{n} lleft( {y_{i} ,widehat{{y_{i}^{t – 1} }} + f_{t} left( {x_{i} } right)} right) + Omega left( {f_{t} } right) $$
(5)
$$ g_{i} = partial_{{hat{y}_{i} (t – 1)}} lleft( {y_{i} ,widehat{{y_{i}^{t – 1} }}} right) $$
(6)
$$ h_{i} = partial_{{widehat{{y_{i} }}(t – 1)}}^{2} lleft( {y_{i} ,widehat{{y_{i}^{t – 1} }}} right) $$
(7)
$$ {text{L}}^{left( t right)} approx mathop sum limits_{i = 1}^{n} left[ {lleft( {y_{i} ,widehat{{y_{i}^{t – 1} }}} right) + g_{i} f_{i} left( {x_{i} } right) + frac{1}{2}h_{i} f_{t}^{2} left( {x_{i} } right)} right] + Omega left( {f_{t} } right) $$
(8)
Final objective function:$$ {hat{text{L}}}^{ i} left( q right) = – frac{1}{2}mathop sum limits_{j = 1}^{T} frac{{(mathop sum nolimits_{{i in I_{j} }} g_{i} )^{2} }}{{mathop sum nolimits_{{i in I_{j} }} h_{i} + lambda }} + gamma T $$
(9)
Equation (9) can be used as the fraction of tree cotyledons, and the tree structure is directly proportional to the fraction. If the result after splitting is less than the maximum value of the given parameter, the cotyledon depth stops growing24,28.AL solves the problems of limited number and high cost of grass hyperspectral labeling samples. The default model of traditional AL is logistic regression, which is mostly studied on the ideal public dataset. However, the actual data has more uncertain noise, which still poses a certain challenge to AL. Consequently, we propose the extreme active learning (EAL) framework to minimize the classification cost of visible-NIR hyperspectral. The framework replaces the logistic regression model with XGBoost. Taking advantage of AL, XGBoost can improve performance with less training marker samples. By jointing of XGBoost and AL, EAL provides significantly better results than AL in field Grassl dataset recognition. Additionally, based on the characteristics of XGBoost, EAL more intuitively enhances the physical essence behind spectral classification than AL. Algorithm 1 summarizes the workflow of EAL framework.Random forest (RF) and decision tree (DT) were used to compare with EAL. RF and DT are frequently used in the field of grassland remote sensing9,29. Furthermore, RF, DT and XGBoost have the same point is that are learning algorithms based on tree structure. DT determines the direction by judging the conditions of the decision node12. RF is an integrated learning of multiple decision trees30. More
