Hyperspectral Satellite Image Classification Based on Feature Pyramid Networks With 3D Convolution

Abstract

Abstract:Recent advances in convolution neural network (CNN) have fostered the progress in object recognition and semantic segmentation, which in turn has improved the performance of hyperspectral image (HSI) classification. Nevertheless, the difficulty of high dimensional feature extraction and the shortage of small training samples seriously hinder the future development of HSI classification. In this paper, we propose a novel algorithm for HSI classification based on three-dimensional (3D) CNN and a feature pyramid network (FPN), called 3D-FPN. The framework contains a principle component analysis, a feature extraction structure and a logistic regression. Specifically, the FPN built with 3D convolutions not only retains the advantages of 3D convolution to fully extract the spectral-spatial feature maps, but also concentrates on more detailed information and performs multi-scale feature fusion. This method avoids the excessive complexity of the model and is suitable for small sample hyperspectral classification with varying categories and spatial resolutions. In order to test the performance of our proposed 3D-FPN method, rigorous experimental analysis was performed on three public hyperspectral data sets and hyperspectral data of GF-5 satellite. Quantitative and qualitative results indicated that our proposed method attained the best performance among other current state-of-the-art end-to-end deep learning-based methods.

Key words:hyperspectral image (HSI), deep learning, feature pyramid network (FPN), spectral-spatial feature extraction

摘要：近年来，卷积神经网络的最新进展促进了物体识别和语义分割领域的进步，进而提高了高光谱图像分类的性能。然而，高维度的特征提取和训练样本的不足严重阻碍了高光谱图像分类的未来发展。在本文中，我们提出了一种基于三维卷积和特征金字塔网络的新算法3D-FPN，用于高光谱图像分类。该框架包含主成分分析、特征提取结构和逻辑回归三部分。首先利用三维卷积构建的特征金字塔结构不仅充分保留了三维卷积提取光谱-空间特征图的优势，而且更关注细节信息，最后进行多尺度的特征融合。该方法避免了模型的过度复杂，能适用于具有不同类别和不同空间分辨率的小样本高光谱分类。为了测试我们提出的3D-FPN的性能，我们对三个公共高光谱数据集和高分五号卫星的高光谱数据进行了严格的实验分析。定量和定性结果表明，我们提出的方法在与其他最先进的端到端深度学习方法比较中展现了最佳性能。

关键词:高光谱图像，深度学习，特征金字塔网络，光谱-空间特征提取

CLC Number:

TP391

CHEN Cheng, PENG Pan, TAO Wei, ZHAO Hui. Hyperspectral Satellite Image Classification Based on Feature Pyramid Networks With 3D Convolution[J]. J Shanghai Jiaotong Univ Sci, 2025, 30(6): 1073-1084.

References

[1] HELDENS W, HEIDEN U, ESCH T, et al. Can the future EnMAP mission contribute to urban applications? A literature survey [J].Remote Sensing, 2011,3(9): 1817-1846.

[2] AHMAD M, KHAN A M. Hyperspectral band selection using unsupervised non-linear deep auto encoder to train external classifiers [DB/OL]. (2017-05-19). https://arxiv.org/abs/1705.06920

[3] EL-SHARKAWY Y H, ELBASUNEY S. Hyperspectral imaging: Anew prospective for remote recognition of explosive materials [J].Remote Sensing Applications:Society and Environment, 2019,13: 31-38.

[4] DUMKE I, NORNES S M, PURSER A, et al. First hyperspectral imaging survey of the deep seafloor: High-resolution mapping of manganese nodules [J].Remote Sensing of Environment, 2018,209: 19-30.

[5] DONOHO D L. High-dimensional data analysis: The curses and blessings of dimensionality [EB/OL]. (2000-08-08).https://sunju.org/teach/TMML-Fall-2021/Donoho-2000.pdf

[6] MELGANI F, BRUZZONE L. Classification of hyperspectral remote sensing images with support vector machines [J].IEEE Transactions onGeoscience andRemoteSensing, 2004,42(8): 1778-1790.

[7] LI J, BIOUCAS-DIAS J M, PLAZA A. Semisupervised hyperspectral image classification using soft sparse multinomial logistic regression [J].IEEE Geoscience and Remote Sensing Letters, 2013,10(2): 318-322.

[8] ZHANG C S, ZHENG Y W. Hyperspectral remote sensing image classification based on combined SVM and LDA [J].Proceedings of SPIE, 2014,9263: 92632P.

[9] FALCO N, BENEDIKTSSON J A, BRUZZONE L. Spectral and spatial classification of hyperspectral images based on ICA and reduced morphological attribute profiles [J].IEEE Transactions on Geoscience and Remote Sensing, 2015,53(11): 6223-6240.

[10] JIA S, SHEN L L, LI Q Q. Gabor feature-based collaborative representation for hyperspectral imagery classification [J].IEEE Transactions on Geoscience and Remote Sensing, 2015,53(2): 1118-1129.

[11] TUIA D, VOLPI M, DALLA MURA M, et al. Automatic feature learning for spatio-spectral image classification with sparse SVM [J].IEEE Transactions on Geoscience and Remote Sensing, 2014,52(10): 6062-6074.

[12] HU W, HUANG Y Y, WEI L, et al. Deep convolutional neural networks for hyperspectral image classification [J].Journal of Sensors, 2015,2015: 1-12.

[13] LI W, WU G D, ZHANG F, et al. Hyperspectral image classification using deep pixel-pair features [J].IEEE Transactions on Geoscience and Remote Sensing, 2017,55(2): 844-853.

[14] DING C, LI Y, XIA Y, et al. Convolutional neural networks based hyperspectral image classification method with adaptive kernels [J].Remote Sensing, 2017,9(6): 618.

[15] MA X R, FU A Y, WANG J, et al. Hyperspectral image classification based on deep deconvolution network with skip architecture [J].IEEE Transactions on Geoscience and Remote Sensing, 2018,56(8): 4781-4791.

[16] ZHONG Z L, LI J, LUO Z M, et al. Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework [J].IEEE Transactions on Geoscience and Remote Sensing, 2018,56(2): 847-858.

[17] ROY S K, KRISHNA G, DUBEY S R, et al. HybridSN: Exploring 3-D–2-D CNN feature hierarchy for hyperspectral image classification [J].IEEE Geoscience and Remote Sensing Letters, 2020,17(2): 277-281.

[18] HANG R L, LI Z, LIU Q S, et al. Hyperspectral image classification with attention-aided CNNs [J].IEEE Transactions on Geoscience and Remote Sensing, 2021,59(3): 2281-2293.

[19] FANG B, LI Y, ZHANG H K, et al. Hyperspectral images classification based on dense convolutional networks with spectral-wise attention mechanism [J].Remote Sensing, 2019,11(2): 159.

[20] RODARMEL C, SHAN J. Principal component analysis for hyperspectral image classification [J].Surveying and Land InformationSystems, 2002,62(2): 115-122.

[21] YUE J, ZHAO W Z, MAO S J, et al. Spectral-spatial classification of hyperspectral images using deep convolutional neural networks [J].Remote Sensing Letters, 2015,6(6): 468-477.

[22] JI S W, XU W, YANG M, et al. 3D convolutional neural networks for human action recognition [J].IEEETransactions on Pattern Analysis and Machine Intelligence, 2013,35(1): 221-231.

[23] LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 936-944.

[24] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks [J].Communications of the ACM, 2017,60(6): 84-90.

[25] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: A simple way to prevent neural networks from overfitting [J].Journal of Machine Learning Research, 2014,15(1): 1929-1958.

[26] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition [DB/OL]. (2014-09-04).https://arxiv.org/abs/1409.1556

[27] BEN HAMIDA A, BENOIT A, LAMBERT P, et al. 3-D deep learning approach for remote sensing image classification [J].IEEE Transactions onGeoscience andRemoteSensing, 2018,56(8): 4420-4434.

[28] LI Y, ZHANG H K, SHEN Q A. Spectral–spatial classification of hyperspectral imagery with 3D convolutional neural network [J].Remote Sensing, 2017,9(1): 67.

[29] LEE H, KWON H. Contextual deep CNN based hyperspectral classification [C]//2016 IEEEInternational Geoscience and Remote Sensing Symposium. Beijing: IEEE, 2016: 3322-3325.

[30] HE M Y, LI B, CHEN H H. Multi-scale 3D deep convolutional neural network for hyperspectral image classification [C]//2017 IEEE International Conference on Image Processing. Beijing: IEEE, 2017: 3904-3908.