Deep Learning Applications for Hyperspectral Imaging: A Systematic Review

Kemal Polat

1. Master Degree Student at School of Natural Sciences, Department of Electrical and Electronics Engineering Bolu Abant Izzet Baysal University, 14280, Bolu, Turkey, Email: [email protected]
2. Department of Electrical and Electronics Engineering, Faculty of Engineering, Bolu Abant Izzet Baysal University, 14280, Bolu, Turkey, Email: [email protected]

Since the acquisition of digital images, scientific studies on these images have been making significant progress. The sizes and quality of the images obtained have increased greatly from past to present. However, when the information contained in these images remains on the visible band (RGB band), the results that can be obtained are limited. For this reason, the need to acquire images with more broadband information has emerged. Hyperspectral Imaging (HSI) method has been developed to meet this need. A hyperspectral image consists of reflections in hundreds of different bands of the electromagnetic spectrum. Each object exhibits a unique reflection characteristic. Due to this characteristic, objects can be separated from each other using hyperspectral imaging.

Hyperspectral cameras are used to obtain this image. The information it contains is much more than an RGB image, so deeper results can be achieved than the human eye can see.

In this respect, it has great importance. Artificial intelligence technologies are used extensively in image processing as well as in many other fields. As a result, classification studies are carried out on hyperspectral images with machine learning methods. Machine learning methods can be considered as the most general terms of supervised machine learning, unsupervised machine learning, and reinforced machine learning. Supervised machine learning methods mainly: Support Vector Machines(SVM), k-Nearest Neighborhood(k-NN), Decision Trees, Random Forest, Linear Regression and Neural Networks(NN). Neural networks are also used as an unsupervised learning method. However, Deep Learning, a specialized method of artificial neural networks, is highly preferred due to its unique structure. Artificial intelligence methods have been widely used in recent years, especially for the classification of hyperspectral images containing complex information. Considering the studies, it is seen that especially deep learning is used intensively. At this point, studies have revealed different types of models. The number of models and their successes are increasing day by day.

Hyperspectral Imaging, HSI, HSI classification, Hyperspectral Image Classification, Deep Learning, HSI with Deep Learning

Akin Ozdemir, Kemal Polat (2020). Deep Learning Applications for Hyperspectral Imaging: A Systematic Review. Journal of the Institute of Electronics and Computer, 2, 39-56. https://doi.org/10.33969/JIEC.2020.21004.

[1] Luo, Yanan & Zou, Jie & Yao, Chengfei & Zhao, Xiaosong & Li, Tao & Bai, Gang. (2018). HSI-CNN: A Novel Convolution Neural Network for Hyperspectral Image. 464-469. 10.1109/ICALIP.2018.8455251.
[2] Mughees, Atif & Tao, Linmi. (2018). Multi Deep Belief Network Based spectral-spatial Classification of Hyperspectral Image. Tsinghua Science & Technology. 24. 10.26599/TST.2018.9010043.
[3] Liang, M.; Jiao, L.; Meng, Z. A Superpixel-Based Relational Auto-Encoder for Feature Extraction of Hyperspectral Images. Remote Sens. 2019, 11, 2454.
[4] Wang, L.; Peng, J.; Sun, W. Spatial–Spectral Squeeze-and-Excitation Residual Network for Hyperspectral Image Classification. Remote Sens. 2019, 11, 884.
[5] Chen, Yushi & Lin, Zhouhan & Zhao, Xing & Wang, Gang & Gu, Yanfeng. (2014). Deep Learning-Based Classification of Hyperspectral Data. Selected Topics in Applied Earth Observations and Remote Sensing, IEEE Journal of. 7. 2094-2107. 10.1109/JSTARS.2014.2329330.
[6] Kramer, M.A. Nonlinear principal component analysis using autoassociative neural networks. AIChE J. 1991, 37, 233–243.
[7] Hu W, Huang Y, Wei L, et al. Deep Convolutional Neural Networks for Hyperspectral Image Classification[J]. Journal of Sensors, 2015, 2015(2):1-12.
[8] Zhong, Z.; Li, J.; Luo, Z.; Chapman, M. Spectral-spatial residual network for hyperspectral image classification: A 3-D deep learning framework. IEEE Trans. Geosci. Remote. Sens. 2018, 56, 847–858.
[9] Ioffe, S.; Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv 2015, arXiv:1502.03167.
[10] Li, Jiaojiao & Xi, Bobo & Li, Yunsong & Du, Qian & Wang, Keyan. (2018). Hyperspectral Classification Based on Texture Feature Enhancement and Deep Belief Networks. Remote Sensing. 10. 396. 10.3390/rs10030396.
[11] Roy, Swalpa & Krishna, Gopal & Dubey, Shiv Ram & Chaudhuri, Bidyut. (2019). HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image Classification.
[12] L. Ma, Y. Liu, X. Zhang, Y. Ye, G. Yin, B.A. JohnsonDeep learning in remote sensing applications: ameta-analysis and review ISPRS J. Photogramm. Remote Sens., 152 (2019), pp. 166-177
[13] Bedini, E. (2017). The use of hyperspectral remote sensing for mineral exploration: a review.
[14] VetriDeepika, K., Johnson, A., & LafrulHudha, M. (2016). Enhancement of Precision Agriculture Using Hyperspectral Imaging.
[15] Elmasry, Gamal & Kamruzzaman, Mohammed & Sun, Da-Wen & Allen, Paul. (2012). Principles and Applications of Hyperspectral Imaging in Quality Evaluation of Agro-Food Products: A Review. Critical reviews in food science and nutrition. 52. 999-1023. 10.1080/10408398.2010.543495.
[16] Kulcke, A., Holmer, A., Wahl, P., Siemers, F., Wild, T., & Daeschlein, G. (2018). A compact hyperspectral camera for measurement of perfusion parameters in medicine. Biomedical Engineering / Biomedizinische Technik, 63, 519 - 527.
[17] Metkar, Monali & Kamalapur, Snehal. (2018). Hyperspectral Imaging. 10.13140/RG.2.2.31364.42887.
[18] Mehta, N.S., Shaik, S., Devireddy, R.V., & Gartia, M.R. (2018). Single-Cell Analysis Using Hyperspectral Imaging Modalities. Journal of biomechanical engineering, 140 2.
[19] Guolan Lu, Baowei Fei (2014), "Medical hyperspectral imaging: a review," J. Biomed. Opt.19(1)010901 https://doi.org/10.1117/1.JBO.19.1.010901
[20] Dale, Laura & Thewis, André & Boudry, Christelle & Rotar, Ioan & Dardenne, Pierre & Baeten, V. & Fernández Pierna, Juan. (2013). Hyperspectral Imaging Applications in Agriculture and Agro-Food Product Quality and Safety Control: A Review. Applied Spectroscopy Reviews. 48. 142. 10.1080/05704928.2012.705800.
[21] Li, Shutao & Song, Weiwei & Fang, Leyuan & Chen, Yushi & Ghamisi, Pedram & Benediktsson, Jon. (2019). Deep Learning for Hyperspectral Image Classification: An Overview. IEEE Transactions on Geoscience and Remote Sensing. PP. 1-20. 10.1109/TGRS.2019.2907932.
[22] P. Ghamisi et al., “New frontiers in spectral-spatial hyperspectral image classification: The latest advances based on mathematical morphology,Markov random fields, segmentation, sparse representation, and deep learning,” IEEE Geosci. Remote Sens. Mag., vol. 6, no. 3, pp. 10–43,Sep. 2018.
[23] L. He, J. Li, C. Liu, and S. Li, “Recent advances on spectral–spatial hyperspectral image classification: An overview and new guidelines,” IEEE Trans. Geosci. Remote Sens., vol. 56, no. 3, pp. 1579–1597,Mar. 2018.
[24] C. M. Bachmann, T. L. Ainsworth, and R. A. Fusina, “Exploiting manifold geometry in hyperspectral imagery,” IEEE Transactions on Geoscience and Remote Sensing, vol. 43, no. 3, pp. 441–454, 2005.
[25] S. M. Davis, D. A. Landgrebe, T. L. Phillips, P. H. Swain, R. M. Hoffer,J. C. Lindenlaub, and L. F. Silva, Remote sensing: the quantitative approach. New York: McGraw-Hill, 1978
[26] Ranjan, Sameer & Nayak, Deepak & Kumar, Santosh & Dash, Ratnakar & Majhi, Banshidhar. (2017). Hyperspectral image classification: A k-means clustering based approach. 1-7. 10.1109/ICACCS.2017.8014707.
[27] Audebert, Nicolas & Saux, Bertrand & Lefèvre, Sébastien. (2019). Deep Learning for Classification of Hyperspectral Data: A Comparative Review. IEEE Geoscience and Remote Sensing Magazine. 7. 10.1109/MGRS.2019.2912563.
[28] Mateen, Muhammad & Wen, Junhao & Nasrullah, Dr & Azeem Akbar, Muhammad. (2018). The Role of Hyperspectral Imaging: A Literature Review. International Journal of Advanced Computer Science and Applications. 9. 51.
[29] Hu, F.; Xia, G.S.; Hu, J.; Zhang, L. Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery. Remote Sens. 2015, 7, 14680–14707.
[30] He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778.
[31] Kemal Polat, Kaan Onur Koc (2020). Detection of Skin Diseases from Dermoscopy Image Using the combination of Convolutional Neural Network and One-versus-All. Journal of Artificial Intelligence and Systems, 2, 80–97. ttps://doi.org/10.33969/AIS.2020.21006
[32] Murat Arican, Kemal Polat (2020). Binary particle swarm optimization (BPSO) based channel selection in the EEG signals and its application to speller systems. Journal of Artificial Intelligence and Systems, 2, 27–37. https://doi.org/10.33969/AIS.2020.21003
[33] Imani, Maryam & Ghassemian, Hassan. (2020). An overview on Spectral and Spatial Information Fusion for Hyperspectral Image Classification: Current Trends and Challenges. Information Fusion. 59. 10.1016/j.inffus.2020.01.007.
[34] Femenias Llaneras, Antoni & Gatius, Ferran & Ramos, Antonio & Marín, Sonia. (2019). Use of hyperspectral imaging as a tool for Fusarium and deoxynivalenol risk management in cereals: A review. Food Control. 108. 106819. 10.1016/j.foodcont.2019.106819.
[35] Feilong, Cao & Guo, Wenhui. (2019). Cascaded dual-scale crossover network for hyperspectral image classification. Knowledge-Based Systems. 105122. 10.1016/j.knosys.2019.105122.
[36] Babellahi, Farahmand & Paliwal, Jitendra & Erkinbaev, Chyngyz & Amodio, Maria & Chaudhry, Mudassir & Colelli, Giancarlo. (2019). EARLY DETECTION OF CHILLING INJURY IN GREEN BELL PEPPERS BY HYPERSPECTRAL IMAGING AND CHEMOMETRICS. 10.1016/j.postharvbio.2019.111100.
[37] Zhang, Mengyun & Li, Changying. (2018). Fully convolutional networks for blueberry bruising and calyx segmentation using hyperspectral transmittance imaging. 10.13031/aim.201801489.
[38] Bei Fang, Ying Li, Haokui Zhang, Jonathan Cheung-Wai Chan(2020). Collaborative learning of lightweight convolutional neural network and deep clustering for hyperspectral image semi-supervised classification with limited training samples. Biosystems Engineering 192(2020),159-175. https://doi.org/10.1016/j.biosystemseng.2020.01.018
[39] Feilong, Cao & Guo, Wenhui. (2019). Deep Hybrid Dilated Residual Networks for Hyperspectral Image Classification. Neurocomputing. 10.1016/j.neucom.2019.11.092.
[40] Dhakal, Ashwin & Shakya, Subarna. (2018). Image-Based Plant Disease Detection with Deep Learning. International Journal of Computer Trends and Technology. 61. 2231-2803. 10.14445/22312803/IJCTT-V61P105.
[41] Qureshi, Rizwan & Khurshid, Khurram & Yan, Hong. (2019). Hyperspectral Document Image Processing: Applications, Challenges and Future Prospects. Pattern Recognition. 90. 12-22. 10.1016/j.patcog.2019.01.026.
[42] Liu, Yuwei & Pu, Hongbin & Sun, Da-Wen. (2017). Hyperspectral imaging technique for evaluating food quality and safety during various processes: A review of recent applications. Trends in Food Science & Technology. 69. 10.1016/j.tifs.2017.08.013.
[43] Calvini, Rosalba & Michelini, S. & Pizzamiglio, V. & Foca, Giorgia & Ulrici, Alessandro. (2020). Exploring the potential of NIR hyperspectral imaging for automated quantification of rind amount in grated Parmigiano Reggiano cheese. Food Control. 112. 107111. 10.1016/j.foodcont.2020.107111.
[44] Zhang, Liu & Rao, Zhenhong & Ji, Haiyan. (2020). Hyperspectral imaging technology combined with multivariate data analysis to identify heat-damaged rice seeds. Spectroscopy Letters. 1-15. 10.1080/00387010.2020.1726402.
[45] Li, Xiaoli & Chen, Kai & He, Yong. (2020). In situ and non-destructive detection of the lipid concentration of Scenedesmus obliquus using hyperspectral imaging technique. Algal Research. 45. 10.1016/j.algal.2019.101680.
[46] Jia, Beibei & Wang, Wei & Ni, Xinzhi & Lawrence, Kurt & Zhuang, Hong & Yoon, Seung-Chul & Gao, Zhixian. (2020). Essential processing methods of hyperspectral images of agricultural and food products. Chemometrics and Intelligent Laboratory Systems. 198. 103936. 10.1016/j.chemolab.2020.103936.
[47] Peón, Juanjo & Recondo, Carmen & Fernández, Susana & Fernandez Calleja, Javier & Miguel, Eduardo & Carretero, Laura. (2017). Prediction of Topsoil Organic Carbon Using Airborne and Satellite Hyperspectral Imagery. Remote Sensing. 9. 1211. 10.3390/rs9121211.
[48] Park, Yongeun. (2020). Drone-based hyperspectral remote sensing of cyanobacteria using vertical cumulative pigment concentration in a deep reservoir. Remote Sensing of Environment. 236. 111517. 10.1016/j.rse.2019.111517.
[49] Asgari Gashteroodkhani, Oveis & Majidi, Mehrdad & Etezadi-Amoli, M. (2020). A combined deep belief network and time-time transform based intelligent protection Scheme for microgrids. Electric Power Systems Research. 10.1016/j.epsr.2020.106239.
[50] Guo, Jifeng & Li, Meihui & Wang, Lin & Yang, Bo & Zhang, Liangliang & Chen, Zhenxiang & Han, Shiyuan & Garcia-Hernandez, Laura & Abraham, Ajith. (2020). Engineering Applications of Artificial Intelligence Estimating cement compressive strength using three-dimensional microstructure images and deep belief network. Engineering Applications of Artificial Intelligence. 88. 103378. 10.1016/j.engappai.2019.103378.
[51] Wang, Gongming & Jia, Qing-Shan & Qiao, Junfei & Bi, Jing & Liu, Caixia. (2019). A sparse deep belief network with efficient fuzzy learning framework. Neural Networks. 121. 10.1016/j.neunet.2019.09.035.
[52] Raza, Mudassar & Sharif, Muhammad & Yasmin, Mussarat & Khan, Muhammad & Saba, Tanzila & Fernandes, Steven. (2018). Appearance based pedestrians’ gender recognition by employing stacked auto encoders in deep learning. Future Generation Computer Systems. 88. 10.1016/j.future.2018.05.002.
[53] Sikdar, Arindam & Chowdhury, Ananda. (2019). Scale-Invariant Batch-Adaptive Residual Learning for Person Re-identification. Pattern Recognition Letters. 129. 10.1016/j.patrec.2019.11.032.
[54] Mandal, Bappaditya & Sultana, Nazneen & Puhan, Niladri. (2018). Deep Residual Network with Regularized Fisher Framework for Detection of Melanoma. IET Computer Vision. 12. 10.1049/iet-cvi.2018.5238.
[55] Gao, Qishuo & Lim, Samsung & Jia, Xiuping. (2018). Hyperspectral Image Classification Using Convolutional Neural Networks and Multiple Feature Learning. Remote Sensing. 10. 299. 10.3390/rs10020299.
[56] Samat, Alim & Du, Peijun & Liu, Sicong & Li, Jun & Cheng, Liang. (2014). (ELMs)-L-2: Ensemble Extreme Learning Machines for Hyperspectral Image Classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 7. 1060 - 1069. 10.1109/JSTARS.2014.2301775.
[57] Luo, Fulin & Huang, Yajuan & Tu, Weiping & Liu, Jiamin. (2019). Local Manifold Sparse Model for Image Classification. Neurocomputing. 10.1016/j.neucom.2019.11.084.
[58] Chen Z, Jiang J, Jiang X, Fang X, Cai Z. Spectral-Spatial Feature Extraction of Hyperspectral Images Based on Propagation Filter. Sensors (Basel, Switzerland). 2018 Jun;18(6) DOI: 10.3390/s18061978.
[59] Kun Tan, Fuyu Wu, Xue Wang. (2018). Xuzhou HYSPEX dataset. IEEE Dataport. http://dx.doi.org/10.21227/t3c9-h862
[60] Bioucas-Dias, Jose & Plaza, Antonio & Dobigeon, Nicolas & Parente, Mario & Du, Qian & Gader, Paul & Chanussot, Jocelyn. (2012). Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 5. 10.1109/JSTARS.2012.2194696.
[61] Manley, II, Paul & Sagan, Vasit & Fritschi, Felix & Burken, Joel. (2019). Remote Sensing of Explosives-Induced Stress in Plants: Hyperspectral Imaging Analysis for Remote Detection of Unexploded Threats. Remote Sensing. 11. 1827. 10.3390/rs11151827.