Learning morphological operators for skin detection

Loris Nanni

¹ DISI, Università di Bologna, Via dell’Università 50, 47521 Cesena, Italy. Email: [email protected]

² DEI - University of Padova, Via Gradenigo, 6 - 35131- Padova, Italy 

^* Corresponding Author: Loris Nanni, Email: [email protected]

Human skin detection, i.e. the process of discriminating “skin” and “non-skin” pixel in an image or a video, is a very important task for several applications including face detection, video surveillance, body tracking, hand gesture recognition, and many others. Skin detection has been widely studied from the research community resulting in several methods based on hand-crafted rules or deep learning. In this work we propose a novel post-processing approach for skin detectors based on trained morphological operators. The first step, consisting in skin segmentation, is performed according to an existing skin detection approach, and then a second step is carried out consisting in the application of a set of morphological operators to refine the resulting mask. Extensive experimental evaluation, performed considering two different detection approaches (one based on deep learning and a handcrafted one), carried on 10 different datasets confirms the quality of the proposed method. To encourage future comparisons the MATLAB source code is freely available in the GitHub repository: https://github.com/LorisNanni.

Skin classification, Skin detection, Skin segmentation, Convolutional Neural Networks, Morphological operators.

A. Lumini, L. Nanni, A. Codogno and F. Berno (2019). Learning morphological operators for skin detection. Journal of Artificial Intelligence and Systems, 1, 60–76. https://doi.org/10.33969/AIS.2019.11004.

[1] A. Lumini, L. Nanni, Fair comparison of skin detection approaches on publicly available datasets, (2018).

[2] S. Naji, H.A. Jalab, S.A. Kareem, A survey on skin detection in colored images, Artif. Intell. Rev. (2018). doi:10.1007/s10462-018-9664-9.

[3] P. Kakumanu, S. Makrogiannis, N. Bourbakis, A survey of skin-color modeling and detection methods, Pattern Recognit. 40 (2007) 1106–1122. doi:10.1016/j.patcog.2006.06.010.

[4] M.J. Seow, D. Valaparla, V.K. Asari, Neural network based skin color model for face detection, in: Proc. - Appl. Imag. Pattern Recognit. Work., 2004. doi:10.1109/AIPR.2003.1284262.

[5] R. Khan, A. Hanbury, J. Stoettinger, Skin detection: A random forest approach, in: Proc. - Int. Conf. Image Process. ICIP, 2010. doi:10.1109/ICIP.2010.5651638.

[6] N. Sebe, I. Cohen, T.S. Huang, T. Gevers, Skin detection: a Bayesian network approach, Proc. 17th Int. Conf. Pattern Recognit. 2 (2004) 2–5. doi:10.1109/ICPR.2004.1334405.

[7] R.P.K. Poudel, J.J. Zhang, D. Liu, H. Nait-Charif, Skin Color Detection Using Region-Based Approach, Int. J. Image Process. 7 (2013) 385.

[8] W.C. Chen, M.S. Wang, Region-based and content adaptive skin detection in color images, Int. J. Pattern Recognit. Artif. Intell. 21 (2007) 831–853. doi:10.1142/s0218001407005715.

[9] T. Xu, Z. Zhang, Y. Wang, Patch-wise skin segmentation of human body parts via deep neural networks, J. Electron. Imaging. 24 (2015) 043009. doi:10.1117/1.JEI.24.4.043009.

[10] H. Zuo, H. Fan, E. Blasch, H. Ling, Combining Convolutional and Recurrent Neural Networks for Human Skin Detection, IEEE Signal Process. Lett. 24 (2017) 289–293.

[11] Y. Kim, I. Hwang, N.I. Cho, Convolutional neural networks and training strategies for skin detection, in: IEEE Int. Conf. Image Process., 2017: pp. 3919–3923. doi:10.1109/ICIP.2017.8297017.

[12] A. Dourado, F. Guth, T.E. de Campos, W. Li, Domain adaptation for holistic skin detection, CoRR. abs/1903.0 (2019). http://arxiv.org/abs/1903.06969.

[13] A. Dourado, F. Guth, T.E. de Campos, W. Li, Domain adaptation for holistic skin detection, CoRR. abs/1903.0 (2019).

[14] W. Song, N. Zheng, R. Zheng, X.B. Zhao, A. Wang, Digital image semantic segmentation algorithms: A survey, J. Inf. Hiding Multimed. Signal Process. 10 (2019) 196–211.

[15] E. Shelhamer, J. Long, T. Darrell, Fully Convolutional Networks for Semantic Segmentation, IEEE Trans. Pattern Anal. Mach. Intell. (2017). doi:10.1109/TPAMI.2016.2572683.

[16] V. Badrinarayanan, A. Kendall, R. Cipolla, SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation, IEEE Trans. Pattern Anal. Mach. Intell. (2017). doi:10.1109/TPAMI.2016.2644615.

[17] O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, in: Lect. Notes Comput. Sci. (Including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 2015. doi:10.1007/978-3-319-24574-4_28.

[18] M. Arsalan, D.S. Kim, M. Owais, K.R. Park, OR-Skip-Net: Outer residual skip network for skin segmentation in non-ideal situations, Expert Syst. Appl. 141 (2020) 112922. doi:10.1016/J.ESWA.2019.112922.

[19] S.L. Phung, A. Bouzerdoum, D. Chai, Skin segmentation using color pixel classification: Analysis and comparison, IEEE Trans. Pattern Anal. Mach. Intell. 27 (2005) 148–154. doi:10.1109/TPAMI.2005.17.

[20] F.E. Sandnes, L. Neyse, Y.-P. Huang, Simple and practical skin detection with static RGB-color lookup tables: A visualization-based study, in: IEEE Int. Conf. Syst. Man, Cybern., 2016: pp. 2370–2375.

[21] M. Kawulok, J. Kawulok, J. Nalepa, B. Smolka, Self-adaptive algorithm for segmenting skin regions, EURASIP J. Adv. Signal Process. (2014) 1–22. doi:10.1186/1687-6180-2014-170.

[22] E. Hassan, A.R. Hilal, O. Basir, Using ga to optimize the explicitly defined skin regions for human skincolor detection, in: Can. Conf. Electr. Comput. Eng., 2017. doi:10.1109/CCECE.2017.7946699.

[23] R.C.. Gonzalez, R.E. Woods, Digital image processing, 2008. doi:10.1049/ep.1978.0474.

[24] J.U. Yun, H.J. Lee, A.K. Paul, J.H. Baek, Face detection for video summary using illumination-compensation and morphological processing, Pattern Recognit. Lett. (2009). doi:10.1016/j.patrec.2009.04.010.

[25] S. Kang, B. Choi, D. Jo, Faces detection method based on skin color modeling, J. Syst. Archit. 64 (2016) 100–109. doi:10.1016/J.SYSARC.2015.11.009.

[26] X. Wang, L. Wang, T. Lei, C. Wang, Face detection based on improved skin model and local iterated conditional modes, in: Proc. - Int. Conf. Nat. Comput., 2016. doi:10.1109/ICNC.2015.7378122.

[27] M.J. Jones, J.M. Rehg, Statistical color models with application to skin detection, Int. J. Comput. Vis. 46 (2002) 81–96. doi:10.1023/A:1013200319198.

[28] J. Ruiz-Del-Solar, R. Verschae, Skin detection using neighborhood information, in: Proc. - Sixth IEEE Int. Conf. Autom. Face Gesture Recognit., 2004: pp. 463–468. doi:10.1109/AFGR.2004.1301576.

[29] S.J. Schmugge, S. Jayaram, M.C. Shin, L. V. Tsap, Objective evaluation of approaches of skin detection using ROC analysis, Comput. Vis. Image Underst. 108 (2007) 41–51. doi:10.1016/j.cviu.2006.10.009.

[30] J. Stöttinger, A. Hanbury, C. Liensberger, R. Khan, Skin paths for contextual flagging adult videos, in: Lect. Notes Comput. Sci. (Including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 2009: pp. 303–314. doi:10.1007/978-3-642-10520-3_28.

[31] J.C. Sanmiguel, S. Suja, Skin detection by dual maximization of detectors agreement for video monitoring, Pattern Recognit. Lett. 34 (2013) 2102–2109. doi:10.1016/j.patrec.2013.07.016.

[32] J.P.B. Casati, D.R. Moraes, E.L.L. Rodrigues, SFA: A human skin image database based on FERET and AR facial images, in: IX Work. Visao Comput. Rio Janeiro, 2013.

[33] W.R. Tan, C.S. Chan, P. Yogarajah, J. Condell, A Fusion Approach for Efficient Human Skin Detection, Ind. Informatics, IEEE Trans. 8 (2012) 138–147. doi:10.1109/TII.2011.2172451.

[34] L. Nanni, A. Lumini, Skin Detection for Reducing False Positive in Face Detection, in: V.M. Petrova (Ed.), Adv. Eng. Res. Vol.16, Nov. Publ. 2017, Nova Publisher, 2017. https://www.novapublishers.com/catalog/product_info.php?products_id=60326&osCsid=.

[35] A. Angelova, Y. Abu-Mostafa, P. Perona, Pruning training sets for learning of object categories, in: Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., 2005: pp. 494–501. doi:10.1109/CVPR.2005.283.

[36] J. Demšar, Statistical Comparisons of Classifiers over Multiple Data Sets, J. Mach. Learn. Res. 7 (2006) 1–30. doi:10.1016/j.jecp.2010.03.005.