Deep Reinforcement Learning-based Service Function Chains (SFCs) Deployment

Limei Peng

¹School of Computer Science and Engineering, Kyungpook National University, Daegu, South Korea

*Corresponding author

Network Function Virtualization (NFV) on top of Mobile Edge Computing (MEC) architecture has gained significant attention for enabling efficient IoT service provisioning. NFV enhances resource utilization by deploying Virtual Network Functions (VNFs) on general-purpose servers, which are organized into ordered sequences known as Service Function Chains (SFCs). However, optimizing VNF placement within SFCs to balance resource utilization and Quality of Service (QoS) remains a major challenge in MEC environments. Existing SFC deployment methods often face limitations in adaptability and efficiency, as they rely on static or heuristic approaches that struggle to handle dynamic network conditions and diverse resource requirements effectively. To overcome these challenges, we propose DeepSFCOpt, a Deep Reinforcement Learning (DRL)-based optimization method for SFC deployment that integrates Graph Convolutional Networks (GCNs) to extract network features and Sequence-to-Sequence(Seq2Seq) models to capture the order of SFCs, enabling adaptive placement strategies that optimize resource allocation and maximize long-term average revenue, thus more effectively meeting the demands of IoT services.

SFC deployment, Virtual function, Deep reinforcement learning, GNN, CARU

Xiao Kong and Limei Peng (2024). Deep Reinforcement Learning-based Service Function Chains (SFCs) Deployment. Journal of Networking and Network Applications, Volume 4, Issue 2, pp. 94–101. https://doi.org/10.33969/J-NaNA.2024.040205.

[1] A. Abouaomar, S. Cherkaoui, Z. Mlika, and A. Kobbane, “Service function chaining in mec: A mean-field game and reinforcement learning approach,” IEEE Systems Journal, vol. 16, no. 4, pp. 5357–5368, 2022.

[2] F. Schardong, I. Nunes, and A. Schaeffer-Filho, “Nfv resource allo-cation: A systematic review and taxonomy of vnf forwarding graph embedding,” Computer Networks, vol. 185, p. 107726, 2021.

[3] J. C. C. Chica, J. C. Imbachi, and J. F. B. Vega, “Security in sdn: A comprehensive survey,” Journal of Network and Computer Applications, vol. 159, p. 102595, 2020.

[4] X. Fu, F. R. Yu, J. Wang, Q. Qi, and J. Liao, “Dynamic service function chain embedding for nfv-enabled iot: A deep reinforcement learning approach,” IEEE Transactions on Wireless Communications, vol. 19, no. 1, pp. 507–519, 2019.

[5] B. Yuan and B. Ren, “Embedding the minimum cost sfc with end-to-end delay constraint,” in 2020 5th International Conference on Mechanical, Control and Computer Engineering (ICMCCE). IEEE, 2020, pp. 2299–2303.

[6] I. Sarrigiannis, K. Ramantas, E. Kartsakli, P.-V. Mekikis, A. Antonopou-los, and C. Verikoukis, “Online vnf lifecycle management in an mec-enabled 5g iot architecture,” IEEE Internet of Things Journal, vol. 7, no. 5, pp. 4183–4194, 2019.

[7] P. Jin, X. Fei, Q. Zhang, F. Liu, and B. Li, “Latency-aware vnf chain deployment with efficient resource reuse at network edge,” in IEEE INFOCOM 2020-IEEE conference on computer communications. IEEE, 2020, pp. 267–276.

[8] O. Alhussein and W. Zhuang, “Robust online composition, routing and nf placement for nfv-enabled services,” IEEE Journal on Selected Areas in Communications, vol. 38, no. 6, pp. 1089–1101, 2020.

[9] J. Pei, P. Hong, M. Pan, J. Liu, and J. Zhou, “Optimal vnf placement via deep reinforcement learning in sdn/nfv-enabled networks,” IEEE Journal on Selected Areas in Communications, vol. 38, no. 2, pp. 263–278, 2019.

[10] H. Xing, Y. Pu, X. Wang, F. Song, Z. Xiao, L. Feng, and L. Xu, “Big data oriented multi-objective sfc placement in dynamic mec: A distributed drl approach,” in ICC 2024 - IEEE International Conference on Communications, 2024, pp. 666–671.

[11] Y. Zhong, D. Zheng, and X. Cao, “A drl approach with network service deployment transformer for reliable sfc deployment,” in ICC 2024 -IEEE International Conference on Communications, 2024, pp. 177–182.

[12] A. Tian, B. Feng, Y. Huang, H. Zhou, S. Yu, and H. Zhang, “Drl-based two-stage sfc deployment approach under latency constraints,” in IEEE INFOCOM 2024 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 2024, pp. 1–6.

[13] D. Xiao, J. A. Zhang, X. Liu, Y. Qu, W. Ni, and R. P. Liu, “A two-stage gcn-based deep reinforcement learning framework for sfc embedding in multi-datacenter networks,” IEEE Transactions on Network and Service Management, vol. 20, no. 4, pp. 4297–4312, 2023.

[14] S. Guo, Y. Du, and L. Liu, “A meta reinforcement learning approach for sfc placement in dynamic iot-mec networks,” Applied Sciences, vol. 13, no. 17, p. 9960, 2023.

[15] D. Hu, S. Lyu, S. Y. Chang, L. Peng, and P.-H. Ho, “Dynamic siot network status prediction,” Journal of Networking and Network Applications, vol. 2, no. 2, pp. 78–85, 2022.

[16] D. Bahdanau, “Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409.0473, 2014.

[17] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., “Human-level control through deep reinforcement learning,” nature, vol. 518, no. 7540, pp. 529–533, 2015.

[18] J. Jia and J. Hua, “Dynamic sfc placement with parallelized vnfs in data center networks: A drl-based approach,” ICT Express, vol. 10, no. 1, pp. 104–110, 2024.