Cloud-based XR Services: A Survey on Relevant Challenges and Enabling Technologies

Theodoros Theodoropoulos

¹ Department of Informatics and Telematics, Harokopio University, Athens, Greece

² ICTFICIAL, Espoo, Finalnd & Ecole Nationale Sup´erieure d’Informatique, Algiers, Algeria

³ University of Oulu, Oulu, Finland

⁴ Eurescom GmbH, Heidelberg, Germany

⁵ OneSource, Coimbra, Portugal

⁶ HPE, Cernusco Sul Naviglio, Italy

⁷ SeeReal Technologies, Dresden, Germany

⁸ ORamaVR & FORTH-ICS & University of Crete, Heraklion, Greece

⁹ ORamaVR & University of Western Macedonia, Kozani, Greece

¹⁰ CNR, Pisa, Italy

*Corresponding author

In recent years, the emergence of XR (eXtended Reality) applications, including Holography, Augmented, Virtual and Mixed Reality, has resulted in the creation of rather demanding requirements for Quality of Experience (QoE) and Quality of Service (QoS). In order to cope with requirements such as ultra-low latency and increased bandwidth, it is of paramount importance to leverage certain technological paradigms. The purpose of this paper is to identify these QoE and QoS requirements and then to provide an extensive survey on technologies that are able to facilitate the rather demanding requirements of Cloud-based XR Services. To that end, a wide range of enabling technologies are explored. These technologies include e.g. the ETSI (European Telecommunications Standards Institute) Multi-Access Edge Computing (MEC), Edge Storage, the ETSI Management and Orchestration (MANO), the ETSI Zero touch network & Service Management (ZSM), Deterministic Networking, the 3GPP (3rd Generation Partnership Project) Media Streaming, MPEG’s (Moving Picture Experts Group) Mixed and Augmented Reality standard, the Omnidirectional MediA Format (OMAF), ETSI’s Augmented Reality Framework etc.

Edge Computing, XR services, Holography, Cloud Computing

Theodoros Theodoropoulos, Antonios Makris, Abderrahmane Boudi, Tarik Taleb, Uwe Herzog, Luis Rosa, Luis Cordeiro, Konstantinos Tserpes, Elena Spatafora, Alessandro Romussi, Enrico Zschau, Manos Kamarianakis, Antonis Protopsaltis, George Papagiannakis, and Patrizio Dazzi (2022). Cloud-based XR Services: A Survey on Relevant Challenges and Enabling Technologies. Journal of Networking and Network Applications, Volume 2, Issue 1, pp. 1–22. https://doi.org/10.33969/J-NaNA.2022.020101.

[1] A. Makris, A. Boudi, M. Coppola, L. Cordeiro, M. Corsini, P. Dazzi, F. D. Andilla, Y. Gonz´alez Rozas, M. Kamarianakis, M. Pateraki, T. L. Pham, A. Protopsaltis, A. Raman, A. Romussi, L. Rosa, E. Spatafora, T. Taleb, T. Theodoropoulos, K. Tserpes, E. Zschau, and U. Herzog, “Cloud for holography and augmented reality,” in 2021 IEEE 10th International Conference on Cloud Networking (CloudNet), 2021, pp. 118–126.

[2] Network Functions Virtualisation (NFV) Release 4 Management and Orchestration Requirements for service interfaces and object model for OS container management and orchestration specification, ETSI GS NFV-IFA 040 - V4.2.1, ETSI, 5 2021.

[3] Network Functions Virtualisation (NFV), Infrastructure Overview, ETSI GS NFV-INF 001 - V1.1.1, ETSI, 1 2015.

[4] Network Functions Virtualisation (NFV) Architectural Framework, ETSI GS NFV 002 - V1.2.1, ETSI, 12 2014.

[5] Network Functions Virtualisation (NFV) Infrastructure Compute Do-main, ETSI GS NFV-INF 003 - V1.1.1, ETSI, 12 2021.

[6] Network Functions Virtualisation (NFV) Infrastructure Hypervisor Do-main, ETSI GS NFV-INF 004 - V1.1.1, ETSI, 1 2015.

[7] Network Functions Virtualisation (NFV) Infrastructure Network Do-main, ETSI GS NFV-INF 005 - V1.1.1, ETSI, 12 2014.

[8] Network Functions Virtualisation (NFV) Management and Orchestra-tion, ETSI GS NFV-MAN 001 - V1.1.1, ETSI, 12 2014.

[9] Zero-touch network and Service Management (ZSM) Reference Archi-tecture, ETSI GS ZSM 002 - V1.1.1, ETSI, 8 2019.

[10] Zero-touch network and Service Management (ZSM) Requirements based on documented scenarios, ETSI GS ZSM 001 - V1.1.1, ETSI, 10 2019.

[11] Zero-touch network and Service Management (ZSM) Closed-Loop Au-tomation Part 1: Enablers, ETSI GS ZSM 009-1 - V1.1.1, ETSI, 6 2021.

[12] Y. Wang, R. Forbes, U. Elzur, J. Strassner, A. Gamelas, H. Wang, S. Liu, L. Pesando, X. Yuan, and S. Cai, “From design to practice: Etsi eni reference architecture and instantiation for network management and orchestration using artificial intelligence,” IEEE Communications Standards Magazine, vol. 4, no. 3, pp. 38–45, 2020.

[13] Y. Wang, R. Forbes, C. Cavigioli, H. Wang, A. Gamelas, A. Wade, J. Strassner, S. Cai, and S. Liu, “Network management and orchestration using artificial intelligence: Overview of etsi eni,” IEEE communications standards magazine, vol. 2, no. 4, pp. 58–65, 2018.

[14] D. M. Gutierrez-Estevez, M. Gramaglia, A. De Domenico, G. Dandachi, S. Khatibi, D. Tsolkas, I. Balan, A. Garcia-Saavedra, U. Elzur, and Y. Wang, “Artificial intelligence for elastic management and orches-tration of 5g networks,” IEEE Wireless Communications, vol. 26, no. 5, pp. 134–141, 2019.

[15] D. Sabella, A. Alleman, E. Liao, M. Filippou, Z. Ding, L. G. Baltar, S. Srikanteswara, K. Bhuyan, O. Oyman, G. Schatzberg et al., “Edge computing: from standard to actual infrastructure deployment and soft-ware development,” ETSI White paper, pp. 1–41, 2019.

[16] Multi-access edge computing (MEC) framework and reference architec-ture, ETSI GS MEC 003 - V2.1.1, ETSI, 8 2019.

[17] Y. Sun, Z. Chen, M. Tao, and H. Liu, “Communications, caching, and computing for mobile virtual reality: Modeling and tradeoff,” IEEE Transactions on Communications, vol. 67, no. 11, pp. 7573–7586, 2019.

[18] T. Taleb, K. Samdanis, B. Mada, H. Flinck, S. Dutta, and D. Sabella, “On multi-access edge computing: A survey of the emerging 5g network edge cloud architecture and orchestration,” IEEE Communications Surveys & Tutorials, vol. 19, no. 3, pp. 1657–1681, 2017.

[19] Q.-V. Pham, F. Fang, V. N. Ha, M. J. Piran, M. Le, L. B. Le, W.-J. Hwang, and Z. Ding, “A survey of multi-access edge computing in 5g and beyond: Fundamentals, technology integration, and state-of-the-art,” IEEE Access, vol. 8, pp. 116 974–117 017, 2020.

[20] X. Jiang, F. R. Yu, T. Song, and V. C. Leung, “A survey on multi-access edge computing applied to video streaming: Some research issues and challenges,” IEEE Communications Surveys & Tutorials, vol. 23, no. 2, pp. 871–903, 2021.

[21] Multi-access Edge Computing (MEC) Framework and Reference Archi-tecture, ETSI GS MEC 003 - V2.1.1, ETSI, 1 2019.

[22] L. M. Contreras and C. J. Bernardos, “Overview of architectural al-ternatives for the integration of etsi mec environments from different administrative domains,” Electronics, vol. 9, no. 9, p. 1392, 2020.

[23] Multi-access Edge Computing (MEC) Study on Inter-MEC systems and MEC-Cloud system coordination, ETSI GR MEC 035 - V3.1.1, ETSI, 6 2021.

[24] B. Confais, A. Lebre, and B. Parrein, “Performance analysis of object store systems in a fog and edge computing infrastructure,” in Trans-actions on Large-Scale Data-and Knowledge-Centered Systems XXXIII. Springer, 2017, pp. 40–79.

[25] I. Clarke, O. Sandberg, B. Wiley, and T. W. Hong, “Freenet: A distributed anonymous information storage and retrieval system,” in Designing privacy enhancing technologies. Springer, 2001, pp. 46–66.

[26] A.-G. Gheorghe, C.-C. Crecana, C. Negru, F. Pop, and C. Dobre, “Decentralized storage system for edge computing,” in 2019 18th Inter-national Symposium on Parallel and Distributed Computing (ISPDC). IEEE, 2019, pp. 41–49.

[27] I. Lujic, V. De Maio, and I. Brandic, “Efficient edge storage management based on near real-time forecasts,” in 2017 IEEE 1st International Conference on Fog and Edge Computing (ICFEC). IEEE, 2017, pp. 21–30.

[28] J. Xing, H. Dai, and Z. Yu, “A distributed multi-level model with dynamic replacement for the storage of smart edge computing,” Journal of Systems Architecture, vol. 83, pp. 1–11, 2018.

[29] Y. Huang, X. Song, F. Ye, Y. Yang, and X. Li, “Fair and efficient caching algorithms and strategies for peer data sharing in pervasive edge computing environments,” IEEE Transactions on Mobile Computing, vol. 19, no. 4, pp. 852–864, 2019.

[30] T. Hou, G. Feng, S. Qin, and W. Jiang, “Proactive content caching by exploiting transfer learning for mobile edge computing,” International Journal of Communication Systems, vol. 31, no. 11, p. e3706, 2018.

[31] Z. Chang, L. Lei, Z. Zhou, S. Mao, and T. Ristaniemi, “Learn to cache: Machine learning for network edge caching in the big data era,” IEEE Wireless Communications, vol. 25, no. 3, pp. 28–35, 2018.

[32] L. Zhang, J. Wu, S. Mumtaz, J. Li, H. Gacanin, and J. J. Rodrigues, “Edge-to-edge cooperative artificial intelligence in smart cities with on-demand learning offloading,” in 2019 IEEE Global Communications Conference (GLOBECOM). IEEE, 2019, pp. 1–6.

[33] S. Sondur, K. Kant, S. Vucetic, and B. Byers, “Storage on the edge: Evaluating cloud backed edge storage in cyberphysical systems,” in 2019 IEEE 16th International Conference on Mobile Ad Hoc and Sensor Systems (MASS). IEEE, 2019, pp. 362–370.

[34] T. Theodoropoulos, A.-C. Maroudis, J. Violos, and K. Tserpes, “An encoder-decoder deep learning approach for multistep service traffic prediction,” in 2021 IEEE Seventh International Conference on Big Data Computing Service and Applications (BigDataService), 2021, pp. 33–40.

[35] M. Caprolu, R. Di Pietro, F. Lombardi, and S. Raponi, “Edge computing perspectives: architectures, technologies, and open security issues,” in 2019 IEEE International Conference on Edge Computing (EDGE). IEEE, 2019, pp. 116–123.

[36] S. Shahzadi, M. Iqbal, T. Dagiuklas, and Z. U. Qayyum, “Multi-access edge computing: open issues, challenges and future perspectives,” Journal of Cloud Computing, vol. 6, no. 1, pp. 1–13, 2017.

[37] B. Varghese, N. Wang, S. Barbhuiya, P. Kilpatrick, and D. S. Nikolopou-los, “Challenges and opportunities in edge computing,” in 2016 IEEE International Conference on Smart Cloud (SmartCloud). IEEE, 2016, pp. 20–26.

[38] R. Yang, F. R. Yu, P. Si, Z. Yang, and Y. Zhang, “Integrated blockchain and edge computing systems: A survey, some research issues and challenges,” IEEE Communications Surveys & Tutorials, vol. 21, no. 2, pp. 1508–1532, 2019.

[39] C. Luo, L. Xu, D. Li, and W. Wu, “Edge computing integrated with blockchain technologies,” in Complexity and Approximation. Springer, 2020, pp. 268–288.

[40] K. Samdanis and T. Taleb, “The road beyond 5g: A vision and insight of the key technologies,” IEEE Network, vol. 34, no. 2, pp. 135–141, 2020.

[41] S. Kekki, W. Featherstone, Y. Fang, P. Kuure, A. Li, A. Ranjan,

D. Purkayastha, F. Jiangping, D. Frydman, G. Verin et al., “Mec in 5g networks,” ETSI white paper, vol. 28, pp. 1–28, 2018.

[42] N. Sprecher et al., “Harmonizing standards for edge computing-a synergized architecture leveraging etsi isg mec and 3gpp specifications,” ETSI White paper No. 36, no. 979-10-92620-35-5, 2020.

[43] E. Coronado, Z. Yousaf, and R. Riggio, “Lightedge: mapping the evolution of multi-access edge computing in cellular networks,” IEEE Communications Magazine, vol. 58, no. 4, pp. 24–30, 2020.

[44] N. Finn, “Time-sensitive and deterministic networking whitepaper,” 2017.

[45] M. Chen, X. Geng, and Z. Li, “Segment routing (sr) based bounded la-tency,” Internet Engineering Task Force, Internet-Draft draft-chendetnet-sr-based-bounded-latency-00, 2018.

[46] F. Chiariotti, S. Kucera, A. Zanella, and H. Claussen, “Leap: A latency control protocol for multi-path data delivery with pre-defined qos guarantees,” in IEEE INFOCOM 2018 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 2018, pp. 166–171.

[47] S. Ha, I. Rhee, and L. Xu, “Cubic: A new tcp-friendly high-speed tcp variant,” SIGOPS Oper. Syst. Rev., vol. 42, no. 5, p. 64–74, Jul. 2008.[Online]. Available: https://doi.org/10.1145/1400097.1400105

[48] 5G, NG-RAN, Architecture description, ETSI TS 138 401 - V15.5.0, ETSI, 5 2019.

[49] S. Niknam, A. Roy, H. S. Dhillon, S. Singh, R. Banerji, J. H. Reed,

N. Saxena, and S. Yoon, “Intelligent o-ran for beyond 5g and 6g wireless networks,” arXiv preprint arXiv:2005.08374, 2020.

[50] “Information technology — Computer graphics, image processing and environmental data representation — Mixed and augmented reality (MAR) reference model,” International Organization for Standardization, Standard, Feb. 2019.

[51] J. Lee, Y. Lee, S. Lee, and G. J. Kim, “Standardization for augmented reality: introduction of activities at iso-iec sc 24 wg 9,” in Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry, 2013, pp. 279–280.

[52] T. Huang and Y. Liu, “3d point cloud geometry compression on deep learning,” in Proceedings of the 27th ACM International Conference on Multimedia, 2019, pp. 890–898.

[53] D. C. Garcia and R. L. de Queiroz, “Intra-frame context-based octree coding for point-cloud geometry,” in 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, 2018, pp. 1807–1811.

[54] T. Wiemann, F. Igelbrink, S. P¨utz, M. K. Piening, S. Schupp, S. Hin-derink, J. Vana, and J. Hertzberg, “Compressing ros sensor and geometry messages with draco,” in 2019 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR). IEEE, 2019, pp. 243–248.

[55] A. Varischio, F. Mandruzzato, M. Bullo, M. Giordani, P. Testolina, and

M. Zorzi, “Hybrid point cloud semantic compression for automotive sensors: A performance evaluation,” arXiv preprint arXiv:2103.03819, 2021.

[56] M. Hosseini and C. Timmerer, “Dynamic adaptive point cloud stream-ing,” in Proceedings of the 23rd Packet Video Workshop, 2018, pp. 25–30.

[57] X. Sun, S. Wang, M. Wang, S. S. Cheng, and M. Liu, “An advanced lidar point cloud sequence coding scheme for autonomous driving,” in Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 2793–2801.

[58] L. Wiesmann, A. Milioto, X. Chen, C. Stachniss, and J. Behley, “Deep compression for dense point cloud maps,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 2060–2067, 2021.

[59] B. Han, Y. Liu, and F. Qian, “Vivo: Visibility-aware mobile volumetric video streaming,” in Proceedings of the 26th Annual International Conference on Mobile Computing and Networking, 2020, pp. 1–13.

[60] C. Portaneri, P. Alliez, M. Hemmer, L. Birklein, and E. Schoemer, “Cost-driven framework for progressive compression of textured meshes,” in Proceedings of the 10th ACM Multimedia Systems Conference, 2019, pp. 175–188.

[61] K. Christaki, E. Christakis, P. Drakoulis, A. Doumanoglou, N. Zioulis,

D. Zarpalas, and P. Daras, “Subjective visual quality assessment of immersive 3d media compressed by open-source static 3d mesh codecs,” in International Conference on Multimedia Modeling. Springer, 2019, pp. 80–91.

[62] Augmented Reality Framework (ARF); AR framework architecture, ETSI GS ARF 004-2 - V1.1.1, ETSI, 8 2021.

[63] D. Graziosi, O. Nakagami, S. Kuma, A. Zaghetto, T. Suzuki, and

A. Tabatabai, “An overview of ongoing point cloud compression stan-dardization activities: video-based (v-pcc) and geometry-based (g-pcc),” APSIPA Transactions on Signal and Information Processing, vol. 9, p. e13, 2020.

[64] H. Edelsbrunner, D. Kirkpatrick, and R. Seidel, “On the shape of a set of points in the plane,” IEEE Transactions on information theory, vol. 29, no. 4, pp. 551–559, 1983.

[65] F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin, “The ball-pivoting algorithm for surface reconstruction,” IEEE transactions on visualization and computer graphics, vol. 5, no. 4, pp. 349–359, 1999.

[66] M. Kazhdan, M. Bolitho, and H. Hoppe, “Poisson surface recon-struction,” in Proceedings of the fourth Eurographics symposium on Geometry processing, vol. 7, 2006.