Title
Arena-style immersive live experience (ILE) services and systems: Highly realistic sensations for everyone in the world
Abstract
Immersive Live Experiences (ILEs) enable audiences at remote sites to feel real-time highly realistic sensations, as if they are at the event site. This article provides a key feature of an implementation of ILE services, called Kirari! for Arena, as a use case of arena-style ILE, and its technical elements developed by NTT Labs. The major functionalities are object extraction from an arbitrary background, object tracking with depth sensors, low-latency synchronized data transport and four-sided pseudo-3D image presentation with depth expression. It also provides evaluations on the experience of Kirari! for Arena audiences, as well as its conformance to International Telecommunication Union, Telecommunication Standardization Sector (ITU-T) standards for ILE.
Keywords
AI, immersive live experience, MPEG media transport, pseudo-3D images, real-time image extraction, standardization
Authors
Jiro Nagao
NTT Service Evolution Labs, Japan
Jiro NAGAO is a Research Engineer, NTT Service Evolution Laboratories, Nippon Telegraph and Telephone Corporation (NTT). He received a B.E., M.E. and Ph.D. in information science from Nagoya University, Japan in 2001, 2003 and 2007. He joined NTT in 2007 and was engaged in research and development of a video recognition system. From 2009 to 2011, he was engaged in a joint research with a research agency in Singapore in application development and evaluation of IPTV services conformant to ITU-T standards. After working as a technical leader of commercial video streaming services at NTT Communications from 2011 to 2017, he is currently engaged in research on immersive media and presentation in NTT Service Evolution Laboratories. Since 2019, he has contributed to the international standardization efforts on immersive live experience (ILE) of ITU-T SG16. He is the Leader of ILE Sub-Working Group of The Telecommunication Technology Committee (TTC), Japan. He is a member of the Institute of Electronics, Information and Communication Engineers (IEICE) and The Japanese Society of Medical Imaging Technology (JAMIT).
Kiyoshi Tanaka
Otsuma Women’s University, Japan
Kiyoshi Tanaka, Associate Professor, Faculty of Social Information Studies, Otsuma Women’s University. He received a B.E., M.E., and Ph.D. in communication engineering from Osaka University, Japan in 1992, 1994, and 2005. He joined Nippon Telegraph and Telephone Corporation (NTT) in 1994 and since then has been engaged in researching video-on-demand systems and metadata-related interactive video systems and services, especially those related to IPTV, digital signage services and immersive live experience. He has also contributed to ITU-T’s international standardization efforts in SG16. After he worked in NTT as a Senior Research Engineer, Supervisor of NTT Service Evolution Laboratories, he moved to Otsuma Women’s University from 2020. He is a member of the Institute of Electronics, Information and Communication Engineers (IEICE), the Human Interface Society in Japan, and the Institute of Image Electronics Engineers of Japan (IIEEJ).
Hideo Imanaka
NTT Advanced Technologies, Japan
Hideo Imanaka is a Senior Manager, Network Innovation Business Headquarters, NTT Advanced Technology Corporation (NTT-AT). He received a B.E., M.E., and Ph.D. in electrical engineering from Mie University in 1985, 1987, and 2001. He joined NTT Telecommunication Network Laboratories in 1987 and from then until 1996, he was engaged in research of fiber optic access network architecture and network operation process reengineering methods. From 1996 to 2003, he had been engaged in enterprise resource planning system integration System Engineer in the Solution Division of NTT Communications. From 2004, he started international standardization activities in ITU-T and the APT standardization program (ASTAP) in NTT Service Integration Laboratories. He was in charge of standardization strategy planning of the entire NTT Group from 2010 to 2015 in NTT R&D planning division. Since joining NTT-AT in 2015, he has been engaged in standardization consulting. He was the Rapporteur of Question 1 of Study Group 13 from 2006 to 2010 for the Next-Generation Network and IPTV standardization work in ITU-T, the chairman of the Internet of Things (IoT) working group in ASTAP from 2012 to 2016 for IoT and smart grid standardization work, and the vice-chairman of Focus Group on IMT-2020 in 2015 for pre-standardization of 5G networks. He is currently the Rapporteur of Question 8 of Study Group 16 in ITU-T for standardization of Immersive Live Experience and the vice-rapporteur of Question 5 of Study Group 2 in ITU-D for international deployment of emergency telecommunications and disaster relief solutions. He received the ITU-AJ award from the ITU Association of Japan in 2009. He is a member of IEICE and the Society of Instrument and Control Engineers (SICE).
