Bringing Culture to Life: Real-Time Immersive Experiences with Free Viewpoint Video

13/11/2025

Researchers from the Information Processing and Telecommunications Center (IPTC) at the Universidad Politécnica de Madrid have developed an innovative system that combines Extended Reality (XR) and Free Viewpoint Video (FVV) to transform how people experience cultural events. The platform allows users to interact in real time with a volumetric video presenter inside a shared virtual environment enriched with 3D cultural artifacts. Through AI-based gesture recognition, the presenter can manipulate virtual objects, creating a dynamic and natural interaction.

Unlike conventional XR platforms that rely on pre-recorded or static content, this system supports live immersive communication, enabling participants to connect as 3D avatars and engage through audio and visual channels. The underlying technology leverages FVV Live, a real-time capture and rendering system that transmits only the 2D perspective needed for each user, ensuring low latency and efficient streaming.

To evaluate the user experience, the team proposes a Quality of Experience (QoE) methodology based on ITU-T standards, focusing on audiovisual quality, interactivity, cybersickness, and usability under different network conditions.

This research opens new possibilities for remote cultural visits, virtual museum tours, educational workshops, and performing arts, where audiences can engage collaboratively and naturally from anywhere in the world. The system demonstrates how advanced immersive technologies can bridge the gap between digital communication and real-world cultural presence.

Bibliographic reference:

Usón, J., Muñoz, V., Cortés, C., Rodríguez, I., Díaz, C., Gutiérrez. J. & Cabrera, J. Enabling Real-Time Collaborative Cultural Experiences with Free Viewpoint Video. 2025 17th International Conference on Quality of Multimedia Experience (QoMEX), Madrid, Spain, 2025, pp. 1-4, doi: 10.1109/QoMEX65720.2025.11219989.

Javier Usón Peirón: ORCID / LinkedIn

Victoria Muñoz Murillo: ORCID / LinkedIn

Carlos Cortés Sánchez: ORCID / LinkedIn

Isabel Rodríguez Márquez: ORCID

César Díaz Martín: GS / ORCID/ LinkedIn

Jesús Gutiérrez Sánchez: GS/ ORCID / LinkedIn

Julián Cabrera Quesada: GS / ORCID/ LinkedIn

LinkedIn: https://www.linkedin.com/company/iptc-upm/

For more information: www.iptc.upm.es