Real-Time Free Viewpoint Video for Immersive Videoconferencing

Resumen

In this work, we propose a demo of an immersive videoconference system using Free Viewpoint Video (FVV) technology. It makes use of the \FVV system, which covers the entire FVV pipeline (capture, view rendering, and visualization) while working in real-time. The \FVV system consists of nine cameras that capture an environment and a view renderer that uses the information from the cameras to generate a synthetic view at an arbitrary point.

It is designed as a hybrid demo. While the capture and rendering processes take place at our premises, \FVV can be visualized through devices connected to the Internet.

The system allows immersive navigation of a virtual scene with 6 degrees of freedom, and interaction with live-captured avatars integrated in such scene. For this purpose, it uses WebRTC connections to update the position of the virtual camera and to receive the \FVV view encoded as a video.

Additionally, the user will be recorded by a simple camera and microphone setup, and the generated streams will be transmitted to our premises through the same WebRTC server. This way, people being recorded by \FVV will be able to see and hear the user, enabling bidirectional communication.