Design and development of a foreground segmentation approach based on Deep Learning for Free Viewpoint Video

Resumen

This master’s thesis is part of the immersive multiview video research of Grupo de Tratamiento de Imá-genes (GTI), more specifically, the FVV-Live system. FVV-Live is a free viewpoint video system capable of working in real-time. It generates a virtual view from the scene, captured by a set of stereo cameras. This lets the user navigate the scene freely. FVV-Live virtual views are synthesized using Depth Image Based Rendering (DIBR), so depth maps need to be generated and transmitted. Real-time foreground segmenta-tion is a key component for this system, as it greatly reduces video bitrate by only transmitting foreground information.

The main aim of this project is to adapt and evaluate neural network architectures to exploit the nature of Free Viewpoint Video systems and obtain high quality foreground masks at an acceptable framerate to later be used in the video transmission and view synthesis of FVV-Live. For this task, a dataset will be generated by annotating images from FVV-like scenes. Results will be compared to the performance of the current real-time foreground segmentation module from FVV-Live.