Simultaneous depth completion and 3D object detection via deep learning for scene reconstruction in autonomous driving scenarios

Abstract

The use of learning-based techniques in the autonomous driving field has grown exponentially in the last years. In particular, great focus has been given to object detection and depth estimation algorithms. Recently, also depth completion has become a competitive research topic in self-driving scenarios. In this work, we propose a new exible two-stages framework based on both depth completion and 3D object detection which takes as input RGB-D data. The first stage, in which the depth completion process is carried out, generates a dense depth map of the analysed scene. This depth map is converted into a pseudo-LiDAR pointcloud which is fed into the second stage to perform the object detection. Between the two stages we introduce a new process which is in charge of estimating the reflectances of the points in the pseudo-LiDAR pointcloud. Three different reflectance estimation alorithms are proposed in this work and their performances are compared. Moreover, the entire framework is trained and evaluated using the prestigious KITTI dataset for autonomous driving applications. The results are finally compared with State-of-The-Art algorithms in the self-driving field. The entire project is implemented in Python programming language using the PyTorch library and some pre-coded networks to make easier the developing process.