M.Sc Thesis

M.Sc StudentPraisler Shachar
SubjectAdaptive LiDAR Sampling and Depth Completion using
Ensemble Variance
DepartmentDepartment of Electrical and Computer Engineering
Supervisor ASSOCIATE PROF. Guy Gilboa
Full Thesis textFull thesis text - English Version


Estimating an accurate depth map of a scene is  essential for navigation and collision avoidance of autonomous vehicles. The introduction of Light Detection And Ranging (LiDAR) sensors has allowed more accurate depth estimation, compared with methods based on stereo images and monocular RGB. Traditional LiDAR technology is based on mechanically rotating transceivers, inducing a fixed sampling pattern. Recently, new LiDAR designs are emerging, which allow programmable scanning, thus opening exciting new possibilities in adaptive sampling research.

In this research, we consider the problem of depth completion, with or without additional RGB image, where a prescribed number of depth points is given (sparse measurements) and the rest should be interpolated. We assume state-of-the-art interpolators are available and focus on the problem of optimal sparse sampling.

The algorithmic challenge is to choose pixel positions strategically and dynamically to maximally reduce the overall depth estimation error. The design is for both daytime and nighttime depth completion, intended for autonomous vehicles with a programmable LiDAR. Our method uses an ensemble of predictors to define a sampling probability over pixels. This probability is proportional to the variance of the predictions of ensemble members, thus highlighting pixels that are difficult to predict. By additionally proceeding in several prediction phases, we effectively reduce redundant sampling of similar pixels. Our ensemble-based method may be implemented using any depth-completion learning algorithm, such as a state-of-the-art neural network, treated as a black box. In particular, we also present a simple and effective random forest-based algorithm, and similarly use its internal ensemble in our design.

We conduct experiments on the KITTI dataset, using a given neural network algorithm and our random forest-based learner for implementing our method. The accuracy of both implementations exceeds the state of the art. Compared with a random or grid sampling patterns, our method allows a reduction by a factor of 4--10 in the number of measurements required to attain the same accuracy.