M.Sc Thesis

M.Sc StudentWertheimer Alon Zsolt
SubjectTowards Applying Machine Learning in Volumetric 3D
DepartmentDepartment of Electrical and Computer Engineering
Supervisor ASSOCIATE PROF. Anat Levin


Light propagating through a non uniform medium scatters as it interacts with particles with different refractive properties such as cells in the tissue. In this work we aim to utilize this scattering process to learn a volumetric reconstruction of scattering parameters, in particular partical densities. We target microscopy applications were coherent speckle effect are an integral part of the imaging process. We focus on the popular confocal setting that utilize a scanning mechanism in which a pair of coherent focused light and viewing point creates high resolution imaging through different scattering mediums. We further extend this setting through our observations and methods for improved reconstruction of densities.

We argue that the key for successful learning is modeling realistic speckles in the training process, for which we build on the development of recent physically accurate speckle simulators. We demonstrate that learning from images created by computer graphics simulations of incoherent sources which doesn't incorporate such speckles fail to reconstruct the densities when coherent images used on such models. Furthermore, we demonstrate the straightforward usage of coherent images which are physically accurate achieving high resolution reconstruction of densities without any reduction of those fluctuations in the images. We also explore how to better incorporate speckle statistics such as the memory effect in the learning framework. Through our examination, we suggest new scanning approaches for enriching the learned models with additional features in the simulated images that yield further improved results in reconstruction.

Overall, this work contributes an analysis of multiple aspects of the network design including  the learning architecture, the training data and the desired input features. We hope this study will pave the road for future design of learning based imaging systems in this challenging domain.