Ph.D Thesis

Ph.D StudentSheinin Mark
SubjectLeveraging Implicit Structure in Artifical Illumination for
Computer Vision
DepartmentDepartment of Electrical and Computer Engineering
Supervisor PROF. Yoav Schechner
Full Thesis textFull thesis text - English Version


Artificial light sources, such as projectors and lasers, are extensively used in computer vision to recover 3D shape, reflectance, and additional scene properties. Mostly, these artificial lights are engineered to have task-specific spatiotemporal structures leveraged to extract scene information. We show that physical effects involved in the creation and propagation of light from artificial light sources to the camera can be exploited to reveal implicit structures useful in a plurality of computer vision tasks. We demonstrate several approaches that leverage physical effects to yield implicit structure.

(a) Accounting for light scattering underwater achieves superior imaging by optimizing camera and light trajectories in space while using unstructured illumination.

(b) Power grid alternating current (AC) causes bulb to flicker. We passively sense this flicker revealing new scene information including the types of bulbs in the scene, the phases of the electric grid up to city scale, and the light transport matrix. This yields unmixing of reflections and semi-reflections, nocturnal high dynamic range, and scene rendering with bulbs not observed during acquisition. Moreover, we provide methods that enable capturing scene flicker using almost any off-the-shelf camera, including smartphones.

(c) Wide-aperture imaging conditions yield significant defocus blur. By capturing a focal-sweep image while simultaneously projecting different temporal textures, the defocus blur is exploited to yield an image having an integrated texture which encodes object depth. Thus, object shape is recovered by decoding the single captured frame.