טכניון מכון טכנולוגי לישראל
הטכניון מכון טכנולוגי לישראל - בית הספר ללימודי מוסמכים  
M.Sc Thesis
M.Sc StudentKaliroff Damian
SubjectSelf-Supervised Unconstrained Photo-Consistent Transform
for Improved Matching
DepartmentDepartment of Electrical Engineering
Supervisor Professor Guy Gilboa
Full Thesis textFull thesis text - English Version


Abstract

Image processing and computer vision tasks often benefit from representations, which are invariant to certain image changes.

Photo-consistency is a highly desired property, essential for tasks based on color and contrast cues, such as matching, registration and recognition. Traditionally, representations were designed in a model-based manner.

Lately, with the rise of deep learning, new data-driven algorithms are proposed to solve this problem.

In this research, we propose a new and completely data-driven approach for generating a photo-consistent image transform, thus creating an illumination invariant representation of an image.

We show that simple classical algorithms which operate in the transform domain become extremely resilient to illumination changes. This considerably improves matching accuracy, outperforming the use of state-of-the-art invariant representations as well as new matching methods based on deep features.

The transform is obtained by training a neural network, trained in a self-supervised manner with a specialized triplet loss, designed to emphasize actual scene changes while attenuating illumination changes.

The transform yields an illumination invariant representation, structured as an image map, which is highly flexible and can be easily used for various tasks. We point out that the utility of our method is not restricted to handling illumination invariance, and that it may be applied for generating representations which are invariant to additional types of nuisance, undesired, image variants.