M.Sc Thesis


M.Sc StudentIsaacs Or
SubjectBoundaries and Region Representation Fusion
DepartmentDepartment of Computer Science
Supervisor PROF. Michael Lindenbaum
Full Thesis textFull thesis text - English Version


Abstract

Generic (non-semantic) image segmentation is an important task in computer vision that is difficult to approach with modern deep learning methods, due to a lack of prior knowledge of the input images' composition and content. We propose a generic segmentation algorithm that uses two methods of image analysis that rely on deep learning: edge detection, which computes an edge-likelihood map for each image pixel, and region representation, which defines a high-dimensional feature vector for each image pixel, such that pixels belonging to the same objects have feature vectors that are spatially close in Euclidean distance.

The proposed Boundaries and Region Representation Fusion (BRRF) algorithm iteratively combines image segments to create a hierarchy of segmentations. The choice of which segments to combine is done by a trained classifier. We use the described deep learning methods along basic geometric and color properties of the segments as a source of features in our classifier. We also employ an additional step in the algorithm that uses context in the region representation between the segments and their surroundings to improve our selection process of the pair of segments to combine.

Our BRRF algorithm attained and passed the state-of-the-art results for most of the quality measures for generic segmentation on two established generic image segmentation datasets: BSDS500 and Pascal Context. We further analyze the features used in our classifier and the additional processes we employ to measure their effect on our algorithm.