M.Sc Thesis

M.Sc StudentShwartz Sarit
SubjectBlind Separation of High Dimensional Sources
DepartmentDepartment of Electrical and Computers Engineering
Supervisors PROF. Yoav Schechner
DR. Michael Zibulevsky


Blind source separation (BSS) is part of a wide range of scientific fields such as acoustics, image processing, medical imaging, and computer vision. In this work we focus on separation of high dimensional mixtures of sources. By high dimensional, we refer to mixtures that have O(101) to O(103) mixing coefficients. This includes pointwise (instantaneous) mixtures of more than two sources, as well as mixtures of spatially blurred sources that have more than one mixing coefficient per source. Our separation criterion is mutual information (MI) of signals. MI is a natural criterion for statistical dependency and is thus commonly used in BSS algorithms.

When dealing with high dimensional mixtures of sources, the separation optimization becomes computationally complex. In such scenarios, existing BSS algorithms become intractable. We present two algorithms that overcome dimensionality problems. The first algorithm addresses the special case of pointwise mixtures. It involves a numerically efficient algorithm for non-parametric kernel (Parzen windows) entropy estimation based on convolutions. In particular, we present an accurate and efficient method for calculating the gradient of the mutual information. This accelerates the optimization needed for source separation and enhances its robustness.

The second algorithm addresses convolutive mixtures of images. Such image mixtures are common in tomography and in reflections. We show that the large optimization problem associated with convolution can be decomposed into several small and simple problems. The factorization is done by a short time Fourier transform (STFT), where signal separation may be done in each frequency channel independently. In addition, we show that the optimization at each frequency band can be solved efficiently by exploiting en a-priori knowledge about the sparsity of the sub-band images and a parametric model of the convolutive process. Moreover, by exploiting the parametric model of the convolutive blur process, we can overcome ambiguities inherent in MI minimization.