M.Sc Thesis


M.Sc StudentMataev Gary
SubjectDeep Image Prior Powered by RED
DepartmentDepartment of Computer Science
Supervisor PROF. Michael Elad
Full Thesis textFull thesis text - English Version


Abstract

Inverse problems in imaging are extensively studied, with a variety of strategies, tools, and theory that have been accumulated over the years. Recently, this field has been immensely influenced by the emergence of deep-learning techniques. One such contribution, which is the focus of this thesis, is the Deep Image Prior (DIP) work by Ulyanov, Vedaldi, and Lempitsky (2018). DIP offers a new approach towards the regularization of inverse problems, obtained by forcing the recovered image to be synthesized from a given deep architecture. While DIP has been shown to be quite an effective unsupervised approach, its results still fall short when compared to state-of-the-art alternatives.


In this work, we aim to boost DIP by adding an explicit prior, which enriches the overall regularization effect in order to lead to better-recovered images. More specifically, we propose to bring-in the concept of Regularization by Denoising (RED), which leverages existing denoisers for regularizing inverse problems. Our work shows how the two (DIP and RED) can be merged into a highly effective unsupervised recovery process while avoiding the need to differentiate the chosen denoiser, and leading to very effective results, demonstrated for several tested problems.


This thesis ends with an additional chapter in which we explore a few more ideas around DIP. First, we present a way to replace the learned network with an equivalent linear operator. This enables using algebraic tools for improving the network outcome. Such improvement is attained by the extraction of the leading singular-vectors of this linear system and then projecting the noisy image onto these singular-vectors.

Second, we explore the relation between DIP and dictionary learning methods, aiming to find a similarity of the processing between the two. On these two fronts, we report very limited success, suggesting that further work may be necessary.