M.Sc Thesis

M.Sc StudentBrifman Alon
SubjectNovel Image and Video Super-Resolution Relying on
Denoising Algorithms
DepartmentDepartment of Computer Science
Supervisor PROF. Michael Elad
Full Thesis textFull thesis text - English Version


Single Image Super-Resolution (SISR) aims to recover a high-resolution image from a given low resolution version of it (the given image is assumed to be a blurred, down-sampled and noisy version of the original image). Video Super Resolution (VSR) targets series of given images, aiming to fuse them to create a higher resolution outcome. Although SISR and VSR seem to have a lot in common, as only the input domain changes between the two, most SISR algorithms do not have a simple extension to VSR, apart for the trivial option of applying the SISR for each frame separately. The VSR task is considered to be a more challenging inverse problem, mainly due to its reliance on a sub-pixel accurate motion estimation, which has no parallel in SISR. Another complication is the dynamics of the video, often addressed by simply generating a single frame instead of a complete output sequence.

In this work we suggest an appealing alternative to the above that leads to a simple and robust super-resolution framework that can be applied to single images and easily extended to video. Our work relies on the observation that denoising of images and videos is well-managed and very effectively treated by a variety of methods. We exploit the Plug-and-Play framework and the Regularization-by-Denoising (RED) approach that extends it, and show how to use these denoisers in order to handle the SISR and the VSR problems. This way, we benefit from the effectiveness and efficiency of existing image/video denoising algorithms, while solving much more challenging problems. We test our SISR framework against the NCSR algorithm that solves for denoising and super-resolution separately, and show how its denoiser can be used in order to perform highly effective super-resolution. Then we turn to video, harnessing the VBM3D video denoiser, we compare our results to the ones obtained by the DeepSR and 3DSKR algorithms, showing a tendency to a higher-quality output and a much faster processing.