Ph.D Thesis

Ph.D StudentProtter Matan
SubjectProcessing Images Sequences without Motion Estimation
DepartmentDepartment of Computer Science
Supervisor PROF. Michael Elad
Full Thesis textFull thesis text - English Version


Digital image restoration, a prominent signal processing field, focuses on improving the quality of images suffering from various degradation effects, such as noise and blur. Separating the true image content from degradation effects and restoring the degradation-free content usually requires modeling the image content.

Restoration of image sequences can obtain better results than restoring each image individually, provided the temporal redundancy is adequately used. Most such algorithms rely on estimating the motion between the frames for merging the data from them.

When motion patterns are very complex, motion estimation tends to be error-prone and inaccurate. Thus, algorithms relying on motion estimation tend to reduce to single-image processing in image areas containing complex motion patterns. Unfortunately, as most sequences exhibit mostly complex motion patterns, relying on motion estimation does not allow fully exploiting the benefits of multiple frames.

A relatively recent trend in image sequence denoising is circumventing motion estimation. Such a feat was made possible by the emergence of powerful image models, extended to also model image sequences. We propose a contribution along these lines, extending sparse and redundant representations modeling to the denoising of image sequences. State-of-the-art results are obtained, indeed proving that motion estimation can be avoided for this task.

Another restoration field relying on motion estimation is super-resolution. It suggests merging several different images of the same scene (i.e., containing motion), each offering a different sampling of it, into a high-quality image (or sequence) of the scene. Motion estimation is used for the merging. Even more so than in denoising, very high accuracy is required, which is not possible in the majority of sequences.

We offer two different approaches to super-resolution without explicit motion estimation, by relying on crude, probabilistic motion estimation instead. This alternative path is indeed able to successfully handle sequences previously considered outside the realm of super-resolution, due to their complicated motion patterns.

Finally, we revisit denoising, focusing on signals obeying the sparse and redundant modeling. It has been shown that averaging several sparse representations achieves better denoising than the sparsest representation alone. This has been explained by relating these two solutions to approximations of the MMSE and MAP estimators, respectively. In general, both estimators cannot be computed directly. We show that in the special case of a unitary dictionary, both estimators enjoy a closed-form formula, with the MMSE out-performing the MAP in this case as well.