Ph.D Student | Shtern Alon |
---|---|

Subject | Shape Correspondence using Spectral Methods and Deep Learning |

Department | Department of Computer Science |

Supervisor | Professor Ron Kimmel |

Full Thesis text |

The interest in acquiring and analyzing three dimensional representations of the world is ever increasing, fueling a wide range of computer vision algorithms in the field of geometry processing. Spectral analysis has become key component in many applications involving non-rigid shapes modeled as two-dimensional surfaces, and recently, convolutional neural networks have shown remarkable success in a variety of computer vision tasks. In this dissertation, we designed a set of methods and tools that use these paradigms for applications such as shape correspondence, nonrigid deformations, and volumetric optical flow.

Finding the correspondences between
pairs of shapes is a fundamental operation in the field of geometry processing.
Measures of dissimilarity between surfaces have been found highly useful for
this task. A powerful approach for measuring distance between two nonrigid
shapes is to embed their two-dimensional surfaces into some common Euclidean
space, defining the comparison task as a problem of rigid matching in that
space. We introduce a novel spectral embedding, named the *Spectral Gradient
Fields Embedding*, which exploits the local interaction between the
eigenfunctions of the Laplace-Beltrami operator and the extrinsic geometry of
the surface. The common embedding relies on the assumption that the
Laplace-Beltrami eigenfunctions computed on each shape independently are
compatible with each other. However, this assumption is often unrealistic. We
address this issue by matching a small number of eigenfunctions, that are
relatively stable, using a *high order statistics* (HOS) approach.

We also analyze the applicability of the spectral kernel distance as a measure of dissimilarity between surfaces, for solving the shape matching problem.

To align the spectral kernels, we
developed the *Iterative Closest Spectral Kernel Maps* (ICSKM) algorithm.
ICSKM extends the Iterative Closest Point (ICP) algorithm to the class of
deformable shapes. Instead of aligning the shapes in the three-dimensional
Euclidean domain, this method estimates the transformation that best fits the
embeddings of the shapes into the spectral domain.

In case the data consists of an
incomplete, occluded and disconnected parts of a shape the approach we took is
to use a small set of representative natural poses. Using these few exemplar
shapes, the method expresses an unseen appearance by a low-dimensional linear
subspace, specified by a redundant dictionary of weighted vertex positions. The
algorithm, called *Fast Blended Transformations* (FBT), finds the
deformation that best fits the partial data, minimizing a nonlinear functional
that regulates the example manifold in a smooth way, and detects the pointwise
mapping between the partial shape and the reference shapes.

Volumetric optical flow is a different way to describe the matching in a three-dimensional dynamic scene. We designed a multi-scale optical flow based architecture for predicting the next frame of a sequence of volumetric images. The fully differentiable model consists of specific crafted modules that are trained on small patches in an unsupervised manner. The approach, called V-Flow, is useful for analyzing the temporal dynamics of three-dimensional images in applications that involve, for example, motion of viscous fluid substances or real magnetic resonance imaging (MRI).