|Ph.D Student||Alon Shtern|
|Subject||Shape Correspondence using Spectral Methods and Deep|
|Department||Department of Computer Science||Supervisor||Full Professor Kimmel Ron|
The interest in acquiring and analyzing three dimensional representations of the world is ever increasing, fueling a wide range of computer vision algorithms in the field of geometry processing. Spectral analysis has become key component in many applications involving non-rigid shapes modeled as two-dimensional surfaces, and recently, convolutional neural networks have shown remarkable success in a variety of computer vision tasks. In this dissertation, we designed a set of methods and tools that use these paradigms for applications such as shape correspondence, nonrigid deformations, and volumetric optical flow.
Finding the correspondences between pairs of shapes is a fundamental operation in the field of geometry processing. Measures of dissimilarity between surfaces have been found highly useful for this task. A powerful approach for measuring distance between two nonrigid shapes is to embed their two-dimensional surfaces into some common Euclidean space, defining the comparison task as a problem of rigid matching in that space. We introduce a novel spectral embedding, named the Spectral Gradient Fields Embedding, which exploits the local interaction between the eigenfunctions of the Laplace-Beltrami operator and the extrinsic geometry of the surface. The common embedding relies on the assumption that the Laplace-Beltrami eigenfunctions computed on each shape independently are compatible with each other. However, this assumption is often unrealistic. We address this issue by matching a small number of eigenfunctions, that are relatively stable, using a high order statistics (HOS) approach.
We also analyze the applicability of the spectral kernel distance as a measure of dissimilarity between surfaces, for solving the shape matching problem.
To align the spectral kernels, we developed the Iterative Closest Spectral Kernel Maps (ICSKM) algorithm. ICSKM extends the Iterative Closest Point (ICP) algorithm to the class of deformable shapes. Instead of aligning the shapes in the three-dimensional Euclidean domain, this method estimates the transformation that best fits the embeddings of the shapes into the spectral domain.
In case the data consists of an incomplete, occluded and disconnected parts of a shape the approach we took is to use a small set of representative natural poses. Using these few exemplar shapes, the method expresses an unseen appearance by a low-dimensional linear subspace, specified by a redundant dictionary of weighted vertex positions. The algorithm, called Fast Blended Transformations (FBT), finds the deformation that best fits the partial data, minimizing a nonlinear functional that regulates the example manifold in a smooth way, and detects the pointwise mapping between the partial shape and the reference shapes.
Volumetric optical flow is a different way to describe the matching in a three-dimensional dynamic scene. We designed a multi-scale optical flow based architecture for predicting the next frame of a sequence of volumetric images. The fully differentiable model consists of specific crafted modules that are trained on small patches in an unsupervised manner. The approach, called V-Flow, is useful for analyzing the temporal dynamics of three-dimensional images in applications that involve, for example, motion of viscous fluid substances or real magnetic resonance imaging (MRI).