Ph.D Student | Matan Sela |
---|---|

Subject | Shape Synthesis and Analysis by Deformable Models |

Department | Department of Computer Science |

Supervisor | Full Professor Kimmel Ron |

The surface separating the interior of a given object or
body from its exterior defines a shape.

A
key challenge in computer vision is to recover the shape from missing, partial,
sparse or noisy observations such as images, depth maps or position
sensors.

Although in theory, surfaces can be arbitrarily embedded in space, the
underlying structure of deformable shapes, such as the human face, is often
driven by just a few degrees of freedom.

For example, it has been shown that the space of the
coarse geometric structure of faces can be well represented in a low
dimensional Euclidean space.

Similarly, the motion of articulated objects, such as the human body, is
controlled by a small set of number of parameters controlling the skeletal
structure of connections between limbs and bones. The goal of this
research is to analyze and synthesize shapes based on low dimensional,
learnable, and axiomatic deformable models.

We begin by proposing a novel method for recovering the coarse structure of a
face from a single image. We demonstrate that training a convolutional neural
network with synthetic images drawn from a statistical deformable model
allows robust reconstructions under large pose, lighting and expression
variations. For recovering subtle structures, such as wrinkles, we devise a
deep deformable model guided by a photometric (axiomatic) image formation
model. In this model, the coarse structure is still restricted to a low
dimensional deformable model. Next, we relieve this constraint and train an
image to image mapping network with samples from the deformable model.
Surprisingly, the network was able to reconstruct unique geometries that cannot
be represented by the model.

Along a similar line of thought, we propose a multi-linear blend skinning
framework for constructing a low dimensional deformation model of articulated
objects from a few isometries. This model allows to effectively
predict a plausible configuration of the shape from a sparse set of
constraints, as well as completing and registering the shape to a
partial observation. Finally, we define a numerical scheme for
computing and optimizing the L1 norm of functions defined over deformable
triangulated models. To handle a Steiffel manifold constraint, we propose
an iteratively reweighted least squares algorithm with an additional projection
step based on the Gershgorin Circle theorem. The framework allows to
efficiently optimize L1 regularized objective functions on the surface
even under non-convex constraints.