טכניון מכון טכנולוגי לישראל
הטכניון מכון טכנולוגי לישראל - בית הספר ללימודי מוסמכים  
M.Sc Thesis
M.Sc StudentRichardson Elad
SubjectLearning to Reconstruct Face Geometries
DepartmentDepartment of Computer Science
Supervisor Professor Ron Kimmel
Full Thesis textFull thesis text - English Version


Abstract

Facial geometry reconstruction is a long researched problem with numerous applications in computer vision and graphics. While the problem might appear simple at a first glance, it has proved to be challenging due to the extensive variation human faces exhibit when considering expressions, poses, textures, and intrinsic geometry. Many approaches tackle this complexity by using multiple sources as input, thus introducing more constraints to the problem. Still, the ill-posed scenario of extracting the facial geometry from a single image remains a difficult problem, where current methods can generally provide only coarse estimations of the facial geometry. In this thesis, we propose to leverage the power of convolutional neural networks (CNNs) to produce a highly detailed face reconstruction from a single image. To that end, we introduce a CNN framework which derives the geometry in a coarse-to-fine fashion. The proposed architecture is composed of two blocks, a network that recovers the coarse facial geometry (CoarseNet), followed by a CNN that refines the facial features of that geometry (FineNet), where the networks are connected using a novel rendering layer. While deep neural networks have proved themselves over complex computer vision tasks, they are heavily dependent on the data, usually requiring an extremely large dataset of annotated examples. Unfortunately, unlike object recognition and detection problems, there are no suitable datasets for training CNNs to perform face geometry reconstruction. To overcome that, we turn to our understanding of image formation and facial geometry models to propose a specialized training regime that uses only synthetic images and unlabeled in-the-wild

facial images. The accuracy and robustness of the proposed model are demonstrated by both qualitative and quantitative evaluation tests.