טכניון מכון טכנולוגי לישראל
הטכניון מכון טכנולוגי לישראל - בית הספר ללימודי מוסמכים  
M.Sc Thesis
M.Sc StudentBridger Dov
SubjectSolving Jigsaw Puzzles with Eroded Boundaries using
GAN Inpainting
DepartmentDepartment of Computer Science
Supervisor Professor Ayellet Tal
Full Thesis textFull thesis text - English Version


Abstract

Jigsaw puzzle solving is an intriguing problem, having a wide variety of applications in archaeology, biology, document restoration and others. This work focuses on a specific variant of the problem - solving puzzles with eroded boundaries. Such erosion makes the problem extremely difficult, since most existing solvers utilize solely the information at the boundaries. Nevertheless, this variant is important since erosion and missing data often occur at the boundaries. This work introduces a novel approach to solve the problem. The key idea is to inpaint the eroded boundaries between puzzle pieces and later leverage the quality of the inpainted area to classify a pair of pieces as "neighbors or not". Inpainting is performed using a Generative Adversarial Network (GAN). An interesting feature of our architecture is that the same GAN discriminator is used for both inpainting and classification; training of the second task is simply a continuation of the training of the first, beginning from the point it left off. We show that a naive approach of training a fresh classifier or learning implicit end to end classification without a prior inpainting stage all together will be substantially outperformed by our integrative approach. Moreover, we show that our puzzle solving results surpass those of the state-of-the-art.