M.Sc Thesis

M.Sc StudentFonaryov Mark
SubjectIn Search of the Minimal Recognizable Patch
DepartmentDepartment of Autonomous Systems and Robotics
Supervisor PROF. Michael Lindenbaum


In contrast to the human recognition processes, which can rely on small and partially visible object to successfully recognize it, performance of neural networks quickly deteriorates when objects are partially occluded or cropped. This raises a natural question: What is the minimal image part (or parts) that suffice for recognizing an object? In this work, we consider a special practical version of this question: what is the minimal size of a square sub-image (patch) that is sufficient for recognition using a convolutional neural network? We consider two question: 1. Globally minimal patch - Here we look for a patch of minimal size that provides correct (and best) categorization. We denote it Minimally recognizable patch, or MRP. 2. Locally minimal patch - Here we look for a patch that suffices for correct categorization, but that its contained sub-patches, do not. This criterion follows the Minimally recognizable configuration (MIRC) specified by Assif, Fetaya, Harari and Ulman in a human perception study. Here we specify it computationally and correspondingly denote it cMIRC. To find the minimal recognizable patches we design a special neural architecture that identifies the most informative patch and classifies the image based on the information within it. As expected, the minimal recognizable patches we found differ between and within categories and increase in size for higher required accuracy. A particularly surprising finding of the MIRC paper, which also motivated this study, is that human recognition accuracy drops sharply and significantly with patch size, exactly for the size separating MIRCs and their sub-patches. Interestingly, and in contrast to previous studies, we found similar sharp changes for MRPs and cMIRCs.