טכניון מכון טכנולוגי לישראל
הטכניון מכון טכנולוגי לישראל - בית הספר ללימודי מוסמכים  
M.Sc Thesis
M.Sc StudentShay Perera
SubjectImage Memorability - Insights and Prediction
DepartmentDepartment of Electrical Engineering
Supervisors Full Professor Tal Ayellet
Professor Zelnik-Manor Lihi
Full Thesis textFull thesis text - English Version


Abstract

Prediction of image memorability by autonomous algorithm has been recently addresses by the research community. Though, the relation between image classification and memorability prediction algorithms haven’t been fully explored. Here, we further delve into the relation between networks for image classification and memorability prediction.



Our study gives rise to three main insights: (i) It suffices to train a regression layer on

top of a CNN for object & scene recognition to achieve on par results with those attained by re-training the entire CNN for memorability prediction. (ii) As object classification CNNs improve, so does image memorability prediction. (iii) Scene classification plays a bigger role in memorability prediction than object classification.


Based on these observations we propose the MemBoost algorithm that achieve state-of-the-art results for memorability prediction and actually reaches the limit of human performance, on the largest existing dataset for image memorability.


Since we achieved what seems to be astonishing results, we further looked deeper

into the factors that impact human performance. Via thorough empirical analysis we

have found out that some of these factors have been overlooked in the existing datasets. We show that changing some of the design decisions in the data collection could lead to data that better represents human memorability. The main conclusion from this study is that reaching human performance does not mean that memorability prediction has been solved. We further provide a list of guidelines for building future datasets.