|M.Sc Student||Perera Shay|
|Subject||Image Memorability - Insights and Prediction|
|Department||Department of Electrical Engineering||Supervisors||Professor Ayellet Tal|
|Professor Lihi Zelnik-Manor|
|Full Thesis text|
Prediction of image memorability by autonomous algorithm has been recently addresses by the research community. Though, the relation between image classification and memorability prediction algorithms haven’t been fully explored. Here, we further delve into the relation between networks for image classification and memorability prediction.
Our study gives rise to three main insights: (i) It suffices to train a regression layer on
top of a CNN for object & scene recognition to achieve on par results with those attained by re-training the entire CNN for memorability prediction. (ii) As object classification CNNs improve, so does image memorability prediction. (iii) Scene classification plays a bigger role in memorability prediction than object classification.
Based on these observations we propose the MemBoost algorithm that achieve state-of-the-art results for memorability prediction and actually reaches the limit of human performance, on the largest existing dataset for image memorability.
Since we achieved what seems to be astonishing results, we further looked deeper
into the factors that impact human performance. Via thorough empirical analysis we
have found out that some of these factors have been overlooked in the existing datasets. We show that changing some of the design decisions in the data collection could lead to data that better represents human memorability. The main conclusion from this study is that reaching human performance does not mean that memorability prediction has been solved. We further provide a list of guidelines for building future datasets.