Object recognition in real-world image scenes is still an open problem. With the growing number of classes, the similarity structures between them become complex and the distinction between classes blurs, which makes the classification problem particularly challenging. Standard N-way discrete classifiers treat all classes as disconnected and unrelated, and therefore unable to learn from their semantic relationships.

In this work, we present a hierarchical inter-class relationship model and train it using a newly proposed probability-based loss function, which we call soft-NLL. This loss function gives a probability weight according to the inter-class taxonomy graph distances. Our hierarchical model provides significantly better semantic generalization ability compared to a regular N-way classifier.

We further proposed an algorithm where given a probabilistic classification model it

can return the input corresponding super-group based on classes hierarchy without any further learning. We deploy it in two scenarios in which super-group retrieval can be useful. The first one, selective classification, deals with the problem of low-confidence classification, wherein a model is unable to make a successful exact classification. In this case, our algorithm returns a corresponding closest super-class prediction. In the second scenario, the proposed algorithm is used for the zero-shot learning problem. In this case, given a novel input, the algorithm returns its hierarchically related group, rather than generating a true unseen group. Extensive experiments with the two scenarios show that our proposed hierarchical model yields more accurate and meaningful super-class predictions compared to a regular N-way classifier because of its significantly better semantic generalization ability.