Interpretable Zero-shot Learning with Infinite Class Concepts
Journal:
arXiv
Published Date:
May 6, 2025
Abstract
Zero-shot learning (ZSL) aims to recognize unseen classes by aligning images
with intermediate class semantics, like human-annotated concepts or class
definitions. An emerging alternative leverages Large-scale Language Models
(LLMs) to automatically generate class documents. However, these methods often
face challenges with transparency in the classification process and may suffer
from the notorious hallucination problem in LLMs, resulting in non-visual class
semantics. This paper redefines class semantics in ZSL with a focus on
transferability and discriminability, introducing a novel framework called
Zero-shot Learning with Infinite Class Concepts (InfZSL). Our approach
leverages the powerful capabilities of LLMs to dynamically generate an
unlimited array of phrase-level class concepts. To address the hallucination
challenge, we introduce an entropy-based scoring process that incorporates a
``goodness" concept selection mechanism, ensuring that only the most
transferable and discriminative concepts are selected. Our InfZSL framework not
only demonstrates significant improvements on three popular benchmark datasets
but also generates highly interpretable, image-grounded concepts. Code will be
released upon acceptance.
