The mutual exclusivity bias of bilingual visually grounded speech models
Journal:
arXiv
Published Date:
Jun 4, 2025
Abstract
Mutual exclusivity (ME) is a strategy where a novel word is associated with a
novel object rather than a familiar one, facilitating language learning in
children. Recent work has found an ME bias in a visually grounded speech (VGS)
model trained on English speech with paired images. But ME has also been
studied in bilingual children, who may employ it less due to cross-lingual
ambiguity. We explore this pattern computationally using bilingual VGS models
trained on combinations of English, French, and Dutch. We find that bilingual
models generally exhibit a weaker ME bias than monolingual models, though
exceptions exist. Analyses show that the combined visual embeddings of
bilingual models have a smaller variance for familiar data, partly explaining
the increase in confusion between novel and familiar concepts. We also provide
new insights into why the ME bias exists in VGS models in the first place. Code
and data: https://github.com/danoneata/me-vgs
