Can LVLMs and Automatic Metrics Capture Underlying Preferences of Blind and Low-Vision Individuals for Navigational Aid?
Journal:
arXiv
Published Date:
Feb 15, 2025
Abstract
Vision is a primary means of how humans perceive the environment, but Blind
and Low-Vision (BLV) people need assistance understanding their surroundings,
especially in unfamiliar environments. The emergence of semantic-based systems
as assistance tools for BLV users has motivated many researchers to explore
responses from Large Vision-Language Models (LVLMs). However, it has yet been
studied preferences of BLV users on diverse types/styles of responses from
LVLMs, specifically for navigational aid. To fill this gap, we first construct
Eye4B dataset, consisting of human-validated 1.1k curated outdoor/indoor scenes
with 5-10 relevant requests per scene. Then, we conduct an in-depth user study
with eight BLV users to evaluate their preferences on six LVLMs from five
perspectives: Afraidness, Nonactionability, Sufficiency, and Conciseness.
Finally, we introduce Eye4B benchmark for evaluating alignment between widely
used model-based image-text metrics and our collected BLV preferences. Our work
can be set as a guideline for developing BLV-aware LVLMs towards a Barrier-Free
AI system.
