Cog-TiPRO: Iterative Prompt Refinement with LLMs to Detect Cognitive Decline via Longitudinal Voice Assistant Commands
Journal:
arXiv
Published Date:
May 22, 2025
Abstract
Early detection of cognitive decline is crucial for enabling interventions
that can slow neurodegenerative disease progression. Traditional diagnostic
approaches rely on labor-intensive clinical assessments, which are impractical
for frequent monitoring. Our pilot study investigates voice assistant systems
(VAS) as non-invasive tools for detecting cognitive decline through
longitudinal analysis of speech patterns in voice commands. Over an 18-month
period, we collected voice commands from 35 older adults, with 15 participants
providing daily at-home VAS interactions. To address the challenges of
analyzing these short, unstructured and noisy commands, we propose Cog-TiPRO, a
framework that combines (1) LLM-driven iterative prompt refinement for
linguistic feature extraction, (2) HuBERT-based acoustic feature extraction,
and (3) transformer-based temporal modeling. Using iTransformer, our approach
achieves 73.80% accuracy and 72.67% F1-score in detecting MCI, outperforming
its baseline by 27.13%. Through our LLM approach, we identify linguistic
features that uniquely characterize everyday command usage patterns in
individuals experiencing cognitive decline.
