Towards deployment-centric multimodal AI beyond vision and language
Journal:
arXiv
Published Date:
Apr 4, 2025
Abstract
Multimodal artificial intelligence (AI) integrates diverse types of data via
machine learning to improve understanding, prediction, and decision-making
across disciplines such as healthcare, science, and engineering. However, most
multimodal AI advances focus on models for vision and language data, while
their deployability remains a key challenge. We advocate a deployment-centric
workflow that incorporates deployment constraints early to reduce the
likelihood of undeployable solutions, complementing data-centric and
model-centric approaches. We also emphasise deeper integration across multiple
levels of multimodality and multidisciplinary collaboration to significantly
broaden the research scope beyond vision and language. To facilitate this
approach, we identify common multimodal-AI-specific challenges shared across
disciplines and examine three real-world use cases: pandemic response,
self-driving car design, and climate change adaptation, drawing expertise from
healthcare, social science, engineering, science, sustainability, and finance.
By fostering multidisciplinary dialogue and open research practices, our
community can accelerate deployment-centric development for broad societal
impact.
