The Rise of Small Language Models in Healthcare: A Comprehensive Survey
Journal:
arXiv
Published Date:
Apr 23, 2025
Abstract
Despite substantial progress in healthcare applications driven by large
language models (LLMs), growing concerns around data privacy, and limited
resources; the small language models (SLMs) offer a scalable and clinically
viable solution for efficient performance in resource-constrained environments
for next-generation healthcare informatics. Our comprehensive survey presents a
taxonomic framework to identify and categorize them for healthcare
professionals and informaticians. The timeline of healthcare SLM contributions
establishes a foundational framework for analyzing models across three
dimensions: NLP tasks, stakeholder roles, and the continuum of care. We present
a taxonomic framework to identify the architectural foundations for building
models from scratch; adapting SLMs to clinical precision through prompting,
instruction fine-tuning, and reasoning; and accessibility and sustainability
through compression techniques. Our primary objective is to offer a
comprehensive survey for healthcare professionals, introducing recent
innovations in model optimization and equipping them with curated resources to
support future research and development in the field. Aiming to showcase the
groundbreaking advancements in SLMs for healthcare, we present a comprehensive
compilation of experimental results across widely studied NLP tasks in
healthcare to highlight the transformative potential of SLMs in healthcare. The
updated repository is available at Github
