IIMedGPT: Promoting Large Language Model Capabilities of Medical Tasks by Efficient Human Preference Alignment
Journal:
arXiv
Published Date:
Jan 6, 2025
Abstract
Recent researches of large language models(LLM), which is pre-trained on
massive general-purpose corpora, have achieved breakthroughs in responding
human queries. However, these methods face challenges including limited data
insufficiency to support extensive pre-training and can not align responses
with users' instructions. To address these issues, we introduce a medical
instruction dataset, CMedINS, containing six medical instructions derived from
actual medical tasks, which effectively fine-tunes LLM in conjunction with
other data. Subsequently, We launch our medical model, IIMedGPT, employing an
efficient preference alignment method, Direct preference Optimization(DPO). The
results show that our final model outperforms existing medical models in
medical dialogue.Datsets, Code and model checkpoints will be released upon
acceptance.
