Achieving GPT-4o level performance in astronomy with a specialized 8B-parameter large language model.

Journal: Scientific reports
Published Date: Apr 21, 2025

Abstract

AstroSage-Llama-3.1-8B is a domain-specialized natural-language AI assistant tailored for research in astronomy, astrophysics, cosmology, and astronomical instrumentation. Trained on the complete collection of astronomy-related arXiv papers from 2007 to 2024 along with millions of synthetically-generated question-answer pairs and other astronomical literature, AstroSage-Llama-3.1-8B demonstrates remarkable proficiency on a wide range of questions. AstroSage-Llama-3.1-8B scores 80.9% on the AstroMLab-1 benchmark, greatly outperforming all models-proprietary and open-weight-in the 8-billion parameter class, and performing on par with GPT-4o. This achievement demonstrates the potential of domain specialization in AI, suggesting that focused training can yield capabilities exceeding those of much larger, general-purpose models. AstroSage-Llama-3.1-8B is freely available, enabling widespread access to advanced AI capabilities for astronomical education and research.

Authors

  • Tijmen de Haan
    International Center for Quantum-field Measurement Systems for Studies of the Universe and Particles (QUP-WPI), High Energy Accelerator Research Organization (KEK), Tsukuba, Ibaraki, Japan. tijmen.dehaan@gmail.com.
  • Yuan-Sen Ting
    Department of Astronomy, The Ohio State University, Columbus, OH, USA.
  • Tirthankar Ghosal
    National Center for Computational Sciences, Oak Ridge National Laboratory, Oak Ridge, TN, USA.
  • Tuan Dung Nguyen
    Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA, USA.
  • Alberto Accomazzi
    Center for Astrophysics, Harvard & Smithsonian, Cambridge, MA, USA.
  • Azton Wells
    Computational Science Division, Argonne National Laboratory, Lemont, IL, USA.
  • Nesar Ramachandra
    Computational Science Division, Argonne National Laboratory, Lemont, IL, USA.
  • Rui Pan
    Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Kowloon, Hong Kong.
  • Zechang Sun
    Department of Astronomy, Tsinghua University, Beijing, People's Republic of China.

Keywords

No keywords available for this article.

External Resources

Popular Topics
Recent Journals