DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Training
Journal:
arXiv
Published Date:
Apr 24, 2025
Abstract
Although large language models (LLMs) have recently achieved remarkable
performance on various complex reasoning benchmarks, the academic community
still lacks an in-depth understanding of base model training processes and data
quality. To address this, we construct a large-scale, difficulty-graded
reasoning dataset containing approximately 3.34 million unique queries of
varying difficulty levels and about 40 million distilled responses generated by
multiple models over several passes. Leveraging pass rate and Coefficient of
Variation (CV), we precisely select the most valuable training data to enhance
reasoning capability. Notably, we observe a training pattern shift, indicating
that reasoning-focused training based on base models requires higher learning
rates for effective training. Using this carefully selected data, we
significantly improve the reasoning capabilities of the base model, achieving a
pass rate of 79.2\% on the AIME2024 mathematical reasoning benchmark. This
result surpasses most current distilled models and closely approaches
state-of-the-art performance. We provide detailed descriptions of our data
processing, difficulty assessment, and training methodology, and have publicly
released all datasets and methods to promote rapid progress in open-source
long-reasoning LLMs. The dataset is available at:
\href{https://huggingface.co/datasets/a-m-team/AM-DeepSeek-Distilled-40M}{https://huggingface.co/datasets/a-m-team/AM-DeepSeek-Distilled-40M}
