A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Law
Journal:
arXiv
Published Date:
May 5, 2025
Abstract
This survey explores recent advancements in reasoning large language models
(LLMs) designed to mimic "slow thinking" - a reasoning process inspired by
human cognition, as described in Kahneman's Thinking, Fast and Slow. These
models, like OpenAI's o1, focus on scaling computational resources dynamically
during complex tasks, such as math reasoning, visual reasoning, medical
diagnosis, and multi-agent debates. We present the development of reasoning
LLMs and list their key technologies. By synthesizing over 100 studies, it
charts a path toward LLMs that combine human-like deep thinking with scalable
efficiency for reasoning. The review breaks down methods into three categories:
(1) test-time scaling dynamically adjusts computation based on task complexity
via search and sampling, dynamic verification; (2) reinforced learning refines
decision-making through iterative improvement leveraging policy networks,
reward models, and self-evolution strategies; and (3) slow-thinking frameworks
(e.g., long CoT, hierarchical processes) that structure problem-solving with
manageable steps. The survey highlights the challenges and further directions
of this domain. Understanding and advancing the reasoning abilities of LLMs is
crucial for unlocking their full potential in real-world applications, from
scientific discovery to decision support systems.
