Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinement
Journal:
arXiv
Published Date:
Dec 30, 2024
Abstract
Code generation is crucial in software engineering for automating the coding
process efficiently. While test-time computation methods show promise, they
suffer from high latency due to multiple computation rounds. To overcome this,
we introduce ThinkCoder, a framework that combines thorough exploration with
optimal refinement. The exploration phase diversifies the solution space by
searching for potential solutions, followed by a refinement phase that enhances
precision. This approach allows us to select the best solution through careful
consideration before taking action, avoiding excessive trial and error. To
further minimize test-time computation overhead, we introduce preference-driven
optimization with Reinforced Self-Training (ReST), which uses exploration
trajectories from ThinkCoder to guide LLM's evolution. By learning preferences,
this approach improves LLM's exploration efficiency, reducing computational
costs while maintaining accuracy. ThinkCoder boosts the performance of multiple
base LLMs, excelling on benchmarks like HumanEval and MBPP. Compared to SOTA
models, it improves Pass@1 by 1.5\% over MapCoder with just 21.7\% of the
computation cost. Against AgentCoder, ThinkCoder achieves a 0.6\% higher Pass@1
after 2 rounds, outperforming AgentCoder's 5 rounds. Additionally, ReST with
success trajectories enhances efficiency, allowing models like LLaMA2-7B to
achieve competitive results using only 20\% of the computational resources.
These results highlight the framework's effectiveness and scalability.
