Bridging the Gap between User Intent and LLM: A Requirement Alignment Approach for Code Generation
Jia Li, Ruiqi Bai, Yangkang Luo, Yiran Zhang, Wentao Yang + 5 more
TLDR
REA-Coder improves LLM code generation by iteratively aligning user requirements, addressing the common issue of LLMs misunderstanding prompts.
Key contributions
- Identifies and addresses the critical flaw where LLMs often misunderstand user requirements for code generation.
- Introduces REA-Coder, an iterative requirement alignment approach for enhancing LLM code generation.
- REA-Coder first aligns requirements, then generates code, and verifies alignment, iterating until correctness.
- Outperforms advanced baselines by up to 30.25% across multiple LLMs and five programming benchmarks.
Why it matters
This paper addresses a fundamental limitation in LLM-based code generation: the assumption that LLMs correctly understand user requirements. By introducing an iterative requirement alignment process, it significantly boosts code generation accuracy. This advancement is crucial for developing more reliable and autonomous code generation tools.
Original Abstract
Code generation refers to automatically producing executable programs from user requirements. Recently, researchers have explored approaches to enhance the correctness of generated code with advanced large language models. Although achieving improvements, existing approaches focus on designing reasoning strategies or post-refinement methods to enhance code generation performance. Despite their differences, all these methods share a common assumption: the LLM can correctly understand the given requirement. However, this assumption does not always hold. To fill this gap, we propose REA-Coder, a requirement alignment approach to enhance the code generation performance of LLMs. REA-Coder involves first identifying the requirement content that does not align with LLMs and aligning the requirements. Then, based on the aligned requirements, LLMs generate code and further verify whether the generated code aligns with the requirements, iterating this process of requirement alignment and code generation until generating correct code or achieving the maximum number of iterations. Experimental results show that REA-Coder outperforms all advanced baselines on four LLMs across five programming benchmarks. Concretely, REA-Coder achieves average improvements of 7.93%, 30.25%, 26.75%, 8.59%, and 8.64% on the five benchmark datasets, demonstrating the effectiveness of requirement alignment for improving the code generation performance of LLMs.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.