Test-Oriented Programming: rethinking coding for the GenAI era
TLDR
Test-Oriented Programming (TOP) proposes a new paradigm where LLMs generate production code, and developers focus on verifying AI-generated test cases.
Key contributions
- Introduces Test-Oriented Programming (TOP), a new paradigm for GenAI-assisted software development.
- Developers verify AI-generated test code from natural language specifications, delegating production code to LLMs.
- Presents a proof-of-concept tool demonstrating TOP's feasibility using two different LLMs.
- Reports promising results and outlines challenges for applying TOP to real-world projects.
Why it matters
This paper introduces a novel approach to software development, shifting the developer's role from writing production code to verifying AI-generated tests. It leverages LLMs for a higher level of abstraction, potentially streamlining the coding process in the GenAI era. This could significantly impact how software is built, making development more efficient.
Original Abstract
Large language models (LLMs) have shown astonishing capability of generating software code, leading to its use to support developers in programming. Proposed tools have relied either on assistants for improved auto-complete or multi-agents, in which different model instances are orchestrated to perform parts of a problem to reach a complete solution. We argue that LLMs can enable a higher-level of abstraction, a new paradigm we called Test-Oriented Programming (TOP). Within this paradigm, developers only have to check test code generated based on natural language specifications, rather than focusing on production code, which could be delegated to the LLMs. To evaluate the feasibility of this proposal, we developed a proof-of-concept tool and used it to generate a small command-line program employing two different LLMs. We obtained promising results and identified challenges for the use of this paradigm for real projects.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.