Conjecture and Inquiry: Quantifying Software Performance Requirements via Interactive Retrieval-Augmented Preference Elicitation
TLDR
IRAP quantifies natural language software performance requirements into mathematical functions using interactive, knowledge-driven preference elicitation.
Key contributions
- Formalizes the problem of quantifying natural language software performance requirements.
- Introduces IRAP, an interactive retrieval-augmented preference elicitation approach for quantification.
- IRAP leverages problem-specific knowledge to guide interaction and reduce cognitive overhead.
- Demonstrates up to 40x performance improvements over 10 SOTA methods in few interactions.
Why it matters
Quantifying natural language software performance requirements is a critical, unaddressed challenge. IRAP offers a novel solution, translating vague requirements into precise mathematical functions. This advances software engineering by enabling more accurate, efficient performance specification, with superior results.
Original Abstract
Since software performance requirements are documented in natural language, quantifying them into mathematical forms is essential for software engineering. Yet, the vagueness in performance requirements and uncertainty of human cognition have caused highly uncertain ambiguity in the interpretations, rendering their automated quantification an unaddressed and challenging problem. In this paper, we formalize the problem and propose IRAP, an approach that quantifies performance requirements into mathematical functions via interactive retrieval-augmented preference elicitation. IRAP differs from the others in that it explicitly derives from problem-specific knowledge to retrieve and reason the preferences, which also guides the progressive interaction with stakeholders, while reducing the cognitive overhead. Experiment results against 10 state-of-the-art methods on four real-world datasets demonstrate the superiority of IRAP on all cases with up to 40x improvements under as few as five rounds of interactions.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.