PICKLES: a Natural Language Framework for Requirement Specification and Model-Based Testing
María Belén Rodríguez, Petra van den Bos
TLDR
PICKLES combines Model-Based Testing and Behaviour-Driven Development for human-readable test specifications and automatic, high-coverage test generation.
Key contributions
- Introduces PICKLES, a natural language framework extending Gherkin for human-readable test scenarios.
- Enables bi-directional translation from PICKLES scenarios to formal models for automatic test generation.
- Combines scenarios into a 'master model' allowing standard MBT algorithms to derive test cases.
- Achieves significantly higher test coverage than BDD in an industrial traffic management case study.
Why it matters
This paper addresses the challenge of creating clear, human-readable test specifications while leveraging the power of automatic test generation. By integrating BDD's clarity with MBT's rigor, PICKLES offers a practical solution for improving software quality. Its ability to achieve higher test coverage from existing scenarios makes it a valuable tool for industry.
Original Abstract
This paper combines methods from the fields of Model-Based Testing (MBT) and Behaviour-Driven Development (BDD) to define a testing approach with human-readable specifications and test cases, as in BDD, while using the modelling techniques and automatic test generation algorithms from MBT. We introduce PICKLES, a Precise Input and Control-flow Keyword-based Language for tEst Scenarios; an extension of Gherkin-style BDD scenarios, specified in structured natural language. We provide a bi-directional translation from Pickles scenarios to formal models, where both specifications and tests are human-readable, and a method to obtain a so-called master model combining all translated scenarios. Standard MBT algorithms can then be applied to automatically derive test cases from it. We implement a prototype of the translation and composition steps, which we use on an industrial case study: a software component from a traffic management system. With it, we illustrate the pipeline and show how Pickles can achieve significantly higher coverage with respect to BDD from the same set of scenarios.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.