Text-to-CAD Evaluation with CADTests
Dimitrios Mallis, Marco Wang, Ahmet Serdar Karadeniz, Elisa Ricci, Anis Kacem + 1 more
TLDR
Introduces CADTestBench, the first test-based benchmark using CADTests for evaluating and guiding Text-to-CAD model generation.
Key contributions
- Proposes CADTestBench, the first test-based benchmark for Text-to-CAD evaluation.
- Introduces CADTests, executable software tests verifying CAD model geometric/topological requirements.
- Benchmarks recent Text-to-CAD methods using the new CADTestBench framework.
- Demonstrates CADTests can guide generation, yielding baselines that surpass current methods.
Why it matters
Text-to-CAD evaluation is a significant challenge with little prior work. This paper provides a much-needed automated testing framework to accurately assess and even guide the generation of CAD models. This can substantially accelerate design workflows.
Original Abstract
Text-to-CAD has recently emerged as an important task with the potential to substantially accelerate design workflows. Despite its significance, there has been surprisingly little work on Text-to-CAD evaluation, and assessing CAD model generation performance remains a considerable challenge. In this work, we introduce a new evaluation perspective for Text-to-CAD based on automated testing. We propose CADTestBench, the first test-based benchmark for Text-to-CAD, based on CADTests, executable software tests that verify whether a generated CAD model satisfies the geometric and topological requirements of the input prompt. Using CADTestBench, we conduct comprehensive benchmarking of recent Text-to-CAD methods and further demonstrate that CADTests can also guide CAD model generation, yielding simple baselines that surpass performance of current methods. CADTestBench code and data are available at GitHub and Hugging Face dataset.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.