SpatialPrompt: XR-Based Spatial Intent Expression as Executable Constraints for AI Generative 3D Design
Yichen Andy Yu, Wanru Li, Qiaoran Wang, Jymon Ross, Gavin Johnson + 2 more
TLDR
SpatialPrompt is an XR system enabling controllable 3D design by converting spatial sketches and voice prompts into executable AI constraints.
Key contributions
- Transforms XR spatial sketches into executable constraints for controllable 3D generation.
- Integrates 3D pen drawing with voice prompts for semantic and stylistic design intent.
- Facilitates iterative refinement and synchronous co-creation in shared XR environments.
- Implemented on Apple Vision Pro, showing intuitive workflow and shared understanding.
Why it matters
This paper introduces a novel XR approach to 3D design, making AI generation more controllable and collaborative. It simplifies complex 3D modeling by allowing intuitive spatial and voice input, enhancing shared understanding in co-creation.
Original Abstract
We present SpatialPrompt, an Extended Reality(XR) system that turns spatial sketches into executable constraints for controllable 3D generation. Users draw rough structures with a 3D pen and add voice prompts for semantic and stylistic intent. The system supports iterative refinement and synchronous co-creation in shared space with color-coded contributions. Implemented on Apple Vision Pro with Logitech Muse and Meshy, a heuristic evaluation suggests that the workflow is intuitive and supports shared understanding in collaborative creation, while revealing needs for faster generation and clearer feedback.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.