BrickCraft: Visuomotor Skill Composition with Situated Manual Guidance for Long-Horizon Interlocking Brick Assembly
Jichuan Yu, Bowei Li, Zhenran Tang, Guanxing Lu, Chuxiong Hu + 2 more
TLDR
BrickCraft is a compositional framework enabling robots to assemble complex interlocking brick structures by decomposing tasks into reusable, spatially guided skills.
Key contributions
- Decomposes complex brick assembly into reusable primitive skills using a relative formulation.
- Introduces 'situated manuals' for explicit spatial guidance, projecting intent onto robot observations.
- Chains spatially grounded visuomotor skills for long-horizon interlocking brick assembly tasks.
- Achieves strong compositional generalization to unseen structures from limited demonstrations.
Why it matters
This paper matters because it addresses a key challenge in robotics: long-horizon, complex assembly. BrickCraft's novel approach of decomposing tasks into reusable, spatially grounded skills significantly improves robotic brick assembly. This framework demonstrates strong generalization, paving the way for more versatile and autonomous robotic construction.
Original Abstract
Autonomous robotic assembly of interlocking bricks demands seamless integration of long-horizon task reasoning, spatial grounding, and fine-grained manipulation. This paper presents BrickCraft, a compositional framework designed for long-horizon and generalizable interlocking brick assembly. BrickCraft models the assembly process using a relative formulation, where each step is anchored to a reference brick within the partial structure, thereby decomposing complex tasks into a finite set of reusable primitive skills. BrickCraft bridges the gap between high-level assembly plans and physical execution through situated manuals, which provide explicit spatial guidance for learned visuomotor skills by projecting the assembly intent onto real-time robot observations. Finally, BrickCraft employs a compositional execution pipeline that chains these spatially grounded skills to accomplish long-horizon assembly tasks. Extensive experimental validations demonstrate that BrickCraft acquires proficient assembly skills from a limited set of demonstrations and exhibits strong compositional generalization to unseen structures. The project website is available at https://intelligent-control-lab.github.io/BrickCraft.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.