ShapeGen: Robotic Data Generation for Category-Level Manipulation
Yirui Wang, Xiuwei Xu, Angyuan Ma, Bingyao Yu, Jie Zhou + 1 more
TLDR
ShapeGen generates diverse robotic manipulation data in 3D to improve category-level generalizability without requiring a simulator.
Key contributions
- Generates shape-diversified robotic manipulation data for category-level policies without a simulator.
- Curates a "Shape Library" by learning spatial warpings between functionally corresponding 3D points.
- Uses the library for "Function-Aware Generation" to create new, plausible demonstrations with minimal human input.
- Improves robotic policies' generalizability to diverse object shapes within a category in real-world tests.
Why it matters
Robotic manipulation policies struggle with the geometric diversity of real-world objects, and collecting enough varied training data is a significant bottleneck. ShapeGen provides an efficient, simulator-free method to automatically generate this crucial shape-diversified data. This advancement enables more robust and generalizable robots capable of interacting with a wider range of objects in unstructured environments.
Original Abstract
Manipulation policies deployed in uncontrolled real-world scenarios are faced with great in-category geometric diversity of everyday objects. In order to function robustly under such variations, policies need to work in a category-level manner, i.e. knowing how to interact with any object in a certain category, instead of only a specific one seen during training. This in-category generalizability is usually nurtured with shape-diversified training data; however, manually collecting such a corpus of data is infeasible due to the requirement of intense human labor and large collections of divergent objects at hand. In this paper, we propose ShapeGen, a data generation method that aims at generating shape-variated manipulation data in a simulator-free and 3D manner. ShapeGen decomposes the process into two stages: Shape Library curation and Function-Aware Generation. In the first stage, we train spatial warpings between shapes mapping points to points that correspond functionally, and aggregate 3D models along with the warpings into a plug-and-play Shape Library. In the second stage, we design a pipeline that, leveraging established Libraries, requires only minimal human annotation to generate physically plausible and functionally correct novel demonstrations. Experiments in the real world demonstrate the effectiveness of ShapeGen to boost policies' in-category shape generalizability. Project page: https://wangyr22.github.io/ShapeGen/.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.