3D-ReGen: A Unified 3D Geometry Regeneration Framework
Geon Yeong Park, Roman Shapovalov, Rakesh Ranjan, Jong Chul Ye, Andrea Vedaldi + 1 more
TLDR
3D-ReGen is a unified framework that regenerates 3D objects from 2D images and initial 3D shapes, enabling controllable enhancement and editing.
Key contributions
- Introduces 3D-ReGen, a unified framework for 3D object regeneration from 2D images and initial 3D shapes.
- Enables diverse tasks like 3D enhancement, reconstruction, and controllable editing.
- Uses a novel VecSet-based conditioning mechanism for consistent fine-grained detail updates.
- Learns regeneration prior self-supervised from existing 3D datasets without extra annotations.
Why it matters
This paper introduces a highly controllable 3D regeneration framework, addressing the limitations of one-shot generators. By conditioning on initial 3D shapes, it unlocks diverse applications like enhancement and editing, making 3D content creation more flexible and precise. The self-supervised learning approach reduces annotation burden.
Original Abstract
We consider the problem of regenerating 3D objects from 2D images and initial 3D shapes. Most 3D generators operate in a one-shot fashion, converting text or images to a 3D object with limited controllability. We introduce instead 3D-ReGen, a 3D regenerator that is conditioned on an initial 3D shape. This conceptually simple formulation allows us to support numerous useful tasks, including 3D enhancement, reconstruction, and editing. 3D-ReGen uses a new conditioning mechanism based on VecSet, which allows the regenerator to update or improve the input geometry with consistent fine-grained details. 3D-ReGen learns a widely applicable regeneration prior from off-the-shelf 3D datasets via self-supervised pretext tasks and augmentations, without additional annotations. We evaluate both the geometric consistency and fine-grained quality of 3D-ReGen, achieving state-of-the-art performance in controllable 3D generation across several tasks.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.