SyncFix: Fixing 3D Reconstructions via Multi-View Synchronization
Deming Li, Abhay Yadav, Cheng Peng, Rama Chellappa, Anand Bhattad
TLDR
SyncFix is a diffusion-based framework that uses multi-view synchronization to fix semantic and geometric inconsistencies in 3D reconstructions.
Key contributions
- Enforces cross-view consistency in diffusion-based 3D scene refinement.
- Formulates refinement as a joint latent bridge matching problem across multiple views.
- Trains on image pairs, generalizing to arbitrary views with quality improving with more views.
- Achieves high-quality 3D reconstructions, surpassing SOTA even without clean references.
Why it matters
Inconsistent 3D reconstructions are a major problem. SyncFix provides a novel diffusion-based approach to fix these issues by ensuring multi-view consistency. It outperforms existing methods, delivering high-quality results even with limited reference data, making it highly practical.
Original Abstract
We present SyncFix, a framework that enforces cross-view consistency during the diffusion-based refinement of reconstructed scenes. SyncFix formulates refinement as a joint latent bridge matching problem, synchronizing distorted and clean representations across multiple views to fix the semantic and geometric inconsistencies. This means SyncFix learns a joint conditional over multiple views to enforce consistency throughout the denoising trajectory. Our training is done only on image pairs, but it generalizes naturally to an arbitrary number of views during inference. Moreover, reconstruction quality improves with additional views, with diminishing returns at higher view counts. Qualitative and quantitative results demonstrate that SyncFix consistently generates high-quality reconstructions and surpasses current state-of-the-art baselines, even in the absence of clean reference images. SyncFix achieves even higher fidelity when sparse references are available.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.