TokenLight: Precise Lighting Control in Images using Attribute Tokens
Sumit Chaturvedi, Yannick Hold-Geoffroy, Mengwei Ren, Jingyuan Liu, He Zhang + 3 more
TLDR
TokenLight enables precise, continuous image relighting by using attribute tokens to control various illumination factors like intensity and 3D light positions.
Key contributions
- Introduces TokenLight, a method for precise, continuous image relighting.
- Uses attribute tokens to control distinct lighting factors like intensity, color, and 3D position.
- Trained on large synthetic datasets, enhanced with real captures for improved realism.
- Achieves state-of-the-art performance, handling complex light-scene interactions robustly.
Why it matters
This paper offers a novel approach to image relighting, providing unprecedented control over lighting attributes. Its ability to handle complex light interactions without explicit inverse rendering makes it highly practical for various applications, advancing realistic image editing and synthesis.
Original Abstract
This paper presents a method for image relighting that enables precise and continuous control over multiple illumination attributes in a photograph. We formulate relighting as a conditional image generation task and introduce attribute tokens to encode distinct lighting factors such as intensity, color, ambient illumination, diffuse level, and 3D light positions. The model is trained on a large-scale synthetic dataset with ground-truth lighting annotations, supplemented by a small set of real captures to enhance realism and generalization. We validate our approach across a variety of relighting tasks, including controlling in-scene lighting fixtures and editing environment illumination using virtual light sources, on synthetic and real images. Our method achieves state-of-the-art quantitative and qualitative performance compared to prior work. Remarkably, without explicit inverse rendering supervision, the model exhibits an inherent understanding of how light interacts with scene geometry, occlusion, and materials, yielding convincing lighting effects even in traditionally challenging scenarios such as placing lights within objects or relighting transparent materials plausibly. Project page: vrroom.github.io/tokenlight/
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.