When Numbers Speak: Aligning Textual Numerals and Visual Instances in Text-to-Video Diffusion Models
Zhengyang Sun, Yu Chen, Xin Zhou, Xiaofan Li, Xiwu Chen + 2 more
TLDR
NUMINA improves numerical alignment in text-to-video diffusion models by guiding regeneration, boosting counting accuracy and CLIP alignment.
Key contributions
- Introduces NUMINA, a training-free framework to align textual numerals with visual instances in text-to-video models.
- Identifies prompt-layout inconsistencies using attention heads to derive a countable latent layout.
- Guides regeneration by refining the layout and modulating cross-attention.
- Improves counting accuracy by up to 7.4% and enhances CLIP alignment on CountBench.
Why it matters
Text-to-video models often fail to generate the correct number of objects. This paper offers a practical, training-free solution to a common limitation. It significantly improves numerical accuracy and content alignment, paving the way for more reliable video synthesis.
Original Abstract
Text-to-video diffusion models have enabled open-ended video synthesis, but often struggle with generating the correct number of objects specified in a prompt. We introduce NUMINA , a training-free identify-then-guide framework for improved numerical alignment. NUMINA identifies prompt-layout inconsistencies by selecting discriminative self- and cross-attention heads to derive a countable latent layout. It then refines this layout conservatively and modulates cross-attention to guide regeneration. On the introduced CountBench, NUMINA improves counting accuracy by up to 7.4% on Wan2.1-1.3B, and by 4.9% and 5.5% on 5B and 14B models, respectively. Furthermore, CLIP alignment is improved while maintaining temporal consistency. These results demonstrate that structural guidance complements seed search and prompt enhancement, offering a practical path toward count-accurate text-to-video diffusion. The code is available at https://github.com/H-EmbodVis/NUMINA.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.