One-Shot Generative Flows: Existence and Obstructions
Panos Tsimpos, Daniel Sharp, Youssef Marzouk
TLDR
This paper explores "straight-line flows" for generative models, showing they exist for Gaussian distributions but are impossible for multi-modal targets.
Key contributions
- Characterizes "straight-line flows" for generative models with independent endpoints using PDEs.
- Constructs explicit straight-line processes for arbitrary Gaussian distributions.
- Proves straight-line flows are impossible for targets with sufficiently well-separated modes.
- Uncovers a fundamental link between sample-path behavior and flow map geometry.
Why it matters
This work provides a foundational theory for "straight-line flows" in generative modeling, which are desirable for their exact integrability. By defining their existence limits, it guides the design of more efficient and robust generative models.
Original Abstract
We study dynamic measure transport for generative modelling in the setting of a stochastic process $X_\bullet$ whose marginals interpolate between a source distribution $P_0$ and a target distribution $P_1$ while remaining independent, i.e., when $(X_0,X_1)\sim P_0\otimes P_1$. Conditional expectations of this process $X_\bullet$ define an ODE whose flow map transports from $P_0$ to $P_1$. We discuss when such a process induces a \emph{straight-line flow}, namely one whose pointwise acceleration vanishes and is therefore exactly integrable by any first-order method. We first develop multiple characterizations of straightness in terms of PDEs involving the conditional statistics of the process. Then, we prove that straightness under endpoint independence exhibits a sharp dichotomy. On one hand, we construct explicit, computable straight-line processes for arbitrary Gaussian endpoints. On the other hand, we show straight-line processes do not exist for targets with sufficiently well-separated modes. We demonstrate this through a sequence of increasingly general impossibility theorems that uncover a fundamental relationship between the sample-path behavior of a process with independent endpoints and the space-time geometry of this process' flow map. Taken together, these results provide a structural theory of when straight generative flows can, and cannot, exist.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.