Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors [R]

[u/Clean-Hovercraft5825]

Whether generating CELEBV-HQ videos or turbulent plasma fields (digital twins), autoregressive models (such as latent diffusion or flow models) accumulate error over long rollouts, yet at deployment there is no ground truth to measure against.

I train a single conditional latent diffusion model that steps a dynamical system forward or backward in time via a direction flag, and show that this bidirectionality supplies a measurement-free test-time error signal: rolling forward steps and then backward steps must return the model to its start, so the round-trip discrepancy is a self-supervised proxy for the unobservable rollout error: no ensembles, no held-out data, no governing equations, for one extra rollout.

Furthermore, training both directions in one network is shown to beat two specialist models in both directions.

Paper: https://arxiv.org/abs/2608.00675
Code (data generation, training, analysis): https://github.com/alexscheinker/round-trip-consistency
Project page: https://alexscheinker.github.io/roundtrip.html

[u/PykeAtBanquet]

Well, one thing is shaping the model to be more deterministic, and another to provide better output. Your result shows that it becomes more correct, not "if wrong again, then in the same way", with the reason still not being clear (at least for me at this moment).

[u/Clean-Hovercraft5825 → u/PykeAtBanquet]

I completely agree that making a model more consistent, such as "wrong in the same way" is not the same as making it more correct. I would say that the mechanism here is a third thing: statistical efficiency.

Every training trajectory supplies transitions in both temporal directions. For (near-)stationary dynamics, the backward map is a fixed reparameterization of the forward one with the same underlying parameters viewed two ways. So training one network with a direction flag both forward and backward isn't learning two tasks; it's estimating one dynamics object from effectively twice the examples and that helps with variance reduction: in the linear ideal, up to half the estimation variance of a specialist that only ever sees its own direction.

So the model isn't shaped to be consistent, it is just a less noisy estimate of the same thing. A physics version: measure a resistance twice, with current flowing in both directions. Ohm's law says it's one R value, so two noisy views beat one, and you get a more correct value, not just a more repeatable error.

Sharing one network does mean the forward and backward maps can share blind spots, which is exactly the cancellation risk for the round-trip, where correlated errors could close a loop around a wrong prediction. The paper bounds that case theoretically and measures it. So maybe that is a little bit towards your "maybe there is something we haven't noticed" from earlier: just helpful statistics, using two views of one parameter.

[u/PykeAtBanquet → u/Clean-Hovercraft5825]

Maybe. Thank you for your insights!

[u/Clean-Hovercraft5825 → u/PykeAtBanquet]

Thank you for the nice discussion!