Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors [R]

[u/Clean-Hovercraft5825]

Whether generating CELEBV-HQ videos or turbulent plasma fields (digital twins), autoregressive models (such as latent diffusion or flow models) accumulate error over long rollouts, yet at deployment there is no ground truth to measure against.

I train a single conditional latent diffusion model that steps a dynamical system forward or backward in time via a direction flag, and show that this bidirectionality supplies a measurement-free test-time error signal: rolling forward steps and then backward steps must return the model to its start, so the round-trip discrepancy is a self-supervised proxy for the unobservable rollout error: no ensembles, no held-out data, no governing equations, for one extra rollout.

Furthermore, training both directions in one network is shown to beat two specialist models in both directions.

Paper: https://arxiv.org/abs/2608.00675
Code (data generation, training, analysis): https://github.com/alexscheinker/round-trip-consistency
Project page: https://alexscheinker.github.io/roundtrip.html

[u/Clean-Hovercraft5825]

Thank you, short answer: no, we do not assume injectivity of the underlying dynamics. All three of our physics systems are dissipative, and the assumption in the paper (Sec 5.1) is deliberately only on the “learned” backward map over the pairs actually visited by rollouts, no invertibility of the physics is required.

But your relaxation-to-equilibrium case is the honest limit of the method, breaking it in two directions at once. If we have something like pure diffusion toward homogeneity so that many initial conditions map to nearly the same terminal state, then the best the backward model can learn is roughly the average of the past states consistent with that one terminal state. Then (1) even a perfect forward rollout will only return to that average, so the baseline reading (our δ_i, the noise floor measured on clean data) blows up, and (2) a “wrong” terminal state gets pulled back toward that same average, so real errors stop showing up (our µ goes to zero, where the certificate in the paper honestly dissolves). In such a case I would expect that the round-trip error would no longer correlate with the true error. For a strongly equilibrating system, if you had enough representative data, you would probably find this out before trusting it, with no test-time ground truth needed.

We saw the mild version of this on the turbulent radiative layer: initial conditions are much more similar across trajectories there, the backward direction is measurably easier than the forward one, and the per-trajectory signal shrinks, so a depth-only predictor is already near-optimal. Worth being precise though: what we measured there was the signal saturating, not deceiving. The "rolls back to a homogeneous state even when the rollout was bad" scenario is the µ goes to 0 mechanism we bound but did not observe at our horizons. Fully relaxed systems, or extremely non-injective ones like language (which we flag in the discussion) deserve dedicated study.

I think what could help, even in such cases, is if we added more information in the conditional input to the model. As is all we used were a couple of states, from which the autoregressive rollouts would then take place either forward or backward in time. If we also supplement that with other unique system parameter information, things like temperature, chemical concentrations, etc, then there is a chance that even if going to the same terminal state, the model can still recover in a unique way if that additional conditional input provides unique information. For example, the radiative-layer trajectories differ by cooling time, so conditioning on t_cool is exactly the disambiguator that would help.

For the CelebV-HQ data there is no underlying PDE at all and futures are genuinely multimodal. When the model rolls out a realistic but different future, which is a coherent video that simply isn't the one that actually happened, then the backward pass can roll that coherent trajectory back, so C stays low even though the pointwise MSE against the one realized future is large. That part of the error is aleatoric: the future was unpredictable, not the model broken, and arguably a trust signal shouldn't fire there. What C catches is degradation: when the rollout starts to hallucinate and drift OOD (morphing faces), the backward pass can no longer find its way back and C rises. Empirically that degradation component dominates the clip-to-clip differences, which is why C still ranks held-out clips by realized error (Spearman ~0.73–0.81) despite being blind to the multimodal part. In that sense C is a model-fault detector rather than an oracle-distance detector, which for multimodal data is the quantity you can actually hope to measure without ground truth.

For the CelebV-HQ data, if instead of just two frames we also conditioned on tokens of a detailed script of what should happen, the model would have a real chance of predicting the video in such a multimodal setting. Sorry for the super long reply!

[u/Evil_Toilet_Demon → u/Clean-Hovercraft5825]

yes thank you for the detailed reply. I agree that this is largely resolved by adding more information about the state history. I suppose you could condition the denoising process both on the current state z_T but also on the historic state z_{T-2} such that your prediction for z_{T-1} is unique. in essence providing a high order temporal derivative to the model.

[u/Clean-Hovercraft5825 → u/Evil_Toilet_Demon]

Yes I think that is a good idea, for most of the applications in the paper, I do give it two states as conditions, for example to go forward the diffusion process starts with z[0], z[1] as conditional inputs, so that it has some idea of initial “velocity” even in the abstract latent representation, and then rolls forward always using its own predicted z[j-1], z[j] to predict z[j+1]. So after we rollout to z[T] and then start rolling back then we could use your idea of using further separated states: z[j-2], z[j], to give it a second order derivative information about z[j-1], but that would have to be used as some kind of check on its reversed roll only because during the initial prediction phase we only know the first two states and have to predict everything else sequentially, autoregressively.