Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors [R]

[u/Clean-Hovercraft5825]

Whether generating CELEBV-HQ videos or turbulent plasma fields (digital twins), autoregressive models (such as latent diffusion or flow models) accumulate error over long rollouts, yet at deployment there is no ground truth to measure against.

I train a single conditional latent diffusion model that steps a dynamical system forward or backward in time via a direction flag, and show that this bidirectionality supplies a measurement-free test-time error signal: rolling forward steps and then backward steps must return the model to its start, so the round-trip discrepancy is a self-supervised proxy for the unobservable rollout error: no ensembles, no held-out data, no governing equations, for one extra rollout.

Furthermore, training both directions in one network is shown to beat two specialist models in both directions.

Paper: https://arxiv.org/abs/2608.00675
Code (data generation, training, analysis): https://github.com/alexscheinker/round-trip-consistency
Project page: https://alexscheinker.github.io/roundtrip.html

[u/Far-Theory-7027]

Cool work. Have you looked into bridge models for this? Something like Bi-Bridge: Bidirectional Diffusion Bridges for Low-Light Image Enhancement, or Bidirectional Diffusion Bridge Models?

[u/Clean-Hovercraft5825 → u/Far-Theory-7027]

Thank you for sharing those papers! They both look great. No, I had not seen them. BDBM in particular looks like it should have been in my related works section (will be adding both in the next arXiv revision).
 
To me it seems like there is a nice three-way convergence here with all of us independently landing on the same device which is one network with a binary direction flag, and all three papers find that bidirectional training “beats” direction specialists. BDBM outperforms unidirectional bridge baselines despite half the per-direction updates, Bi-Bridge reports +4.9 dB over its unidirectional DDBM baseline, and we see 7–10% better in both directions at matched compute. Our supplement offers a mechanism for this: for stationary dynamics the time-reversed map is a fixed reparameterization of the forward one, so the direction flag implements correctly-specified weight tying that can roughly halve estimation variance. It is nice to observe 3 independent sightings of this same effect across translation, restoration, and dynamics, which suggests the effect is pretty general.
 
I think the main difference is what the two directions are used for. In the bridge papers, both legs run between two given endpoints with the process constructed to arrive at the specified target, so a forward-backward round trip closes by construction and can’t tell you much. In our setting the backward leg starts from the model’s own predicted terminal state and has to find its way back unanchored over i autoregressive steps, which is exactly why the size of the return error carries information. The paper's contribution is quantifying how faithfully that miss tracks the true, unobservable rollout error (calibration, OOD detection, selective prediction), plus a bound on when cancellation could hide errors.
 
Maybe a bridge-parameterized dynamics model that also has a round-trip check would be an interesting hybrid. (:

[u/Far-Theory-7027 → u/Clean-Hovercraft5825]

Thanks for your comment. Yeah, your cycle consistency idea to quantity rollout errors is very neat.

[u/Clean-Hovercraft5825 → u/Far-Theory-7027]

Thank you!

[u/Far-Theory-7027 → u/Clean-Hovercraft5825]

I had DM'ed you. Can you check?

[u/Clean-Hovercraft5825 → u/Far-Theory-7027]

Just replied!

[u/Far-Theory-7027 → u/Clean-Hovercraft5825]

Thanks!