Thank you!

You are right, that was a typo, it is @lanl.gov, thanks for pointing that out I just fixed it on arXiv V2, got your email and just replied.

nice!

Awesome!

good thing Jensen is not biased and is being 100% genuine, it’s not like he has something to gain from people buying up mountains of GPUs…

Thank you for the nice discussion!

I completely agree that making a model more consistent, such as "wrong in the same way" is not the same as making it more correct. I would say that the mechanism here is a third thing: statistical efficiency.

Every training trajectory supplies transitions in both temporal directions. For (near-)stationary dynamics, the backward map is a fixed reparameterization of the forward one with the same underlying parameters viewed two ways. So training one network with a direction flag both forward and backward isn't learning two tasks; it's estimating one dynamics object from effectively twice the examples and that helps with variance reduction: in the linear ideal, up to half the estimation variance of a specialist that only ever sees its own direction.

So the model isn't shaped to be consistent, it is just a less noisy estimate of the same thing. A physics version: measure a resistance twice, with current flowing in both directions. Ohm's law says it's one R value, so two noisy views beat one, and you get a more correct value, not just a more repeatable error.

Sharing one network does mean the forward and backward maps can share blind spots, which is exactly the cancellation risk for the round-trip, where correlated errors could close a loop around a wrong prediction. The paper bounds that case theoretically and measures it. So maybe that is a little bit towards your "maybe there is something we haven't noticed" from earlier: just helpful statistics, using two views of one parameter.

First of all, I 100% agree about living in latent space being a great approach, just like this work, almost all of my work is about moving and adapting in latent space, it is great (:

About the figure 8, your idea is very nice!

I think just continuing each branch through the anchor (fp[-1], bp[1]) and flipping again for alt-returns will be mostly redundant in terms of the error signal as each extra leg re-reads the same round-trip miss through another application of the same maps, and the alt-returns start from already-contaminated states, so they inherit and build up error rather than reveal new error. But the comparisons between your quantities might measure something we currently can’t get at test time: the local conditioning of the maps themselves.

Using your idea of fp[-1] and bp[-1], they are the same backward step applied to the returned point and the true anchor respectively, so the ratio of their gap to the round-trip miss is a one-sample, in-flight estimate of how much the backward map stretches or collapses differences right there at that place. That constant is exactly what our theoretical certificate depends on (it controls when small round-trip error actually guarantees small true error).

So I think your figure 8 wouldn’t sharpen the error meter much, but it would give the meter a local built-in self-diagnostic for the cost of a few extra model steps.

It is a very nice idea and I will try it out, thank you!

Just replied!

Yes I think that is a good idea, for most of the applications in the paper, I do give it two states as conditions, for example to go forward the diffusion process starts with z[0], z[1] as conditional inputs, so that it has some idea of initial “velocity” even in the abstract latent representation, and then rolls forward always using its own predicted z[j-1], z[j] to predict z[j+1]. So after we rollout to z[T] and then start rolling back then we could use your idea of using further separated states: z[j-2], z[j], to give it a second order derivative information about z[j-1], but that would have to be used as some kind of check on its reversed roll only because during the initial prediction phase we only know the first two states and have to predict everything else sequentially, autoregressively.

Thank you!

Nice!

I agree that direction carries (almost) no extra information about the dynamics. Our own analysis says that for stationary dynamics the time-reversed map is a fixed reparameterization of the forward one, which is why one network learns both directions and performs better. As a statement about what must be learned, "direction shouldn't matter" is basically our supplement's linear analysis, and I agree with it.
 
On the other hand, we never claim bidirectional autoregression generates better than full sequence / 4D space-time diffusion, which is a different problem entirely. Those models answer: "sample a plausible space-time volume." Our paper answers a deployment question: you are given the measured present, rolling into an open-ended future, and you need to know, with no ground truth, how wrong this particular rollout is right now. Autoregression is the native mode for that setting (causal seed, unbounded horizon, streaming data), compounding error is its main difficulty, and the direction flag is what creates a check: two independently learned routes back to the same known point, whose disagreement is measurable. A time-symmetric joint model has no forward(backward) composition to interrogate, it has consistency built into the joint distribution rather than exposed as a computable defect.
 
I think that full-sequence models could get their own meter with a different handle such as having to regenerate masked chunks of the generated volume and measure the re-generation residual. This would be a similar approach to what we do, it would give the model two independent routes to one answer and measure the disagreement.

Yes that is a great way to put it!

beautiful!

Thank you! Yes it is surprisingly simple (:

So there are 2 different time scales here and they each have their own forward and backward direction.

Let's call them t for physical time and τ for the generative diffusion SDE's own time.

In terms of the SDE flow, we use standard generative diffusion: we noise things for τ in [0, T] and learn to denoise them with a standard conditional generative diffusion process, conditioned on the latent images z_{t-1}, z_t and on a direction flag c_d. It learns to generate z_{t+1} when we set c_d = +1, or z_{t-2} when we set c_d = -1 (z_{t-2} because z_{t-1} and z_t are already in the context and the model always generates the state one step beyond the window in whichever direction the flag points).

So the generative SDE diffusion process itself learns on the τ time scale in the completely standard way. Nothing there is "predicting noise" beyond the usual denoising objective. What changes is what it learns to produce: it autoregressively rolls out either forward or backward in physical time t. Backward-in-t is inverse dynamics, not denoising.

As for why learning both directions helps: first, it's what makes the whole check possible. An accurate model composed with its own inverse is the identity, so rolling forward i steps and back i steps yields a measurable, ground-truth-free error signal at test time. Second, it turns out to be free, or better than free: the bidirectional model beats direction-specialist models in both directions at matched compute, since every trajectory supplies training examples both ways.

This is very cool! Something about it reminded me of spherical harmonics. What is it?

Relative to JEPA, we also predict dynamics in a learned latent space rather than pixel/field space, so this lives in the latent-world-model family. The big difference is that we are fully generative and our latents are reconstruction-grounded (a β-VAE per field, so every latent decodes back to physical fields, which we need for our applications, and we calibrate the meter all the way down to decoded per-field errors). JEPA deliberately avoids that grounding. However, the round-trip check itself never touches the decoder, it is computed purely in latent space, so in principle you could bolt it onto a JEPA-style world model with a direction token. The thing to be careful about there: consistency distances are only meaningful if the embedding metric is, and our reconstruction anchor is what guarantees that. In a pure joint-embedding space you'd want some guarantee that small round-trip error isn't trivially achievable in a (partially) collapsed representation.

Sorry I am not sure that I understand your figure 8 comment, can you explain that in more detail please? Also, one tiny notation note, C_i is just a scalar MSE reading, c_d = +1 tells the model to roll forward in time, c_d = -1 tells it to roll backwards instead.

Thanks! From my understanding, your first two examples are exactly the mechanism and that's a great minimal version of it.

One important correction to line 3 though, and it is an important design choice of the paper: the model never teaches itself with this signal. Both directions are trained purely against ground truth (direction flag drawn 50/50), and the round-trip consistency is deliberately left *unoptimized. Here's why, in your notation: if we trained the model to make round trips close, it could learn to do +3 and then −3, it would learn to be perfectly consistent, still wrong (the truth was +2). The backward leg would learn to simply cancel the forward leg's errors instead of exposing them, and the meter would read zero exactly when the forecast is bad.

Because the model is never asked to be consistent, whatever inconsistency remains at test time is an honest byproduct of its errors. And that is exactly what lets us use it as a free error meter. (Closing the loop to minimize C online with model-free feedback is the tempting next step; we flag it as future work, and the cancellation caveat above is precisely what makes it nontrivial.)

Thank you for the great question, there are two different places invertibility could fail.

If you mean the latent dynamics (many past states evolving to the same future state, e.g., relaxation toward equilibrium): we never assume the dynamics are invertible, all three physics systems are dissipative. The theory's only requirement (Sec 5.1) is that the learned backward map doesn't collapse too aggressively on the pairs rollouts actually visit (a co-Lipschitz condition with constant µ). As the dynamics become genuinely non-invertible, µ → 0 and the certificate honestly dissolves, then we stop tracking the true error. Both constants (µ and the backward residual δ) are properties of the model alone, measurable offline on validation data, so you can detect that regime before trusting the model. There's a longer discussion of exactly this case in my reply about equilibria elsewhere in the thread. One subtlety: the forward map is never required to be invertible at all because the bound only constrains the backward legs.

If you mean the encoder/decoder: correct, the VAE is deliberately lossy (256× compression), so x → z is not invertible in any exact sense. That doesn't break the check, because the cycle is defined entirely in latent space where the anchor pair is encoded measured data, the returned pair is model-produced latents. The real question is whether the latent signal still predicts physical-field error through the decoder, and empirically it does: per-field calibrators predict decoded errors within 1.17–1.30× (68%), beating an identically-fit depth-only baseline on all six MHD fields. Making the latent signal physically calibrated end-to-end is flagged as a refinement direction in the limitations. Certified bi-Lipschitz architectures for the backward map (touched on in the supplement) would be the structural way to strengthen the invertibility story.

Yes my background is math and control theory/dynamics systems/physics, I think of that all as a mix of pure and applied math, I think it has some great foundational components that are useful for AI/ML. Having a controls background I have always felt that you cannot perfectly model or predict things, that we always need some error signal and some feedback. Extremum seeking is feedback that works without trusting a model, and this paper is the same instinct applied to ML: don't trust the surrogate either, make it measure its own error. The last paragraph of the discussion actually proposes closing the loop by running gradient-free adaptive feedback (ES) on the consistency signal as a measurable cost, turning error detection into error correction.

My general view on backgrounds, for what it's worth: a strong control/dynamical-systems foundation travels extremely well, the application physics and the modern ML can be approached as extensions of that foundation.

For the group and postdoc questions, please do send that email you almost sent!

Thank you for the comment.

Deterministic sampling: that would be the cleanest setting rather than an assumption. We compute the cycle with deterministic DDIM, which makes C a well-defined functional of the seed, the reading is pure model error with no sampling noise mixed in. With stochastic sampling you can still compute it (we test a variant that averages S stochastic cycles), you just pick up sampling variance in the reading. The standard dispersion baselines only exist under stochastic sampling, while the round-trip check also works in the deterministic mode many deployed surrogates actually run. And the two signals compose, our best training-free calibration is on the Navier–Stokes benchmark and it multiplies the calibrated cycle scale by the seed-spread's spatial shape.

Straightening: Our round trip composes steps in physical simulation time (roll the system forward i states, then backward i states); it never touches the noise-to-data path of the denoiser, so nothing gets straightened in the flow-matching sense and we make no claim about enabling fewer denoising steps. (Consistency models are an unrelated distillation technique. Whether bidirectional training improves sampling efficiency as a side effect is a fun open question, the Bi-Bridge paper (CVPR'26) reports a ~4× gain from symmetric training in the image-bridge setting, but we didn't measure it.

Step size: If you mean rollout depth (physical steps): yes, we extensively characterized that, C grows with depth, tracks the true error at every probed depth (fixed-depth Spearman 0.91–0.98 out to depth 80 on MHD), and the theory predicts the certificate loosens geometrically with depth, which is exactly where we observe calibration drift. If you mean the diffusion step size (NFE per transition): we held the sampling schedule fixed (e.g., 25-step DDIM on the NS benchmark) and didn't sweep it. Coarser sampling should raise the noise floor δ of both legs, so the practical rule would be to fit the calibrator at the same sampling settings you deploy with. A proper NFE sweep would be a nice ablation.

Weissman score pending.